Ecommerce Search Planning and Evaluation Framework, Version 1
How to Plan and Select Search, Navigation, and Discovery Solutions for Ecommerce Web Sites
By Susan E. Aldrich, August 28, 2008

NETTING IT OUT

We define ecommerce search as the technologies companies deploy to connect customers with products and answers. This includes search, navigation, and discovery; plus merchandising, searchandising, and tools to manage the customer experience.

Great search and navigation have the potential to deliver huge benefits. We have spoken with companies that increased cart size by 270 percent and eliminated 60 percent of customer support calls by implementing effective ecommerce search. Whether or not your results reach these heights (or exceed them) depends on how poor your seeker experience is today, the quality of your content, and how effective your organization is at managing the seeker experience once you’ve got the right technologies in place.

Search is far from being just a technology solution, because information and seekers’ needs are constantly evolving. You must assign responsibility for the quality of the seeker experience, monitor the quality, and take actions to improve it. You must also assign responsibility for the quality of your information collections, monitor the quality, and take actions to improve information quality.

To simplify the selection of products that can contribute to ecommerce implementation, we’ve developed our framework for evaluating ecommerce search products and architectures. This framework describes evaluation criteria in the areas of seeker interfaces, seeker experience management, marketing management, information collection management, architecture, and product and company viability. We also provide our recommendations on search-related metrics and responsibilities.

Since 2003, we have been using our earlier evaluation frameworks to assess vendors’ offerings for search and navigation, for ecommerce, enterprise, and customer service applications. To date, we’ve assessed offerings from ATG, Autonomy, Celebros, Endeca, FAST, Fredhopper, Google, InQuira, IBM, Mercado, Microsoft, Northern Light, Oracle, SLI Systems, and Thunderstone, as well as solutions from EasyAsk (acquired by Progress Software), iPhrase (acquired by IBM), Knova (acquired by Consona), Verity (acquired by Autonomy), and WebSideStory (acquired by Omniture).

This framework also provides our assessment of goals and metrics for key stakeholders in successful search and navigation. As with our earlier frameworks, we plan to evaluate leading solutions against our criteria and then prepare a side-by-side comparison and a ranking of the solutions.

ECOMMERCE SEARCH REQUIREMENTS

Ecommerce search offerings have for some years taken on user interface, marketing, and merchandising tasks that are common to ecommerce platforms. Recently, they have begun to expand into Internet search marketing, adding search engine optimization (SEO) and even search engine marketing (SEM) capabilities. This is good news for retailers and customers, because the resulting solutions have greatly improved the ecommerce search experience.

Forty percent of visits start with search. You deliver a great search experience if you have consistent, complete, high quality product information; merchandising skills and resources; resources to monitor, test, and improve the customer search experience from Internet to cart. Two out of three companies need to improve their product information quality to support search. One in eight companies has no resource monitoring the search experience. Not surprisingly, many companies fail to consider the range of requirements they will need in order to achieve the quality of search experience their customers need.

As you consider an ecommerce search solution, keep in mind that it must address the requirements of several groups of stakeholders. The key stakeholders—those who will use the technology or rely on the technology to achieve their goals—include the following groups:

• Seekers. Customers who use the technology to fill their needs, and the partners and employees who support them.

• Marketing Managers. Corporate, brand, and product marketing, as well as merchandisers, who use search marketing tools to manage lead and revenue generation.

• Information Collection Managers or Stewards. Employees in the role of information collection stewards use search management tools to manage the contents, organization, quality, and findability of various collections (e.g., customer support knowledgebase, corporate Web pages, and the product catalog).

• Customer Advocates. Managers responsible for the quality of relationships with customers and partners who negotiate with internal service providers (such as technologists and information collection managers) for better quality of customer experience.

• IT Architects, Developers, and Operations. Technologists who deploy and support the search engine indexing, retrieval, reporting, tuning, and integration.

Three groups—seekers, customer advocates, and information collection managers—represent roles rather than titles or job descriptions. At many companies, the responsibilities of collection managers and advocates have never been considered or assigned. This is not at all logical, but it is definitely traditional. Outside of the publishing realm, companies haven’t needed information collection managers until the Internet made everyone a publisher. The concept of assigning responsibility for findability of information seems foreign to many executives, who will mistakenly dump the job on IT staff. Yet it is business people who know which product or document solves which problems in their area of specialty, and who know which products should be presented to customers.

What’s harder to make sense of is the disinclination to identify customer and partner advocates. Shouldn’t a company want to measure the quality of customer experience or the efficiency of partner relationships? Is it not important to know the obstacles to doing business that the company’s policies have erected? Wouldn’t you want to know if the cross-channel, cross-lifecycle customer experience was improving or devolving? At many companies, the answer is apparently, “of course not!” At many companies, the VP of sales is responsible for all customer experience, for all segments, as long as the person is in the throes of buying. During those parts of the day that a person seeks help with previous purchases, well, the head of customer support owns that experience. Unless the person’s question is best answered by information about a product line, in which case his experience is owned by corporate marketing. In short, the customer is in charge of the quality of his entire customer experience, and it is up to him to find a way around any obstacles and inconsistencies. If he is really struggling, he can go straight to the person who does own the cross-channel, cross-lifecycle customer experience, the CEO.

Based on my own customer experiences, more companies should be contacting us for help fixing their disjointed customer experience.

The goals and metrics of these stakeholders will be addressed in more detail as we analyze the planning and evaluation criteria for ecommerce search solutions.

Search Supports the Customer Lifecycle

In our view, search and findability are critical to the customer experience, across the customer lifecycle, and it is a key tool for suppliers in keeping each customer active in all phases of the lifecycle: plan, explore, select, buy, use, maintain, and replace.

Customers are likely to interact directly with your site search for ecommerce, customer self-service, customer support, and learning. They indirectly experience your internal search engines when contacting customer support to resolve problems or to review invoices, shipping status, order history, and order status. Customers would like all of “their” information available from their own personalized page or portal. You may not yet have the pieces in place to offer that, but your ecommerce search solution, especially in a B2B environment, must be able to support the full range of information retrieval needs.

B2B has Special Requirements

In our experience with our clients, we’ve seen a repetition of a set of requirements specific to the B2B arena. Some of the B2C requirements can seem foolish in the B2B realm – great product images, terrific cross-sell offers, customer reviews. Likewise, some B2B requirements are nonsensical from the B2C perspective. However, this is an ecommerce framework, so we need to look at both aspects. We’ll call out the items that are strictly B2B. The most important of these is support for customer and partner portals and support for multi-tier distribution networks. While ecommerce search solutions have not as yet tackled these problems, there are features of the solutions that can make it easier for their customers to be successful. These features are included in our criteria.

Requirements Derived from Customer Scenarios®

We apply our Customer Scenario® methodology to determine the requirements on which we base our evaluation criteria. We see two distinct groups of people whose success depends on your search capabilities: The first consists of seekers, those people who are your customers, partners, employees, and other stakeholders; the second group consists of information owners, who have a variety of goals from quality of customer experience to cost cutting to relationship deepening.

ECOMMERCE SEARCH FRAMEWORK

Our framework for ecommerce search solutions has six categories of planning and evaluation criteria. (See Illustration 1.) The first four of these categories address capabilities that interact to deliver the seeker’s search and navigation experience. The six evaluation categories are:

• Seeker Interface. Seeker experience addresses what types of searches or queries can be performed and how the query is translated and executed. It also addresses how the results are organized and presented, including capabilities that ensure that the user is neither overwhelmed with choices nor presented with no results or guidance.

Ecommerce Search Planning and Evaluation Matrix

© 2008 Patricia Seybold Group

Illustration 1. This diagram presents our evaluation matrix for ecommerce search solutions.

This report continues...