Tealeaf: Visibility. Insight. Answers.

The Golden Teapot Award - Customer Experience Management Excellence

First off, I have to say that I wasn't there personally to see this magic moment, however the story is so compelling that I want to share it with you.

For years now, we've been proud of the fact that the Tealeaf solution is passive. In “techie” terms that means:

• We take no CPU or other resources
• We add no latency to the site
• We add no risk – because we're “passive”

But, just because Tealeaf is passive, doesn't mean that the usage of Tealeaf has to be.

And for that, I am awarding my own little prize - the Golden Teapot for interesting Tealeaf “story-of-the day” -  to our customer Gold Medal Travel, a leading independent travel site with headquarters in the UK. (Sorry guys, there is no actual award, but as you’ll see below, they are being handsomely rewarded for their innovation….)

At our recent UK Customer Forum event (a great event we hold annually in London that brings together our UK customers and prospects), Gold Medal spoke about how they dramatically increased their conversion rate - and thus their revenues and profits -  with a novel use of Tealeaf.

Gold Medal created Tealeaf events to monitor whenever somebody makes it to the final step of the process of creating an itinerary. They also created events that fire when somebody adds such a trip but doesn't complete the booking process, for whatever reason. 

Further, they allocated 6 service agents from their call center (out of more than 200) as “outbound” revenue recovery agents, and setup the Tealeaf alert console to send these folks a notification whenever somebody made it to the final step, but didn't complete the booking process (i.e. an abandonment). 

Now, when their potential customer “abandons” an itinerary at the last step of the process - when they are showing clear “intent” to buy - they get a phone call within minutes asking if they need any assistance with their transaction. Oftentimes these abandonments are not because the user wants to abandon, but rather are forced to, by errors, issues or even having entered their credit card information incorrectly. Regardless, they want to book with Gold Medal, they just can’t.

Call it 'Chutzpah' or 'Pride' – Gold Medal actually demonstrated this process live at the UK Customer Forum.

They got on their live site, went through the process of selecting all the elements for a trip and entered the information for a phantom user – but used the mobile phone number of a random event attendee. 

They closed the browser and “abandoned” after they had entered an invalid credit card number that had no chance to go through.  38 seconds later (we timed it with another user’s iPhone stopwatch), the mobile number rang, as a service agent contacted them to see if they needed help with their online purchase. It was a cool demo (and unfortunately for this very diligent agent, this one was not a “real” customer).  

However, Gold Medal shared that over the months since building out this process, the results are mind-blowing:

• Their conversion rate at the final step has doubled from 15% to 30%
  
  
• The productivity of the “outbound” agents responding to “failed” transactions is much higher than that of the inbound team, which relies on the customer to call them.

It's important to note that we're not talking about site errors or performance problems here - we're talking about optimizing a business process. 

Sometimes, the human touch - at the right time - and with the right context (i.e. their session) can pay mind blowing dividends. 

So can creatively realizing that solving abandonment is not always about testing pages or fixing issues.

Customer Experience is really all about servicing the customer and finding optimized ways to do it.

Well done.

by: Robert Wenig, Founder and CTO

How to Prioritize Customer Experience Issues Based on Business Impact

It’s almost always a given that constraints exist on both the time and budget that are available to optimize your customers’ experience. You probably have a long list of issues to address, voices in the room and possibly even competing interests.

Where do you start?

Once you get past the urgent issues, it can be hard to tell what’s most important. I have found that the missing link for most ebusinesses is quantification—without quantifying the business impact of each customer experience issue, you may not prioritize the most important ones. And lack of quantification may even prevent you from addressing some issues at all.

My previous blog entries have talked about best practices that help you to become aware of customer experience issues and to understand why these issues are occurring. In the process of sharing real-world examples of these best practices, I also touched on how you would quantify the impact of a particular issue. This quantification leads to the final best practice I’m going to discuss in this series: prioritize your customer experience issues based on the business impact.

For every issue, you need to be able to answer the following questions:

• When did this issue start?
• How many visitors per day experience it?
• What is the difference in conversion rates (or other key task completion rates) between visitors who experience the issue and those who don’t?
• Given the number of visitors affected, to what losses in sales/profits do these changes in conversion rates translate? In other words, how much business is being lost every day because of this issue?
• How does lost sales/profits compare with other issues? How does it compare with the cost of resolving the problem?

Armed with this knowledge, you can build a clearly quantifiable business case that helps you set customer experience optimization priorities and improve revenue.

Let’s look at an example. Tealeaf customer Art.com generates significant traffic via search engines. Many of these indexed links send a user directly to specific product pages within their web site. However, the inventory on the site is incredibly dynamic, with products constantly going in and out of stock. Because the search engine indices can’t keep up, the first impression many customers get is a “sorry this product is no longer available” message. While reviewing customer sessions in Tealeaf, the user experience team at Art.com saw a number of customers getting this experience. Obviously, this is not the favorable first impression they wanted to impart, but no one really knew the magnitude of the problem.

The team realized that determining the extent of this problem would be very simple using Tealeaf. To their surprise, they found that 15,000 to 20,000 customers per day were seeing this unfriendly message. By reviewing multiple user sessions, seeing their purchase behavior and experiencing the “anti-shopping” experience first hand, it was an easy decision to re prioritize a better solution to this issue. The company immediately focused on a redesign that would give customers a more positive and actionable screen that recommends alternative products even if the original product is currently out of stock.

What are the best practices for quantifying the business impact?

1. Once a customer experience problem has been identified, use your customer experience management solution to determine the number of visitors impacted and the impact on conversion rates (or task completion rates) for those visitors.
2. Monetize the outcome by using a measure for the average value lost by customers not completing this task (for example the average shopping cart value for a checkout process). The number of customers impacted during a defined time period, the drop in conversion rates based on the issue and the average value for these lost transactions will enable you to calculate the approximate overall loss for a given period of time because of this issue.

For Art.com, this customer experience management best practice was what made it a no-brainer to invest in turning a formerly unfriendly, negative experience into a more positive, pro-shopping experience for its customers. As an additional benefit, Art.com was able to realize a greater return on their SEO/SEM customer acquisition programs. You too will benefit from knowing that you’re investing your resources where they will make the biggest improvements to customer experience.


-- John Dawes, Vice President, Product Management

Real-Time Response to Known Technical Issues

Every IT person I know can name at least ten generic conditions and errors that lead to poor customer experience. It’s not hard to figure out that if a customer is seeing messages like “Invalid SQL” or “Sorry, Page Not Found,” he or she is probably not accomplishing the task that he or she wanted to accomplish.  This customer experience management best practice is about putting processes in place to identify technical and application issues, track those issues and trigger a response by IT whenever a significant number of customers experience them.

You should be looking for the following types of things:

• Known error pages such as the global error page.
• Known application or system error messages, such as “Sorry, please try again later (status code xxxx).” Even if you think that you know all of these messages, you should work with your development team to verify the list and put a process in place to update the list as the site changes.
• Unexpected error messages or pages: It is hard to anticipate every error in advance; virtually every Tealeaf customer I talk to has seen unexpected errors at some point. In addition to looking for standard messages such as “SQL Error,” our customers have found it helpful to search their customer sessions for keywords such as “sorry” or “apologize.” They have found new error conditions in this way.
• Known bad status codes, such as a 500 error.
• Slow page performance: For example, a page that takes over ten seconds to be delivered to the user is certainly creating a very poor customer experience.

You can track many of the conditions I describe above with existing site management tools. For others, such as the unexpected error messages, you will need a customer experience management solution like Tealeaf. However you track them, you should think about:

• How will IT be able to be notified of the condition, and at what point should someone respond?
  A single instance of “Sorry, Page Not Found” is probably not worth paging someone in the middle of the night. You should define thresholds for each issue and set up a way for alerts to go out when the thresholds are exceeded. Many Tealeaf customers base these thresholds on previous activity, triggering an alert when the percentage of customers getting an error page exceeds 10% of the maximum percentage in the past month.
• How will IT diagnose the issue?
  Sometimes the underlying cause may be clear right away, but other times it may take a while to reproduce. A customer experience management tool can save a lot of time by providing a visual context for where customers are encountering obstacles.
• What is the real impact of a particular technical problem on my business?
  For example, can you correlate slowdowns in response time to task completion or conversion rates? An increase in the time it takes to get search results from two to three seconds may have no impact on a shopper’s purchase decision, but the same increase in response time for displaying the product details page could have a major impact on conversion rates.

One of our customers, ARC, has seen a lot of benefit from taking this kind of proactive approach to known technical issues. Originally, the company’s web-based applications were built so that when an internal error did occur, the application automatically displayed an “application error, please call support” message. Following instructions, ARC’s customers (travel agents) who saw this message would call the ARC customer service center. But the Customer Service Representatives (CSRs) handling these calls had no visibility into the site’s problems; as a result, they would have to escalate all of these issues to production support. Production support would then research the problems and report their findings back to the CSRs, who in turn would get back to the travel agents who had called. The process was reactive, cumbersome and costly—to say the least.

ARC’s production support team used Tealeaf’s real-time alerting capability (read the ARC case study) to establish a new, proactive approach to the problem. Now, every time one of the applications generates an “application error” message, the production support team is instantly notified—allowing them to intervene immediately. Often, this means that before a travel agent has time to call into the customer service center, the issue has either been fixed, or the CSR is at least already aware of the problem and can communicate appropriately with incoming callers.

To summarize this best practice, here are the steps you should take to respond to known technical issues in real-time:

1. Define the known technical issues to track. Start with the list of issues described above and then determine what is appropriate for your site. Getting a full list can require involvement from your development team, who will know about known exceptions that might be shown to users. Keeping this list up to date requires a defined process with your development team so that new error messages can be added as the site changes. Some ebusinesses even have a standard format for error pages or a standard comment in the page so that they can set up their customer experience management solution to identify future error pages in advance.
2. If you have a customer experience management solution, use it to search for keywords like “sorry” and “apologize” that often accompany error messages. Do this on a regular basis. I think that you will uncover new errors that aren’t being tracked.
3. Define thresholds for known errors and create a system so that the right people get real-time alerts when the thresholds are exceeded. You can determine the appropriate thresholds based on deviations from historical norms.
4. Define owners: who should receive these alerts and what steps should be taken when they receive each of them. For some sites, these alerts should go to the network operations center which can monitor them 24x7 in real-time. Once they receive an alert, the operations team can pull in the appropriate teams as needed.

These steps give your error messages the right context—customer experience—to help you explain the underlying problems and get them fixed.

-- John Dawes, Vice President, Product Management

Investigate Application Log Errors for Customer Experience Impact

If your web site is critical to your business, you have probably put in place tools and systems to monitor your applications at all times and respond to problems. Almost all of our customers use performance management tools to track application performance, response time, error rates, and similar IT-oriented metrics. In addition, error logs contain valuable information about what happened on your site. The trick is to be able to analyze the information in these logs not just for its own sake but from the perspective of customer experience.

After all, application problems do not always translate into poor customer experience while problems that do cause poor customer experience do not always get logged, for example:

Your server fails for a full hour...

• Were customers actually affected?
• Did failover work properly and hide the whole event from the customers?
• If there is a whole log file full of errors from your application servers about an inability to process user credentials, did customers see errors or did exception handling work properly?

The real metric that matters, of course, is customer experience—what impact a problem had on conversion rates, sales, and customer loyalty. As a customer experience management best practice, you should always investigate the application errors in a log file to get real answers. If a server went down, for example, find out:

• How many customers had active sessions on that server when it went down?
• Did they lose state (such as losing their shopping cart)?
• Did they see strange error messages?

You can also look for users who attempted to go to that server while it was down, perhaps because they had a bookmark that went directly to it. And most importantly, you can analyze the overall impact to your business: as a group, did they convert at a lower rate than expected?

One of our customers, a financial services organization, has taken this concept even farther. They track all of their service level agreements based on their impact on customers. As a standard operating procedure, any time the firm has a server outage or other problem, staff go into Tealeaf to investigate and quantify the impact on actual customers. This approach ensures that service level agreements have real meaning to IT and that the business owners clearly understand their value. I think that a lot of IT organizations could benefit from taking an approach like theirs.

-- John Dawes, Vice President, Product Management

Digging Into A/B and Multivariate Testing Results

Ebusinesses can clearly see the value of testing before making design decisions or implementing online marketing strategies. That’s why A/B and multivariate testing are so popular today. But with either an A/B or multivariate method, you are making critical decisions based on one metric alone: "Did the variable perform well or not?"

In review of everything involved in the experiment, do you think that ONE metric is really enough?

One of the best examples of the danger of relying on one metric comes from Marianina Manning’s Web Analytics Princess blog. Marianina is responsible for web analytics and customer experience at Rightmove. The blog entry describes how she investigated the results of a multivariate test with Tealeaf. Marianina wasn’t sure why property renters were converting at a higher rate than property buyers—on the same form in her application. After reviewing some sessions in Tealeaf, Marianina discovered that a content feature about energy efficiency, which appeared only occasionally, was distracting attention from the call to action.

Rightmove is not alone—many ebusinesses have told me that they found unexpected behavior or unplanned results when they conducted deeper customer behavior analysis during multivariate testing. Often, they find unexpected downstream consequences to a change or test in their application, such as a promotional price not showing up at checkout or navigation that doesn’t make sense to some customers based on the new application flow.

As a best practice, you should look not just at which variable performed best but also ask why a particular variable worked well or poorly.

1. Review a select group of sessions—for example, users who got the new functionality (that is being tested before being rolled out to all users on the site), but who still abandoned the purchase process.
2. Use a customer experience management solution to replay a sampling of those sessions to see what actually happened. The number of sessions to replay is dependent on how large a difference was seen in conversion rates (or task completion rates) in the variable that performed well or poorly. Use the same techniques described in my “Where Do I Start Customer Behavior Analysis? I Have So Many Sessions!” blog entry.
3. After investigating why the variable did well or poorly, you might come to a decision to revise the test and run it again before fully deploying the best-performing variable.

Customer behavior analysis is a very valuable complement to your A/B and multivariate testing processes. Whether your test is as small as trying different home page promotions or as big as selecting a new design for a checkout process, you need to be sure that you really understand what’s happening. Customer behavior analysis does just that.

-- John Dawes, Vice President, Product Management

Investigate Changes or Differences in Conversion Rates - Another CEM Best Practice

Online businesses live and die through conversion rates and task success rates. Most of my customers can tell me what their key metrics are at any point in time. And they know that when they see a sudden change in conversion rates or other success rates, they need to investigate why right then.

The vast majority of the ebusiness leaders that I talk to have invested in web analytics and/or other tools to give them a top-level view of the health of their ebusinesses and to stay on top of site trends. Web analytics tools do a great job of ‘slicing-and-dicing’—helping you to find the customer segment that’s most representative of a change. You can look at differences across specific products or product categories; new versus existing customers; geographic region; day of the week; where the customer came from (for example, from Google versus from a banner ad); and virtually any other segment that you define in the tool.

However, web analytics tools don’t tell you why customers in these segments behaved differently. That’s where a customer experience management solution comes into play—in supplementing this quantitative data with qualitative data for full customer behavior analysis. (Note: if you want to delve more into how customer experience management tools and web analytics complement each other and where they overlap, read the white paper “Customer Experience Management and Web Analytics: From KPIs to Customer Transactions” by Eric Peterson of Web Analytics Demystified, Inc.)

Also, in my experience the customer experience problems that are hardest to understand are ones that involve factors that the ebusiness is not tracking—and therefore not tagging for web analytics. No ebusiness can tag every component of its customer experience for web analytics; but it is the things that aren’t being tracked that are often the most likely to go wrong because no one is thinking about them at all.

Here’s an example. A major automotive retailer noticed a high number of abandonments on the last step of the checkout process but had no way to determine what was causing the problem with its existing site tools. When the company replayed a few of these abandoned sessions in Tealeaf, they recognized a pattern: no debit card user was able to complete a transaction successfully. Further analysis showed that its web application was not built to handle debit cards and their required PINs. Because debit cards were accepted in the company’s stores, they were automatically added to the list of payment options on the site. However, a form field and logic to accept PIN numbers had been overlooked. This is the kind of unanticipated problem that you can’t tag for; and therefore can’t be analyzed or even identified using web analytics tools.

To implement all of the best practices for investigating changes or differences in conversion rates, you should deploy both a web analytics tool and a customer experience management tool in order to get the full picture of what is happening on your sites and why.

Here are the steps that you should take:

1. Define what constitutes a significant change to each of your key task completion rates by analyzing the existing benchmarks you have in place and define a process for monitoring these KPIs on a regular basis.
2. Put in place processes for investigating any significant change immediately and to investigate any unexpected, significant difference in task completion rates between segments. Always ask why.
3. Use your web analytics tool to isolate the difference to as small a customer segment as possible.  Enlarging the percentage difference in task completion rates between segments will make it easier to identify the problem.
4. Use a customer experience management tool to replay a sample of sessions in the problem segment to visualize the customer experience and look for unexpected behavior. (See my previous blog entry, “Where Do I Start Customer Behavior Analysis? I Have So Many Sessions!” for guidance on estimating the number of sessions to view).
5. Look at the complete user sessions, not just the step where the problem occurred. Often, a problem occurs long before it results in an abandonment.
6. After gaining visibility into the problem from replay, the qualitative data you see may give you insights on how to refine the problem. By searching for other sessions where the same thing went wrong you can refine problems with the type of variables that web analytics do not track, as in our retailer example above (searching for the type of credit card).
7. Once you have insight into the problem and have refined it, quantify the impact of it. The impact of the problem should correlate with the overall change in conversion rate.

These best practices are how you turn your key metrics into action: you must go from which and where to discovering why.

-- John Dawes, Vice President, Product Management

CEM Best Practice: Observe and Review Actual Customer Behavior

The most customer-centric businesses I have worked with don’t just listen to their customers; they systematically review their customer experience on a regular basis. Retailers do this by observing shoppers in their stores; banks do it by listening to interactions in the call center. For an ebusiness, there is no better way to understand the challenges that your customers face than by actually watching them use your site.

Because Tealeaf gives you a way to conduct direct observation of your online customers, many of our customers have established a weekly customer experience review as a customer experience management best practice. In doing so they:

1. Gather all of the site’s key stakeholders in a weekly meeting.
2. Select one suspected trouble spot to investigate. For example, they may have seen a downturn in new registered users and want to review the registration process.
3. Find customer sessions which demonstrate potentially bad experiences. For example, they could search for customers that abandoned the registration process after trying it three times.
4. Select 20 or so sessions from this list and watch their actual experience going through the registration process.
5. Use Tealeaf to look for other potential problems beyond the one that drove this investigation. Otherwise, you only end up finding the problems you are looking for. Tealeaf customers have found that the most interesting issues or obstacles facing their customers are often ones uncovered during these investigations.
6. Use the insight gained to clarify what problems or issues with this process need improvement.
7. Choose two or three specific things to investigate in more depth based on what came out of this review.

For example, one of our customers knew that users were having problems with the registration process, but was not sure why. The company decided to evaluate this part of its site during its regular customer experience review. After replaying several abandoned registrations in Tealeaf, it found a common thread: customers were seeing an “invalid email address” message and trying to enter their email addresses in new and creative ways—all to no avail. The company then looked up some of the email accounts that customers were entering and found that these people already had accounts.

The customer, an online travel provider, knew that its vacation travelers visited the site infrequently and often could not remember that they already had registered accounts. The insight the company got through its customer experience review made it clear that they needed to design the registration process with existing users in mind. Simply by changing the “invalid email address” message to “email already in use” and implementing better processes for existing users to recover their usernames and passwords, our customer was able to drive a significant increase in conversion rates.

In my experience, most ebusinesses understand the strengths and weaknesses of their sites and have hypotheses about where they need to make improvements. Regular customer experience reviews, supported by tools for observing online customer behavior, make it possible to test these hypotheses with real-life data. Ebusinesses that follow this best practice are often able to address suspected problems before they have a major impact on business. And, they have been able to see what is working well so that they can leverage good ideas across the site.

-- John Dawes, Vice President, Product Management

Investigating Information Coming From Voice of Customer (VOC) Surveys and Public Forums

Customers are often your best source of information as to why your site isn’t delivering as much value as you (or they) would like. And they are already talking to you. All you need is a systematic way to listen to them and turn their feedback into something that is actionable.

Tealeaf customers have been telling me how they use customer behavior analysis to take their voice of the customer (VOC) survey data to the next level. For example, one online travel provider’s customer survey contained complaints from some customers that their search results listed hotels in the wrong city (for example, they would search for hotels in Atlanta and get hotels in Chicago). The provider could not reproduce the issue; fortunately, they decided to analyze it further.

Using Tealeaf, our customer discovered that the users who complained had actually started their searches from an affiliate partner’s site (View the complete OnDemand Webinar). This one affiliate was appending extra characters at the end of the search string, and therefore the search was not providing the right results. To add insult to injury, the travel provider was paying the affiliate for these bad searches! 

Similarly, a major European airline was able to take advantage of data from public forums to improve its customer experience. This customer regularly monitors FlyerTalk.com, where forum members were posting that its web site was terrible because they couldn’t book tickets on it. The posters would get the message “town invalid” when they tried to complete their booking forms. This feedback made no sense to the airline because its web analytics and performance management systems showed only a slight uptick in bookings failures. But after using Tealeaf to search for online customers who received the “town invalid” message and replaying some of these failed sessions, the airline quickly discovered the problem: the town field on the address form was not accepting hyphens and dashes. When flyers from these hyphenated towns were getting the resulting “town invalid” message, the majority of them abandoned their purchases. Fixing this single customer experience flaw recaptured at least £30,000 per week in lost bookings.

These examples show why deeper analysis of customer survey data—and any customer experience data that you can access—is a customer experience management best practice. For survey data, you should:

1. Check the data for problem areas that affect customer experience.
2. Review free-text feedback responses that either highlight problem areas or prompt you to ask why?
3. Look for segments of customers associated with interesting free-text responses or low-scores in a particular area. With a customer experience management solution like Tealeaf, you can then replay representative sessions and look for customer experience obstacles that can explain the scores or feedback you received. It’s important to note that the more integrated that VOC survey data is with your customer experience management system, the more it will be able to do what it’s supposed to do—guide improvements. Ideally, you should be able to pivot directly from a survey response into the representative session in order to understand the full context of the experience which garnered the feedback.

For customer data that comes from public forums, you should:

1. Take advantage of any names, email addresses or other information given by customers—any information that can be used to find their online experience. A customer experience management solution can help you to reproduce the experience they actually had and understand their feedback.
2. Use these forums to look for trends in complaints. If you do see such a trend, you can often find problems based on where the customer was on your site, what they did (and commented about) or other information about them that’s available, such as their location. A customer experience management solution like Tealeaf enables you to search for individual customer sessions and drill-down based on all of these types of attributes.

Once you identify an issue, you can then move on to refine, quantify and diagnose that issue.

There has never been a time when businesses of all kinds had more information about what their customers think of them, whether through low-cost VOC surveys or information that’s freely available in blogs, communities and a variety of other public forums. Are you ready to listen?

Learn more: Download the 'Uniting Voice with Customer Experience Webinar' and view the Tealeaf CEM Integration Solution

--John Dawes, Vice President, Product Management

Customer Experience Management Best Practice: Monitor Your Customer Experience KPIs

When I talk to ebusiness leaders, they can almost always name a number of factors that affect their customers’ online experience. The most customer-centric of them have set up Key Performance Indicators (KPIs) that measure the incidence of these factors and alert them to changes or trends. At the simplest level, this can be just tracking overall page performance metrics on the site; but there are many other metrics that end up having a bigger effect on customer experience.

For example, one of our online travel provider customers uses Tealeaf to track a KPI for validating itinerary purchases. Many tasks happen during the final step in a travel purchase: accessing airline and hotel reservation systems, validating information entered, processing credit card information, and more. Small glitches in any of these tasks can prevent the transaction from being processed completely. Simply by tracking whether customers went to a purchase confirmation page after they clicked on the purchase button, our customer was able to identify glitches that represented millions of dollars of lost business.

The most common KPIs I see are set up around “known bad behaviors” such as:

• Multiple attempts of key processes without success: For example, a user tried to log in three times and failed, a user searched for a product three times without getting any results, a user went through the purchase process three times but never purchased, etc.
• Multiple attempts to submit a form without success: For example, a user received a form validation message such as “enter valid address” more than three times, a user was asked to accept payment terms more than three times, etc.

Or, they are based on “known bad experiences” that can be tracked:

• Technical issues such as slow page performance, global error pages, or unexpected error messages such as “SQL exception.”
• Business issues: These vary depending on the type of site and industry but could include seeing an out-of-stock message on a retail site, having an itinerary re-priced on a travel site, or having account qualification issues on a financial services site.
• Customer issues: Actual customer behavior that indicates a bad experience such as going to the customer feedback form or help page.

A final set of KPIs are based on unexpected outcomes. These are outcomes that deviate from the defined business processes on the site, as in the itinerary purchase validation example above. The most common areas I see tracked involve:

• Validating steps with heavy back-end processing, for example checkout on a retail site or purchase itinerary on a travel site. If the next page shown to the user is not a purchase confirmation, something went wrong.
• Validating steps after partner activity, for example payment processing for a credit card. Once this partner activity completes, the expected behavior would be that the customer would return to the site’s process.

All of these different KPIs can be rolled up into a single overall customer experience KPI that you can easily track for your site.

People ask me if you have to deploy a customer experience management tool like Tealeaf to track KPIs; the answer is no. You can get started by tracking the KPIs that can be captured with existing web analytics or performance management tools. But there are many important KPIs that can’t be tracked without a customer experience management tool such as Tealeaf. Tealeaf is particularly effective at tracking KPIs of customer behavior and unanticipated events such as unexpected outcomes, unexpected error messages or unexpected behaviors.

Here are my recommended best practices for monitoring your customer experience KPIs:

1. Work with your entire ebusiness team to define appropriate customer experience KPIs for different organizations and groups. You should limit the number of KPIs that any one group must track. Keep in mind that not everyone will look at every KPI.
2. Once you determine what is appropriate, you can assign owners to track each one and take action if they deviate from their norms. Norms will be dependent on the KPI, but you should define what is expected and how to identify spikes and significant changes.
3. Establish an easy way for each and every team member who is responsible for KPIs to monitor them. You may want to set up dashboards and define a standard frequency for responsible parties to do visually monitoring. Better yet, set up alerts so that people are immediately notified of changes. Both dashboards and alerts are standard features of a customer experience management tool like Tealeaf.
4. Roll-up customer experience KPIs to report to management. Management should not look at individual KPIs for unexpected outcomes, but they should care about an overall KPI for site user experience. You might want to create scorecards that roll up individual KPIs and assign a grade in each major category. This can be much more valuable to management than tracking very basic customer experience metrics like page performance.
5. Investigate all changes in KPIs. When a KPI goes out of its historical norm, the assigned owner should take action immediately, performing customer behavior analysis until the cause of the change becomes clear.

The most important best practice for customer experience KPIs is to track them; too many ebusinesses do not because they don’t know how to capture the information (view a video on how and why Tealeaf captures all data). Once you take this first step, you’ll be able to see the value of this important customer experience management best practice.

-- John Dawes, Vice President, Product Management

Customer Experience Management Best Practice: Take Action on Customer Service Complaints

It’s easy for ebusinesses to set up processes for collecting online customer complaints. Whether it’s through online forms or contact centers, customers are more than willing to take the time to complain about problems they experience online. According to a recent Harris Survey, 53% of users will report an issue they experienced online. The hard part is making sure that you can respond appropriately and put these complaints to good use.

The ebusinesses that I have worked with often do a great job investigating customer complaints and acting upon them. The most customer-centric of them have defined customer listening initiatives that formalize complaint resolution processes and take them to the next level. The shortcomings that I have seen in these initiatives happen when the person or team following up on the complaint can’t pinpoint exactly what the customer experienced online. Intermittent site malfunctions may be hard to reproduce, and usability problems might not be apparent until you’re in the eyes of the beholder.

Take the example of Thomson Holidays, the UK’s most popular leisure travel retailer. Last year Thomson launched a new web site to provide enhanced functionality for its one million visitors per week. After the new site went live, Thomson began receiving emails and calls from customers reporting that the confirmation screen and email they received included strange data instead of their personal information. For example, instead of saying, “Dear John” the email would say, “Dear First Name.” When Thomson’s IT team tried to reproduce the problem they were unable to isolate the issue.

Thomson turned to Tealeaf to conduct further analysis of the problem (read the Case Study). Because the call center had the names of people reporting the issue, it was easy to pick a few examples, and then use Tealeaf to replay those visits. Tealeaf enabled Thomson to discover that, occasionally, the site would malfunction between the “enter personal details” and “payment” steps in the booking process. When this happened the customer’s information would be replaced with the “Dear First Name” data rather than the actual details. The visibility Tealeaf provided allowed Thomson Holidays to halt a customer experience flaw that was increasing inquiries to the call center and eroding customer satisfaction. Furthermore, because Tealeaf allowed Thomson to view and replay affected sessions, Thomson’s IT team no longer had to try and reproduce the problem, and could instead focus on higher-value enhancements to the site.

Customer experience management best practices include formalizing the process of tracking and escalating customer issues that originate in the contact center. I recommend that you:

1. Have a defined process for what information needs to be gathered (name, account number, etc.) to find the customer’s session with a customer experience management tool. With Tealeaf for example, even anonymous sessions can be found if information unique to that session is gathered, for example what they searched for and the approximate time of their session.
2. Make sure that there is a way for call center representatives to escalate issues for investigation. Most customers use a trouble ticket system, but regardless of the system there should be a process for capturing the necessary information and escalating the issue to the appropriate team.
3. Have a person responsible for periodically examining trouble tickets to look for trends and identify common problem areas.
4. Use a customer experience management tool to search for customer sessions that demonstrate the problem area and replay these sessions to see the actual customer experience that triggered a complaint.
5. Use the insight you gain from conducting customer behavior analysis to identify the issue.

Once you identify an issue, you can then move on to refine, quantify and diagnose that issue.

To summarize, the customer who takes the time to complain is one who perceives a significant problem in your customer experience. It is up to you to build the right processes and systems to turn one customer’s frustration into many customers’ benefit.

Also, in case you missed it, we recently hosted a webinar with OpinionLab and our joint customer, hotels.com, where we discussed another best practice –- how to take action on Voice of Customer survey data -- which you may find useful to review.

-- John Dawes, Vice President, Product Management