Tealeaf: Visibility. Insight. Answers.

Investigate Changes or Differences in Conversion Rates - Another CEM Best Practice

Online businesses live and die through conversion rates and task success rates. Most of my customers can tell me what their key metrics are at any point in time. And they know that when they see a sudden change in conversion rates or other success rates, they need to investigate why right then.

The vast majority of the ebusiness leaders that I talk to have invested in web analytics and/or other tools to give them a top-level view of the health of their ebusinesses and to stay on top of site trends. Web analytics tools do a great job of ‘slicing-and-dicing’—helping you to find the customer segment that’s most representative of a change. You can look at differences across specific products or product categories; new versus existing customers; geographic region; day of the week; where the customer came from (for example, from Google versus from a banner ad); and virtually any other segment that you define in the tool.

However, web analytics tools don’t tell you why customers in these segments behaved differently. That’s where a customer experience management solution comes into play—in supplementing this quantitative data with qualitative data for full customer behavior analysis. (Note: if you want to delve more into how customer experience management tools and web analytics complement each other and where they overlap, read the white paper “Customer Experience Management and Web Analytics: From KPIs to Customer Transactions” by Eric Peterson of Web Analytics Demystified, Inc.)

Also, in my experience the customer experience problems that are hardest to understand are ones that involve factors that the ebusiness is not tracking—and therefore not tagging for web analytics. No ebusiness can tag every component of its customer experience for web analytics; but it is the things that aren’t being tracked that are often the most likely to go wrong because no one is thinking about them at all.

Here’s an example. A major automotive retailer noticed a high number of abandonments on the last step of the checkout process but had no way to determine what was causing the problem with its existing site tools. When the company replayed a few of these abandoned sessions in Tealeaf, they recognized a pattern: no debit card user was able to complete a transaction successfully. Further analysis showed that its web application was not built to handle debit cards and their required PINs. Because debit cards were accepted in the company’s stores, they were automatically added to the list of payment options on the site. However, a form field and logic to accept PIN numbers had been overlooked. This is the kind of unanticipated problem that you can’t tag for; and therefore can’t be analyzed or even identified using web analytics tools.

To implement all of the best practices for investigating changes or differences in conversion rates, you should deploy both a web analytics tool and a customer experience management tool in order to get the full picture of what is happening on your sites and why.

Here are the steps that you should take:

1. Define what constitutes a significant change to each of your key task completion rates by analyzing the existing benchmarks you have in place and define a process for monitoring these KPIs on a regular basis.
2. Put in place processes for investigating any significant change immediately and to investigate any unexpected, significant difference in task completion rates between segments. Always ask why.
3. Use your web analytics tool to isolate the difference to as small a customer segment as possible.  Enlarging the percentage difference in task completion rates between segments will make it easier to identify the problem.
4. Use a customer experience management tool to replay a sample of sessions in the problem segment to visualize the customer experience and look for unexpected behavior. (See my previous blog entry, “Where Do I Start Customer Behavior Analysis? I Have So Many Sessions!” for guidance on estimating the number of sessions to view).
5. Look at the complete user sessions, not just the step where the problem occurred. Often, a problem occurs long before it results in an abandonment.
6. After gaining visibility into the problem from replay, the qualitative data you see may give you insights on how to refine the problem. By searching for other sessions where the same thing went wrong you can refine problems with the type of variables that web analytics do not track, as in our retailer example above (searching for the type of credit card).
7. Once you have insight into the problem and have refined it, quantify the impact of it. The impact of the problem should correlate with the overall change in conversion rate.

These best practices are how you turn your key metrics into action: you must go from which and where to discovering why.

-- John Dawes, Vice President, Product Management

CEM: The Time is Now

Ventana Research recently posted a noteworthy blog about CEM and its evolution

At Tealeaf, we have witnessed the growth of CEM over nearly a decade now, as more and more transactions move online. People now are talking a lot about “Web 2.0,” which points to a next wave of the Internet – one in which people have higher expectations than ever when they go online. Ten years ago, people hoped they could complete their transaction, but were probably patient if things veered off track. Today, people demand more and companies are realizing that they can’t just throw money at site performance issues -- they have to be able to *see* their customers’, and *understand* their experiences in order to deliver upon their site’s promises.

As online users’ expectations have increased, so too has the use of CEM. As Ventana said, there are whole customer management teams being created now. We’ve seen many of our customers designate CEM-specific roles.  These executives are helping their organizations to become more customer-centric through the common view of their online user. Being able to actually see every session from each customer (and combine this qualitative insight with quantitative analysis) helps them understand their sites, explore opportunities for improvement and innovation, and provide the most effective service possible. It also provides irrefutable answers to some of the previously nebulous questions about online behavior and site issues to drive requirements and set priorities.

To Ventana’s point about spanning the line of business, helping an entire organization become more customer-centric bridges departments together to achieve overarching business goals. And whereas previously CEM was used primarily by ebusiness and IT, we’re seeing expanded use in customer service, and legal and compliance departments especially. One of our large travel customers is using Tealeaf all across their enterprise – spanning more than six departments.

For example, Tealeaf cxReveal allows customer service teams to replay both live and recent customer interactions so that they can improve their first resolution rates. And Tealeaf cxVerify – by preserving complete records of all online customer interactions -- has been extremely valuable to departments supporting customer dispute resolution and fraud investigations, as well as audit and compliance requirements.

There is no question that the Web is still not perfect (but what ever is?). We sponsor an annual Harris Survey. In 2007 (for the third year running) the survey results showed that 9 out of 10 users conducting transactions have experienced problems. Read the summary release or better yet, download the complete set of findings.

However, we’re encouraged to see that more and more businesses are recognizing the need for online visibility, creating CEM-specific roles and best practices. Beyond the spread of CEM across teams, we’re also seeing a spread across application type (from ecommerce focused systems to B2B portals and HR applications, for example) and into diverse markets. I’d be shocked if any of us haven’t, at some time, felt the frustration of not being able to complete an online transactions, so this is all a step in the right direction!

-- Geoff Galat, Vice President, Marketing & Product Strategy

CEM Best Practice: Observe and Review Actual Customer Behavior

The most customer-centric businesses I have worked with don’t just listen to their customers; they systematically review their customer experience on a regular basis. Retailers do this by observing shoppers in their stores; banks do it by listening to interactions in the call center. For an ebusiness, there is no better way to understand the challenges that your customers face than by actually watching them use your site.

Because Tealeaf gives you a way to conduct direct observation of your online customers, many of our customers have established a weekly customer experience review as a customer experience management best practice. In doing so they:

1. Gather all of the site’s key stakeholders in a weekly meeting.
2. Select one suspected trouble spot to investigate. For example, they may have seen a downturn in new registered users and want to review the registration process.
3. Find customer sessions which demonstrate potentially bad experiences. For example, they could search for customers that abandoned the registration process after trying it three times.
4. Select 20 or so sessions from this list and watch their actual experience going through the registration process.
5. Use Tealeaf to look for other potential problems beyond the one that drove this investigation. Otherwise, you only end up finding the problems you are looking for. Tealeaf customers have found that the most interesting issues or obstacles facing their customers are often ones uncovered during these investigations.
6. Use the insight gained to clarify what problems or issues with this process need improvement.
7. Choose two or three specific things to investigate in more depth based on what came out of this review.

For example, one of our customers knew that users were having problems with the registration process, but was not sure why. The company decided to evaluate this part of its site during its regular customer experience review. After replaying several abandoned registrations in Tealeaf, it found a common thread: customers were seeing an “invalid email address” message and trying to enter their email addresses in new and creative ways—all to no avail. The company then looked up some of the email accounts that customers were entering and found that these people already had accounts.

The customer, an online travel provider, knew that its vacation travelers visited the site infrequently and often could not remember that they already had registered accounts. The insight the company got through its customer experience review made it clear that they needed to design the registration process with existing users in mind. Simply by changing the “invalid email address” message to “email already in use” and implementing better processes for existing users to recover their usernames and passwords, our customer was able to drive a significant increase in conversion rates.

In my experience, most ebusinesses understand the strengths and weaknesses of their sites and have hypotheses about where they need to make improvements. Regular customer experience reviews, supported by tools for observing online customer behavior, make it possible to test these hypotheses with real-life data. Ebusinesses that follow this best practice are often able to address suspected problems before they have a major impact on business. And, they have been able to see what is working well so that they can leverage good ideas across the site.

-- John Dawes, Vice President, Product Management

Investigating Information Coming From Voice of Customer (VOC) Surveys and Public Forums

Customers are often your best source of information as to why your site isn’t delivering as much value as you (or they) would like. And they are already talking to you. All you need is a systematic way to listen to them and turn their feedback into something that is actionable.

Tealeaf customers have been telling me how they use customer behavior analysis to take their voice of the customer (VOC) survey data to the next level. For example, one online travel provider’s customer survey contained complaints from some customers that their search results listed hotels in the wrong city (for example, they would search for hotels in Atlanta and get hotels in Chicago). The provider could not reproduce the issue; fortunately, they decided to analyze it further.

Using Tealeaf, our customer discovered that the users who complained had actually started their searches from an affiliate partner’s site (View the complete OnDemand Webinar). This one affiliate was appending extra characters at the end of the search string, and therefore the search was not providing the right results. To add insult to injury, the travel provider was paying the affiliate for these bad searches! 

Similarly, a major European airline was able to take advantage of data from public forums to improve its customer experience. This customer regularly monitors FlyerTalk.com, where forum members were posting that its web site was terrible because they couldn’t book tickets on it. The posters would get the message “town invalid” when they tried to complete their booking forms. This feedback made no sense to the airline because its web analytics and performance management systems showed only a slight uptick in bookings failures. But after using Tealeaf to search for online customers who received the “town invalid” message and replaying some of these failed sessions, the airline quickly discovered the problem: the town field on the address form was not accepting hyphens and dashes. When flyers from these hyphenated towns were getting the resulting “town invalid” message, the majority of them abandoned their purchases. Fixing this single customer experience flaw recaptured at least £30,000 per week in lost bookings.

These examples show why deeper analysis of customer survey data—and any customer experience data that you can access—is a customer experience management best practice. For survey data, you should:

1. Check the data for problem areas that affect customer experience.
2. Review free-text feedback responses that either highlight problem areas or prompt you to ask why?
3. Look for segments of customers associated with interesting free-text responses or low-scores in a particular area. With a customer experience management solution like Tealeaf, you can then replay representative sessions and look for customer experience obstacles that can explain the scores or feedback you received. It’s important to note that the more integrated that VOC survey data is with your customer experience management system, the more it will be able to do what it’s supposed to do—guide improvements. Ideally, you should be able to pivot directly from a survey response into the representative session in order to understand the full context of the experience which garnered the feedback.

For customer data that comes from public forums, you should:

1. Take advantage of any names, email addresses or other information given by customers—any information that can be used to find their online experience. A customer experience management solution can help you to reproduce the experience they actually had and understand their feedback.
2. Use these forums to look for trends in complaints. If you do see such a trend, you can often find problems based on where the customer was on your site, what they did (and commented about) or other information about them that’s available, such as their location. A customer experience management solution like Tealeaf enables you to search for individual customer sessions and drill-down based on all of these types of attributes.

Once you identify an issue, you can then move on to refine, quantify and diagnose that issue.

There has never been a time when businesses of all kinds had more information about what their customers think of them, whether through low-cost VOC surveys or information that’s freely available in blogs, communities and a variety of other public forums. Are you ready to listen?

Learn more: Download the 'Uniting Voice with Customer Experience Webinar' and view the Tealeaf CEM Integration Solution

--John Dawes, Vice President, Product Management

Customer Experience Management Best Practice: Monitor Your Customer Experience KPIs

When I talk to ebusiness leaders, they can almost always name a number of factors that affect their customers’ online experience. The most customer-centric of them have set up Key Performance Indicators (KPIs) that measure the incidence of these factors and alert them to changes or trends. At the simplest level, this can be just tracking overall page performance metrics on the site; but there are many other metrics that end up having a bigger effect on customer experience.

For example, one of our online travel provider customers uses Tealeaf to track a KPI for validating itinerary purchases. Many tasks happen during the final step in a travel purchase: accessing airline and hotel reservation systems, validating information entered, processing credit card information, and more. Small glitches in any of these tasks can prevent the transaction from being processed completely. Simply by tracking whether customers went to a purchase confirmation page after they clicked on the purchase button, our customer was able to identify glitches that represented millions of dollars of lost business.

The most common KPIs I see are set up around “known bad behaviors” such as:

• Multiple attempts of key processes without success: For example, a user tried to log in three times and failed, a user searched for a product three times without getting any results, a user went through the purchase process three times but never purchased, etc.
• Multiple attempts to submit a form without success: For example, a user received a form validation message such as “enter valid address” more than three times, a user was asked to accept payment terms more than three times, etc.

Or, they are based on “known bad experiences” that can be tracked:

• Technical issues such as slow page performance, global error pages, or unexpected error messages such as “SQL exception.”
• Business issues: These vary depending on the type of site and industry but could include seeing an out-of-stock message on a retail site, having an itinerary re-priced on a travel site, or having account qualification issues on a financial services site.
• Customer issues: Actual customer behavior that indicates a bad experience such as going to the customer feedback form or help page.

A final set of KPIs are based on unexpected outcomes. These are outcomes that deviate from the defined business processes on the site, as in the itinerary purchase validation example above. The most common areas I see tracked involve:

• Validating steps with heavy back-end processing, for example checkout on a retail site or purchase itinerary on a travel site. If the next page shown to the user is not a purchase confirmation, something went wrong.
• Validating steps after partner activity, for example payment processing for a credit card. Once this partner activity completes, the expected behavior would be that the customer would return to the site’s process.

All of these different KPIs can be rolled up into a single overall customer experience KPI that you can easily track for your site.

People ask me if you have to deploy a customer experience management tool like Tealeaf to track KPIs; the answer is no. You can get started by tracking the KPIs that can be captured with existing web analytics or performance management tools. But there are many important KPIs that can’t be tracked without a customer experience management tool such as Tealeaf. Tealeaf is particularly effective at tracking KPIs of customer behavior and unanticipated events such as unexpected outcomes, unexpected error messages or unexpected behaviors.

Here are my recommended best practices for monitoring your customer experience KPIs:

1. Work with your entire ebusiness team to define appropriate customer experience KPIs for different organizations and groups. You should limit the number of KPIs that any one group must track. Keep in mind that not everyone will look at every KPI.
2. Once you determine what is appropriate, you can assign owners to track each one and take action if they deviate from their norms. Norms will be dependent on the KPI, but you should define what is expected and how to identify spikes and significant changes.
3. Establish an easy way for each and every team member who is responsible for KPIs to monitor them. You may want to set up dashboards and define a standard frequency for responsible parties to do visually monitoring. Better yet, set up alerts so that people are immediately notified of changes. Both dashboards and alerts are standard features of a customer experience management tool like Tealeaf.
4. Roll-up customer experience KPIs to report to management. Management should not look at individual KPIs for unexpected outcomes, but they should care about an overall KPI for site user experience. You might want to create scorecards that roll up individual KPIs and assign a grade in each major category. This can be much more valuable to management than tracking very basic customer experience metrics like page performance.
5. Investigate all changes in KPIs. When a KPI goes out of its historical norm, the assigned owner should take action immediately, performing customer behavior analysis until the cause of the change becomes clear.

The most important best practice for customer experience KPIs is to track them; too many ebusinesses do not because they don’t know how to capture the information (view a video on how and why Tealeaf captures all data). Once you take this first step, you’ll be able to see the value of this important customer experience management best practice.

-- John Dawes, Vice President, Product Management

Customer Experience Management Best Practice: Take Action on Customer Service Complaints

It’s easy for ebusinesses to set up processes for collecting online customer complaints. Whether it’s through online forms or contact centers, customers are more than willing to take the time to complain about problems they experience online. According to a recent Harris Survey, 53% of users will report an issue they experienced online. The hard part is making sure that you can respond appropriately and put these complaints to good use.

The ebusinesses that I have worked with often do a great job investigating customer complaints and acting upon them. The most customer-centric of them have defined customer listening initiatives that formalize complaint resolution processes and take them to the next level. The shortcomings that I have seen in these initiatives happen when the person or team following up on the complaint can’t pinpoint exactly what the customer experienced online. Intermittent site malfunctions may be hard to reproduce, and usability problems might not be apparent until you’re in the eyes of the beholder.

Take the example of Thomson Holidays, the UK’s most popular leisure travel retailer. Last year Thomson launched a new web site to provide enhanced functionality for its one million visitors per week. After the new site went live, Thomson began receiving emails and calls from customers reporting that the confirmation screen and email they received included strange data instead of their personal information. For example, instead of saying, “Dear John” the email would say, “Dear First Name.” When Thomson’s IT team tried to reproduce the problem they were unable to isolate the issue.

Thomson turned to Tealeaf to conduct further analysis of the problem (read the Case Study). Because the call center had the names of people reporting the issue, it was easy to pick a few examples, and then use Tealeaf to replay those visits. Tealeaf enabled Thomson to discover that, occasionally, the site would malfunction between the “enter personal details” and “payment” steps in the booking process. When this happened the customer’s information would be replaced with the “Dear First Name” data rather than the actual details. The visibility Tealeaf provided allowed Thomson Holidays to halt a customer experience flaw that was increasing inquiries to the call center and eroding customer satisfaction. Furthermore, because Tealeaf allowed Thomson to view and replay affected sessions, Thomson’s IT team no longer had to try and reproduce the problem, and could instead focus on higher-value enhancements to the site.

Customer experience management best practices include formalizing the process of tracking and escalating customer issues that originate in the contact center. I recommend that you:

1. Have a defined process for what information needs to be gathered (name, account number, etc.) to find the customer’s session with a customer experience management tool. With Tealeaf for example, even anonymous sessions can be found if information unique to that session is gathered, for example what they searched for and the approximate time of their session.
2. Make sure that there is a way for call center representatives to escalate issues for investigation. Most customers use a trouble ticket system, but regardless of the system there should be a process for capturing the necessary information and escalating the issue to the appropriate team.
3. Have a person responsible for periodically examining trouble tickets to look for trends and identify common problem areas.
4. Use a customer experience management tool to search for customer sessions that demonstrate the problem area and replay these sessions to see the actual customer experience that triggered a complaint.
5. Use the insight you gain from conducting customer behavior analysis to identify the issue.

Once you identify an issue, you can then move on to refine, quantify and diagnose that issue.

To summarize, the customer who takes the time to complain is one who perceives a significant problem in your customer experience. It is up to you to build the right processes and systems to turn one customer’s frustration into many customers’ benefit.

Also, in case you missed it, we recently hosted a webinar with OpinionLab and our joint customer, hotels.com, where we discussed another best practice –- how to take action on Voice of Customer survey data -- which you may find useful to review.

-- John Dawes, Vice President, Product Management