Tealeaf: Visibility. Insight. Answers.

Where Do I Start Customer Behavior Analysis? I Have So Many Sessions!

When I talk to ebusiness leaders about Tealeaf, they immediately see the value of being able to replay what each customer is doing and seeing in real-time on every page of the site. But the thought of having thousands if not millions of individual customer sessions to review can be overwhelming. Is looking for customer experience issues like looking for a needle in a haystack?

The answer I give to that question is emphatically no. Just because you can’t look at all of your customers’ sessions—or everyone who abandoned a shopping cart—doesn’t mean that you shouldn’t look at any sessions! Call center managers have long known the value of listening in on a sample of calls, and retailers regularly observe shoppers in the store. And online, it is even easier to get a tremendous amount of value from reviewing a limited number of customer sessions because you can choose the best ones before observation begins.

If a customer experience issue is large enough to have a significant impact on your business, then that issue will be affecting a significant number of your customers. For example, a retail site would not see its order rate drop significantly if only a few customers were having an issue. In fact, an issue that lowers your order rate by just 5% must be affecting at least one in twenty customers. By reviewing a sample of only 15 to 30 sessions, the probability of identifying this issue is extremely high (almost 80%). If the change in order or conversion rates is even higher, then even fewer sessions need to be reviewed.

The laws of statistics can tell our retailer how many sessions to look at for a given change in conversion rate, if the underlying problem is causing 100% of shoppers to abandon their purchases.

For Example:

• If the change in conversion rate is 10% instead of 5%, the retailer would need to look at just 15 sessions to have an 80% chance of finding the issue.


• If the change in conversion rate is 15%, it should only take reviewing 10 sessions to have that same 80% chance.

One of the main purposes of our customer experience management best practices is to point you right to the sessions you should be looking at. In other words, best practices help you determine which ones are most likely to give you the best insight—the insight that leads to the biggest improvements in online customer experience. Instead of looking for a needle in a haystack, you’re looking for a pitchfork in a hay pile.

Your Turn
Share your best practices for discovering which sessions to look at for your customer behavior analysis.

-- John Dawes, Vice President, Product Management

The Qualitative and The Quantitative Faces of Customer Behavior Analysis

In my previous entry, I talked about the three phases of Tealeaf’s online customer experience optimization methodology: Visibility, Insight and Answers.  Visibility is how you become aware of customer experience issues; Insight is where you understand the root cause and business impact of each issue; and Answers is about taking action on your insights. The cornerstone of the critical Insight phase is what we call customer behavior analysis. I think it’s important for me to explain in more detail what customer behavior analysis is.

Regardless of how you become aware of potential customer experience issues, customer behavior analysis is how you answer the why questions about the web site: Why do more customers abandon the credit card application on the second step rather than the first step? Why are customers searching for products multiple times and still not adding items to the shopping cart?

To provide you with the most insight, customer behavior analysis must have two pillars: the qualitative and the quantitative. Because without qualitative data, the numbers from quantitative data are just numbers. How often does an anomaly in the numbers get written off as a data problem because it can’t be validated in the real world? And at the same time, qualitative data that can’t be quantified can cause you to focus on the wrong things. You might ignore a huge problem that’s not very visible and invest in site changes that really don’t translate into higher sales. It’s the connection between the qualitative and the quantitative aspects of customer behavior analysis that makes customer experience management both data-driven and actionable.

Customer behavior analysis is usually an iterative process, where both of its pillars work together in concert. Let me illustrate this with a real-world example. One of our customers, a leading communications company, noticed a rise in the number of customers going through the subscriber process multiple times before abandoning. They needed to know why, and traditional segmentations weren’t giving them the answer.

• The first step in their customer behavior analysis was to identify which customers abandoned the process. This information might come from a web analytics tool or from searching in a customer experience management tool like Tealeaf.
• Next, they looked for visual evidence  that something was not working as expected. With Tealeaf’s visual replay capabilities, they simply watched some of the sessions of abandoning subscribers. Before long, the company discovered an endless loop in the “new subscriber” process. Once new subscribers enter their payment information, the last step before order completion, they’re supposed to enter a date for equipment delivery. However, when new subscribers neglected to schedule a date, they ended up in an endless loop.
• Finally, they quantified the problem by looking for other sessions where the same thing went wrong. If you have found the right problem, the number of users affected will reflect the number of users who didn’t complete the process successfully. This analysis also gives you the information you need to determine the problem’s actual business impact. In this case, the company found that the endless loop was putting several potential new customers per day into a ‘no man’s land,’ representing more than $220K in lost revenue per year.

From this example, you can see that customer behavior analysis is a systematic way of uncovering and understanding customer experience issues from both a qualitative and quantitative perspective. While you can conduct customer behavior analysis without a customer experience management tool like Tealeaf, our customers have found that they save tremendous amounts of time and effort by not having to reproduce issues manually or search log files to determine who has been affected by an issue. And by having quantitative business impact at their fingertips, they find it much easier to set their priorities around making the biggest improvements to customer experience.

-- John Dawes, Vice President, Product Management

Why Ebusinesses Need Online Customer Experience Optimization Best Practices

Over the last few years, I have talked to hundreds of people about customer experience management – including Tealeaf customers, prospective customers, and industry analysts and observers. Virtually every one of the people I talk to knows that online customer experience has a lot of room for improvement. After all, no one has ever been able to build and maintain a dynamic web site that works perfectly every moment of every day for every customer. Between implementing new content, changing technology, managing internal stakeholders, and designing for different types of customers, you could never produce a 100 percent error-free site that is highly usable for every visitor.  We have all experienced problems ourselves – not being able to log into an account or add an item to a shopping cart, getting lost in a site’s navigation, or seeing a confusing error message.

So, I find little disagreement in the industry that ebusinesses need to focus on customer experience management. What most of them haven’t been able to do is to take a systematic, quantifiable approach to the problem. Megan Burns of Forrester Research found the same thing when she and her team conducted a customer experience survey earlier this year. In spite of the fact that 91 percent of executives surveyed said that online customer experience is either very important or critical to their companies in 2008, only 12 percent said their organizations take a disciplined approach to customer experience.

Why can’t ebusinesses take a disciplined approach? In my experience, it’s because they don’t have:

1. Defined processes for discovering and fixing customer experience flaws, and
2. A way to collect the kind of information that’s necessary in order to make truly data-driven decisions.

In the absence of defined processes and without a complete picture of user experience, people must rely on opinions and hunches when making major decisions about their web site optimization efforts. Or worse yet, they simply ignore suspected customer experience issues that they cannot explain and/or cannot quantify.

This is why I started a project to bring together the collective knowledge of Tealeaf and its customers into a set of best practices for online customer experience optimization. Over the next few weeks, I will be blogging about these online customer experience optimization best practices. The best practices fall into three phases: what we call Visibility, Insight and Answers.

• Visibility best practices are about setting up the right processes, systems and even mindset to make sure that you become aware of customer experience issues. I will draw on some real-world examples to illustrate the types of problems that Tealeaf customers have uncovered through systematic customer listening initiatives, proactive customer experience reviews, ongoing site metrics reviews and regular application health monitoring.
• Insight best practices will show you how to take a deep look at each customer experience issue until you understand both the root cause of the issue and its business impact. The cornerstone of this process is what I call customer behavior analysis.
• Answers best practices are about use the insights gleaned in the first two phases to implement changes that deliver real improvements to your online customer experience.

I believe all ebusinesses can benefit from putting in place the kind of disciplined approach that our 12 best practices represent. Stay tuned in for more details and I also encourage you to dig deeper by reading our white paper, Establishing a Disciplined Approach to Online Customer Experience Optimization.

I’m looking forward to your comments, as well as your questions, regarding our best practices. Every site is unique so I encourage all of you to share your own challenges and successes to date, and share your feedback as you roll these best practices out within your organization.

-- John Dawes, Vice President, Product Management

A common Tealeaf question and a new way to reply

As the CTO of Tealeaf, I’m fortunate to talk to many organizations interested in improving their customer’s experience.  Whether I’m speaking to the CIO, the VP of eBusiness or the team managing the online channel the question always arises when discussing online customer experience:  “Why does Tealeaf capture everything, what is the data and just what is it used for?"

I’d guess I’ve had this conversation thousands of times since Tealeaf was founded in 1999 and I enjoy seeing how companies can immediately grasp how visibility into their customer can really change the way they conduct their business.  Because I haven’t had a chance to talk to everyone who might be interested in how Tealeaf operates I decided to visually walk someone through how Tealeaf captures the complete user experience.

How does Tealeaf Capture the Complete User Experience?
Here is my answer in video format...

I'm interested to know your thoughts, comments and what questions you come up with next...

-- Robert Wenig, Founder, CTO, Member of Board