Tealeaf and Web Analytics: Quantitative + Qualitative

In the new whitepaper, Customer Experience Management and Web Analytics: From KPIs to Customer Transactions, authored by Eric Peterson of Web Analytics Demystified, Eric talks about the different ways of understanding the online experience. He covers the quantitative measures (traditional web analytics tools like Omniture, Webtrends, Visual Sciences) and the qualitative measures -- Customer Experience Management (What is CEM? - i.e. Tealeaf). In short, he makes the conclusion that both are foundational and both are needed.  The “What” to compliment the "Why".

So, let's build on the foundation of his conclusion -- what are the benefits of bringing together Q+Q (Quantitative and Qualitative).

Does Q+Q = 2Q or do we get Q squared?
As long as Q is a positive integer greater than 1, you want Q squared, right? After all, this is CTO chatter…

Some background on the situation
In earlier blog entries I talked about where Tealeaf data comes from, Tealeaf captures every byte of the HTTP/HTTPS requests and the HTTP/HTTPS response. We need all of this data to power replay of a session. We're able to capture the complete user experience passively -- i.e. with no risk, latency, additional CPU or code changes.

In general, a web analytics tool operates on a subset of the data. This data comes from page tags, weblogs or server side sensors.

Page tags can be an interesting analytical tool -- properly done, pagetags may contain information that is not normally on the page -- and the information can be better suited for analysis by encoding more structure and context. Good pagetags require code changes and proper foresight -- i.e. you have to know what information that you want/need to capture in advance. For many of us, changing a pagetag across a large site could take days to weeks to months to implement and that doesn’t factor time to then collect enough data to properly analyze.

Q Squared -- What are the benefits?

• Drill down

• A web analytics tool will do a good job of understanding customer fall-off. They will show you a funnel analysis of where people stop in a buying process. You will be able to see how many people entered the funnel and at what point people stopped/diverged, looped, etc. This answers the question of "What Happened" -- but it leaves you thirsting for more. With our combined solution, we can now pivot from "What to Why" -- i.e. we can drill down and launch visual replay.
• How can you trust aggregate data if you can't drill down to the actual sessions and replay them?

• Drill up

• What happens when some of the users on your site experience an obstacle in completing their transaction? A small percentage of people get 'Price Not Available' when they attempt to book a flight.
• Did you even know that your pricing engine could respond with such a message? Since this message appeared on the page, the page had a valid, good HTTP status code. It's doubtful that your pagetag could have picked up this message because it would have required scanning the HTTP response for a message that you didn't even know that could happen.
• Obviously, in such a situation, it would be impossible to understand the behavior of visitors who experienced 'Price Not Available' because this dimension was not known/observed by your web analytics solution.

With Tealeaf, because we capture the complete user experience, we could search for all people who received this message. We can then export a result set of sessions back into your web analytic product. In effect, we can dynamically create a new dimension of 'Price Not Available' by creating a synthetic filter or dimension on-demand through the integration.

Synthetic Page Tags      
Tealeaf Events are beneficial. In a Tealeaf environment, you create events by establishing rules to evaluate each and every page. Tealeaf events aid in creating structured, analytical information out of unstructured content. Since tealeaf is passive and invisible to the application infrastructure, events can be easily created/changed on demand. Tealeaf uses these events to create KPI's (Key Performance Indicators) and to aid in search and session scoring.

Taking the example above of 'Price Not Available', we can easily create an event for this. Out of this event, we can send a fake page tag containing this payload up into your web analytics solution -- without having to modify the original application. Pretty darn cool, if I say so myself.

Historical data is valuable.   
Since a web analytics tool is operating on a subset of the data -- you have to define ahead of time what data to capture. If you don't capture the data at execution time -- it's lost forever -- and obviously you can't analyze what you don't see. One of the most commonly requested Tealeaf features has been the ability to event on historical data. People want to be able to make observations on past data and create analytical insight. We've come up with utilities to replay/respray old data and persist these new observations into an analytical shell.

Q Squared -- let the revolution begin.

-- Robert Wenig, Founder, CTO, Member of Board