If your web site is critical to your business, you have probably put in place tools and systems to monitor your applications at all times and respond to problems. Almost all of our customers use performance management tools to track application performance, response time, error rates, and similar IT-oriented metrics. In addition, error logs contain valuable information about what happened on your site. The trick is to be able to analyze the information in these logs not just for its own sake but from the perspective of customer experience.
After all, application problems do not always translate into poor customer experience while problems that do cause poor customer experience do not always get logged, for example:
Your server fails for a full hour...
- Were customers actually affected?
- Did failover work properly and hide the whole event from the customers?
- If there is a whole log file full of errors from your application servers about an inability to process user credentials, did customers see errors or did exception handling work properly?
The real metric that matters, of course, is customer experience—what impact a problem had on conversion rates, sales, and customer loyalty. As a customer experience management best practice, you should always investigate the application errors in a log file to get real answers. If a server went down, for example, find out:
- How many customers had active sessions on that server when it went down?
- Did they lose state (such as losing their shopping cart)?
- Did they see strange error messages?
You can also look for users who attempted to go to that server while it was down, perhaps because they had a bookmark that went directly to it. And most importantly, you can analyze the overall impact to your business: as a group, did they convert at a lower rate than expected?
One of our customers, a financial services organization, has taken this concept even farther. They track all of their service level agreements based on their impact on customers. As a standard operating procedure, any time the firm has a server outage or other problem, staff go into Tealeaf to investigate and quantify the impact on actual customers. This approach ensures that service level agreements have real meaning to IT and that the business owners clearly understand their value. I think that a lot of IT organizations could benefit from taking an approach like theirs.
-- John Dawes, Vice President, Product Management
Comments