Real-Time Response to Known Technical Issues

Every IT person I know can name at least ten generic conditions and errors that lead to poor customer experience. It’s not hard to figure out that if a customer is seeing messages like “Invalid SQL” or “Sorry, Page Not Found,” he or she is probably not accomplishing the task that he or she wanted to accomplish.  This customer experience management best practice is about putting processes in place to identify technical and application issues, track those issues and trigger a response by IT whenever a significant number of customers experience them.

You should be looking for the following types of things:

• Known error pages such as the global error page.
• Known application or system error messages, such as “Sorry, please try again later (status code xxxx).” Even if you think that you know all of these messages, you should work with your development team to verify the list and put a process in place to update the list as the site changes.
• Unexpected error messages or pages: It is hard to anticipate every error in advance; virtually every Tealeaf customer I talk to has seen unexpected errors at some point. In addition to looking for standard messages such as “SQL Error,” our customers have found it helpful to search their customer sessions for keywords such as “sorry” or “apologize.” They have found new error conditions in this way.
• Known bad status codes, such as a 500 error.
• Slow page performance: For example, a page that takes over ten seconds to be delivered to the user is certainly creating a very poor customer experience.

You can track many of the conditions I describe above with existing site management tools. For others, such as the unexpected error messages, you will need a customer experience management solution like Tealeaf. However you track them, you should think about:

• How will IT be able to be notified of the condition, and at what point should someone respond?
  A single instance of “Sorry, Page Not Found” is probably not worth paging someone in the middle of the night. You should define thresholds for each issue and set up a way for alerts to go out when the thresholds are exceeded. Many Tealeaf customers base these thresholds on previous activity, triggering an alert when the percentage of customers getting an error page exceeds 10% of the maximum percentage in the past month.
• How will IT diagnose the issue?
  Sometimes the underlying cause may be clear right away, but other times it may take a while to reproduce. A customer experience management tool can save a lot of time by providing a visual context for where customers are encountering obstacles.
• What is the real impact of a particular technical problem on my business?
  For example, can you correlate slowdowns in response time to task completion or conversion rates? An increase in the time it takes to get search results from two to three seconds may have no impact on a shopper’s purchase decision, but the same increase in response time for displaying the product details page could have a major impact on conversion rates.

One of our customers, ARC, has seen a lot of benefit from taking this kind of proactive approach to known technical issues. Originally, the company’s web-based applications were built so that when an internal error did occur, the application automatically displayed an “application error, please call support” message. Following instructions, ARC’s customers (travel agents) who saw this message would call the ARC customer service center. But the Customer Service Representatives (CSRs) handling these calls had no visibility into the site’s problems; as a result, they would have to escalate all of these issues to production support. Production support would then research the problems and report their findings back to the CSRs, who in turn would get back to the travel agents who had called. The process was reactive, cumbersome and costly—to say the least.

ARC’s production support team used Tealeaf’s real-time alerting capability (read the ARC case study) to establish a new, proactive approach to the problem. Now, every time one of the applications generates an “application error” message, the production support team is instantly notified—allowing them to intervene immediately. Often, this means that before a travel agent has time to call into the customer service center, the issue has either been fixed, or the CSR is at least already aware of the problem and can communicate appropriately with incoming callers.

To summarize this best practice, here are the steps you should take to respond to known technical issues in real-time:

1. Define the known technical issues to track. Start with the list of issues described above and then determine what is appropriate for your site. Getting a full list can require involvement from your development team, who will know about known exceptions that might be shown to users. Keeping this list up to date requires a defined process with your development team so that new error messages can be added as the site changes. Some ebusinesses even have a standard format for error pages or a standard comment in the page so that they can set up their customer experience management solution to identify future error pages in advance.
2. If you have a customer experience management solution, use it to search for keywords like “sorry” and “apologize” that often accompany error messages. Do this on a regular basis. I think that you will uncover new errors that aren’t being tracked.
3. Define thresholds for known errors and create a system so that the right people get real-time alerts when the thresholds are exceeded. You can determine the appropriate thresholds based on deviations from historical norms.
4. Define owners: who should receive these alerts and what steps should be taken when they receive each of them. For some sites, these alerts should go to the network operations center which can monitor them 24x7 in real-time. Once they receive an alert, the operations team can pull in the appropriate teams as needed.

These steps give your error messages the right context—customer experience—to help you explain the underlying problems and get them fixed.

-- John Dawes, Vice President, Product Management