Improving End-User Experience

I've come across this writeup from ComputerWorld.

Computerworld - Do you find that despite the small fortune you have spent on products to manage your servers, networks and applications, your business constituencies still complain about the performance, usability and availability of key business applications?

If so, it might be time to expand your performance management horizons to include the end-user experience. Traditional monitoring and management tools focus on factors that impact but are blind to the actual end-user experience. While they are excellent at measuring application availability, they provide no insight into real application performance from the perspective of the end user. And what is that perspective? As far as end users are concerned, an application is "not performing" whenever it does not work as expected — when it’s slow, when it’s continually spitting out error messages or when the user interface is so counterintuitive the only option is to create a work-around.

A focus on the end user is the key to driving adoption and efficient and effective use of your critical business applications. This process starts with deciding on your approach. There are three basic types of end-user experience management products: scripted synthetic agents, passive network appliances and passive client agents.

Scripted synthetic monitoring

Scripted synthetic monitoring works by installing an agent on a PC to monitor a client-side application. The agent executes a script that emulates the actions of a user. The script can do anything that a real user can do, and you can have a script do things that you would not expect a real user to do. A script could run through 100 key transactions at 5:00 every morning and verify that the system is working before the real users show up.

Despite the flexibility of synthetic agents, there are some downsides. First, someone has to develop and maintain all of those scripts. The time and effort required to maintain (and test) these scripts means that this approach is best suited to verifying the most important transactions in the most important applications. Also, the synthetic transactions create a maintenance task to reverse the effects of the transaction. But, most critically, while the synthetic transaction measures the performance delivered to the scripting engine, it doesn’t measure the actual end-user experience. So, at best, it is a proxy for the end-user experience.

Passive network appliances

Network appliances that measure end-user experience attach to the same switch that supports the appliances used to balance the load for your Web servers. Since they attach to a mirror or spanned port on the switch, these appliances are able to see all of the Web traffic that flows in and out of each switch. Custom hardware and some very sophisticated software crack into the Web and network protocols and allow you to see any transaction that you define to the appliance. Most of these appliances focus upon the Web layer of an application system, and therefore focus upon HTTP/HTTPS. All of these products allow you to see how many transactions of interest are occurring. You can see the response times of those transactions, any errors that show up in the HTTP protocol stream, and the path that your users are taking through your Web-based applications.

The great advantage of this approach is the noninvasive nature of the network appliance, coupled with complete coverage for those Web-based applications and transactions you define to the appliance. The fact that no software is required on the client side, means that this approach is ideal for business-to-commerce Web applications. Jonas Hirshfield, director of infrastructure development at Blackboard Inc., points out the value of such an approach: "As a hosting provider, we needed insight into how our product was delivered to our customers — beyond how fast disk drives were spinning, and CPU and memory utilization. We use Coradiant to understand how users interact with the software and how the environment responds to them. TrueSight gives us the ability to quickly show a quantitative report that indicates performance issues that are most often outside of the Blackboard network. That is really beneficial to the team."

A factor that can limit the value of the appliance approach is the effort required to configure these devices to recognize the applications and transactions of interest amid all the Web traffic flowing through the switch. And since there is no client-side insight, the user experience is measured only from the perspective of the Web server and the load balancer, not the actual end users themselves.

Many companies balance the pros and cons by using a variety of tools, which is the strategy deployed by Meijer Inc., a grocery and general merchandise retailer operating in the Midwest. "The end-user experience has always been the Holy Grail," said Tim Osbeck, director of ITS e-commerce at Meijer. "We use a combination of Compuware Vantage scripted robots and a network probe to give us quantifiable data for the best of both worlds. It’s data that everyone believes, so we can have a common dialogue. It’s the one truth on how applications perform here at Meijer."

Passive client agents

Passive client agents are installed on the actual end user’s desktop and capture a record of the real end-user experience. Typically, the agent works in tandem with a profile or a template for each application to be monitored (whether it’s a "fat" Win32 application or Internet Explorer). The advantage of a client agent is that it is the only approach that enables monitoring of the actual end-user experience from the perspective of end users themselves. The constraint of this approach is that for global coverage, you have to install agents on every user’s desktop. So, this approach works best when the users are employees or partners or strategic B2B customers. This approach is just not a fit for large-scale B2C applications, which are better served by the network appliance approach. The depth of information these products deliver typically means that you want to focus them on your most significant applications — not all activity occurring on the end-user desktop.

The combination of the user-base coverage and application depth provided by this approach is tough to beat. More sophisticated client agents can expand the perspective of the user experience beyond transaction response times and capture usage metrics, user errors and system errors. With this information, companies can manage the performance achieved by end users, as well as the experience that they receive. This is also the only technique capable of capturing real user interaction with the application before, during and after the transaction.

British Telecom is one company that leverages a passive client agent for end-user experience monitoring. "We had been looking to deploy a response time monitoring tool, when we evaluated Knoa EPM, which monitors the complete user experience with a passive agent approach," said Stuart Smith, director of CRM application performance at British Telecom. "The insight into actual end-user experience gives us benefits across the board, helping target response time issues, solve end-user problems faster, identify usage and adoption issues, and even make priority decisions on ongoing application investments."

Is end-user monitoring right for you?

According to a recent Forrester Research Inc. study, 67% of the surveyed organizations are more focused on the end-user experience than they were two years earlier. If you’re not looking at the potential of this application, it might be time to expand your performance management horizons to include the end-user experience. Agilent Technologies Inc. is one company that is demonstrating the business impact of end-user monitoring. "Wily CEM has illuminated the true user experience to our support staff and business leaders," said Dave Williams, Siebel functional architect. "This enables us to align business processes with performance measurements.

End-User Performance Management

I'm setting a new site to discuss about end-user performance management. I know this is a general topic, but I would like see how does this topic goes. In business nowadays, most of us are talking about focusing on customer centric. So how do business measure their performance or quality of services to the end-users? Its get interesting where most of the business use their own metric in this measurement. Some may say they outsource to external parties to perform measurement for them. For large company or organization, they might have its in-house team to do these. The query comes into my mind when I think of could there me any IT system that able to help in this measurement.

I was surfing around on the internet to get more info on such tool. But most of the measurement tools in the market could provide the require measurement but its also involve a lots of complex integration. This mean needs long cycle of implementation and resources. It would be great if there a non-complex tool that able to do this. It would save a lot of headaches. I'll share my finding on this tools in my coming post.