What sort of metrics should the modern Service Desk measure? Consider this analogy – When my children were toddlers, our pediatrician measured things like their height and weight and head circumference and what percentile they were compared to other children. But now that they are older those things do not matter nearly as much, and the doctor has different ways to measure their relative health.
A similar transition is in order when it comes to measuring the effectiveness of the Service Desk. The Service Desk has for long been considered a lynchpin of IT operational efficiency and gatekeeper of senior technical resources. These days, the “all grown up” Service Desk or “Responsive Automation Center” possesses a greater degree of technical acumen. It is not only expected to quickly respond to and resolve incidents; it is also charged with preventing incidents in the first place and eliminating recurring incidents. In addition to driving operational excellence, it is expected to improve both internal user satisfaction and external customer engagement by engineering user-friendly self-help solutions. Given this expanded role for the Service Desk, it is high time we question the metrics we have been using to measure success. In this article, we discuss the “Responsive Automation Center” and present a new perspective on metrics to measure the health of the Service Desk and adjacent practices.
What Is the Responsive Automation Center?
The Responsive Automation Center (RAC) represents the latest evolution of the Service Desk, which plays a central role in supporting organizations with mature IT Service Management Models and those undergoing digital transformation. Our whitepaper, It’s about Time for a Change: Introducing the Responsive Automation Center, provides a comprehensive discussion of RAC. For now, let’s provide a brief overview.
The RAC is more expansive in scope than Service Desks of generations past. Although the RAC still interacts with users, the main value it offers is bringing solutions closer to the customer – in other words, making it easier for users and third-party customers to resolve their own issues. This is primarily accomplished by using
4 Dimensions of the Responsive Automation Center
advanced automation, workflow management platforms, and high-quality knowledge articles. Through automation and “shift left” approaches, the RAC gives the knowledge agents that work for it the time and empowerment to focus on resolving the most complex incidents, orchestrating requests that are not fully automatable and require “high touch,” and leveraging data to gain insights regarding operations and customer experience.
As such, it does not rely on tiered escalations. Instead, it prevents incidents using a combination of advanced monitoring and self-healing technologies . When incident prevention is possible, it deflects routine user incidents, requests, and questions by providing user-friendly self-help tools and service catalog items supported by automated workflows. More than this, RAC knowledge agents possess a great level of technical knowledge. Instead of triaging incidents and escalating them using a tiered approach, they rely on their advanced technical skills to resolve incidents in the first pass. Unlike traditional service desks, the RAC owns both reactive (multiple related active incidents) and proactive (trend analysis of historical incidents) Problem Management.
The business model of the RAC is constructed around four domains, each of which supports the overall value proposition and each of which can be measured.
What Are Typical Service Desk Metrics that still Need To Be Tracked in a Responsive Automation Center?
Although the RAC requires enhanced metrics that support its elevated capabilities, many typical Service Desk metrics are still relevant and need to be tracked. These include:
- Average queue time
- Incident response time
- Incident resolution time
- Mean time to resolution (MTTR)
- Cost to resolve incident
- Number and percent of correctly (and incorrectly) categorized incidents
- Number and percent of incidents correctly (and incorrectly) prioritized
Although these metrics are still relevant, the context around each and the emphasis placed on each has changed. In fact, some of these can be used as temporary metrics to assess progress in transitioning to a RAC. For example, average queue time relates to how long a user spends waiting on hold on the telephone to speak with a human agent (or how long an item of work such as an e-mailed incident or request waits in the backlog before it is actioned). Goals of the RAC include preventing incidents from happening in the first place and increasing user self-help tools. As an organization transitions to a RAC, the volume of interactions with the Service Desk can be expected to decrease dramatically over time. Ideally, an organization transitions from having a Service Desk that handles up to 70% of user interactions (and only 30% addressed through self-service and auto-healing) to a scenario where a RAC only handles 30% of interactions while automated and self-help tools cover a full 70% of incidents and requests. (Some organizations go even further and have a full 90% of interactions addressed by non-human agents).
Incident response time remains an important metric to track. However, in a fully realized RAC, most responses to incidents are automated and, in many cases, nearly instantaneous. Likewise, tolerances for incident resolution time and MTTR are lower since the RAC aims to minimize downtime through auto-detection and recovery. While “acceptable” MTTR is highly contextual, it is fair to say that expectations for incident resolution are falling from days and hours to minutes and even seconds. In this case, tracking MTTR is performed to ensure that the RAC continues to meet these goals and to identify and prevent backsliding in early stages of RAC adoption.
Throughout the history of service desks, categorization of incidents and requests has been used to ensure that work was escalated to the correct team.
- Traditionally, cases where a federated Service Desk – one where major business units such as Human Resources and Facilities provide their own specialized support – is used (or when technical experts do not sit on the Service Desk), this will still be the case. Solutionists need to know whether the categorization is accurate so they can continually improve automated workflows and route work to the correct specialists.
- In a model where technical experts work directly for the RAC, categorization allows the RAC to perform proactive problem management by identifying failure trends with specific services and technologies.
RAC Metrics place an increased focus on “Shift-Left”
RAC metrics place a “shift-left” approach front and center. They are focused on measuring the effectiveness of preventing and deflecting incidents and automating service requests except in situations where human interaction adds value.
In a shift-left culture, we move work closer to the source empowering users and employees with tools and processes to find solutions to their problems and make decisions that help them do their jobs more effectively. This not only frees up time so technical subject matter experts can focus on the complex work they enjoy, but it also creates a culture of employee engagement and empowerment.
Obvious metrics candidates include:
- Number and percent of incidents detected by monitoring tools
- Number and percent of incidents remediated by monitoring tools without human intervention
- Number and percent of incident tickets reported by users to the RAC and/or directly to other specialist teams
Additionally, RAC metrics give us insight into how other practices (such as knowledge management and service catalog management) contribute to the health of the RAC. These metrics, while perhaps not so obvious, include:
- Number and percent of incidents able to be resolved by the customer without escalation
- Number of self-service tools available to end-users
- Change in the number and percent of self-service knowledge articles
- Change in the use of self-service knowledge articles
- Change (as rated by end-users and technical staff) in the quality of knowledge articles
- Change in the use of the service catalog for self-service requests
- Change in the number of automated workflows created for self-service requests
- Change in the velocity of creating new service catalog items and retiring outdated items
RAC Metrics Focus on the relationship between Operational Excellence and the Customer and User Experience
While customer and user experience has always been important, RAC metrics place an increased focus on gaining an “outside-in” view of the Service Desk and service management. In other words, the RAC is particularly concerned about the impact of improving specific areas of operations on customer satisfaction. As previously referenced, low MTTR is often a key driver of customer satisfaction. It is, however, only one factor. The RAC also needs to be concerned with qualitative aspects of the overall customer experience such as whether customers are interested in “high-touch,” “low touch,” or “no touch” interactions, which communication channels customer prefer, ease of submission for service requests, and ease of use for self-service incident wizards.
The RAC Measures Employee Skills and Engagement
As critical as it is to measure customer and user experience, so too is it important to develop and track metrics around RAC staff. RACs require employees to have a higher level of technical skills than those on the traditional Service Desk. Staff are expected to be able to investigate and resolve complex incidents that once had been escalated to Tier 2. They are also expected to be proficient with a number of monitoring and cloud-based tools. In some industries, the Service Desk may be called upon to support specialized applications. At a minimum, all RAC staff should receive ITIL 4 Foundation certification and some roles could benefit from additional ITIL 4 training such as Create Deliver, and Support. Beyond this, technical training should focus on the particular needs of the organization. Metrics should support a continuous learning approach. Here are some suggestions:
- Number and percent of RAC staff with ITIL 4 Foundation certification
- Number and percent of RAC staff with upper-level ITIL 4/ITSM certifications
- Number and percent of RAC staff that have taken tool-specific training in the last quarter
- Number and percent of RAC staff that have taken continuing education classes in the last quarter
- Number and percent of RAC staff that have completed required security training
In addition to measuring employee learning, RAC leadership recognizes that employee engagement goes hand-in-glove with customer satisfaction. Engaged employees tend to foster a superior customer experience. What is more, the stresses associated with Service Desk work are changing. In prior years, a primary staff stressor was dealing with the volume of incoming work. Though the RAC receives far fewer overall tickets, the complexity of incidents and requests that require human intervention is greater and the tolerance for restoring service is lower. Leadership must measure RAC staff engagement around key drivers such as:
- How do staff rate the level of collaboration on the RAC and with other teams
- How do staff rate the training opportunities available to them
- How effectively are staff able to accomplish their jobs using specific tools
What Service Desk Metrics Will Be Measured Less Going Forward?
In the era of the RAC, it is less a matter of eliminating previously tracked metrics and more a matter of evaluating existing metrics in a new light and tracking additional metrics. That said, measuring outputs for their own sake is not particularly useful. Likewise, although governance is always important, metrics used strictly for Service Desk process governance are not as critical as other metrics we have discussed. A few outdated metrics that immediately come to mind are:
What are the New RAC Metrics?
RAC metrics are a combination of “tried and true” legacy metrics as well as metrics inherited from adjacent practices. Specific key performance indicators (KPIs) measure progress towards the goals of:
- Preventing incidents
- Reducing downtime
- “Shifting Left” (moving solutions closer to the end-user or customer)
- Minimizing the amount of human interaction required to resolve incidents, fulfilling requests, and answering questions
- Improving customer and employee experience
- Promoting people empowerment and continuous learning
The table below provides a starter list of Critical Success Factors (CSFs) and more than thirty RAC key performance indicators (KPIs). Certain legacy KPIs are temporary in nature and are used to track progress during a transition to a RAC. Leaders should not feel compelled to track every KPI in this table. Instead, it is better to track a few KPIs that are relevant to your current state and goals. (Please note that organizations often have nuanced definitions for any given term and not all best practices use the same definitions.)
| CSF | KPI | Description | Why It Is Important |
|---|---|---|---|
| Preventing Incidents | Percent change in incidents detected by monitoring tools | Monitoring and event management tools act as an early warning system and can detect incidents before they cause disruption. | · A key aspect of the RAC is to deflect non-complex incidents from human agents. This frees up knowledge agents to focus on more complex and meaningful work. An increase in incidents detected by monitoring tools is a lagging indicator of progress in transitioning to a RAC. · A decrease could indicate that tool thresholds must be re-evaluated. |
| Percent change in incidents remediated by monitoring tools without human intervention | Beyond simply detecting incidents, state-of-the-art tools can resolve incidents without involving Service Desk agents or other technicians. | · Self-healing functionality allows human knowledge agents to focus on higher-value tasks and more complex incidents. Auto-resolution also tends to minimize downtime/improve MTTR. · A decrease could indicate that the tools are not properly calibrated. |
|
| Number and percent of new problems raised by proactive problem management | The RAC, unlike previous Service Desk iteratives, is directly involved with trend analysis and identifying problems using historical data analysis. | · New problems raised and investigated by the RAC demonstrates an active commitment to continual improvement. | |
| Number and percent of problems resolved | Beyond simply raising problems, the RAC (and other teams) must prioritize and resolve problems. | · Resolving problems prevents future recurrence of incidents which improves user satisfaction and operational efficiencies. | |
| Minimizing human/manual intervention | Percent change in incidents reported by users to the RAC or a specialist team | Incidents reported by users and customers indicates a failure to prevent the incident and the inability of a monitoring tool to identify the incident. | · When organizations transition to a RAC, an increase in incidents reported by monitoring tools should be accompanied by a corresponding decrease in incidents reported by users or customers. This is an indicator of progress when adopting a RAC. |
| Number and percent change of incident types addressed with human intervention more than once. | Recurring incidents indicate a failure to devise prevention or self-healing solutions. | · The RAC is effective when it frees up human agent time to focus on higher value work. Whenever a human agent receives an incident, the team has the dual goals of resolving the incident and creating automated solutions to prevent the incident from recurring or to enable self-healing. This is an extension of traditional problem management. | |
| Excellent Customer and Employee experience | Number and percent change in self-service tools available to end-users | Self-service tools include wizards that ask users questions to help them resolve their own incidents and automated workflows to place requests. | · When it is easy for the user/customer to resolve their own issue, it reduces workload for the RAC. This, in turn, allows the RAC to focus on more complex work. · A reduction in this number could indicate successful efforts to standardize technology by removing redundant tools. |
| Number and percent change in quantity of self-service knowledge articles | Knowledge articles help users and customers to find answers for themselves. | · An increase in knowledge articles suggests a commitment to empowering the user to resolve their own issues. · A decrease could indicate knowledge management (KM) is retiring obsolete articles. |
|
| Percent change in use of service catalog for self-service requests | The service catalog is a primary means for IT to communicate service offerings to users and customers. Properly designed and maintained catalogs enable users to submit their own requests without having to engage a human agent. | · The more users leverage the service catalog to make requests for simple services and products, the less human knowledge agents need to be involved. This not only frees up knowledge agents to focus on higher-value work, it also improves customer satisfaction when the catalog is “user friendly” and requests are easy to make. | |
| Number and percent change in automated workflows created for self-service requests | Ideally, an automated workflow exists for every type of user request. | · RAC knowledge agents, DevOps teams, and others in IT should actively work to automate as many workflows as possible. Frictionless workflows increase user satisfaction and remove toil from RAC staff. | |
| Excellent Customer and Employee Experience | Percent change in RAC staff satisfaction with tools and technology | In order to perform well, RAC staff must be satisfied that the tools they use are fit for purpose, fit for use, and are intuitive. | · Part of overall RAC staff engagement relates directly to the ease-of-use and utility of the tools they use on a day-to-day basis. · Decreased levels of satisfaction can point out opportunities for continual improvement. |
| Percent change in customer/user overall satisfaction | A major aspect of transitioning to a RAC is to improve customer and user satisfaction. Satisfaction can be measured in many ways, including short surveys, conversations, and customer focus groups. | · Customer and user satisfaction is notoriously difficult to measure. However, the customer and user perspectives of the RAC (and IT at-large) are the most important. · Decreased levels of satisfaction can point out opportunities for continual improvement. |
|
| Percent change in First Contact Resolution (FCR) | FCR means that an incident is resolved by the first person with whom the user speaks without resorting to any escalation. | · In some RAC models, “centralized” knowledge agents working directly for the RAC will be able to resolve complex incidents. In other federated service desk models, complex incidents will be directly immediately to specialized teams. FCR decreases escalations, which are costly, cause delays, and decrease customer satisfaction. | |
| Percent change in average queue time | Average time telephone callers spend waiting on hold to speak with a RAC agent. Organizations that use e-mail or other channel for incidents and service requests sometimes include the amount of time it takes to respond to an incident in the average queue time metric. In this case, average queue time is essentially the same as incident response time. | · Informs staffing plans. · Imperfect proxy for customer satisfaction (longer wait times mean lower customer satisfaction). |
|
| Percent change in incidents that achieve service level targets for response times | Many organizations have service level agreements (SLAs) in place that specify service-level targets for how quicky an incident must be acknowledged and actioned by a human. Although, strictly speaking this metric is part of the Service Level Management Practice, it is directly related to the health of the RAC. | · Can serve as a lagging indicator of customer satisfaction. When SLAs are breached, customers are less likely to be satisfied. · Note: Achievement of SLAs does not necessarily mean that customers are satisfied. |
|
| Percent change in incidents that achieve service level targets for resolution time | A common SLA specifies how quickly an incident needs to be resolved to the customer’s satisfaction. Once again, although this is a Service Level Management metric, it is a key indicator of RAC health. | · Can serve as a lagging indicator of customer satisfaction. When SLAs are breached, customers are less likely to be satisfied. | |
| Percent change in service request response time | How long it takes for an intelligent agent (a human being or a workflow management system) to acknowledge and respond to a service request raised by a user. | · Shorter response times tend to lead to increased user satisfaction. | |
| Percent change in service request resolution time | How long it takes to fulfill a service request to the user’s satisfaction. | · Quicker resolution times lead to increased user satisfaction. | |
| Percent change in service requests that achieve service level targets for response times | Many organizations have defined service level agreements for how quickly it takes for a human or intelligent agent to acknowledge that a service request has been raised by a user. | · Can serve as a lagging indicator of customer satisfaction. | |
| Percent change in service requests that achieve service level targets for fulfillment times | A common SLA for specific service requests defines how quickly a request needs to be resolved to the customer’s satisfaction. Once again, although this is a Service Level Management metric, it is a key indicator of RAC health. | · Meeting and exceeding user expectations for request fulfillment is a major driver of satisfaction. | |
| Percent change in customer satisfaction score for incident management | Customers and users rate their satisfaction with incident handling based on a variety of factors including quality and speed of resolution as well as friendliness of response. | · Though subjective, by definition, customer satisfaction provides an “outside-in” perspective of the RAC. | |
| Percent change in customer satisfaction score for service request management | Similar to incident, customers and users rate their satisfaction with request fulfillment based on quality and speed. Additionally, they may rate the ease of submitting requests and tracking their status. | · In the context of RAC, the ease of locating orderable items, submitting requests, and tracking their end-to-end status is nearly as important as request fulfillment time. When feasible, each of these should be measured discretely. | |
| Empowered People and Continuous Learning Culture | Percent change in use of knowledge articles by end users | Simply creating new knowledge articles does not mean the users and customers will use them. Leaders must ensure that users are aware of knowledge articles and find them helpful. | · As users increasingly make use of high-quality knowledge articles, questions to the Service Desk should decrease. |
| Number and percent change in RAC staff with ITIL 4 Foundation certification | ITIL 4 is the leading global best practice for IT service management. | · ITIL 4 Foundation is a baseline certification in service management. It is desirable that 100% of RAC staff earn ITIL 4 Foundation certification. | |
| Number and percent of RAC staff with upper-level ITIL 4/ITSM certifications | Continuing training and education in ITIL 4 show a dedication to the objectives of the RAC. | · As ITIL 4 is the global standard for ITSM, continuing certification training for the RAC staff show a commitment to life-long learning and continued user/customer-service excellence. | |
| Number and percent of RAC staff that have taken continuing education courses in the last quarter | The RAC needs to be up-to-date with changes to technologies and ways of working. Continuing education can take the form of training on specific tools and technologies as well as work methods (agile, DevOps, project management, customer service, ITIL, etc.) | · Consistent education on the part of RAC staff demonstrates a commitment to continuous learning and helps knowledge agents better support users and customers. | |
| Number and percent of RAC staff that have completed required security and privacy training | As the scope of RAC responsibilities expands, security and privacy risks are greater. Specific training should be tailored to workplace needs based on business outcome, governance, risk, and compliance. | · 100% of RAC staff should have appropriate security and privacy training. | |
| Minimized MTTR | Percent change in incident response time | How long it takes for a human being to acknowledge and respond to an incident reported by a user. Note: Some Service Desks count automated incident response in this metric. In our definition we specifically exclude it. [DC2] [KJ3] [DC4] | · Informs staffing plans. · Helps to measure operational efficiency. |
| Percent change in incident resolution time | How long it takes for an incident to be resolved to the user’s satisfaction. | · Helps to measure operational efficiency. · Imperfect proxy for customer satisfaction (longer resolutions times mean lower customer satisfaction). |
|
| Percent change in mean time to resolution (MTTR) | The average elapsed time from when an incident is opened until the incident is resolved to the user’s satisfaction. Note: MTTR is often used in relation to particular services and systems. In this context, if there are multiple incidents associated with the same system, the MTTR aggregates the downtime associated with each occurrence such that: MTTR = sum of all time to recovery for each occurrence / number of incidents. | · Helps to measure operational efficiency. · Key driver of customer satisfaction for desktop support. |
|
| Percent decrease in number of Incorrectly categorized incidents | Incidents are typically categorized by services and/or by technology impacted (e.g., Telecommunications, Network, Database, etc.). | · Helps to measure operational efficiency by ensuring that incidents are escalated to the correct technical support specialists. · Serves as an input into trend analysis for problem management. |
|
| Percent decrease in number of incorrectly prioritized incidents | Incidents are typically prioritized based on the triage technique which considers impact and urgency. Incidents with the greatest impact on the business generally have the highest priority. | · Helps to determine whether resources and efforts are appropriately and efficiently allocated. | |
| Percent change in ticket reassignments | Number of times a ticket (typically an incident) is reassigned to another team or individual. | · Hand-offs usually result in delays and risk knowledge loss during transfer. A RAC goal is no reassignments. | |
| Cost Effective | Percent change in cost to resolve incident | How much it costs to resolve an incident. In a tiered approach, the average cost at each tier is multiplied by the time the incident remained in that tier. The cost at each tier is then aggregated to produce a total cost. In a tiered approach, it costs less to resolve an incident at the Service Desk level since resource costs are lower. This is a heuristic or best effort metric since average costs are being used per tier (i.e., the cost of tier 2 and tier 3 resources is not the same across all teams or specialties). In a fully-realized RAC, the cost to resolve an incident should be more stable and predictable since tiered escalations are rare and RAC costs are essentially fixed. | · Helps to measure operational efficiency. · Aids in financial analysis and budgeting. |