Made to Measure: KPIs that matter

How do we turn performance data into better delivery and improved client satisfaction? Here we investigate the much debated topic of key performance indicators.

Key performance indicators (KPIs) are very much at the heart of current localisation processes and have come to be an important feature of successful performance. They represent measurable values that you can collect to gauge how well your organisation is doing. For example, how many deliveries were late? How many spot-checks did not return a pass? How many quality issues were found? And how many defects were left unresolved? These values can then be fed into a framework and analyzed against a set of defined targets.

Organisations generally choose KPIs amongst a set of metrics that can generate a positive change in their operations. For example, more scorecard passes are a
measure for good language performance; fewer invalid queries imply a good understanding of processes; and a low number of unresolved defects will improve the end user experience.

From metrics to KPIs

There are many metrics in the localisation process that can be tracked, but not all metrics are KPIs. A metric becomes a KPI when you can collect a given set of values, link them to an achievable target, run an analysis, measure, rate and finally convert your metrics into actions. KPIs are used to evaluate your success rates and reach your performance goals.

For instance, the number of deliveries you have completed in any given year is a useful metric to track the transactions made for every project, but this is not a performance indicator. Actually, it is the number of on-time deliveries that can be used as an indicator of your organisational effectiveness and customer satisfaction level. It can also be a call for corrective actions if deliveries are late more often than a specified level.

Sharpening the focus: when less is more

Sometimes there is plenty of enthusiasm for measuring the diverse aspects of production. Some years ago, one of our clients set a new focus on KPIs which would capture and measure metrics to analyze supplier performance and set them on a successful path. They initially identified over one hundred unique KPIs, which were then organised and presented in performance charts every quarter. Each indicator was in turn analyzed and discussed in depth. But this effort soon lost momentum and meaning.

After a time, the client made an important shift in approach. It was decided that a focus on key targets was more important than monitoring a large number of different KPIs. As a result, the KPIs were reduced to include consolidated data from higher level aspects of production and monitoring became more manageable. Less was more, in this case. KPI review times became shorter and analysis cycles got more to the point.

What makes KPIs effective?

The example illustrates a common situation occurring when trying to define the right KPIs. It is easy to exaggerate and line up lots and lots of indicators. This is because on the one hand you are dealing with complex human aspects of your organisation, and on the other hand there are no pre-emptive ways to gauge the effectiveness of KPIs upfront. There are however some useful guidelines.

As a start, consider how data is collected and stored in your organisation and how easy it is to extract it from your data repositories. Do you have easily accessible databases? Can you arrange data in simple, easy to understand, replicable patterns? Most databases have very good functionalities when it comes to filtering and reporting. A well-structured report can form the basis for your collection of KPIs.

Relevant, understandable, measurable

To start with, single out those KPIs that are relevant to the level of service you want to influence. Are you interested in monitoring the translation quality of your marketing material? In which case, why don’t you start to collect, analyze and present data on stylistic edits made by final reviewers? The customer satisfaction index should improve considerably when it is seen that the number of edits reduces over time. Or, for example, are you concerned that speed will win over quality in the context of defect management? Then why don’t you run an analysis of your defects history and extract data on how much early prevention your team was able to address? The sooner you detect and address defects, the easier your project life cycle will be.

Crucially, your data must be easy to understand. Reading an array of columns and rows without being able to convert them into actions is not the best use of anybody’s time. Effective KPIs give answers to questions and trigger actions, but do not prompt additional questions. On reading your KPIs, your audience needs to see the underlying issues instantly, take immediate action, make proactive decisions, take control of the issues and promote the desired positive outcome.

An effective KPI must be measurable. This will uncover positive or negative deviations from defined targets. KPIs often measure success against a threshold. For example, you set a performance threshold to measure the language quality of your content. Consequently, any result below that threshold will trigger an analysis of why the translation teams are underperforming.

Varieties of indicators

Not all indicators need to come directly from quantitative measurements. Indicators can be of different types and may include:

  • qualitative indicators, possibly measuring promoters and detractors answering surveys such as: “how satisfied are you with the performance of your language supplier?”
  • process indicators, measuring the performance of the key processes that influence customer expectations, including questions such as: “how often was your translation management system unavailable during peak production time?”
  • leading indicators, often measured against results coming from trials or pilot projects and addressing questions such as: “will my processes be better if this change is produced?”

Effective KPIs are generally not measured against generic questions such as: “Did your team achieve what was planned for this year?”

KPIs must also be meaningful, achievable and timely to your organisation. To this end, it is worth considering answering the following questions. Will your staff understand the reasons behind the indicators? Will they feel engaged in achieving the targets? Is there a risk of demotivating your employees if you set unachievable targets? And are the results collected and analyzed over an appropriate timeline? If they are too frequent, this might water down the results. If they are too infrequent, this might make it too difficult to capture significant patterns.

KPIs @ Alpha: an intelligent approach

At Alpha we are always willing to embrace any KPIs defined by our customers. In recent years, they have become more stringent and diversified, and we have approached them with an open mind, often assisting and working together with our clients in the definition and collection of suitable KPIs.

We have instantly adopted reporting dashboards generated by our clients in an effort to understand and resolve performance shortfalls and improve our overall success rate and at the same time increase our clients’ satisfaction indices.

We have also worked to define our own internal KPIs. We have carefully isolated metrics which can uncover performance variances in our internal operations and offer us with an opportunity to align our strategic objectives with the desired direction we intend to take as a well-performing competitive organisation.

Every client survey or feedback is fed into a dashboard that we use to benchmark our performance across the whole range of clients. On a quarterly timescale, we can observe the evolution of our performance for Quality of Deliverables, Organisational Maturity, Level of Service and Costs.

Language quality data is captured via a randomised and anonymous approach: we collect information on our language departments’ performance across the whole organisation and action corrective measures when we identify underperforming language teams.

The efficiency of our engineering departments is captured by analyzing the time it takes to perform regular and repetitive tasks; by the number of process improvements requested and actioned; by the number of defects left unresolved in our bug tracking databases; and through a careful analysis of the support questions asked by our engineers.

We also go as far as to capture Financial KPIs (Profit; Cost; Revenue vs. Target); Customer KPIs (Customer Satisfaction and Retention; Ability to Increase Activities with our Clients); Process KPIs (Percentage of Process Defects, Adherence to Schedules, Timely Deliveries); and Staff KPIs (Employee Turnover Rate; Employee Satisfaction and Maturity).

Moving forward

This framework is, however, not static. As KPI requirements are in continuous evolution, we regularly need to adapt them to make sure they are current and constantly aligned with our goals. We accomplish this by questioning the validity of each KPI at regular intervals so as to refine existing metrics and identify more relevant KPIs.

Defining KPIs is not an exact science. KPIs rely on time factors. Moreover, they are closely dependent on human behavior and organisational performance. Like most
human activities, it is sometimes only via a trial-and-error exercise that you will find out if a KPI really works for you. To help you with this, we can add our many years of experience in creating, maintaining and refining effective KPIs for ourselves and for our clients.

We hope you enjoyed this article! Find out more about Alpha’s range of localisation services!