Key Performance Indicators

Key Performance Indicators define factors the institution needs to benchmark and monitor.  Assessment techniques provide the mechanism for measuring and evaluating the defined factors to evaluate progress or impact.  KPIs specify what is measured and assessment techniques detail how and when it will be measured. KPI is a measure used to define and evaluate how successful an organization is. Typically is expressed in terms of making progress towards its long-term organizational goals. KPI incorporates information on the sources, calculations and definitions for each measure and sets out the timetable for submission of monthly data.

KPIs assist an organisation to define and measure progress toward organisational goals and objectives. Once an organisation has analysed its mission and defined its goals, it needs to measure progress towards those goals. KPIs provide a measurement tool.

KPIs assist an organisation to measure that it is ‘on track’ – most often, that it is working towards and attaining a beneficial outcome or improvement. In many cases, KPIs are used in projects and to measure service delivery.

There are as many KPIs as ways in which they can be constructed. For example, in an Electronic Document and Records Management (EDRM) project, KPIs could be used to measure client uptake as the system rolls out. Another example is to measure the timeliness and quality of service delivery – in this case, KPIs may be used to measure that records services meet agreed delivery times for correspondence in accordance with a Service Level Agreement (SLA).

KPIs are quantifiable measurements that reflect the critical success factors of a business.

  • Key Drivers that have a major impact on the performance of the business
  • A handful of numbers that give the owner an “at a glance”view of the business
  • Keep their finger on the pulse of the business
  • Identify hot spots that need attention
  • Act quickly to drive the business forward

Key Performance Indicators: Using Them Effectively

Key performance indicators (KPIs) are just one of the ways of using measurement and evaluation in KM initiatives. They give a very focused view that is most useful for monitoring KM activities for progress in the desired direction. They do not substitute for the other measurement and evaluation activities listed above.

Monitoring via KPIs can provide useful inputs to impact evaluation, but unless KM activities have a direct quantitative output such as sales results or direct cost savings (mostly they do not) they do not in themselves provide sufficient data to evaluate and assess the positive impact of KM. KPIs almost always need to be supplemented with some qualitative analysis to understand the background drivers for the trends and results displayed by the KPIs.

It is a particular risk in using KPIs (especially if you do not extend them with impact evaluation techniques) that your KPIs give you an illusion of progress. KPIs typically monitor activities and quantifiable outputs (such as documents created). KPIs can be good at reporting on KM efforts in tangible ways via numbers and trendlines, but they do not substitute for evaluating the performance of KM in terms of positive impact on the business. Counting beans (or documents) alone does not tell you whether your KM efforts are paying off. So KPIs are not enough and focusing on them should not distract from the real question, which is one of organizational performance.

Key Performance Indicators Measurement is Not Static

Secondly, at the beginning of any new KM initiative, your measurement system will evolve with the activity itself. You will have two, perhaps three measurement horizons.

  1. Monitoring Investment: Before you start the activity cycle, you will be most interested in the investments and inputs required to launch and sustain the activity. If this involves any complexity, such as multiple investments of money, time and effort from different places, you may need to monitor the investment inputs to ensure that they are taking place when required.
  2. Monitoring Adoption: When you launch the activity, you want to check whether or not the activity is being taken up according to plan. You will focus on evidence of activity levels, and you will be most interested in examining the trends (increasing).
  3. Monitoring Health: Once an activity is established you will be less interested in trends (though you will continue to monitor them for health) and your focus will shift towards benchmarking your activity levels against other similar organizations and looking for factors that can strengthen the activities and the outputs. It is at this stage that you will extend your monitoring beyond activity levels and start to focus on monitoring and evaluating value creation from the activity. At the investment and adoption stages, value creation is not a major target of attention.

This monitoring cycle can vary in duration from a few months to up to a couple of years depending on the type of activity and complexity of the change being introduced. For this reason, it is important to be able to build individual sets of KPIs whenever a new activity, program or system that is introduced, where the purpose of the KPIs is defined, the three stage activity cycle is defined and the duration of each stage is anticipated; and where the switch of focus between the three stages is properly planned and actioned. Examples of different KPIs for different types of initiative are given below together with a template to use in drawing them up.

Understand What the KPI’s Mean 

KPIs almost always require qualitative analysis to support their interpretation. At the investment stage (if being monitored), the trigger for a qualitative analysis will be a variation from plan. At the adoption stage, the trigger for a qualitative analysis will be a trend contrary to expectations.

At the health stage, the trigger for a qualitative analysis will be any significant variation in activity levels or a large gap between a comparable external benchmark and the actual performance; because this is also the stage at which the KM activity is expected to create value, proxies for value creation need to be introduced, and you will need to make a link between your monitoring of KPIs and your business impact assessments, using the other measurement and evaluation mechanisms apart from KPIs (such as story collection, MSC, management survey etc).

Examples of these qualitative supporting activities for specific types of KM activity are given below. These are given for illustrative purposes only, and should be selected carefully to support your objectives and match your resources.

Key Performance Indicators Definition

KPIs assist an organisation to define and measure progress toward organisational goals and objectives. Once an organisation has analysed its mission and defined its goals, it needs to measure progress towards those goals. KPIs provide a measurement tool.

KPIs assist an organisation to measure that it is ‘on track’ – most often, that it is working towards and attaining a beneficial outcome or improvement. In many cases, KPIs are used in projects and to measure service delivery.

There are as many KPIs as ways in which they can be constructed. For example, in an Electronic Document and Records Management (EDRM) project, KPIs could be used to measure client uptake as the system rolls out. Another example is to measure the timeliness and quality of service delivery – in this case, KPIs may be used to measure that records services meet agreed delivery times for correspondence in accordance with a Service Level Agreement (SLA).

Key Performance Indicators Characteristics

Other KPI characteristics identified in the literature are listed below. A KPI does not need to satisfy all of these characteristics to be useful to the organisation and characteristics may overlap. A KPI should be:

  • relevant to and consistent with the specific organisation’s vision, strategy and objectives (see Figure 2);
  • focused on organisation wide strategic value rather than non-critical local business outcomes – selection of the wrong KPI can result in counterproductive behaviour and sub optimised outcomes;
  • representative – appropriate to the organisation together with its operational performance;
  • realistic – fits into the organisation’s constraints and cost effective;
  • specific – clear and focused to avoid misinterpretation or ambiguity;
  • attainable – requires targets to be set that are observable, achievable, reasonable and credible under expected conditions as well as independently validated;
  • measurable – can be quantified/measured and may be either quantitative or qualitative;
  • used to identify trends – changes are infrequent, may be compared to other data over a reasonably long time and trends can be identified;
  • timely – achievable within the given timeframe;
  • understood – individuals and groups know how their behaviours and activities contribute to overall organisational goals;
  • agreed – all contributors agree and share responsibility within the organisation;
  • reported – regular reports are made available to all stakeholders and contributors;
  • governed – accountability and responsibility is defined and understood; and
  • resourced – the program is cost effective and adequately resourced throughout its lifetime.

Be Realistic: KPIs have a Cost

Monitoring and measurement are powerful ways of keeping track of your investments/efforts and alerting you to important changes (both good and bad) in your KM initiatives. However, they also have a cost.

In some cases (eg system KPIs) you may need to commission special reporting tools to generate the reports that you need. Somebody will need to collect data and analyse it. You may need to conduct an audit. If there are frequent changes you will need to follow up with qualitative analysis to explore the reasons. Hence it is essential only to choose the minimum number of KPIs to achieve your monitoring and evaluation objectives, and consistent with your resources.

KPIs May Bias Apparent Activity Levels

KPIs are often used to influence action, especially if they are linked to performance reviews and recognition and reward systems. This may sometimes produce unintended effects, or a tendency to game the KPIs being monitored, at the expense of important aspects of KM that cannot be easily measured.

An example of an unintended effect might be the linking of storage costs with file quotas, where in order to limit the costs of storage space on servers, an organization might impose quotas, eg on email space or size of network drives available to a department. Research shows that this does not result in rationalization of documents (which is the intended effect) but very often a flight of documents to “invisible” storage such as CDs, thumb drives and PC hard disks. Faced with a KPI that penalizes certain behaviours, staff will often improvise a strategy that is invisible to the measurement system.

An example of gaming KPIs might be the linking of rewards to numbers of knowledge assets submitted. Research shows that there are significant spikes in quantities of new documents contributed just before performance reviews, but also that the quality of the knowledge assets is extremely variable. Rewards and penalties should not normally be associated tightly to quantitative KPIs for this reason. Qualitative analysis, in particular the contextual information gathered from anecdotes and examples, is essential for understanding the true drivers behind numerical KPIs.

Key Performance Indicators Example

Participants for the study were drawn from two call centers in northwest Ohio.  Call centers were chosen because they are becoming an increasingly common environment for a variety of organizations.  As a result, many call center jobs are being created across the nation.  In addition, because of their propensity for electronic monitoring of individual employees, call centers have available multiple and objective measures of key performance indicators.

Why are KPIs important?

  • Gives a good indication of performance
  • Commonly used in business
  • Metrics to define and measure business goals

The first call center (A) was a regional office of a national promotions organization employing approximately 138 customer service associates.  Sixty-three percent of the total study respondents (n=91) came from this center.  The second call center (B), employing approximately 75 customer service associates, was a local provider of cable television and internet services.  It provided 37% of the total study respondents (n=54).  Each call center gave objective performance data directly to the researchers, but permitted us to collect individual data via questionnaire from voluntary participants.  In both cases we were able to collect similar data from both organizations on the individual employees and from the organizational performance records.  Thus, for the analyses done here, the data from both centers were aggregated, resulting in a sample size of 145.

The average age of the respondents across both samples was 32.47 years, and their average tenure in the organization was 3.86 years.  Ninety percent of all respondents were female.

Key PerformancemIndicators-Example

Other Examples:
  • GNP (Gross National Product)
  • ARPU (Average Revenue Per User)
  • Dow Jones Index
  • Expressed as a %
  • Accepted as a “report” on status of economy
  • Longitudinal (trend over time)
  • Comparisons can be made (states, counties)
  • Made up of smaller components (regional, industry)
  • It’s difficult to know WHAT to do about it from this one measure
  • Has understood limitations (underemployed are left out, chronic unemployed are left out, seasonal employment etc)
  • Despite limitations we still know it works (as it rises we see people losing jobs, foreclosures go up, our enrollments go up etc…it has face validity)

KPI vs. Balanced Scorecard

A note about terminology: what we call Balanced Scorecard is also called “kpi”. KPI is for “Key Performance Indicators”. What is the difference between KPI and Balanced Scorecard? Actually there is no much difference. What we call “Indicator” is equal to what we call “Metric”. What metrics does Balanced Scorecard include? The most important one – the “key” metrics. And finally, the goal of Balanced Scorecard is to measure, yes, the performance of your business, focusing on some specific aspects.

It’s a time now to think about scorecards and kpi as a set of key metrics, which help you to measure the performance of your business. When we talk about benchmarking, we actually talk about comparing products’ or businesses’ performance, so benchmarking is also based on key metrics and their comparison.

Using Balanced Scorecard

How to use Balanced Scorecard concept to measure a business performance? It’s actually easy. You will need to pass tree simple steps.

  1. First, you will need to design a set of proper metrics, which will describe your business well. It is very important step, as it will affect all your future estimations, so be carefully and as some expert to help you or consider purchasing ready-to-use metrics.
  2. Second, the metrics should be grouped. There should not be too many metrics and groups of metrics. It would be great if you will have four or five metric groups and about 3-5 metrics in each group. You will need to set the importance values for every metrics, you will need to describe the way, how do you measure the metric value, you will need to set a target values for metrics.
  3. The final step is calculating the performance, using your estimation of metric values, their weights and weights of their groups. The total values will tell you how the business is performing within the certain viewpoint.

It’s important to understand benefits and limitations of estimating business performance in this way.

  • The main limitation is that your metrics will never describe all your business, so what you can see in numbers is some kind of abstraction.
  • The main benefit, is having performance described in numeric values. If you have these values you will be able to record performance, to benchmark your performance and to control your business.

Key Performance Indicators Measurement Interval

  • KPI Incentives can often be measured over the industry standard 12 month rolling average.  This provides a measurement of performance averaging the last 12 months as opposed to a measurement of performance for only one month.
  • The first 11 months should be measured on a graduated scale; that is, the first months measure equals the first month’s availability; the second month’s measure should be the average of the first two months; etc.

Use Key Performance Indicators in KM Initiatives

Measuring and evaluating KM performance and impact can be used to serve a number of purposes. It can be used to:

  • Monitor implementations for progress compared to plans
  • Gather evidence of beneficial impact
  • Control the release of resources for new phases
  • Communicate with stakeholders and retain their support and involvement
  • Learn from past activity to feed into new plans

The intention and purpose of the measurement and evaluation activity will affect the type of data collection instruments that you use. Some will be open and qualitative using techniques such as storytelling (eg assessment of beneficial impact needs to be open so that unanticipated benefits can be captured); some will be closed and qualitative using techniques such as surveys with assessment questions (eg assessment by managers of the degree to which KM supports business objectives or has met agreed targets); some will be closed and quantitative using mechanisms such as activity reports (eg determining the degree of takeup of a KM activity).

This guide focuses on just one aspect of KM measurement, ie the use of key performance indicators (KPIs) to monitor progress and perhaps control the release of resources. It is important to recognize that this forms only a part of the whole KM measurement and evaluation picture which is touched on, but not dealt with in any depth here.