Measuring programming progress by lines of code is
like measuring aircraft building progress by weight.
Business metrics—the quantifiable measurements by which a company’s performance is gauged—are part of the broad area of business intelligence (BI) or business analytics (BA).
In this regard, metrics are the application of a mathematical expression to a set of data to analyze it and obtain a figure. This figure helps to quantify a business process and, consequently, determine its performance status.
Metrics are a core component of obtaining a view of the business in figures. Some key metrics that many organizations consider important are profit, net income, cost of goods, and services sold.
Almost everyone agrees about the importance and benefits of being able to measure performance in order to make it possible for an organization to improve business performance, make more sales, or increase customer satisfaction.
But metrics when defined and managed improperly can not only give poor results but also mislead and create more problems than they solve. Why? In the important 1998 paper Metrics: You Are What You Measure!, it’s explained as follows:
Every metric, whether it is used explicitly to influence behavior, to evaluate future strategies, or simply to take stock, will affect actions and decisions. If a brand manager knows that, in his or her company’s culture, a “good brand is a high share brand,” he or she will make decisions to maximize market share—even if those decisions inadvertently sacrifice long-term profit or adversely affect other brands in the company’s portfolio.
The authors cite several other examples of how metrics can bias performance and outcome measurements. Thus, defining and using inadequate metrics can potentially be quite misleading, if not fatal, for an organization.
The Pitfalls of Designing Metrics
The above-mentioned document describes seven pitfalls as examples of how metrics can give counter-productive results. In brief, they are as follows:
Designing metrics is a difficult process, and it is not always possible to avoid all these pitfalls. But understanding the potential pitfalls can go a long way toward minimizing their effect.
To be relevant, metrics must evolve. Some metrics created to serve a specific purpose might not be useful over time. I would add an additional pitfall to the list: treating metrics, once created, as written in stone. The most important drivers of metrics are improvement and maximizing gains (profit, performance, customer satisfaction, etc.). Once a metric is identified as no longer effective, it must be reviewed, modified, or even replaced to better reflect actual business conditions and performance.
Guidelines for Good Metrics
Good metrics are hard to identify. There is no one-size-fits-all set of metrics that will suit all organizations. Each company must design its own metrics according to its purpose, its structure, and its business goals. Along with being mindful of the pitfalls, here are some guidelines for implementing successful metrics:
These guidelines will help keep your metrics focused, manageable, and useful. Additionally, from the perspective of managers and IT departments tasked with measuring performance, the chances of formulating good metrics are vastly improved by understanding the business as a whole. Understanding the business case for implementing metrics plays a basic role in formulating the right metrics.
Achieving Balanced Metrics
Accuracy is important, but it is not necessarily the only component of a good metric. We try to design metrics with a high degree of accuracy and precision.
In scientific fields, accuracy and precision represent different measures. Accuracy is the degree of closeness of measurements to a quantity’s true value. Precision is the degree to which repeated measurements under the same conditions produce the same results.
Creating a precision metric does not necessarily mean creating an accurate one. But just because a value is repeatable doesn’t necessarily make it the number, or even the measure, we are looking for.
The key to obtaining a metric that truly measures what we want to measure is striking the right balance between the following elements:
Finally, metrics can be the core of an effective program for measurement and improvement within your organization, but can also be misleading and deceptive, generating more losses than profit. To achieve better performance, look to design metrics that truly represent what you need to measure and that are as accurate and precise as possible, as well as aligned to your organization’s strategy.
METRICS MUST BE APPLIED TO SALIENT PARAMETERS
In Axiomatic Design terminology, the key saliences are associated with functional requirements (FR).
I’m optimistic that more will appear triggered by my recent post submission to Forbes based on a revival by my friend R.Mallory Starr of a long-lost color version of a picture that we published back in the 1980s.
Brian (in Sweden)
Alias Sir George the Dragon Slayer
Knighted in Canadian Dragons’ Den 2009
But good metrics can be destroyed if people are payed based on the mesurement. Because, they will always find a way to up the value out of the reality to increase the money in their pockets.
To resume : Don’t pay people based on the value of your metrics !
First of all thank you for your comments,
Just a quick comment, getting good metric is precisely about having the right balance between business performance and human productivity, it is true that paying people based on eh value of metrics can be risky, but it is also to not play attention to metric results, as this might end in to a chaotic situation were there is no reliable way to monitor and/or control performance.
In this regard Brian has a good point mentioning and Axiomatic Design approach as a mean to comply with both functional requirements and customer needs.
That’s why it’s important to listen to stake holders and users to really understand whats is owe to be measure and how to do it.
Thank you for you comment, great cartoon!
In regards to the Axiomatic Design, I haven’t used the method formally wih¿thin metric and KPI, normally I’ll take this approach combined with a rapid application development methodology (specially to prototype if possible), being able to provide fast results as well as fast evolution of the metric as necessary.
But agree with you that salient parameters are key to develop reasonable metrics!
Thank you again for your comments.
[…] Measuring programming progress by lines of code is like measuring aircraft building progress by weig… […]