August 20

Shortly after I published Part 1 of this series, I received a column posted on CIO.com, "What Keeps Top CIOs Sleepless" .

The author asked CIOs at CIO Magazine's leadership conference in Huntington Beach, California, "What technology keeps you up at night?" What she heard apparently surprised her: It's not technology - it's people. Why? In summary, . . . Operational excellence and its underlying technology are simply a given. Line-of-business peers want CIOs to maintain that operational excellence, while spotting ways that the company can use IT to drive business value innovation, i.e., use technology as a lever to win new business, transform itself, or improve in significant ways. CIOs simply won't be able to pull it off without a good IT team. (I recommend the article.)

So, while the conversational focus was on people, the underlying theme was the value of IT

Measuring the Value of IT

Our topic in this three-part series of newsletters is measuring value. To continue the thread, let's explore a framework for measuring the value of IT different from the balanced scorecard approach we discussed last month. This scheme, a cause-and-effect chain of measurement, consists of four levels of metrics.

• Investment   Simply put, (a) how much time and money did you put into IT, and (b) how close were you to budget? There's no value revealed in answering the first of these questions, regardless of how impressive the number is. The second question is a measure of managerial maturity, but zero variance from budget means nothing to a customer who didn't get that last feature that would have made a difference.
• IT Capabilities  What did you get, literally, for the money and time? This measures physical devices, business applications, and IT processes (like a helpdesk). There's still no value here. If you're in the "utility" business of networks, email, and file services, this is all you can talk about, unfortunately - messages delivered and terabytes stored - and any connection to "higher" measures of value (below) are tenuous at best.
• Direct Impact  How did IT's customers actually benefit? If you can answer this using objectively measurable quantitative metrics, you've hit paydirt. Did your application or service improve upon collections? Shorten sales cycle time? Reduce cost of inventory? There may be other effects that can be harder to measure but are nevertheless positive, such as contributions to the company's value proposition or to new ways of doing business.
• Company Effects  How can you indirectly affect the company? Save it money or bring more revenue; help it gain market position; perhaps help it survive; or help it merge? The unfortunate news is that this is very hard to measure convincingly, since it's impossible to control all the market and internal company variables.

The cause-and-effect value chain of measurements in these categories should be clear: You spend assets to gain capabilities that have some effects on business processes that affect the company's bottom line. The last category shows IT's actual value; others are predictors, and the farther back you go in the chain, the weaker the predictive value.

Since we are in the business of helping our clients improve profitability and competitiveness through better management and use of information technology, we are naturally drawn towards metrics in the fourth category. But they're hard to come by for most IT work. We'll take direct business process effects that the business believes in.

By the way, we're discussing after-the-fact measurement. Some of the same metrics in the third and fourth categories are forecasted for justification purposes. I would be careful in projecting those in the fourth category, though; somewhere in my past was an IT organization that had a list of proposed projects, total savings for which was projected to be more than the company's total IT spending!

Taking the Measurement

So, how do you take the measurements? Many in the first and fourth group are available from your IT management spreadsheets and financial system reports. System management and asset management reports should help with the second.

The third is more interesting. These are metrics embedded in the business processes of the business community. If you build or enhance an application or a service, why not build measurement into the application? During the specification phase, define how the application you're building or the service you're setting up automatically takes measurements and reports the results. Make it a mandatory feature, and don't allow funding cuts to downgrade the feature to "nice-to-have". Make it a standard practice to build measurement in. That by itself is an IT contribution to value.

Ill-Chosen Metrics

Speaking of metrics, Verizon was running TV commercials some months ago, drawing a bead on cable-provider phone service. (Verizon phones always work, even when the power goes out.) The commercial went on to say that Verizon handles over a billion calls a day with 99.9% reliability.

Sounds great, but it means, on average, one million calls per day go awry. I don't think I'd advertise that.