But even conventional technical measurements often have a direct bearing on business outcomes. "If our systems aren't up, we aren't generating revenue," says Pete Gibson, vice president of IT infrastructure and operations at Phoenix-based Cendant Hotel Group, which operates 6,400 hotels.

It's up to IT leaders to translate those metrics into tangible business terms. "Business and IT people alike have a hard time making percentages feel real," says Stanley. "You don't feel a percentage of systems uptime. You feel the business or customer impact of an outage."

To address that, Harrah's is creating new metrics that will quantify the negative effect that a system outage has on revenue or other aspects of the company's business, says Stanley.

As companies have become more sophisticated about which metrics to evaluate, they've also determined which ones to drop. For example, until a few years ago, DHL Express' IT organization used to track the number of e-mail messages that employees were receiving to determine if there was a link between developing effective ideas and communication. "It turned out to be a pointless exercise," says Bentley. "We're not afraid to abandon metrics."

That flexible attitude has also led the IT group at DHL Express to experiment with more sophisticated measurements. Instead of just assessing the number of IT projects that are delivered on time, within budget and within scope, IT also measures what the business cares most about: the percentage of projects that have met their business objectives, says Bentley.