What Gets Measured Gets Done: The Achilles Heel of Metrics

Twelve West in Portland, upload by Prep.77

The opening chapters of The Carbon Efficient City make an important plea for measurement. Rightly so, Hurd and Hurd point out that for all the talk of reducing carbon emissions from the built environment, progress will only be made once we codify what we mean and how we get there. The discussion of USGBC’s LEED illustrates this point perfectly. Many building projects had worked “sustainability”  and “green” into their marketing efforts, but it wasn’t until LEED became a national standard that substantial, measurable progress was made toward more efficient development.

But a discussion of metrics points to another vital issue: sustainability can be exceptionally difficult to measure. Sustainability encompasses myriad aspects of the environment, economy, and society that work toward long term stability of ecosystems. Thus, for every BTU and MPG that is quantifiable (even if challenging), there are things such as diversity and walkability that are nearly impossible to measure and rank objectively.

Why does this matter? One could argue that we should just focus on the things we can measure now and work on the more difficult metrics later or slowly over time. True, but this leads to what is called in the policy world, the What Gets Measured Gets Done (WGMGD) problem. Below are two examples of how WGMGD can generate results that aren’t always the outcomes we are shooting for.

Rendering of the Bullitt Center

The first example comes from the Bullitt Center project and is the classic WGMGD problem; issues that aren’t measured get overlooked. The Bullitt Center, currently under construction at the intersection of Capitol Hill and the Central District (map), intends to be the “greenest commercial building in the world”. As you can imagine this is no small feat; the building incorporates every green feature imaginable.

But with all of it’s good intentions, this project still had its critics. In February 2011, an appeal was filed against the development in what some called a “solar rights issue”. To generate sufficient solar power the original plans for the building had solar panels that extended out from the building, blocking views of the sky from the public sidewalk. As one design review board member put it, “it could be a potential black hole at night”. Luckily the Bullitt Center was able to scale back the solar panels to satisfy these concerns, but the story illustrates WGMGD well. Creating more energy efficient building undeniably has trade-offs, such as solar access for power or human enjoyment. But since these tradeoffs are often hard to quantify, we often overlook what we’re giving up. There may not be a metric for public access to light and air, but it is hard to imagine a sustainable city without it.

A less obvious problem of WGMGD is the issue of homogeneity. Even if we create some metric for hard to quantify aspects of sustainability, they are generally one dimensional. They have trouble using what Hurd called systems thinking, understanding the complex nuances of systems that make them function so well. Take for example the LEED for Neighborhood Development. This ambitious project attempts to measure the sustainability of geographic areas rather than single buildings. One of the measures of walkability is the number of intersections per square mile (see page 26). Neighborhoods are awarded 1 point for 250 intersections/mile2 and 5 points for more than 400 intersections/mile2. In many ways this makes sense because the more streets there are, the easier it is to get to destinations.

Cul De Sac in Toronto's Cabbagetown courtesy of reCities

But is walkability really only about efficiently getting from Point A to Point B? Isn’t there something to be said for unbroken streetscapes? The Champs-Elysees seems to attract a decent amount of walkers. Yet some of its blocks are nearly three times longer than the much applauded block lengths of Portland. Similarly (dare I say it) even the cul-de-sac and other blocked street ends can act as wonderful local refuges for neighbors. All this is to say that a sustainable city requires variety and diversity. Metrics systems, with their inherent ordinality, have trouble accounting for this necessary miscellany.

Now, I’m not suggesting that we should forget about metrics all together. In particular areas standardized measurements are key to making our cities more sustainable. But it is important to understand that metrics are one tool, and don’t work for everything. Alongside measurement systems we need other tools, such as community engagement and local empowerment. These things can help us ensure that our sustainability techniques respond correctly to the intricate web of relationships and connections that make our communities great places to live and play. Overall, I hope this conversation helps you think about a theme I want to explore throughout this course. Cities are like ecosystems and for all the planning and retooling of our urban environments from the top, we need to maintain freedom and mobility at the bottom and let places adapt to changing environments in ways that may not seem intuitive to us now. But more on that later…

Chess Match in the "Parking Lot", Rome


One thought on “What Gets Measured Gets Done: The Achilles Heel of Metrics

  1. This is an interesting observation Josh. It seems that the underlying reason behind the use of measurement is alluded to in your final paragraph: you mention ‘community engagement and local empowerment’ as tools for making better cities, and it’s hard to make a case that they aren’t. But they are really difficult to define, and you’d probably need to figure out how to measure them to see if they are successful. So you’re back to measuring. Is your argument really against using measurement as the only tool, or is it really an argument for expanding what gets measured?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s