Gaining elevation

A/B Street is both a game and a very powerful tool that allows you to download a detailed street map of an area and test out changes to the configuration of roads:

A short video of A/B Street in action showing a user adding bike lanes and changing traffic signal patterns

It uses OpenStreetMap as a data source. OSM has impressively rich data on things like the number of lanes a street has, where bike lanes and turn restrictions go, and so on, but one thing it lacks is detailed elevation data. The developers noticed that it was suggesting absurd routes for cyclists in Seattle, including routes that no-one who’s actually tried cycling would ever repeat because of the hills involved. So they brought me in to figure out how to add elevation data.

I wrote a simple Python tool that reads in paths as plain lists of coordinates, and writes out statistics for each one: the start and elevations, and cumulative elevation gain and loss along each path. A/B Street incorporates this into a data load by exporting each road segment as a series of points every metre along the way, getting as fine-grained a picture of a route’s hilliness as the source data allows.

By default, elevation_lookups uses data from the Shuttle Radar Topography Mission, because that source provides fairly high quality data for most of the world’s land. I learned about SRTM while making this tool, and I’m still sort of star-struck that we have a global dataset like this, freely available to anyone who has a use for it. But it is also relatively low resolution, so I built in the ability to override the data source with higher resolution sources where known. It comes preconfigured with examples of both raster (LIDAR source) and vector (contour source) datasets for the Seattle area.

This is an open-source project. I hope it can be useful for other applications, and I have more ideas for it then I have time to implement. I’d love contributions from anyone this appeals to, and have some suggested starting points (not all requiring programming skills!).

I also ran into some technical surprises, which are below the fold in case the information is useful to anyone else

Staying relevant when the ground shifts

Before the pandemic,Crisis map legend, showing emergency services, which businesses were open, where to get sanitiser and places to volunteer I was working on a major update for Washington Hometown’s recreation mapping work. They’ve long been a major provider of outdoor recreation data, but wanted to put more of that data into public-facing maps that they host. They partnered with TOTAGO for mobile maps, and I built a map generator for their desktop maps so that they would be able to quickly publish new thematic maps by just updating some configuration files rather than having to write code each time.

This flexibility turned out to be more important than any of us had anticipated, because before the project wrapped up the pandemic hit. In the first months of Washington State’s lockdown, a lot of public land was either entirely closed to the public or had very limited services. Suddenly being able to publish a spring hiking map just didn’t seem relevant or even appropriate any more. But WHT’s speciality is keeping data current as things change, and they applied the same mindset to the map themes themselves. At the height of public confusion about where to find COVID tests, sanitiser supplies, and so on, they released a “crisis” map with that information updated daily.

As things calmed down, they quickly pivoted away from the crisis map (quickly enough that I didn’t even get a screenshot of the working map!), and started focusing on the ever-changing list of which public lands were open, closed, or somewhere in between. Now that we all know that outdoor activities are relatively safe, there are far fewer closures, but still enough that it’s valuable to have someone keeping track.

I was impressed with my client’s ability to keep this project relevant when I was afraid that the pandemic would sink it. And in the end it’s been a great validation of the map generator itself, which has helped them to stay agile.

A map of central Washington State, showing mostly open public land with a few closures

Fitting more housing into the city

Seattle is suffering from a deep housing affordability crisis, with more and more people being priced out of living there. At the same time, it’s been deeply resistant to changes in zoning that would allow enough new housing to be built. One of the examples of this is that it has a program called “Encouraging Backyard Cottages”, and has gone through at least two rounds of legal reforms to support that, but figuring out whether one can be built on any given lot still involves going through a long checklist about the exact dimensions of the site and intricacies of zoning.

In theory, DADUs (Detached Accessory Dwelling Units, the much less appealing legal term for “backyard cottages”) allow a lot of small, affordable housing to be added to single-family zoned neighbourhoods and spread out enough to not feel like a radical change in the streetscape. In practice, the complicatedness of the process adds enough of a barrier that relatively few have been built so far. Hatchback Cottages has a plan to solve this with a set of ready-to-build designs and a package of support to help people through the process.

Even with their expertise, assessing a site under the arcane rules is a time consuming process. But computers are good at applying lots of rules and calculating all the measurements, so Hatchback contracted me to run a GIS analysis assessing every residential lot in Seattle for suitability.

A map of one city block with individual lots shaded green if they could have a DADU and red if they couldn't.  Existing building outlines and the area in which a DADU might fit are also shown.

Fortunately for us, Seattle and King County publish very comprehensive and regularly updated open data about zoning and development, so I had a lot to work with. The analysis takes into account existing building footprints, lot characteristics and potential complicating factors like steep slopes and landslide hazard areas. It will never be a complete replacement for a knowledgeable human looking at the site, but by ruling out all the sites that definitely won’t work it saves my client a lot of time. Now the experts can solely focus on sites that have a relatively good chance of working out.

Pacific Northwest climate trends

The University of Washington Climate Impacts Group has built a new visualization tool for the state climatologist, which makes it easier than before to explore the whole climate record for Washington and some data for neighboring US states. The data itself is nothing new,  but being able to explore it can help to get a handle on what’s going on.  I rather arbitrarily picked my life to date as the range to compare, and got the sobering result that every single monitoring station has registered an increase in temperature since I was born; most statistically significant.

A map of Washington, Oregon, Idaho and western Montana, showing temperature increases at every station, and line charts for each state showing the overall trend

Of course, there’s no scientific basis for picking that particular range, but I wasn’t cherry-picking either (I’d have picked 1985-2015 if that were my goal). I just wanted dates that felt personally meaningful, and even as someone who considers myself relatively well-informed about climate change the sheer scale of the trend was breathtaking.

Allen Coral Atlas

As a child, I adored the National Geographic magazines.  I collected them for long enough to fill a bookshelf, and I think it’s fair to say that they had a pretty big influence on how I’ve ended up making a living.  So it was particularly satisfying to see a project I’ve been involved with since the summer get a writeup in National Geographic:

Inside the daring plan to map every coral reef from space

It’s a wonderfully ambitious project—using imagery that wasn’t available 5 years ago—and terrifyingly urgent.  Coral reefs are particularly sensitive to the impacts of climate change and ocean acidification, and massively important as both habitat and shoreline protection.  We have very little time to left to figure out ways to make them more resilient, or lose a major source of protein and watch storms do more and more damage as the reefs’ protection is lost.

The project also lost two of its champions this year: Paul Allen and Ruth Gates.  I wish I had had the chance to get to know Dr. Gates.  Among other things, she set a great example of how to engage with work this sad and frightening and never be crushed by the weight of it.  The project continues, and feels like a fitting memorial to both.

My role is to integrate the processed data we get from two different research groups: one infers depth from the satellite imagery, and the other classifies areas of reef by types of sea floor and what’s growing on them.  I write automation that turns the depth data into false-colour imagery, and prepares everything to be displayed in the web map. Here’s a snapshot of that depth imagery:

Bathymetry around Karimun Jawa, Indonesia

So far we just have a single snapshot in time of 6 reefs, but the real challenge will be scaling that to all the reefs in the world, updated regularly.  You can explore it yourself and read about the methods and partnership at http://allencoralatlas.org/ .

Because it’s a limited preview so far and most areas aren’t covered yet, panning and zooming around the globe isn’t very satisfying.  I recommend clicking on the place names in the “Mapped Areas” list to see where we actually have data.  And for a first look I prefer to turn all of the data layers off, zoom to a location, and then turn them back on one at a time starting from the bottom of the list.

My comments on the proposed addition of a citizenship question to the US Census

I’m sorry to leave this till the last minute, but there are just over 24 hours left to make public comments on an important mistake that the US Census Bureau is set on making. The Bureau has proposed, over the objections of many of its own staff, to add a question about citizenship to the 2020 survey. This is a terrible idea which is almost certain to cause immigrants to be undercounted because of fear, especially in the context of the vicious hostility the US has been displaying against immigrants over the past couple of years. Even if that were an unintended consequence, it would be reason enough to oppose this change, but there is also strong reason to believe it’s being done in bad faith. Meanwhile, the 2020 Census was already in danger from good old fashioned neglect, underfunding and incompetence, problems that will only be compounded by adding a question that discourages responses.

As someone who uses Census statistics regularly in my work, I am concerned that if this is allowed to go through it will leave us unable to rely on the 2020 count as the accurate snapshot we depend on it being. As a citizen and resident of this country, I am afraid that what we’re seeing is a deliberate attempt to skew future elections and federal government resource allocations by systematically undercounting the population of areas with more immigrants.

If you read this in time, please make a public comment.  It doesn’t have to be a long reasoned argument—even just saying “please don’t add the citizenship question” may be worth something—but here’s what I wrote, trying to put it in terms that the Census Bureau is officially supposed to care about, in case it helps you to write yours: Continue reading “My comments on the proposed addition of a citizenship question to the US Census”

“A Renewable America”

A map I made just went live at arenewableamerica.org .

This was a relatively small project, but there are two things I’m particularly happy with. One is that it’s the first time I’ve hand-drawn boundaries with enough detail to actually publish – all the convoluted parts of the Grid Campaign layer that don’t correspond to state lines are hand drawn from a reference map:

A map showing the very convoluted shapes of 3 of the US's "Regional Transmission Organizations"

The other is that it’s the first time in years I’ve been able to work on promoting renewable energy. After leaving a PhD program, my first work outside academia was an internship for a local renewable energy nonprofit, but since then I hadn’t found a way to contribute to this field until this little project showed up.  It feels good to circle back to a cause I’ve never stopped caring about.

“Mapas interactivos para el análisis de la inclusión financiera en México”

One of the pleasures of working on FINclusionLab has been getting my work translated into the languages of most of the countries we cover: French, Spanish, and Turkish. By happy coincidence these are also most of the languages that I either speak or am making a serious effort to learn. I am not fluent enough in any of them to do the translating myself, but I can at least understand the translations that I’m applying, and read domestic press coverage when it comes out. It’s not always easy to get key stakeholders to actually use the tools we’re making for them, so it was particularly nice to see Mexico’s national bank trumpeting the release of the financial inclusion dashboards we made:

La CNBV presentó los mapas interactivos para el análisis de la inclusión financiera en México

[Google’s translation is not bad, albeit even wordier than an already government-speak heavy press release]

A screenshot of the Mexico dashboard showing a map and chart about the spread of correspondent banking state-by-state

“Deciphering the Data Deluge”

My biggest project for the past few years has been an ongoing series of workbooks about access to financial services in Africa, Asia & Latin America. I do this work as a subcontractor to an NGO called MIX, whose CEO recently gave an interview concisely explaining why we do this work and what it’s useful for. This paragraph gets to the heart of it:

FINclusion Lab creates single datasets and databases where previously siloes existed. For example, the data – which primarily includes access point location and demand-side data like population density, cellular coverage, poverty rates and the like – is usually found in project documents (PDFs), or separate online locations managed by regulators, or even individual Excel files from financial institutions. Bringing it all together in one place allows users – often regulators, financial institutions or others – to conduct analyses across different types of data including service points (geo-coded data), credit/deposit usage and demographics. It also allows users to visualize the data across geographies and drill down to more specific locales. Because we publish this data in a highly interactive format, users can explore the data based on their specific questions or interests. For example, a user can explore a particular district or type of financial service provider, or pick a reference period to view trends.

I’ve also been working behind the scenes on the infrastructure we use to conduct and share analyses, simplifying the toolchain and making updates & translations easier to apply. This included rebuilding a venerable Tableau template from the ground up, and here’s the first country workbook we’ve published in the new template:

MIX Zambia country workbook

Navigating within Palestine

Palestinians in the Palestinian-administered parts of the West Bank live in a completely separate reality from Israelis just a few miles away, to the point that many roads are only open to one of those populations. This turns out to be a real challenge for crowdsourced navigation apps like Waze and Google Maps. Although the linked article makes much of the countries each service is based in, I find this more interesting as a study in the accidental politicisation of maps. I don’t think Waze is trying to handicap Palestinians; it’s stuck in a model of trying to show one consensus reality, when what’s a great route for one driver may be illegal or unacceptably dangerous for another.

Jericho in Google Maps and OpenStreetMap