“Deciphering the Data Deluge”

My biggest project for the past few years has been an ongoing series of workbooks about access to financial services in Africa, Asia & Latin America. I do this work as a subcontractor to an NGO called MIX, whose CEO recently gave an interview concisely explaining why we do this work and what it’s useful for. This paragraph gets to the heart of it:

FINclusion Lab creates single datasets and databases where previously siloes existed. For example, the data – which primarily includes access point location and demand-side data like population density, cellular coverage, poverty rates and the like – is usually found in project documents (PDFs), or separate online locations managed by regulators, or even individual Excel files from financial institutions. Bringing it all together in one place allows users – often regulators, financial institutions or others – to conduct analyses across different types of data including service points (geo-coded data), credit/deposit usage and demographics. It also allows users to visualize the data across geographies and drill down to more specific locales. Because we publish this data in a highly interactive format, users can explore the data based on their specific questions or interests. For example, a user can explore a particular district or type of financial service provider, or pick a reference period to view trends.

I’ve also been working behind the scenes on the infrastructure we use to conduct and share analyses, simplifying the toolchain and making updates & translations easier to apply. This included rebuilding a venerable Tableau template from the ground up, and here’s the first country workbook we’ve published in the new template:

MIX Zambia country workbook

Navigating within Palestine

Palestinians in the Palestinian-administered parts of the West Bank live in a completely separate reality from Israelis just a few miles away, to the point that many roads are only open to one of those populations. This turns out to be a real challenge for crowdsourced navigation apps like Waze and Google Maps. Although the linked article makes much of the countries each service is based in, I find this more interesting as a study in the accidental politicisation of maps. I don’t think Waze is trying to handicap Palestinians; it’s stuck in a model of trying to show one consensus reality, when what’s a great route for one driver may be illegal or unacceptably dangerous for another.

Jericho in Google Maps and OpenStreetMap

Clues in cuneiform

I just read about some very cool big data archaeology. A group of economists and historians constructed a dataset from 4,000-year-old cuneiform tablets, adding up to hundreds of records of trade transactions between cities. They treated trade volumes as proxies for the distance between cities, presumably calibrated using trade between pairs of cities with known locations. But many of these cities are lost to modern people, and that’s where this gets really interesting: if archaeological site A traded with lost city B, the researchers could plot a radius around A for the likely distance to B. With enough known locations, they can start to narrow down where the lost cities must be:

Narrowing down the location of a lost city

Source: Washington Post. Original paper: Trade, merchants, and the lost cities of the Bronze Age.

What would it take to predict climate change inundation well?

Last week I ranted about a poorly executed report claiming to show which properties around the US are at risk of climate change driven flooding. Peter Abrahamsen responded with:

Also, mean sea level, however it’s measured, doesn’t tell you who’s at risk of flooding.

and then pointed me at a couple of much better examples, most notably the Victoria, BC Capital Regional District’s climate change modeling project. I think it’s worth looking at all the other questions that would have to be answered to really get this right. Before diving in, I want to make clear that I’m not piling on Zillow any more: last week’s post was about flaws they should have figured out as a real estate data company, while most of what’s below will be details that they never claimed expertise in or took a position on. Continue reading “What would it take to predict climate change inundation well?”

Blindly trusting data will leave you all at sea

Zillow recently published a report on how many houses are at risk from sea level rise around the US. It’s a good idea, but looking closely at where I live reveals some… issues in their analysis. Here’s a screenshot from the Seattle Times’ article with the local angle on the report:

Homes that Zillow thinks are at risk of flooding from sea level rise in Seattle

If you know Seattle, you can probably spot the first problem. For those who don’t: the Eastern shore, and the cluster N of the word “Seattle” on that map are all lakefronts, separated from the sea by the rather large Ballard Locks. The local article’s been updated to mention that, but Zillow’s downloadable data hasn’t. And even the local article misses a detail: if sea levels were to rise by a foot more than the analysis assumed, the highest tides would still only be 4¾ inches over the lock gates—a problem for some houses for sure, but nothing like the 7 feet of flooding we’d have on our sea side, and much less than the amount the lake is already allowed to rise and fall by.

But there’s another problem, unrelated to quirks of Seattle’s Herculean engineering, and much more worrying for the reliability of this analysis. Continue reading “Blindly trusting data will leave you all at sea”