Dotspotting.org is officially live with some new features and swanky new cartography.
Cross-posted on PBS Idea Lab
The project has been in a partially-completed stage for a few months now, and I’ve blogged about the project before. We’ve got a few new things to announce:
- The url is now http://dotspotting.org; no more of this stamen subdomain stuff
- The cartography is completely revised, with a severe black-and-white style we’re calling toner, in a gentle nod to the halftone process that newspapers use (the project’s funded by the Knight News Challenge after all). We’ve intentionally stripped back the level of detail at most zoom levels in favor of a style that lets the dots on top shine through, while hopefully remaining legible enough for lots of uses.
- We’ve torn the guts out of the uploading process, which was giving us no end of heartache, and rebuilt it from scratch. It turns out, much to my chagrin (and having been told this ad nauseum by various people at Stamen) that geocoding of is hard, dude and best left to the professionals. Another, less gentle way of saying this is that we’ve switched over from Yahoo’s absolutely rassa-frassin’ useless geocoding service and started using Google’s amazing service, and so.
- We’ve also moved from a model where all of this was happening server-side, which is how we orignally had hoped things could work, to one where the process is split into two, distinct processes: first, we check to see what kind of data you’re trying to submit, second, we try and turn it into geographically relevant data, and third, we run it through google’s geocoder. My advice to anyone who is embarking on this kind of venture in future is simple: do not try and do this stuff server side unless you want to make it the core of your business. You will go down an endless rabbit hole and it will take you months to come out.
- We’ve changed the way we think about geocoding, after taking some hard knocks. Rather than trying to accommodate a kind of abstract geographic information space of addresses and file formats and latitudes and longitudes, we’ve decided to pick a few, representative set of data formats that are pegged to specific examples of data out in the real world, make it work for those, point people at those formats, and work backwards from there. All of the files that dotspotting can handle are now in a folder for people to browse, and the FAQ has specific examples of how to work with these files. I hadn’t fully understood the complexities involved in geocoding different data formats, and that’s meant that we had to hold things up for much longer than they needed to be. Mea culpa.
But it’s live now, and we’re working with a few different groups to make it even better, in the spirit of using real data to solve real problems. Onwards!
