There’s been a lot of talk about TileMill and CartoCSS lately, with good cause. TileMill makes it very easy generate beautiful map tiles using the Mapnik engine and CartoCSS provides a familiar method to author the cartographic representation of spatial data. As Brian Timoney points out, CartoCSS has the added bonus of making best practices shareable via copy-and-paste.
Naturally, the best way to take advantage of TileMill is to export your tiles to MBTiles and use MapBox hosting. If that’s not an option, you can pretty easily self-host with TileStream. That said, there are some organizations that, due to larger GIS workflows, IT policies, and a host of other legitimate reasons, need or choose to use ArcGIS Server to do map hosting. For those organizations, TileMill is still an option to create attractive basemaps, within certain constraints.
So I set out to see if I could bridge the gap between the two. Two blog posts pointed the way. A while back, Dan Dye blogged about how he had forked mb-util and added support for exporting WeoGeo tilepacks from MBTiles. Also, a co-worker of mine, Eric Mahaffey, had blogged some time ago about how to use Arc2Earth to manage tile caches across air-gapped networks. Using these posts for guidance, I was pretty sure I had all the pieces I needed.
First, I decided to fork Dan’s code (I love GitHub) and add support for exporting ArcGIS tiles. Essentially, I followed Dan’s pattern and added a choice for an ArcGIS schema as a command-line option. My forked version can be found on GitHub.
I then fired up TileMill (0.10.0 for Windows) and created some tiles at zoom levels 4 through 8 using the “Control Room” sample that comes with TileMill. I chose this for two reasons: First, I am not very good at cartography and this sample looks attractive and distinct from the generic ArcGIS sample I used. Second, it’s very well-known to TileMill users so it worked well for this demo. I won’t go into how I created the tiles as this post is not intended to be a TileMill tutorial.
Once I had exported the tiles to an MBTiles file, I ran the updated mb-util Python utility to export the tiles in an ArcGIS structure. The command-line usage is shown below. One note: I did not generate the “level” folders in accordance with ArcGIS convention. If you are familiar with ArcGIS, you know that the level numbers are relative so that, if your service only contains levels 3, 4 , and 5, the will be levels L00, L01, L02 respectively. Instead, the utility names them according to their original TMS levels (L03, L04, L05 in this case). So, you may need to rename your level folders when you rehost to ArcGIS Server but this should entail a maximum of 19 manual edits, which seemed like a reasonable compromise.
So mb-util created the “_alllayers” folder and its child file system for us. Because I’m not creating a conf.xml yet, I can create it by hand or use ArcGIS to create some dummy tiles with a configuration. Once that’s done, we are simply engaged in a file copy operation (picking up at step 4 of Eric’s post). For this demo, I created an actual map using one of the ArcGIS samples so that it would be apparent that the operation worked. In practice, I wouldn’t go to that much trouble up front. The image below shows the map service using the original tiles.
After I copied the tiles from TileMill into the correct location, I was able to refresh the window and see the boundary between the original tiles and the those from TileMill. Again, this was just for demo purposes. In practice, the whole cache would be from TileMill. The image below shows the updated cache, displayed using the ArcGIS Javascript API.
I apologize for screenshots but my EDN license precludes making my ArcGIS Server public for a live demo.
So, it’s perfectly possible to create nice basemaps using TileMill for use in ArcGIS server, provided you adhere to two constraints: 1) You use Web Mercator for your spatial reference and 2) You make sure your ArcGIS cache is the “exploded” variety.
Why would you do this? It is certainly possible to build great cartographic representations in ArcGIS but those representations are locked away in ArcGIS map documents, styles, layer packages and such; all of which are some proprietary binary format such as persisted COM objects. While this may not be the worst thing in the world, CartoCSS, being a text format, more easily lends itself to configuration control. Organizations that want to maintain configuration control over map templates and such can take advantage of standard tools such as git or SVN to do so. I have found that organizations with strict IT policies are also exactly the same kinds of organizations that to maintain control over content and styling so this is actually a surprisingly important factor.
I’ll probably play with this more as time and customer demand allow, but it’s good to know this option is available for providing flexibility with ArcGIS Server.
Very likely to be something I’m doing wrong/differently. But when we uploaded the tiles for a single city the server was returning 404 errors – looking at the requests showed that it was trying to serve non-exixtent tiles like R00000000/C00000000.png at each zoom level. We concluded, rightly or wrongly I can’t be sure, that it was expecting relative naming – and fixing that got it working (but I wouldn’t claim to understand exactly why it worked, so there may be other underlying issues with what I did). Cheers!
It may be a version issue or something else I did wrong, but the mb-util fork didn’t quite work for me on ArcGIS server 10 – apparently it requires that the Row folders and Column file names are, like the Level folders, relative. Certainly it’s trivial to rename the Levels, but manually renaming 30,000+ folders and files, especially when they’re in hex format, is not so simple.
I’m not a programmer, so submitting my own fork to github was a little beyond me, but I was able to fix the issue by adding a step to calculate the minimum x and y at each zoom level, store these in a dictionary keyed by the level and subtract these from the level, row and column name calculations in your fork. Very hacky, but it worked.
Thanks very much for the idea to even try this – I would never have figured it out without your code to base it on. Cheers!
Interesting. I have run this a few times with ArcGIS Server 10 and haven’t experienced that yet. I wonder if my data (North America) is masking this from me. I’ll check it out. Thanks.
So I was concerned that my previous tests, being focused in North America, were skewing my results. I created a test map service using Australia and had ArcGIS Server generate tiles for just the extent of the continent of Australia and I got the row and column IDs that I would expect. They did not appear to be relative.
This leads me to believe that there are probably some other preconditions that I need to document with the current code. Thanks for your feedback.