Python

Data Over Software

by

One of the first tasks I ever had in my then-new GIS career was doing AML development in ARC/INFO 6.x for a data production project. My code parsed DXF exported from AutoCAD R11 for DOS and then assigned attributes based on things like layer, color, line weight, feature type, and others. It also georeferenced the data based on tic marks captured in AutoCAD. The end result was multiple ARC/INFO coverages that were fully populated from data templates based on the AutoCAD characteristics. From there, QA analysts tailored the data from defaults, if necessary.

After that, I did a lot of work in AML to build a cartographic production system for a water utility. That had me building a GUI using ARC/INFO forms and developing customized editing tools with ArcEdit in ArcPlot mode.

As you can imagine, I dug deeply into AML. I learned a lot about GIS – in which I had no formal training. Because AML essentially batched the same commands the analysts used at the command line, all of this development made me quite proficient with ARC/INFO. Those were fun times. Because I needed to learn GIS, this period had a lot of value for me.

As a software developer, however, there was a big drawback that is evident in the full name of AML – Arc Macro Language. All of the time and effort I was investing into building proficiency in AML was usable in exactly one place. The same was true when ArcView came along with its proprietary object-oriented language, Avenue.

Read more

Personal Geospatial Workflows, May 2014 Edition

by

I have been spending the past few weeks dealing more with data and mapping than I have in quite a while. It’s given me a chance to regain my footing with map-making, reconnect with some end-user tools like Arc2Earth, and build a little more proficiency with things like GDAL, QGIS, and TileMill. Of course, I’ve been able to sneak in some coding as I’ve identified gaps in my workflow.

Read more

A #LazyWeb Compendium of Python Resources for Beginners

by

A friend who is in the midst of a career change and moving into the GIS world asked me for some pointers to resources for getting started with Python. I threw the question out to Twitter (with a similar variation also posted to Google+):

I got a couple of requests to summarize any information I received, which seemed reasonable. I got quite a few responses and here are some links:

Read more

Personal Geospatial Workflows

by

I’ve had a couple of people ask me recently about the geospatial tools I use. Year-over-year, that answer changes but here’s how I answer that right now:

As a Federal contractor, I spend a lot of time working with the Esri stack during my work day. A few years ago, I added a few open-source geospatial tools into my tool set and, since then, have also done a respectable amount to consulting work them as well. The balance between the two varies over time, depending on the requirements of individual customers and projects. Lately, commercial customers have seemed much more interested in open-source tools while my government customers are sticking with Esri. Since those observations are based on the the extremely heavy filter of my own recent experience, I’d be hesitant to draw any larger conclusions from them.

I’ve always believed that proficiency with a wide range of tools makes me a better consultant and integrator, so I am always exploring and trying new things. With those commercial customers, and in my own personal side projects, my recent workflows have gelled around a core set of tools, both commercial and open-source:

Read more

Cutting Tiles for ArcGIS Server Using TileMill

by

There’s been a lot of talk about TileMill and CartoCSS lately, with good cause. TileMill makes it very easy generate beautiful map tiles using the Mapnik engine and CartoCSS provides a familiar method to author the cartographic representation of spatial data. As Brian Timoney points out, CartoCSS has the added bonus of making best practices shareable via copy-and-paste.

Naturally, the best way to take advantage of TileMill is to export your tiles to MBTiles and use MapBox hosting. If that’s not an option, you can pretty easily self-host with TileStream. That said, there are some organizations that, due to larger GIS workflows, IT policies, and a host of other legitimate reasons, need or choose to use ArcGIS Server to do map hosting. For those organizations, TileMill is still an option to create attractive basemaps, within certain constraints.

So I set out to see if I could bridge the gap between the two. Two blog posts pointed the way. A while back, Dan Dye blogged about how he had forked mb-util and added support for exporting WeoGeo tilepacks from MBTiles. Also, a co-worker of mine, Eric Mahaffey, had blogged some time ago about how to use Arc2Earth to manage tile caches across air-gapped networks. Using these posts for guidance, I was pretty sure I had all the pieces I needed.

Read more

Using Dynamic Non-Spatial Data In GeoCommons

by

In my previous post, I described how I used a Python script to scrape power outage information from a local web site and convert it into an RSS feed. In this post, I’ll show how I used GeoCommons to visualize the changing information over time.

The process starts by creating a data set in GeoCommmons based on a URL link to the feed created in the previous post. The general process for doing that can be found here in the GeoCommons documentation.

Read more

Prying Data Open

by

In the aftermath of Hurricane Irene, I was trying to get information from my local electric cooperative about outages. There were many (including my neighborhood) and I wanted to see the scale of the problem. It turns out, they have a page with a map that shows current outages by zip code.

Old school outage map

It’s pretty old-school as far as web maps go but it gets the job done. Their day job is making electricity, not web maps, so I won’t critique it too much. One thing I did notice is that the map seems to be dynamically generated (as do the tables on the page) from some inaccessible data source. I search and tried to find some kind of feed, to no avail.

Read more

Triggered Notifications Using PostGIS

by

My project work the last few months has kept me away from a lot of my favorite open-source tools and I was starting to get hives. Specifically, it had been a while since I had worked with PostgreSQL and PostGIS and I was missing the experience, so I dreamed up something to do.

I do a lot of work implementing situational awareness systems for my customers and one common requirement is automated notification of events. I decided that I wanted to roll a completely FOSS approach to sending an SMS notification based upon the results of a spatial query. This post will discuss the basic wiring to make it all work. I’ll probably add more advanced features in subsequent posts but I’ll be sticking to the basics for now.

Read more

Piling On About Python

by

First and foremost, I am not a Python expert. I am not even sure I could effectively play one on TV. As I mentioned at the beginning of the year, I am trying to beef up my Python skills.

Secondly, what I have to say in this post isn’t particularly original or unique. Others have said it before.

So I am posting this because I have fielded some form of this question at least ten times (no exaggeration) in the past couple of months: “What skill do you recommend most for someone getting into GIS today?”

Read more