HIFLD Open January 2017 Updates

I just got an announcement in my inbox of a major update to HIFLD Open. A number of new data sets have been added, along with updates to many others. The announcement also addressed HIFLD Secure, but I won’t touch upon that here. From the flyer attached to the email, here are the updates. If you are so inclined, it’s time to get scraping.

Continue reading “HIFLD Open January 2017 Updates”

HIFLD Open

Among the many things I did at the Esri Federal GIS Conference was that I attended the Homeland Infrastructure Foundation-Level Data (HIFLD) community meeting. HIFLD is now an FGDC Subcommittee, being led my Mr. David Alexander, head of the Department of Homeland Security (DHS) Geospatial Management Office (GMO). Now that the alphabet soup is out of the way (though there will be more), I’ll get to the significant news of the meeting: the announcement of the availability of the HIFLD Open infrastructure data sets for general access and download. Feel free to browse and get the data here: https://hifld-dhs-gii.opendata.arcgis.com/

hifld_open2

HIFLD Open uses the Esri Open Data platform to make the data sets available. In yesterday’s meeting. Andrew Turner of Esri stated that it took just a month to implement the site and put the data online. As my friend Archie Belaney points out, this is an illustration of the fact that the impediments to getting this done before now have had more to do with anthropology than technology. The HIFLD data started out as HSIP Gold, which included a mixture of government and commercial data sets. The commercial data was distributed under a wide federal-only license. Because all of the data was only distributed via snail-mail DVDs, it meant that the license restrictions of the commercial data sets hindered the distribution of the government data as well. Physical media trumps all. HSIP Freedom was an attempt to make it easier to get some of the HSIP data to state and local users, but byzantine workflows again meant to protect access to commercial data sets proved unworkable. HSIP was always hindered by the ever-present cadre of single-issue voters who fretted about the potential vulnerabilities presented by the aggregation of such data.

Luckily, the culture (anthropology) of open data has rendered those dissenters mute. States and localities have been publishing much more detailed data sets for some time now. Thankfully, the new stewards of HIFLD and the data formerly known as HSIP have understood this, reorganized the data into segments that can be freely distributed (Open) and that which requires access control for any number of reasons. As a result, data that should have always been freely available is no longer bound up by restrictions on other data that happen to reside on the same physical media. The government leadership deserves credit for recognizing and solving this cultural issue and the technical leadership and contract support (Ardent Management Consulting) deserve credit for making it happen so quickly.

As my career has progressed, I have become much more removed from HIFLD than I was at the outset. It’s now something I watch from afar, mainly for nostalgia. As a technologist, my experience with HIFLD and HSIP was a lesson in the primacy of anthropology over technology. Numerous times, technical solutions were presented and subsequently discarded as proponents could not overcome the concerns of those single-issue voters. As a result, I was very happy to see this significant step represented by HIFLD Open toward the fulfillment of the vision that was articulated for HIFLD years ago.

Working with Node and the GeoServer REST Configuration API

I’ve been working with a mix of technologies lately that includes Node and GeoServer. I’ve recently begun integrating the two by using Node to manipulate GeoServer’s configuration through the REST API it provides for that purpose. One task I’ve been working on automating is the registration of vector layers stored in PostGIS with GeoServer to make them available via WMS, WFS, and the various other services provided by GeoServer.

geoserver_admin

Continue reading “Working with Node and the GeoServer REST Configuration API”

Maryland Council on Open Data

Back in May, I had the honor of being appointed to the newly established Maryland Council on Open Data. The Council had its inaugural meeting in Baltimore yesterday and was heavily attended, including attendance by Governor Martin O’Malley. I’ll discuss his remarks to the group later.

As the first meeting of a new group, it went off largely as I expected. The agenda consisted primarily of an overview of the establishing legislation, a review of ethics requirements, demos of the existing open data portals, discussion of the history of open data in Maryland, and remarks from the Governor.

Continue reading “Maryland Council on Open Data”

Prying Data Open

In the aftermath of Hurricane Irene, I was trying to get information from my local electric cooperative about outages. There were many (including my neighborhood) and I wanted to see the scale of the problem. It turns out, they have a page with a map that shows current outages by zip code.

Old school outage map

It’s pretty old-school as far as web maps go but it gets the job done. Their day job is making electricity, not web maps, so I won’t critique it too much. One thing I did notice is that the map seems to be dynamically generated (as do the tables on the page) from some inaccessible data source. I search and tried to find some kind of feed, to no avail.

Continue reading “Prying Data Open”