I spent the past couple of days at the Esri Federal GIS Conference, still referred to by many as the “FedUC,” in Washington, DC. The primary reason I went was to attend some customer meetings. The FedUC draws many people (6,000 in this year’s case) from around the country and it’s a convenient place to do in-person meetings in today’s mostly-remote world.
I didn’t attend many sessions, but one that caught my eye was an update on the status of the Geospatial Management Office (GMO) at the US Department of Homeland Security (DHS). While I never worked with the GMO, I worked with adjacent organizations during the years that DHS was being created and the current state of the GMO interested me.
An example data set from HIFLD Open
One of the newer duties in the portfolio of the GMO is the management of the Homeland Infrastructure Foundation-Level Data (HIFLD) data sets. This is a collection of open and proprietary geospatial data that is useful for homeland security activities, mostly centered on critical infrastructure protection. The GMO manages the distribution of this data, with the open data sets available here and the proprietary data managed through a secure system to ensure compliance with license agreements. HIFLD is a long-standing and successful program that meets the needs of its user community and that, rather than the data itself, is the focus of this post.
This was of great interest to me because I was in the room at the earliest inception of what became HIFLD. HIFLD was often referred to as a “coalition of the willing” – a phrase I’ll come back to. I don’t want to oversell my contribution to the early days of HIFLD. I was a contractor providing geospatial support to one of the earliest of those willing and, as such, had a front-row seat as it got off the ground.
Not long after 9/11, organizations were created to address homeland security needs – which mostly still needed to be defined. These organizations pre-dated those we know well now – DHS, NORTHCOM and others. I was a contractor supporting a group that had been working on critical infrastructure protection for several years and we were pulled in early.
What became HIFLD was conceived to solve the “M: drive” problem. Everyone had an M: drive on their file server. The “M” stood for “maps” and the drive contained a grab-bag of shapefiles and MXDs. Early exercises made it clear that everyone had different shapefiles and their MXDs all looked different. Additionally, homeland security required collaboration with local partners and existing systems didn’t make two-way sharing easy – they actually made it impossible.
So what became apparent by early 2002 was that everyone needed to be looking at the same thing at the same time and that data and information needed to move freely, within reasonable security controls. In short, there needed to be a homeland security common operational picture (COP). As an aside, the DHS COP got an award at the FedUC this week.
The coalition of the willing was formed to solve this problem and it was soon named HIFLD. For years, it was a working group that met periodically, each time with more participation as the coalition expanded. Its early goal was to achieve consensus on what the data in the COP needed to be and then go get that data and make it available. The early champions of the problem HIFLD was attempting to solve kept it going out often out of sheer will. Most participants had no real funding for this effort early on, but were willing to put their resources toward it because the problem was important.
The interesting part was how those who were willing kept expanding, almost like concentric rings in a tree. When it became apparent that commercial data sets would be needed, NGA took the lead on figuring out how to get appropriate licenses for use among the participants. The GMO took over that role in 2022. IT restrictions at the time made it nearly impossible for government agencies to host servers to distribute the data, so it was sent as batches of DVDs for a long time. No one ever thought that was ideal, but the problem was important so they were willing to participate in that process. The HIFLD portal ran for a long time on a single-processor Dell “server” sitting next to my desk in the office of the company I was working at during that period.
A screenshot of the original HIFLD portal from 2006 (courtesy of the Internet Archive Wayback Machine)
As the HIFLD grew and the coalition gained more momentum, it became clear that this self-organized public/private community needed to be formalized and it was eventually chartered under FGDC. Also during that period, technology and policy caught up to enable sharing. The open data sets are now freely available on a site powered by ArcGIS Hub. The proprietary data is distributed on the DHS Geospatial Information Infrastructure (GII), which handles necessary access controls and security to enable compliance with licensing and security requirements. HIFLD is no longer a working group. It is a catalog of data sets with an invested community to keep it going. It is, from what I can see, everything it was envisioned to be in 2002.
As a contractor, your work often takes you to other things. That’s exactly what happened to me and HIFLD – my work evolved and eventually I moved on. It’s been over a decade since I attended a HIFLD meeting, but I have kept watching it from afar. This week, as I stood in the room and watched all the hands go up when asked who knew what HIFLD was, I did not know or recognize a single person – not even the DHS staff leading the session. That’s as it should be. If something is truly important, then its early champions should be able to move on as it continues to move forward and evolve. HIFLD has crossed that chasm.
That is the real lesson I took away from the session this week. There is a societal narrative that government can no longer do hard things and that public and private are inherently at odds with each other. HIFLD stands as a counterpoint to that narrative. Getting it going was hard, growing it and keeping it alive was hard, maintaining it going forward will be hard. It certainly isn’t as hard as going to the moon or building the interstate highway system, but we probably need to stop giving so much weight to such superlatives. There are a lot of hard problems to solve and HIFLD shows a way to do it. With tenacity, patience, and, yes, willingness, it is possible for public and private entities to collaborate, find common ground, and solve hard problems.
Header image: Jim Evans, CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0, via Wikimedia Commons
