GIS – geoMusings

Attribute Transfer in PostGIS Using Spatial SQL

2020-04-062020-04-06 by Bill Dollins

Data conflation is a meat-and-potatoes task in most GIS workflows. There are numerous reasons one might need to get data from one data set into another. Perhaps you want to attach a new geometry type to existing attributes or a table. Or maybe you need to pull attributes from one or more data sets into a single, “master” version. I have seen this latter use case referred to as “attribute transfer.” In an interactive, desktop setting, this can be tedious, but it’s a task at which spatial SQL excels.

Here is a simple example that uses just one line of spatial SQL (or two lines if you need to add the column) to do the heavy lifting. First, some table setting. This example takes the number of confirmed COVID-19 cases from the Johns Hopkins university county-level data (a point data set) and transfers it to a polygon data set of the US counties. There’s one caveat at the end of this post.

Watching COVID-19 Data for Your County with PostgreSQL and Node

2020-04-022020-03-31 by Bill Dollins

I have addressed the topic of triggered notifications a couple of times on this blog previously. I’ve taken the opportunity to apply the technique to a current use case – the ability to get notifications whenever the confirmed count of COVID-19 cases changes in my county or surrounding ones.

I am basing this workflow on the Johns Hopkins University county-level data available as an ArcGIS feature service here: https://services1.arcgis.com/0MSEUqKaxRlEPj5g/arcgis/rest/services/ncov_cases_US/FeatureServer

Using the “f=geojson” parameter, it is possible to download the data in a format (GeoJSON) that is readily consumable by OGR. As a result, I was able to initiate a core workflow using the following steps.

Fulcrum Query API and Koop

2020-03-202020-03-20 by Bill Dollins

I don’t write code as much as I used to, but I have to return to it every so often to keep my sanity. With the current world situation, there are a lot of dashboards going up, many of which are based on the Esri Operations Dashboard or ArcGIS Hub.

I got thinking about a previous crisis in which Fulcrum and Esri’s Koop were used to great effect and started wondering if I could make the interaction between the two easier. Koop, at its core, takes GeoJSON and transforms it to be consumed by clients that can handle ArcGIS feature services.

Fulcrum has two primary ways to expose data as GeoJSON feeds – data shares and the Fulcrum Query API. Koop has a provider that handles arbitrary GeoJSON feeds quite well. It can be found here. In order to use that provider with Fulcrum, the only option is to use Fulcrum data shares. That works really well, but data shares have a couple of drawbacks. First, they are always public. Second, there is a limit to the number of shares that can be enabled from an account. Third, the data is shared all or nothing, so columns cannot be filtered.

Milestones, Goals, and the Power of No

2019-11-252019-11-21 by Bill Dollins

It’s been a few weeks since I hit my latest milestone, but life was fairly full in the immediate aftermath. On the plus side, the extra time was good for reflection.

The milestone to which I am referring is that, on October 27, 2019, I ran the Marine Corps Marathon in Washington, DC. The race itself and the training leading up to it were the hardest physical things I have ever done. The weather during the actual race was crazy, with torrential rains for my first 14 miles or so. My finish time was a lot slower than I’d hoped, but I don’t care because I finished. I learned a lot and am already looking to do another.

Looking back on the process of training for the race, it was very much a clarifying experience. As the daily and weekly mileage ramped up, my time management skills were put to the test. It was no longer possible for me to squeeze in the mid-week training runs during a lunch hour, so it required more communication with my co-workers. To a one, they were supportive.

On the other end of the day, I needed to make sure things didn’t extend in a way that impinged on time with my family. Our kids are older and fully self-sufficient, so a lot of the logistics that were needed when they were younger were no longer a factor. At this stage, I just wanted to make sure I was getting meaningful time with them and my wife.

Ron Lake – The Man In the Arena

2019-09-112019-09-06 by Bill Dollins

It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.
Theodore Roosevelt

I never met Ron Lake, but the majority of my career has taken place in the geospatial technology landscape he helped create. He is best known as the author and chief advocate of Geography Markup Language (GML), the XML encoding of geographic objects that underpins most of the Open Geospatial Consortium’s web standards and is a standard itself. This fact made him a pioneer, a visionary, and a source of controversy.

It is easy, for those of us who have worked with geospatial tools for a long time, to sit at a keyboard in 2019 and forget how much harder it used to be. By that, I mean working with geospatial data. It’s still probably more difficult than it should be, but it used to be so much harder.

Hello, Discipline, My Old Friend

2019-08-202019-08-16 by Bill Dollins

It wasn’t that long ago that I was “always on” in terms of work. There is a certain ethos in the DC area that rewards that outlook and I was fairly good at it. This was compounded by the fact that I was a partner in my company and the buck always stops with the owners. Therefore, you always answered the phone/email/text/DM. A few years ago, I began to realize that this approach wasn’t serving me well over the long haul, especially in terms of my health.

What I needed to do was get reacquainted with discipline. Because I had fully bought into the idea that a business owner is never “off,” I had allowed discipline to atrophy. When all 24 hours of each day are available to accomplish tasks, then “close of business” means “before 11:59pm.” If you make your deliverable before the person you have committed to shows up for work in the morning, it counts.

There are obvious problems with this approach. First, work is always lingering in the background. Second, you are never fully engaged in any activity (including work). I coached soccer games, ran practices, and many other similar activities, but the block of code that I couldn’t quite finish before heading out to practice, or the proposal inputs due by midnight were never far away. Conversely the practice, or board meeting, or dinner party was always lingering in my mind when I should have been focusing on a proposal or my code.

Evolution and Leadership

2019-08-152019-06-12 by Bill Dollins

My first management position was at a restaurant. It was a chain steakhouse in a one-horse town in Maryland. A few months into that role, I was the sole manager on duty for a weekend night shift that was going poorly.

Feeling my frustration mounting, I did the most mature and leaderly thing my 19-year-old mind could think of – I tipped over the break table, causing everything on it to crash loudly to the floor.

I felt a momentary rush of power as people scurried away from my obvious anger, setting off to do whatever they could do to right the sinking ship that was this shift. Word apparently spread quickly throughout the staff because, before I had walked another ten feet through the kitchen, I was confronted by a member of the waitstaff.

I forget her exact words, but they boiled down to “You’re being an idiot. This is your shift. If you want it fixed, go fix it or find another job.” She was right, and jarringly so. I’m pretty sure I didn’t fix that shift, but I never flipped a break table again in my restaurant career. Thus began a lifelong interest in leadership and management.

I should also note that, several years later, the woman who confronted me agreed to marry me and we will celebrate 25 years of marriage next year.

Catching Up with Microsoft

2019-05-07 by Bill Dollins

Recently, I’ve gotten back in touch with .Net in the form of .Net Core. I’ve been shaking off some the coding rust and building some tools to help with data handling related to the Foresight data service at Spatial Networks. It’s been fun to get my hands dirty again and also interesting to see how .Net has evolved over the past few years.

It’s been a few years since I’ve done a lot with .Net and, after spending some time in the Node ecosystem, this was my first foray into .Net Core. The application I was working on just wasn’t coming together correctly in Node, so I started prototyping out the logic flow in .Net Core, with the intent to port it back to Node when I had a good reference implementation. The more I kept using .Net, the more impressed I got, so I just kept the application there.

The One About PostGIS

2019-03-29 by Bill Dollins

Desktop GIS – The Evergreen Topic

2019-02-08 by Bill Dollins

It seems that I tend to revisit the state of desktop GIS every so often. With the continued advancement of “web GIS,” as well as the increased power of mobile platforms, proliferation of spatial analysis techniques into non-traditional environments, the ubiquity of spatial databases, and a host of other factors, it’s tempting to speculate on the long-term prospects of traditional desktop GIS software. This seems especially true when the software in question originates in Redlands, California.

I was brought back to this topic by a recent discussion on Twitter, initiated by my friend, Atanas Entchev.

https://twitter.com/atanas/status/1093489694082510853

The ensuing discussion grew legs and continued much longer than I would have thought. The core of the discussion centered around confusion in Esri’s messaging or, more accurately, subsequent interpretation of Esri’s messaging with regard to the status of ArcGIS Desktop. Long story short: much ado about nothing. Esri is releasing new versions of ArcGIS Pro and ArcMap. There are primary sources reaffirming their commitment to desktop GIS, so we can all go back to what we were doing. Awesome.