geoMusings

FME, Salesforce, and BigQuery

2020-09-052020-09-04 by Bill Dollins

More often that not in my current role, opportunities to get my hands dirty come from the data side of our operation rather than the engineering side. These days, the data side involves corporate data rather than a lot of geospatial data. If I were to be guided by my own personal inertia, I’d drift toward traditional geospatial data 99% of the time, but working with other data stores and building pipelines involving them is good exposure.

Most recently, I’ve been working a lot with Salesforce and other data sources to support customer success operations. Customer success, as a discipline, is relatively new, having grown out of the SaaS market from the best practices involved in nurturing and growing customers post-sale as part of the SaaS land-and-expand model.

SaaS typically begets SaaS – meaning that you won’t often find SaaS companies using on-prem versions of business tools. This presents interesting challenges for data integration and analytics. Fifteen years ago, there’d most likely be a server room with installs of various vertical systems that had been configured to use whatever the organization’s blessed database platform was. In the event that an individual system didn’t support that database, there might be some nominal ETL performing a one-way sync so that the necessary charts and graphs could be made as needed.

Your Culture Will Adjust

2020-05-262020-05-26 by Bill Dollins

All hail the mighty water cooler – the oft-praised bastion of corporate culture, where the strategies of organizations great and small are made or destroyed in the hushed tones of whispered conversations among those who gather for daily hydration.

By now, everyone who can has been working remotely for several weeks. Companies are starting to ponder what to do as the world opens back up. A few have already committed to fully remote work going forward while many others of all sizes are trying to divine how they will move forward.

Recent Diversions

2020-05-222020-05-22 by Bill Dollins

Work in the new all-remote world has actually been quite busy, and I realize I am very fortunate to be able to say that. But we know what they say about all work and no play. For me, play often involves cracking open an IDE, especially since work for me isn’t centered on that anymore. This post is a loose roundup of extracurricular activities that have gotten some attention lately.

Rural Broadband – An Anecdotal Look

2020-05-192020-05-19 by Bill Dollins

I spent the better part of a decade and a half building geospatial applications in support of infrastructure analysis. Not infrastructure in the modern tech sense of containers and cloud providers and orchestration, but infrastructure in the classic sense of roads and rail and telecommunications. If we consider infrastructure through the lens of the ISO/OSI layered model, I spent a lot of time looking at the physical layer.

Personal Geography

2020-05-062020-05-06 by Bill Dollins

Human geography or anthropogeography is the branch of geography that deals with the study of people and their communities, cultures, economies, and interactions with the environment by studying their relations with and across space and place.
Wikipedia (https://en.wikipedia.org/wiki/Human_geography)

I was catching up with my friend and former boss, Tony Quartararo, a couple of days ago when our discussion got around to behavioral changes we have made in response to the COVID-19 pandemic and subsequent lock-downs and social distancing guidelines. Not surprisingly, given our backgrounds, geography figured in strongly.

The use case we discussed was grocery shopping. Both of us admitted to limiting our choices of venue to those that we knew from memory, so that we could gather groceries and proceed to the exit as expeditiously as possible. This has the effect of geographically constraining our choices to our immediate locality. Even in the case of large chain grocers, the floorplan of each store varies widely.

Tips For Your Best Zoom Experience

2020-05-012020-05-01 by Bill Dollins

Or Skype, or Google Meet, or GoToMeeting, or whatever.

As I bounce around social media, I keep running across a lot of spurious advice on how to project a “professional” impression as you, like everyone else, participates in video calls from home. This seems to be particularly true on LinkedIn.

Most of that advice is hogwash.

Nonessential

2020-04-282020-04-28 by Bill Dollins

The cursor flashes, waiting for a command.Any command.An editor window is open,Empty.The fan does not hum, with nothing to cool.The keyboard simply restsAs dust,Illuminated by the stripes of sunlightmoving across the desk,Settles between its idle keys. It all taunts meWith the truth: There is no map I can make,No chart,No graph,No query,No post nor comment; … Read more Nonessential

Attribute Transfer in PostGIS Using Spatial SQL

2020-04-062020-04-06 by Bill Dollins

Data conflation is a meat-and-potatoes task in most GIS workflows. There are numerous reasons one might need to get data from one data set into another. Perhaps you want to attach a new geometry type to existing attributes or a table. Or maybe you need to pull attributes from one or more data sets into a single, “master” version. I have seen this latter use case referred to as “attribute transfer.” In an interactive, desktop setting, this can be tedious, but it’s a task at which spatial SQL excels.

Here is a simple example that uses just one line of spatial SQL (or two lines if you need to add the column) to do the heavy lifting. First, some table setting. This example takes the number of confirmed COVID-19 cases from the Johns Hopkins university county-level data (a point data set) and transfers it to a polygon data set of the US counties. There’s one caveat at the end of this post.

Watching COVID-19 Data for Your County with PostgreSQL and Node

2020-04-022020-03-31 by Bill Dollins

I have addressed the topic of triggered notifications a couple of times on this blog previously. I’ve taken the opportunity to apply the technique to a current use case – the ability to get notifications whenever the confirmed count of COVID-19 cases changes in my county or surrounding ones.

I am basing this workflow on the Johns Hopkins University county-level data available as an ArcGIS feature service here: https://services1.arcgis.com/0MSEUqKaxRlEPj5g/arcgis/rest/services/ncov_cases_US/FeatureServer

Using the “f=geojson” parameter, it is possible to download the data in a format (GeoJSON) that is readily consumable by OGR. As a result, I was able to initiate a core workflow using the following steps.

Fulcrum Query API and Koop

2020-03-202020-03-20 by Bill Dollins

I don’t write code as much as I used to, but I have to return to it every so often to keep my sanity. With the current world situation, there are a lot of dashboards going up, many of which are based on the Esri Operations Dashboard or ArcGIS Hub.

I got thinking about a previous crisis in which Fulcrum and Esri’s Koop were used to great effect and started wondering if I could make the interaction between the two easier. Koop, at its core, takes GeoJSON and transforms it to be consumed by clients that can handle ArcGIS feature services.

Fulcrum has two primary ways to expose data as GeoJSON feeds – data shares and the Fulcrum Query API. Koop has a provider that handles arbitrary GeoJSON feeds quite well. It can be found here. In order to use that provider with Fulcrum, the only option is to use Fulcrum data shares. That works really well, but data shares have a couple of drawbacks. First, they are always public. Second, there is a limit to the number of shares that can be enabled from an account. Third, the data is shared all or nothing, so columns cannot be filtered.