FME, Salesforce, and BigQuery

More often that not in my current role, opportunities to get my hands dirty come from the data side of our operation rather than the engineering side. These days, the data side involves corporate data rather than a lot of geospatial data. If I were to be guided by my own personal inertia, I’d drift toward traditional geospatial data 99% of the time, but working with other data stores and building pipelines involving them is good exposure.

Most recently, I’ve been working a lot with Salesforce and other data sources to support customer success operations. Customer success, as a discipline, is relatively new, having grown out of the SaaS market from the best practices involved in nurturing and growing customers post-sale as part of the SaaS land-and-expand model.

SaaS typically begets SaaS – meaning that you won’t often find SaaS companies using on-prem versions of business tools. This presents interesting challenges for data integration and analytics. Fifteen years ago, there’d most likely be a server room with installs of various vertical systems that had been configured to use whatever the organization’s blessed database platform was. In the event that an individual system didn’t support that database, there might be some nominal ETL performing a one-way sync so that the necessary charts and graphs could be made as needed.

Read more

Watching COVID-19 Data for Your County with PostgreSQL and Node

I have addressed the topic of triggered notifications a couple of times on this blog previously. I’ve taken the opportunity to apply the technique to a current use case – the ability to get notifications whenever the confirmed count of COVID-19 cases changes in my county or surrounding ones.

I am basing this workflow on the Johns Hopkins University county-level data available as an ArcGIS feature service here: https://services1.arcgis.com/0MSEUqKaxRlEPj5g/arcgis/rest/services/ncov_cases_US/FeatureServer

Using the “f=geojson” parameter, it is possible to download the data in a format (GeoJSON) that is readily consumable by OGR. As a result, I was able to initiate a core workflow using the following steps.

Read more

Fulcrum Query API and Koop

I don’t write code as much as I used to, but I have to return to it every so often to keep my sanity. With the current world situation, there are a lot of dashboards going up, many of which are based on the Esri Operations Dashboard or ArcGIS Hub.

I got thinking about a previous crisis in which Fulcrum and Esri’s Koop were used to great effect and started wondering if I could make the interaction between the two easier. Koop, at its core, takes GeoJSON and transforms it to be consumed by clients that can handle ArcGIS feature services.

Fulcrum has two primary ways to expose data as GeoJSON feeds – data shares and the Fulcrum Query API. Koop has a provider that handles arbitrary GeoJSON feeds quite well. It can be found here. In order to use that provider with Fulcrum, the only option is to use Fulcrum data shares. That works really well, but data shares have a couple of drawbacks. First, they are always public. Second, there is a limit to the number of shares that can be enabled from an account. Third, the data is shared all or nothing, so columns cannot be filtered.

Read more

Y2K and Today

Even though I work from home full time, I have a little extra time available since most things have been closed by a gubernatorial executive order. During downtime, I’m doing a little tidying, and it’s amazing the things you find and realize you’ve held onto.

At this time twenty years ago, I was probably be in some conference room receiving this plaque along with a couple dozen other people as our efforts related to Y2K had concluded. Our effort was small – to monitor potential degradation to physical infrastructure related to any unfixed Y2K bugs. The real work had been occurring for a few years prior, to get the world ready for the rollover. I couldn’t help but think about the parallels to the current COVID-19 situation.

Read more

Lessons on Working from Home

I have worked from home full-time for the past three years. I know a few people who have done so for a lot longer (decades), but, regardless of how long anyone I know has worked from home, we seem to have many of the same observations.

Thanks to the coronavirus, we are becoming a lot more familiar with the concept of “social distancing” and work from home may become more prevalent for a while. When I worked in a cubicle, I romanticized the idea of work from home. Now that I’ve done it for a while, I realize it is a unique work experience with its own benefits and drawbacks. Rather than enumerate those in detail, I’ll address some practices that I have adopted and evolved during my time at home.

Read more

Milestones, Goals, and the Power of No

It’s been a few weeks since I hit my latest milestone, but life was fairly full in the immediate aftermath. On the plus side, the extra time was good for reflection.

The milestone to which I am referring is that, on October 27, 2019, I ran the Marine Corps Marathon in Washington, DC. The race itself and the training leading up to it were the hardest physical things I have ever done. The weather during the actual race was crazy, with torrential rains for my first 14 miles or so. My finish time was a lot slower than I’d hoped, but I don’t care because I finished. I learned a lot and am already looking to do another.

Looking back on the process of training for the race, it was very much a clarifying experience. As the daily and weekly mileage ramped up, my time management skills were put to the test. It was no longer possible for me to squeeze in the mid-week training runs during a lunch hour, so it required more communication with my co-workers. To a one, they were supportive.

On the other end of the day, I needed to make sure things didn’t extend in a way that impinged on time with my family. Our kids are older and fully self-sufficient, so a lot of the logistics that were needed when they were younger were no longer a factor. At this stage, I just wanted to make sure I was getting meaningful time with them and my wife.

Read more

Reclaiming Twitter

Approximately six months ago, I was at a crossroad with Twitter. Unfiltered, it has become too toxic and negative to continue to allow into my life. My dilemma is that, after 11 years on the platform, there are a host of people I’ve never met “IRL” whom I consider friends or with whom I want to maintain a connection. Twitter has always been that connection.

What originally drew me to Twitter was an ease of interacting with a community of technologists and geographers, who shared tools, techniques, and knowledge. Over the years, as Twitter has “grown up” and captured the wider public imagination, in addition to changes in the behavior of the platform itself, the content of my feed has skewed more to the political and the negative – especially after the 2016 election.

This trend seems to apply regardless of ideology. People with opinions that span the political spectrum seem to take to Twitter to leave bite-sized and brash statements that bolster whatever position or candidate they support. In some ways, the structure of Twitter encourages this, even with the advent of longer tweets and threads as first-class citizens.

In the three years since the last election, I’ve become incredibly familiar (more than I ever wanted to be) with the political and social positions of a lot of people I follow, and not a single position has changed or evolved in that time. It has become a digital Forum Romanum, with countless self-styled tribunes shouting speeches into the passing crowd.

I didn’t need to continue to allow this in my life.

Read more

Ron Lake – The Man In the Arena

It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.

Theodore Roosevelt

I never met Ron Lake, but the majority of my career has taken place in the geospatial technology landscape he helped create. He is best known as the author and chief advocate of Geography Markup Language (GML), the XML encoding of geographic objects that underpins most of the Open Geospatial Consortium’s web standards and is a standard itself. This fact made him a pioneer, a visionary, and a source of controversy.

It is easy, for those of us who have worked with geospatial tools for a long time, to sit at a keyboard in 2019 and forget how much harder it used to be. By that, I mean working with geospatial data. It’s still probably more difficult than it should be, but it used to be so much harder.

Read more

Hello, Discipline, My Old Friend

It wasn’t that long ago that I was “always on” in terms of work. There is a certain ethos in the DC area that rewards that outlook and I was fairly good at it. This was compounded by the fact that I was a partner in my company and the buck always stops with the owners. Therefore, you always answered the phone/email/text/DM. A few years ago, I began to realize that this approach wasn’t serving me well over the long haul, especially in terms of my health.

What I needed to do was get reacquainted with discipline. Because I had fully bought into the idea that a business owner is never “off,” I had allowed discipline to atrophy. When all 24 hours of each day are available to accomplish tasks, then “close of business” means “before 11:59pm.” If you make your deliverable before the person you have committed to shows up for work in the morning, it counts.

There are obvious problems with this approach. First, work is always lingering in the background. Second, you are never fully engaged in any activity (including work). I coached soccer games, ran practices, and many other similar activities, but the block of code that I couldn’t quite finish before heading out to practice, or the proposal inputs due by midnight were never far away. Conversely the practice, or board meeting, or dinner party was always lingering in my mind when I should have been focusing on a proposal or my code.

Read more