I recently had a project requirement to export the contents of a delta table in Databricks to several formats, including shapefile and file geodatabase, with the output being placed in object storage mounted to DBFS. I set up the logic in a notebook, with the intent to use geopandas, which provides an easy wrapper around … Continue reading Exporting Data with GeoPandas in Databricks
Tag: Python
Experimenting with RAG Using ChatGPT and DuckDuckGo
I had a project over the summer that required me to do a deep dive into various concepts around AI. Approximately 200 academic papers later, I came away with a laundry list of things I wanted to try. I am primarily interested in the capabilities of LLMs, but I am not particularly interested in the … Continue reading Experimenting with RAG Using ChatGPT and DuckDuckGo
Image Similarity with PgVector
At FedGeoDay in April, I attended a workshop conducted by my friend Steve Pousty on the use of vector databases. Steve is a great teacher and I learned a lot that day. I’ve been on a bit of a journey since. I’ve gotten some project work with AI and I find vector databases to be … Continue reading Image Similarity with PgVector
Just Enough Geospatial
As I mentioned earlier this year, I spend a lot of time working in Python these days. More and more, my work seems to be about performing geospatial tasks with the minimal amount of geospatial dependencies possible. My most recent work has been generating contours from a set of sampled elevation points. I’ve done this … Continue reading Just Enough Geospatial
Export WordPress Posts to PDF
I've been working a project recently to investigate training an LLM (LocalGPT, in this case) to help analyze a document library. (More on that in the future.) For ingest, it handles PDF files really well. I needed some well-known (by me) content to kick the tires for initial prototyping so I decided to dump all … Continue reading Export WordPress Posts to PDF
Data Over Software
One of the first tasks I ever had in my then-new GIS career was doing AML development in ARC/INFO 6.x for a data production project. My code parsed DXF exported from AutoCAD R11 for DOS and then assigned attributes based on things like layer, color, line weight, feature type, and others. It also georeferenced the … Continue reading Data Over Software