By: Sean Gillies on March 16 2023
Can Planet data or the Planet platform be used with Python? It can, and not by accident. Planet is working to make it so. This post attempts to explain what the Python Geospatial Stack is and Planet’s role in keeping the stack in good shape so that developers continue to get value from it.
Planet’s customers don’t only use Planet products via commercial or open source desktop GIS or partner platforms. Many of you are integrating Planet’s products and services into custom-built GIS systems, which use the open source Python Geospatial Stack. This post attempts to explain what the Python Geospatial Stack is and Planet’s role in keeping the stack in good shape so that developers like you continue to get value from it. Can Planet data or the Planet platform be used with Python? It can, and not by accident. Planet is working to make it so. You can help, too.
What is the Python Geospatial Stack?¶
The Python Geospatial Stack is a set of Python packages that use a smaller set of non-standard system libraries written in C/C++: GEOS, PROJ, and GDAL. GEOS is a library of 2-D computational geometry routines. PROJ is a library for cartographic projections and geodetic transformations. GDAL is a model for computing with raster and vector data and a collection of format drivers to allow data access and translation on your computer or over your networks. Planet engineers, including myself, have contributed significantly to these system libraries. Principal Engineer Frank Warmerdam is the author of GDAL, a longtime maintainer of PROJ version 4, and a major contributor to GEOS.
Shapely, Pyproj, Fiona, Rasterio, GeoPandas, PDAL, and Xarray are the core of the stack. Shapely draws upon GEOS. Shapely is useful to developers for filtering asset catalog search results, comparing catalog item footprints to areas of interest, and much more. Pyproj wraps PROJ. Pyproj is helpful for calculating the projected area of regions that are described in longitude and latitude coordinates, and more. Fiona and Rasterio are based on GDAL and allow developers to read and write vector and raster data. PDAL translates and manipulates point cloud data, GeoPandas and Xarray use Fiona and Rasterio and provide higher levels of abstraction for analysis of column-oriented tabular and gridded scalar data, respectively.
How did the stack come about?¶
Why are so many organizations using Python for GIS work? Isn’t it slow? Only an instructional language? Timing explains a lot. When Bruce Dodson started looking at alternatives to Avenue, the scripting language of Esri's ArcView GIS version 3, Python was ready. Python had a good extension story in 2000, meaning that although Python was relatively slow, it was fairly easy to extend with fast code written in C.
When you compute and dissolve the convex hulls of multiple shapes using Shapely, all of the intensive calculation is done by native code, not Python bytecode. The open source native code (GDAL, GEOS, PROJ, etc) just happened to rise up at the same point in time. Timing really is everything.
To close the loop, the Geospatial Stack has helped make the Python language sticky. Developers chose Python from many options for the availability of solid, feature-rich, decently-documented libraries in a particular domain, and end up staying for all the other nice things about the Python language and community. The reason why Python is the second best language for many domains is that the same discovery happened in Numerical Analysis, Machine Learning, Image Processing, and elsewhere.
How is Planet helping?¶
Software and software communities need care and feeding. Planet is involved through financial sponsorship, project governance, code maintenance, and builds of binary distributions.
In 2021, Planet became an inaugural platinum sponsor of GDAL. Planet is helping to pay for infrastructure costs and for the labor of full-time maintainers for GDAL and affiliated projects like GEOS and PROJ. In addition, two Planet engineers (Frank Warmerdam and I) serve on GDAL’s project steering committee. The impact of this sponsorship is huge. GDAL has a better and faster build system, which makes it easier to contribute to and which reduces the cost of every bug fix and new feature. Most importantly, the sponsorship makes it possible for GDAL’s longtime maintainers to avoid burnout, stay involved, and mentor their eventual successors.
Planet’s Developer Relations team is mainly involved at the Python level. I’m the release manager for Fiona and Rasterio. I make sure that binary distributions (aka “wheels”) for Linux, MacOS, and Windows are uploaded to Python’s Package Index so that when developers run “pip install rasterio” almost everybody gets pre-compiled Python packages with GDAL, GEOS, and PROJ batteries included. Packages that “just work” for many cases on laptops and in hosted notebooks.
A major new version of Shapely was released at the end of 2022 through collaboration with core GeoPandas developers. Shapely 2.0 adds vectorized operations that radically speed up GeoPandas and keep this piece of the stack relevant as projects like GeoParquet start to change the nature of vector data.
Fiona 1.9.0 was released on Jan 20, 2023 and also provides a boost to GeoPandas. Planet’s Developer Relations team is currently building new tools that integrate with this new version of the Geospatial Stack’s vector data package. They will be announced soon.
Multiple teams at Planet helped make Rasterio 1.3.0, Shapely 2.0.0, and Fiona 1.9.0 successful releases by testing beta releases of these packages. Planet is financially sustaining the open source projects that form the foundation of the stack. The DevRel team is engaged with the open source communities that write the stack’s code. At Planet we consider the health of the Python Geospatial Stack to be a big factor in the overall experience of our platform.
Next Steps¶
Are you using Fiona, Rasterio, or Shapely? Upgrade to the latest versions and try them out. Chat with us about any ideas or issues with using these packages with Planet data at https://community.planet.com/developers-55. Follow us on twitter @planetdevs.
