OGC Made Easy: Tiling Spec
By: Tim Schaub on May 12 2023
Looking for help getting started with the Open Geospatial Consortium (OGC) API Tiles specification? This primer can get you going.
OGC Made Easy: Tiles¶
The Open Geospatial Consortium (or OGC) publishes standards related to geospatial information services and formats. The organization recently published “OGC API - Tiles - Part 1: Core” – a specification aimed at web map tile services. The standard weighs in at 137 pages and depends on another 273 page standard, so if you are new to OGC specifications, getting started can be intimidating. The goal of this post is to provide a quickstart for tiled map providers who might want to implement the standard but are struggling to know where to begin.
What’s in a name?¶
The “OGC API - Tiles - Part 1: Core” specification name is quite a mouthful. The rest of this post will use "OGC Tiles" as a shorthand, but it is worth picking apart the name first.
The previous generation of OGC standards had titles like "Web Map Service" (WMS), "Web Feature Service" (WFS), and "Web Coverage Service" (WCS). These were affectionately referred to as the WxS standards. The "OGC API" prefix in this new specification’s name refers to the new generation of standards designed to work in harmony with existing web standards and best practices – so things like content negotiation and HTTP status codes aren’t redefined by the standards themselves.
The "Tiles" part of the name is what this standard is really about. Before the Google Maps era, maps on web pages were often provided by services that would generate map images of arbitrary places on demand. It is generally faster and more efficient to render map images ahead of time and to cut these images up into regular grids of tiles. The OGC Tiles specification describes how mapping clients (like web pages) can get information about tilesets and how to request the tiled data itself.
The "Part 1: Core" suffix in the standard’s name implies that additional parts will be published in the future describing functionality beyond the core. The core concerns itself with how to provide metadata about a tileset, how to describe the tiling scheme, and how to relate a tileset to a geospatial data collection that might have other representations. The core also discusses how a tileset with a temporal dimension can be handled.
Prior art¶
For the last 18 years since the launch of Google Maps, people who develop mapping applications on the web have been consolidating on a common scheme for serving tiled map data. The tiles in this scheme fit into a conceptual pyramid where each tile can be identified by three parameters: zoom level (Z), column (X), and row (Y). This has become known as the XYZ tiling scheme.
In an XYZ tileset, the 0/0/0 tile at the top usually covers the whole world, and each level below contains four tiles for every one tile above. Most XYZ tilesets use the same Web Mercator projection as Google Maps. XYZ tiles are usually available with a URL that looks like this: https://example.com/tiles/{z}/{x}/{y}.png
. In this URL template, {z}
is a placeholder for the zoom level, {x}
for the column number, and {y}
for the row number. The available zoom, column, and row values generally start with zero. The zoom level may extend to something in the 20s for "street level" data.
Of course all of these things (extent, projection, URL template, maximum level) may vary from tileset to tileset. The Mapbox TileJSON specification formalizes how some of this variety can be described in a simple JSON document. One notable constraint of TileJSON is that tilesets must use the Web Mercator projection popularized by Google Maps.
Conformance¶
The OGC Tiles specification builds on what people have been doing for years with XYZ tilesets and goes beyond what is possible to describe with TileJSON. The standard is organized into conformance classes that build on one another to handle increasingly complex use cases.
The good news is that if you have an existing XYZ tileset, you are (most likely) already OGC Tiles compliant! The Core Conformance Class boils down to three unconditional requirements:
- Tiles must be available using a URL built from a template like
https://example.com/tiles/{z}/{x}/{y}.png
(see Requirement 1 for more detail). - Successful responses must have a
200
status code (see Requirement 5 for more detail). - Responses for tiles that do not exist must have a
400
or404
status code (see Requirement 6 for more detail).
The Core Conformance Class in OGC Tiles has additional requirements that only kick in under certain conditions. For example, if you also conform to the Core Conformance Class from the separate “OGC API - Common - Part 1: Core” standard, then requirements 2, 3, and 4 in the Tiles standard say that you have to use {tileMatrix}
instead of {z}
, {tileRow}
instead of {y}
, and {tileCol}
instead of {x}
in your tile URL template (but the core doesn’t have any requirements about how this tile URL template is advertised).
If you have an XYZ tileset and you pass the list of requirements above, then you can say that you implement the Core Conformance Class from the "OGC API - Tiles - Part 1: Core" specification. The OGC Common standard describes a formal way to advertise which conformance classes are implemented. For example, if you conform with the Core Tile requirements and the Core Common requirements, you would advertise this with a document like this:
{
"conformsTo": [
"http://www.opengis.net/spec/ogcapi-common-1/1.0/conf/core",
"http://www.opengis.net/spec/ogcapi-tiles-1/1.0/conf/core"
]
}
The Core Conformance Class from the "OGC API - Common - Part 1: Core" standard requires that a conformance document like the one above is available at a /conformance
path relative to the root of your service. So a service at https://example.com/
would make the conformance document available at https://example.com/conformance
.
Beyond the core¶
The Core Conformance Class in OGC Tiles adds a bit of formality to what people have already been doing with XYZ tilesets. If you want to provide additional metadata about your tilesets, you can make use of the conformance classes beyond Core. The next set of requirements to consider are part of the Tileset Conformance Class. You might choose to implement these requirements if your tileset doesn’t cover the whole world, if you want to let clients know about a limited set of available zoom levels, or if you use an alternative projection, for example.
The Tileset Conformance Class depends on a separate standard, “OGC Two Dimensional Tile Matrix Set and Tile Set Metadata” (referred to as TMS below), to define the structure of the tileset metadata. Examples are included below, but if all you want is the JSON Schema for tileset metadata, you can find that here: https://schemas.opengis.net/tms/2.0/json/tileSet.json.
Here is an example tileset metadata document that describes the XYZ OpenStreetMap tiles:
{
"title": "OpenStreetMap",
"dataType": "map",
"crs": "http://www.opengis.net/def/crs/EPSG/0/3857",
"tileMatrixSetURI": "http://www.opengis.net/def/tilematrixset/OGC/1.0/WebMercatorQuad",
"links": [
{
"title": "Tile URL Template",
"href": "https://tile.openstreetmap.org/{tileMatrix}/{tileCol}/{tileRow}.png",
"rel": "item",
"type": "image/png",
"templated": true
},
{
"title": "Tiling Scheme",
"href": "https://ogc-tiles.xyz/tileMatrixSets/WebMercatorQuad",
"rel": "http://www.opengis.net/def/rel/ogc/1.0/tiling-scheme",
"type": "application/json"
}
]
}
The optional title
provides users with an easy way to identify the tileset. You can also provide an optional description
with more information. The dataType
is a required property. Values can be map
(for rendered raster data), vector
(for vector tile data), or coverage
(for raw raster data). The crs
is an identifier (specifically a URI) that indicates which coordinate reference system is used. The tileMatrixSetURI
is an identifier for the tiling scheme used. The coordinate reference system and tiling scheme shown corresponds to the setup popularized by Google Maps (zoom level zero has one 256 x 256 pixel tile covering the world in Web Mercator; every level below has four tiles for each tile above). The links
property includes a link to the tiling scheme definition (also referred to as tile matrix set). As an alternative to linking to a separate document with the tiling scheme definition, the definition can be included in the tileset metadata using a tileMatrixSet
property (this is Requirement 17 of the TMS standard). Finally, and most importantly, the links
list includes the tile URL template for requesting individual tiles.
One of the most useful reasons to implement the requirements of the Tileset Conformance Class is to let clients know that a tileset is not available for the whole world. The boundingBox
property of the tileset metadata serves this purpose, and including it enables a client to automatically zoom to exactly where there are tiles available. Here is an example bounding box member that indicates that tiles are only available north of the equator and west of the prime meridian (see the tileset-bbox.json for a complete example that includes the whole world as a bounding box):
"boundingBox": {
"crs": "http://www.opengis.net/def/crs/OGC/1.3/CRS84",
"lowerLeft": [-180, 0],
"upperRight": [0, 90]
}
The tileset metadata can also include a tileMatrixSetLimits
property. It is recommended that tilesets with limited extent (compared to the extent of the tiling scheme) use this property in addition to the boundingBox
to describe those limits.
The tileMatrixSetLimits
property is also useful to describe a limited set of zoom levels. For example, if only zoom levels 2
and 3
were available for a particular tileset, the limits could be described with this metadata (see the tileset-limits.json for a complete example):
"tileMatrixSetLimits": [
{
"tileMatrix": "2",
"minTileRow": 0,
"maxTileRow": 1,
"minTileCol": 0,
"maxTileCol": 1
},
{
"tileMatrix": "3",
"minTileRow": 0,
"maxTileRow": 3,
"minTileCol": 0,
"maxTileCol": 3
}
]
The above tileMatrixSetLimits
only limit the available zoom levels or tile matrix levels. To limit the available extent as well, adjust the minimum and maximum column and row values. The JSON schema for this property is available here: https://schemas.opengis.net/tms/2.0/json/tileMatrixLimits.json.
Next steps¶
The OGC Tiles standard describes additional sets of requirements beyond the Core and Tileset Conformance Classes. For example, the Tileset List Conformance Class includes requirements for describing a list of tilesets with a subset of the metadata required for an individual tileset (see Requirement 9 and 10). In addition, the OGC Common standard includes requirements that can be useful if your tilesets are related to other geospatial data collections. The OGC API family of standards leverages OpenAPI and JSON Schema for describing services. If your tileset is provided by a service that already makes use of OpenAPI, it can be useful to implement the OpenAPI Conformance Class of the OGC Common standard.
Because these requirements are spread across multiple specifications with slightly different conventions and vocabulary, it can be a challenge to get started if you want to implement a compliant tile service. We put together a go-ogc library and a xyz2ogc utility to help bootstrap the process. You can use the xyz2ogc generate
command to produce metadata documents describing an existing XYZ tileset that satisfies the requirements of the conformance classes discussed above. See https://ogc-tiles.xyz/ for a complete set of metadata documents representing some common XYZ tilesets.
Be sure to share your experiences using OGC Tiles with us at https://community.planet.com/developers-55. Follow us on Twitter @PlanetDevs. Sign up for Wavelengths, the Planet Developer Relations newsletter, to learn more about our workflows and get announcements about what is new.
State of STAC
By: Tim Schaub on August 31 2022
STAC-ing up! We gathered statistics on over 550 million assets. Check out the summary of our project and register to be included in our next crawl.
The number of imagery providers hosting data online has increased over the years. Search interfaces and metadata formats have also grown to describe and provide access to that data. The SpatioTemporal Asset Catalog (STAC) specification aims to provide a unified framework for describing and linking to earth observation data, with the goal of increasing the interoperability of search tools and making it easier to access the data.
Since its release in May of 2021, the specification has been gaining popularity, with massive archives of publicly accessible geospatial data being hosted with STAC compliant metadata. The USGS hosts STAC metadata for the entire Landsat Collection 2 archive. Microsoft’s Planetary Computer provides a STAC API with access to imagery archives of Landsat, MODIS, Sentinel, and more.
Planet sees STAC as a key component in making earth observation data more accessible to developers, and helping advance it is a core part of our “Open Initiatives” as previously described. STAC’s adoption has felt very rapid, but we wanted more concrete metrics to establish a baseline for its actual adoption. Building up knowledge of how data is exposed as STAC can help us understand how the standard is being used and how mature various extensions are. This, in turn, can help us prioritize what we work on to increase its uptake. So to better understand how STAC is being used, we recently deployed a crawler to visit publicly accessible STAC endpoints, recording millions of statistics about the catalogs, collections, items, and assets linked from those endpoints.
STAC in a Nutshell¶
The STAC specification describes a few different resource types: catalogs, collections, and items. Items represent the individual spatio-temporal data entries and have references to the data assets. Collections are used to group similar items. Catalogs are the top-level entry and can also be used as sub-groupings of collections or items.
STAC implementations come with two different flavors of interface: static files and API. The static flavor can be thought of as an online folder or directory of JSON documents linking to one another. The API flavor is designed to provide more efficient pagination through large collections of items and may provide more advanced search functionality.
Crawl Summary¶
Starting with just 27 endpoints, sourced from the amazing stacindex.org, our crawl visited over 120 million items, spread across almost four thousand collections. We gathered statistics on over 550 million assets referenced by those items. About 97% of the collections were from static catalogs, while over 99% of the items and assets came from STAC API implementations.
Static | API | Total | |
---|---|---|---|
Catalogs | 10,084 | 26,041 | 36,125 |
Collections | 3,736 | 131 | 3,867 |
Items | 1,077,616 | 122,258,062 | 123,335,678 |
Assets | 1,497,447 | 550,133,772 | 551,631,219 |
The numbers suggest that people use collections far more frequently in static catalogs, presumably providing a way for users to explore reasonably small batches of items. While the STAC API implementations are dominated by catalogs with relatively small numbers of collections. Those API collections include many more items compared with their static counterparts. Because the STAC API provides search capabilities, catalogs and collections don’t need to be used as navigational aids in finding relevant items.
STAC Versions¶
The core STAC specification reached a stable 1.0 version in May of 2021, and the STAC API specification had its first release candidate toward 1.0 in March of 2022. Of the catalog, collection, and item resources we crawled, 88% conform with version 1.0.0. About 12% are still using version 1.0.0-beta.2. And a few stragglers are still on versions 1.0.0-rc.2, 1.0.0-rc.3, and 0.8.1.
Version | Catalogs | Collections | Items | 1.0.0 | 35,984 | 3,258 | 108,967,770 |
---|---|---|---|
1.0.0-rc.3 | 0 | 0 | 10,000 |
1.0.0-rc.2 | 2 | 0 | 0 |
1.0.0-beta.2 | 123 | 597 | 14,352,545 |
0.8.1 | 16 | 12 | 5,345 |
It is encouraging to see the number of implementations that comply with the stable 1.0.0 version. Perhaps those still using an earlier version have a path to upgrade.
STAC Extensions¶
The core STAC specification limits itself to describing the spatial and temporal aspects of the data—specifying metadata fields for the geographic location and time range associated with the data. STAC extensions provide a way to add information specific to a given domain. Extensions exist for adding metadata specific to electro-optical data, for adding additional coordinate reference system information, for adding detail about raster bands, and more.
Of the 120 million STAC items crawled, 86% referenced at least one extension to the STAC core. In total, 46 different extensions were used by the catalogs, collections, and items visited in the crawl.
This information collected on extension usage should be useful to help give real metrics to establish the proper maturity classification of proposed extensions—suggesting which might be candidates to graduate to a stable classification or which might be candidates for deprecation due to lack of use.
STAC Asset Types¶
Assets in STAC provide access to the underlying spatio-temporal data. For STAC items representing satellite imagery, the assets could be GeoTIFF rasters. Items may have assets that refer to additional JSON or HTML documents. The STAC asset metadata includes a “type” property advertising the asset media type. Values like “image/tiff” represent a generic TIFF image; “image/tiff; application=geotiff; profile=cloud-optimized” represents the Cloud Optimized GeoTIFF variety.
Asset Type | Count |
---|---|
application/json | 101,831,216 |
application/xml | 87,647,164 |
image/tiff; application=geotiff; profile=cloud-optimized | 75,945,405 |
image/png | 72,572,728 |
image/jpeg | 50,259,307 |
text/plain | 41,762,282 |
text/html | 41,139,833 |
image/vnd.stac.geotiff; cloud-optimized=true | 39,807,777 |
application/x-hdf | 16,489,141 |
application/netcdf | 10,252,878 |
image/jp2 | 3,639,899 |
application/gml+xml | 3,619,721 |
application/vnd.laszip+copc | 3,313,116 |
image/tiff; application=geotiff | 946,859 |
application/x.mrf | 902,162 |
text/xml | 620,780 |
image/tiff | 402,485 |
application/wmo-GRIB2 | 143,509 |
application/x-ndjson | 143,509 |
application/x-netcdf | 127,042 |
(missing) | 25,051 |
application/html | 10,000 |
application/vnd.google-earth.kml+xml | 9,249 |
application/gzip | 9,249 |
application/geopackage+sqlite3 | 5,854 |
application/geo+json | 2,846 |
application/vnd+zarr | 1,119 |
application/octet-stream | 569 |
application/x-parquet | 411 |
application/zip | 58 |
At first glance, it is surprising to see the high counts of asset types that refer to additional metadata – the “application/json” and “application/xml” asset types, for example. We had anticipated that STAC assets would refer primarily to the spatiotemporal data instead of additional metadata describing that data. As it turns out, many implementations use assets to point to additional metadata that either doesn’t fit into STAC or provides an alternate representation of the same metadata found in the STAC resources. In total, just under 50% of the 550 million assets could be classified as metadata assets as opposed to actual data assets.
Considering all of its varieties together, TIFF is the most common asset type, representing over 20% of all assets crawled. About 99% of the TIFF assets are advertised as GeoTIFFs. It is possible the remaining TIFFs are GeoTIFFs as well, but simply use “image/tiff” as their asset type. Of the GeoTIFFs, over 99% are of the Cloud Optimized variety. It appears that people are still not sure about what media type is appropriate for Cloud Optimized GeoTIFF (COG). Although there is not yet an officially specified MIME type for COGs, there is growing consensus around “image/tiff; application=geotiff; profile=cloud-optimized” as a candidate for specification.
Under the Hood¶
The service used to perform the crawl made heavy use of go-stac, which has been expanded with a full set of “crawler” utilities.
The crawling done by go-stac is now quite robust against errors and failures, and multiple end-points can be used, with a task queue to go through large amounts of data concurrently. The validate
and stats
commands in the go-stac command-line interface use the same crawling backend, so it’s now a great tool to use with large STAC catalogs.
The crawl results were stored in a BigQuery table, and we’ll work to make the resulting data available to the community. It will be great to see what other insights might come from this data.
What’s Next¶
We hope that reporting on real world implementations helps the community understand how the STAC specification is being used. We are looking forward to running another crawl and updating the results as more implementations are rolled out and the STAC spec continues to evolve.
While the linked nature of STAC makes it possible to crawl and gather statistics on publicly accessible resources, it is not the most efficient way to summarize this information – particularly for very deep collections and relatively small page sizes. We have proposed a lightweight stats extension for reporting counts of resource types, versions, extensions, and asset types accessible in catalogs and collections. In future crawls, when the crawler encounters stats like those described in the extension, it will stop crawling deeper and increment counts based on what is advertised in the stats metadata. The stats extension is not yet finalized, so try it out with your catalogs and API implementations, and share any feedback or contributions in our issue tracker.
And please upgrade to STAC 1.0.0 and STAC API 1.0.0.rc.1 and register on stacindex.org to be included in our next crawl!