Subscribe to the newsletter for updates:
TLDR: use machine readable GeoTIFFs to make user interfaces more responsive and remove the need for server interactions.
A common user interaction in GIS applications is selecting a bounding box or polygon:
Your application may then use this polygon to do things like:
Users of your service expect that any action will complete in a reasonable amount of time. It's smart to build in a limit for the amount of data processed on the server. One simple way to do this is an area limit, for example: 100 square kilometers.
Using total area doesn't work well for datasets of man-made objects like OpenStreetMap. The density of features has huge variance in different parts of the world. In the image above, the area on the left is only part of Paris and contains about 10 million nodes. The area on the right is most of the U.S. state of Minnesota, and also has about 10 million nodes. A system limit designed for the densest areas would be too restrictive.
Instead of having an area limit, we can first test the area for the count of features, and then provide feedback in the UI if the limit is exceeded. Common ways to count are:
This information can be exposed to a web browser via API.
The previous approach has the major drawback of needing a remote, blocking server call. We can instead design a quicker way to estimate the number of features without a dynamic query. This allows for near-instant feedback in the UI, and less moving parts, at the cost of being a coarse approximation.
The GeoTIFF format is a natural fit for this problem. It's readable by all GIS libraries, supports compression, and embeds a geographic projection.
Linked here is a global GeoTIFF. Each pixel is one level 12 Web Mercator Tile, and each sample is a 16 bit unsigned OpenStreetMap node count (in thousands). This is too dark to be visible to the human eye, but normalizing the color range results in this image:
The resulting compressed GeoTIFF is only 212 KB, which is reasonable to deliver to web browsers. The source code for generating this GeoTIFF is available at github.com/protomaps/OSMEstimator.
import pyproj from rasterio import mask from shapely.geometry import shape RASTER = rasterio.open('osm_nodes.tif') PROJECT = partial(pyproj.transform,pyproj.Proj(init='epsg:4326'),pyproj.Proj(init='epsg:3857')) transformed = transform(PROJECT,shape(geom)) #geom is a GeoJSON-like dict masked = mask.mask(RASTER,[transformed],all_touched=False) estimate = masked.sum() * 1000
The choice of zoom level 12 and the accuracy of the estimation rely on query areas being significantly larger than the pixels of the GeoTIFF. Different applications may want to modify this code for: