Tips for optimising large GeoJSON files

The other day Terence Eden, who makes the excellent Open Benches, asked:

I think I need some #geospatial help.

I have a #GeoJSON file which is ~10MB. Even when compressed it is about 2MB.

It provides a cluster of ~28k points displayed on a #Leaflet map.

Is there a sensible way to make this more efficient? Can I start will a small GeoJSON of just the clusters and then "stream" in more detailed files when users zoom in?

This is a great question and I've had to deal with the same issue myself a lot over the years. By the time I saw it there had already been quite a few responses. But most responses suggested pretty complicated/involved alternatives (GeoTIFF, vector tiles, PostGIS etc). Although some of those would reduce the size of the data (by using more compact formats) some of that saving would be lost due to larger Javascript libraries that you need to be able to use those formats. So I set about trying to optimise the GeoJSON itself. I'm sharing my approach here in case it is useful for others.

Here's the very top of Terence's 9.2MB GeoJSON file (I've added some whitespace for readability):

{
"type": "FeatureCollection",
"features": [{
	"id": 1,
	"type": "Feature",
	"geometry": {
		"type": "Point",
		"coordinates": [-1.240306, 51.729153]
	},
	"properties": {
		"created_at": "2017-07-11T21:12:50+01:00",
		"popupContent": "Donated by the Godfrey Family\u003Cbr \/\u003E\r\nin memory of\u003Cbr \/\u003E\r\n\u0022THE OXFORD KID\u0022",
		"media": [[]]
	}
},{
	"id": 2,
	"type": "Feature",
	"geometry": {
		"type": "Point",
		"coordinates": [-1.236847, 51.730831]
	}
...

Tips

Here are my tips for shrinking the size of GeoJSON files.

1. Truncate coordinates

The first thing I always check in a large GeoJSON file is the precision of the latitude/longitude coordinates. Five decimal places gives you around 1m precision and is therefore fine for many (if not most) use cases. Any more decimal points are just a waste of bytes; some large GeoJSON files can have around 25-40% of their file due to unnecessary precision in coordinates. However, I'd had a conversation with Terence a few years ago and, as you can see, they are only 6 decimal places in this file. So, in this case, there wasn't a huge amount to be saved from that - but every little helps.

2. Remove whitespace

Often there can be a lot of whitespace (spaces, tabs, newlines etc) in GeoJSON files to make it a little easier for us humans to read. However, if your GeoJSON is large it is preferable to remove as much of that as possible. I will sometimes still leave a newline before each GeoJSON feature but otherwise remove indentation and spaces between properties and coordinates. Our GeoJSON Minify tool will do this for you. The Open Benches GeoJSON file has around 945kB of unneeded whitespace.

3. Remove unnecessary properties

Next on my check list is removing unnecessary properties. Our GeoJSON Minify tool tells me that popupContent was taking up 3.6MB, created_at 1.2MB, and media another 373kB. Neither media nor created_at were being used on the front page of Open Benches so they could go. I also noticed that the popupContent values often contained escaped characters (e.g. "\u003E") taking up six characters instead of one. There were also a lot of "\r\n" values which wouldn't actually get rendered in the page so could be zapped too. Using the variable name "popupContent" 28,000 times also takes up some space so I suggested shortening it to "popup" (or even "p" to save another 100kB). Doing all this took the file down to 6.2MB.

4. Simplify polygons

Open Benches only has point data but often a GeoJSON may contain polygons. The more points used to make the polygons the larger the file size. You can use a useful tool such as MapShaper to simplify your polygons. It gives you a few different simplification algorithms to choose from and then a slider to adjust the amount of simplification applied. See how much simplification you can get away with without distorting things too much.

You can also use MapShaper to merge polygons into one larger polygon. Once you've loaded your GeoJSON file, open the MapShaper console and type "dissolve" to merge all the polygons. When you export from MapShaper you can also set "precision=0.00001" in the command line options to limit the coordinates to 5 decimal places.

If your polygons are UK geographies such as Local Authority boundaries or LSOA boundaries check if there are lower resolution versions of the ONS Geoportal. ONS often have four or five different resolutions for each geography type. For instance, their 2021 LSOA boundaries are available as BFE (full resolution), BFC (full resolution clipped to the coastline), BGC (20m clipped), BSC (200m clipped). If 200m is good enough, go with that file as it will be much smaller. Just make sure to run the Geoportal's GeoJSON output through our GeoJSON Minifier to reduce coordinate precision as they default to 15 decimal places (roughly atomic precision).

As an aside, the ONS Geoportal has a tendency to return coordinates as Eastings/Northings even though the GeoJSON standard has required WGS84 (latitude/longitude in decimal degrees) since 2016. It'd be nice if ONS could match the standard. Update 2024-07-10: we have now created a tool that helps fix the common problems with ONS Geoportal GeoJSON files.

Further thoughts

I have a lot of time for GeoJSON but it is quite a bulky format with a lot of repetition in it. For the Open Benches example, I realised that converting to a simple TSV (tab-separated values) or CSV (comma-separated values) file could dramatically reduce the file size further. In a TSV, the "id", "latitude", "longitude" and "popupContent" values would only take up 3.6MB in total. There would then need to be a few lines of Javascript code to parse the TSV file and convert it into GeoJSON in the browser but that is tiny compared to saving a further 3MB.

Finally, I realised that the popup content didn't actually need to be loaded with the page as most people wouldn't click on a marker and certainly not a large fraction of the 28,000 markers. So the popup content could be dropped and the Leaflet map could, instead, load the few hundred bytes of specific popup content as-and-when anyone clicked on a map marker. Shifting that content out took the initial file size down to 670kB from 9.2MB - almost a 93% reduction.

In the end, Terence decided to go with a TSV format but kept the bench inscriptions in the file because that fitted his needs better. Still, it let him reduce the load on the visitor from 9.3MB to 2.7MB (2MB to 1MB gzipped).