The Hex Election
In November 2019 ODI Leeds was approached by New Statesman and Britain Elects to create a live visualisation of the 2019 General Election results for use before and during election night. We re-used and extended existing visualisation code from former elections. The challenge was the need to collect data as results came in, rather than presenting previously collected static data. We decided to invest some effort in data collection tooling to ease the process.
First the hexes
It seems that it was the hexmap election! There were many examples across the media - most prominently a large physical map in the entrance to BBC Broadcasting House - and it's great to see such a visualisation being used more broadly. We re-used the hex cartogram of UK constituencies we created in 2017 and code we'd made for former elections, petitions, and House of Commons Divisions.
A hex layout is always a compromise of arrangement and we think some layouts were more effective (and, quite frankly, more attractive) than others. We're not beyond reproach on this topic; we know that our layout was initially created with our geographic biases and areas of ignorance. This is where our open approach differs though; our constituency layout is open. We have previously had improvements to the layout of Northern Ireland (by someone from Belfast) and we were tweeted on the day before the election taking us to task about the layout of constituencies in Cornwall and Devon. Our commenter (themselves from Cornwall) was able to update the layout using our HexJSON Builder just hours before the results started coming in. By allowing community feedback - and providing the tools to do it - we all get a better map.
Then the data collection
We wanted to create a form-based tool to collect the data as we felt the chance of errors slipping in to the data collection or breaking the data pipeline would be too great if data collectors were entering directly into a spreadsheet. The latter of these was the largest concern: all it needed was for someone to inadvertently restructure the spreadsheet or otherwise corrupt it and the visualisation would stop working. Not what any of us wanted!
Of course the high-pressure data collection conditions increased the risk of errors late in the evening and by a team of journalists attempting to interpret multiple media sources of results.
We felt that an app presenting a simple data collection process would offer a reduced cognitive load on the people entering data whilst protecting from data corruption. We could also build in some fairly complex data validation and were able to extend, repair and replace functionality simply.
This last point became critical as the results began to roll in. We had (quite late on in the build) added functions to the tools to create formatted tweets containing the constituency results. As the first results were processed, it became apparent that there were formatting problems with the tweets. We were able to quickly push out a fix. Had this been based on a spreadsheet, it's possible that this wouldn't have been resolved in such a simple manner.
The techy details
The main collector app was a single-page web app based on Vue.js. This presented existing results and enabled the creation of new results. Collected data was stored, one JSON object per constituency, in an Amazon Web Service S3 bucket. Some reference data sourced from the Electoral Commission, data.parliament.uk, They Work For You and Democracy Club was loaded into the tool when it started up. The individual objects were processed to a form better suited to presentation as well as a summary CSV file using AWS Lambda functions, triggered periodically.
Once all the code was up and running together, the ODI Leeds team had great fun testing the whole process by the entering fake election results to try to break things. We even managed to run some performance tests by automating the data entry to see how the system performed under increased load. We are happy to report that it passed with flying colours.
All the code for the infrastructure can be found in the election-2019-infrastructure repo on GitHub. The code for the app is in the election-2019-collector repo. Some static assets were hosted on a GitHub Pages site managed from the election-2019-static repo.
Results
We've published the 2019 results - as collected on the night - under a Creative Commons license. There were errors made, of course, and with the benefit of hindsight, perhaps some more visual feedback could have been given to help the people collecting the data spot some of the trickier data entry issues (transposed digits, additional zeroes, etc).
