Yorkshire Water 'Natural Knowledge' dashboard

As Data Scientists at Yorkshire water, the Open Data Institute (ODI) Leeds Six Capitals Hackathon was a great way for us to combine our love of open data, and some of the techniques we use every day. We developed a prototype dashboard which allows the public to explore and understand the green and blue natural capital across Yorkshire.

What tools and data did we use?

Where possible, we used open-source software and datasets to align with ODI values and increase the accessibility of our dashboard development. The Data Science team at Yorkshire Water work with these tools and datasets every day, so it was the perfect opportunity to pull all these together in to a single piece of work.

1) After collating our data sources, we transformed them into spatial data that could be displayed on a map. We used QGIS and the ODI postcode converter to do this. This gave us one shape file, containing outlines of each postal sector (e.g.LS1) and a single CSV file with XY co-ordinates for each pollution/beach/reservoir/volunteering/carbon datapoint.

2) To extract the volunteering data for our dashboard, we needed to do some webscraping using the Beautiful Soup package in Python. Beautiful Soup provides the tools to scrape a webpage of information and process the data in to the segments and formats that you want.

3) We then created an RShiny dashboard, made up of two key components: the user interface (UI) and the server. The UI covers the aspects of the dashboard that the user can control and interact with (e.g. typing in a postcode). The server is the part of the script which produces the output (e.g. the map).

4) We used the leaflet package to create the base map, combined with our postcode shape file. A Postcode text input box was created to allow the user to type in their postcode, following which the map reactively adjusts and zooms to the relevant part of the map. The postal sector was then highlighted as a box within the interactive map.

5) Finally,a series of tick boxes were created to allow the user to select information of interest (Get Active, Get Involved or Get Understanding). The reactive dashboard then selects the chosen elements from the large data file anddisplays pins on the map. The pins display relevant information when the user clicks on them.

What's next?

The RShiny package is a powerful tool which allows dashboards to be created with small amounts of code. Importantly, our dashboard has been developed with a central database so that additional datasets could easily be added. An organisation would simply need toprovide a postcode and any relevant detail (e.g. community events, volunteering events) and the information could be added to the master data CSV file. Itwould also be possible to add in extra levels of postcode detail, to allow the user to search for a specific postcode, rather than a postal sector.

Extra info

Here is a list of relevant tutorials for some of the software we discussed:

Most open-source datasets used within this project can be found here:

https://datamillnorth.org/