Skip to main content

Re-opening the bins

The Leeds Bin App was made by Tom and Dan at imactivate. It was built on top of open data about bin collections published by Leeds City Council. It is very simple: you search for your address and it tells you your bin collection dates. It can add reminders to your device's calendar too so that you don't miss the collection. As of 21st August 2019, over 1 million lookups have been done for over 13,000 unique postcodes. The app has a multiplier affect because, as long as some people on a street have it, others can tell which bin to put out by following the lead of their neighbours.

The bin app is obviously helpful for those of us that can't remember which bin to put out this week, but it can also help Leeds City Council. Every year, around Christmas/New Year, they have to contact every household in Leeds to let them know about changes to bin collections over the holiday period. Posting that many letters can lead to a cost in the hundreds of thousands of pounds. If they are able to keep people up-to-date about bin collections for a minimal cost, that money can go into other things. Especially as council budgets get reduced. So bin apps make citizens lives easier and save money for councils.

Although Leeds Bin App originally used a dataset published on Data Mill North, copyright issues arose and it turned out the dataset wasn't as open as thought. Originally, Leeds City Council were publishing a file with the premises ID (used for bin collections), the address line, the locality, and the postcode. This allowed the citizen to search by postcode and then narrow down the address in a process familiar from online shopping. However, having both the address and the postcode infringed on Royal Mail's copyright as they own the Postcode Address File which is not open data. As a result, the Leeds dataset no longer includes postcode and the Leeds Bin App now has to use closed data in order to work.

Recently I decided to try to replicate what Tom and Dan do but only using open data sources. I thought I'd also use this as an opportunity to make a progressive web app and, for extra fun, work out how to insert anonymisation into the process right from the start.

Finding an address whilst protecting privacy

The first task was to re-imagine an address look-up that doesn't involve postcode. It has been suggested that people prefer to search by postcode but, in 2019, many people are used to entering the first line of their address into online forms. It is increasingly likely that their street name is in their predictive text. Typing the street name is much quicker than it might have been in previous decades. However, whereas postcodes are unique, street names are not. To break that ambiguity, we need to rely on the locality to make sure people pick the correct street. We could make a type-ahead search (that people are used to with Google etc) to make it easier for people too. With what seemed like a workable solution to avoid postcodes, the next consideration was how to do this in a privacy-first way.

The simplest and most privacy-protecting solution would be to send the premises data for the whole of Leeds to the browser. That file comes in at 33MB so is not practical. There needed to be a solution that split the data into small enough chunks to be useful whilst also preserving a level of anonymity. In the end I came up with the following method:

  1. Sort all the street names in a local authority alphabetically (ignoring spaces and non A-Z characters).
  2. Clean up the names of streets and localities (which are all in UPPERCASE) by using the OS Open Names dataset to get all the cases correct (rather than just assume that every word starts with a capital letter).
  3. Split the list of streets up into chunks of at least 50 making sure that all streets that start with the same 3 letters are in the same file e.g. for Leeds, every street starting AAA to ACO goes in one file, every street from ACP to AIN goes in a second, and so on.
  4. Create an index for these chunks of addresses so that any three letters can be mapped to one specific street file.

Once the user has entered the first three letters of their street name (and in many cases before that), the browser can work out which file to request. This means that the server only gets to know the range of the first three letters i.e. Street Lane is in the file premises-STP-SUM.csv. These files range from 12kB to 148kB (but are largely under 50kB) and contain every street address, the locality, the postcode district, the latitude/longitude, and every house along with the corresponding premises ID e.g. 

Street LaneRoundhayLS1753.8443-1.52463:JKI9;4:GSKG;5:FRXG;6:JLTA;7:JLTC;8:JMN7;10:JLTG;11:JMTB;12:MB4A;13:JLTH;14:FRXL;15:JMDK;16:K8UG;18:JME0;19:GWHK;20:K91Z;21:LZTP;23:EVPG;24:JMDR;25:MB4H;26:JMDV;31:JK74;31:JK77;42:JK7E;46:JL8B;48:IYN0;50:J5SI;50:MA21;50:MA22;57:MA26;61:MA2B;61:MA2C;64:JP7G;74:IQ9P;74:J5SL;74:JFL3;74:JP7R;76:EYH5;79:NV2S;80:EXHR;80:M5JR;84:JL0R;86:J5SN;88:J5SO;102:GQG8;104:MB4F;106:JL11;108:JL13;110:JL14;110:JL15;118:JKU3;120:FTHR;125:JP4W;127:JP4S;128:MB45;129:JP4Q;130:MB4B;132:JL86;133:JMJ7;133:LVFS;134:GSKX;138:JP3Y;140:JP3L;142:JP3O;144:JP4F;146:FTHO;148:JP48;150:JNUI;151:JNUK;152:JNUF;153:JNUN;155:JNUB;156:JNUD;157:JNUS;158:J5SS;159:JLNE;161:MB4G;162:JKPO;163:EYGU;164:JLN5;165:JNUT;167:JNUW;169:JLN4;170:JLN7;171:EZHM;172:EZHN;173:JLHL;174:GSKD;174:MB48;175:JLHR;176:JLHT;177:M4W2;178:GSKC;180:JLH8;181:JLHF;182:JNTJ;183:JNTT;184:JNTX;185:JNU6;186:JNU0;187:JNU3;188:GSU5;189:JNK7;190:JNTG;191:JNTR;192:JNTL;194:JNTO;196:JP3R;196:NME8;198:JOZV;200:JOZW;202:EX89;204:JP02;206:JP03;208:JOZZ;210:JKRD;212:JOZI;341:JOZL;343:M2ZP;345:JOZQ;349:EX8H;351:JP9L;353:JP9N;355:EX8D;357:JP08;361:JP9A;363:JP9C;365:JP9G;367:JKQ7;369:JKQA;371:JKQE;373:JKQG;375:JKQI;377:JKHE;379:JKH7;400:JKR8;401:JKRA;402:JKQQ;403:ILDM;404:ILDJ;405:ILDG;406:FSI3;407:JO34;408:JO37;409:FSI4;410:M2ZD;411:IMBW;412:IMBQ;413:IMBT;414:IMBV;415:GTQO;416:JNPP;418:IMBZ;433:IMBJ;435:GTQN;436:JNSJ;438:JNSG;439:JNSS;440:LUU7;441:JNS4;442:JNS7;443:JNS9;444:EX8J;445:JNSD;446:JNSA;447:JNTB;448:JNTE;448:JO2S;449:JO2U;450:JO2W;451:M2ZC;453:JNSZ;455:GTS1;456:JNT5;458:JNT7;460:JMCY;461:J5ST;461:LZTN;463:JMCQ;465:EZIF;470:JMD3;472:JMDB;474:JMCT;476:EZI3;478:JM1A;480:JM15;482:JM18;483:EZI4;484:JM0O;485:JM0Q;486:M2ZT;487:JM0V;488:JM0Y;489:JM19;490:LZTM;491:JMC8;492:JMC5;493:M2ZV;494:JMCG;495:JMCA;496:EZI5;497:JIDD;498:JMBX;499:M2ZU;500:JMC2;501:JOF6;502:JNNH;503:JNNK;504:JNNO;505:JNNQ;506:JNNS;507:JNDU;509:JNDX;511:JNNE;512:JNNA;513:JIM1;514:JNNV;515:LZU2;516:JIM4;517:JNOD;518:JNOG;519:JNOI;520:JNO1;521:M300;522:JNNZ;523:JIM3;524:JNO2;525:JNO4;526:JNO9;527:JOOV;528:JOOX;529:JOOS;530:JOP3;531:JOP6;532:JOOZ;533:M424;534:JOOH;535:JOF8;536:JOON;537:JOOP;539:JOPJ;541:JOPK;543:JOPM;545:JOPO;547:JOPR;549:JOPE;551:JOP9;553:JOPC;555:JOPH;557:EWN5;559:JOPG;561:JNTQ;563:JNQ8;565:JP6J;567:M2ZY;569:JP6P;571:JO01;573:GRDE;575:JP76;1 LIME TREE LODGE, 426:E15F;1 PARK VIEW COURT:KAWS;1 PARK WAY LODGE, 424:DFNB;10 PARK VIEW COURT:JMP7;11 PARK VIEW COURT:JMP2;12 PARK VIEW COURT:EVPM;12A PARK VIEW COURT:JMNQ;14 PARK VIEW COURT:KAWR;15 PARK VIEW COURT:JMP8;16 PARK VIEW COURT:EVPO;17 PARK VIEW COURT:M1HK;18 PARK VIEW COURT:MF6B;19 PARK VIEW COURT:JIDI;2 LIME TREE LODGE, 426:M5WC;2 PARK VIEW COURT:JMOL;2 PARK WAY LODGE, 424:DFNC;20 PARK VIEW COURT:JMND;21 PARK VIEW COURT:JMNN;3 LIME TREE LODGE, 426:E15G;3 PARK VIEW COURT:JMOP;3 PARK WAY LODGE, 424:FRQL;4 LIME TREE LODGE, 426:E15H;4 PARK VIEW COURT:EVPL;4 PARK WAY LODGE, 424:DFND;5 LIME TREE LODGE, 426:E15I;5 PARK VIEW COURT:JMPE;5 PARK WAY LODGE, 424:DFNF;6 LIME TREE LODGE, 426:L7VN;6 PARK VIEW COURT:JMPF;6 PARK WAY LODGE, 424:DFNG;7 LIME TREE LODGE, 426:E15J;7 PARK VIEW COURT:JMPC;7 PARK WAY LODGE, 424:DFNH;8 PARK VIEW COURT:JLUW;9 PARK VIEW COURT:MBPA;ANNEXE, 128:NP5A;APARTMENT 1, 31:LYFJ;BASEMENT FLAT, 20:JIDT;BASEMENT FLAT, 24:O2LN;BASEMENT FLAT, 27:K921;BASEMENT FLAT, 33:MB46;BASEMENT FLAT, 35:H9XA;BASEMENT FLAT, 48:MA20;CARETAKERS FLAT OVER OFFICE BLOCK BETH HAMIDRASH HAGADOL SYNAGOGUE:ILDA;DUNLEARY:JLJU;FIRST FLOOR FLAT, 16:JMDL;FIRST FLOOR FLAT, 154:KAIM;FIRST FLOOR FLAT, 452:KBSH;FIRST FLOOR FRONT FLAT, 17:K8UI;FIRST FLOOR REAR FLAT, 17:K91X;FIRST FLOOR, 94:L7DS;FLAT 1 RICHMOND HOUSE:KA2K;FLAT 1, 1:MBP7;FLAT 1, 9:L7DU;FLAT 1, 28:H9WN;FLAT 1, 30:L320;FLAT 1, 33:K922;FLAT 1, 72:KAIK;FLAT 1, 81:MA2G;FLAT 1, 467:E1EW;FLAT 1, 469:LBNJ;FLAT 1, 471:J5SU;FLAT 10 RICHMOND HOUSE:JLIH;FLAT 11 RICHMOND HOUSE:JLIB;FLAT 12 RICHMOND HOUSE:JLIE;FLAT 14 RICHMOND HOUSE:EYH8;FLAT 15 RICHMOND HOUSE:JLBH;FLAT 16 RICHMOND HOUSE:JMBK;FLAT 17 RICHMOND HOUSE:KD8O;FLAT 18 RICHMOND HOUSE:MF6A;FLAT 19 RICHMOND HOUSE:LZT7;FLAT 2 RICHMOND HOUSE:JLI0;FLAT 2, 1:L7DT;FLAT 2, 9:L7DV;FLAT 2, 28:H9WX;FLAT 2, 30:LAS3;FLAT 2, 33:HPFQ;FLAT 2, 35:K924;FLAT 2, 72:J5SK;FLAT 2, 81:MA2H;FLAT 2, 467:E2LU;FLAT 2, 469:KAIN;FLAT 20 RICHMOND HOUSE:FTI6;FLAT 21 RICHMOND HOUSE:JK7W;FLAT 22 RICHMOND HOUSE:JLBE;FLAT 23 RICHMOND HOUSE:FTI8;FLAT 24 RICHMOND HOUSE:MB47;FLAT 25 RICHMOND HOUSE:LZTL;FLAT 26 RICHMOND HOUSE:JKI2;FLAT 27 RICHMOND HOUSE:JKI4;FLAT 28 RICHMOND HOUSE:JKI5;FLAT 29 RICHMOND HOUSE:FTR2;FLAT 3 RICHMOND HOUSE:JLIL;FLAT 3, 9:L7DW;FLAT 3, 28:H9WP;FLAT 3, 30:DXH2;FLAT 3, 33:K923;FLAT 3, 35:KBSG;FLAT 3, 72:MA2E;FLAT 3, 467:E2LW;FLAT 3, 469:NT0X;FLAT 30 RICHMOND HOUSE:KD8N;FLAT 31 RICHMOND HOUSE:JKI6;FLAT 32 RICHMOND HOUSE:JMBM;FLAT 33 RICHMOND HOUSE:JMBO;FLAT 34 RICHMOND HOUSE:JMBQ;FLAT 4 RICHMOND HOUSE:JLIP;FLAT 4, 9:L5VQ;FLAT 4, 30:MFSF;FLAT 4, 467:E2LX;FLAT 4, 469:NT0Y;FLAT 5 RICHMOND HOUSE:JLIM;FLAT 5, 9:JLJW;FLAT 5, 467:EVUH;FLAT 6 RICHMOND HOUSE:MBP8;FLAT 6, 467:EVUI;FLAT 7 RICHMOND HOUSE:MB49;FLAT 8 RICHMOND HOUSE:KD8M;FLAT 9 RICHMOND HOUSE:JLI5;FLAT A, 131:MAO3;FLAT AT THOMAS OSBORNE:HT11;FLAT AT, 53:LJ68;FLAT AT, 64:HT8M;FLAT B, 131:J5SR;FLAT C, 131:MA2I;FLAT GROUND FLOOR, 154:MB4C;FLAT, 92:J5SP;FLAT, 359:EZEK;FLAT, 381:KBZ0;GARAGE COURT PARK VIEW COURT:E7PC;GARDEN FLAT, 50:J5SJ;GROUND FLOOR FLAT, 27:MA1S;GROUND FLOOR FLAT, 35:M5IJ;GROUND FLOOR FLAT, 48:HNAX;GROUND FLOOR FRONT FLAT, 17:K8UH;GROUND FLOOR FRONT, 44:M121;GROUND FLOOR REAR FLAT, 17:K91Y;GROUND FLOOR REAR, 44:M8QG;HIGHGATE:LKQ3;MEETING HOUSE, 136:MB44;OWNERS ACCOMMODATION BEECHWOOD HOTEL, 3234:LOFJ;PARK VILLE:JLTF;STABLE FLAT, 30:MA1X;THE STABLES, 28:MA1U;TOP FLAT, 44:K926;TOP FLAT, 76:EYH2;TOP FLAT, 94:M4V7;TOP FLAT, 471:J5SV;TOP FLOOR FLAT, 17:KBSF;TOP FLOOR FLAT, 27:K920;TOP FLOOR FLAT, 96:J5SQ;WOODLAND VIEW:JLJV;WOODLAND VILLA:JMTD;

Here I've taken the extra step of encoding the premises ID in base64 in order to save some bytes. Each of these files contain multiple streets in multiple parts of the city. The result is that the server has returned the address whilst not really knowing where the user is searching for. When the web page has a premises file, it can create a type-ahead search contained entirely within the browser. So the first stage of the type-ahead is to narrow down by street name. The second step is to limit those results by house number/name. Hopefully it is just as intuitive and just as fast to use as Leeds Bin App's postcode and street address steps.

Once the user has selected a specific address we then need to get the collection dates for it. But we don't want to send the premises ID to the server as that undoes all the privacy we just preserved. So, I've packaged up the collection dates data in the same way as the addresses. The browser just requests a file named jobs-STP-SUM.csv which contains all the jobs for all the premises with street addresses in the range STP to SUM. The server hasn't learned anything new about the address in this step as the key information sent (the range of the first three letters) is identical to before. In summary, we've now got a process that can find bin collections quickly, entirely with open data, and in a way that the server never really knows where you looked up.

Appiness

Now that I had a privacy-protecting mechanism for looking up addresses, the next step was to make a progressive web app. That meant creating a Service Worker and a Manifest file neither of which I have done before. Mozilla Developer Network and Google have some useful documentation on making a Manifest. It also turns out that the Developer tools in Chrome have a useful feature named Lighthouse that can help spot issues. After a few basic mistakes I got it working. That means you should be able to "pin to home screen" or even be prompted by your browser to "install" the page as an "app" once you've visited it a few times.

I kept imactivate's simple input field process to focus the user's attention. I've hidden everything else in a menu. I've added a cookie (only read by the browser and not sent to the server) which saves the form inputs so they can be auto-completed next time you come back. If you've disabled cookies that isn't a problem; you just have to fill in the form again yourself. Here's the result:

Alert! Alert! Alert!

A big point of a "bin app" is to be alerted when your bin collection is about to happen. That means waiting perhaps several weeks before alerting the user. I'm currently exploring how best to do this. One option - and the one I'm using at the moment - is to create iCal files that the user can download to add to their device's calendar. This isn't the cleanest option and doesn't work on all devices. Another option I'm exploring is to keep the alerts entirely within the app using Web Notifications.

You may have noticed Web Notifications become a thing in recent months as more and more websites prompt you to allow notifications. I've become very used to automatically saying "No" to these because they often occur when I'm not expecting them. That learned behaviour makes it more likely that people accidentally don't allow notifications when they want them. This might be mitigated by only asking to allow notifications as a result of the user clicking a button that lets them know they will be asked to allow notifications. Hopefully that is enough to make them be more conscious when the browser prompts them.

Having worked out how to create a notification I ran into another issue. An "app" such as this is mostly "off". That seems like a blocker. But, it turns out that the Service Worker should stay running in the background. I've experimented with sending the bin collection dates to the Service Worker and then running a setInterval event to check if any notifications need to be created. This only seems to last until the browser deletes cookies (which includes the Service Worker) and that seems to occur more frequently than bin collections so doesn't seem to be reliable enough. For the moment, I'll have to stick with the iCal solution. If you've had experience of setting a notification for a future time in a web app I'd be interested to hear how you did it.

Differences

After creating a fully open bin app, I tried testing it with the addresses of people I know. As a result I've found some differences in results between the imactivate app and my results. Initially I thought I might be doing something wrong but, after talking to Dan, it looks as though there are differences in the source data. I've fed that back to Bartec who operate the bin collections on behalf of the council. Hopefully they can find the issues and make sure the open and closed data sets are consistent.

Summary

This process has resulted in me creating a friendly "rival" to Tom and Dan's bin app to provide some competition to the bin app market. My version is entirely web-based. It only uses open data. It only uses static files so can run on Github pages. It also goes out of its way to make sure the server knows as little as possible about the address the user has searched for.

Next steps might include adding useful features around recycling. To start with I could include the city council's open data set of their recycling sites. But that is a limited dataset and, for instance, doesn't include all the battery recycling sites. I'm not sure there is a complete dataset of those in existence but that could be crowd-sourced through Open Street Map. What else would you like to see in a bin app? Let us know on Twitter.