COVID Cases Monitor
I wanted to produce a small display for simple items of data streamed/taken from the internet. My initial plan was to look at environmental monitoring, but this was Feb 2020 and covid-19 hit the UK about this time. I saw that the UK government were making the cases data publicly available & so I set myself the challenge of making a low cost display to show the covid daily and cumulative cases for my local area or for the country. This code will only work for the UK data, but the principles might be useful to other applications.
I’ve done a post on using the TTGO 2.19″ EPaper ESP32 Unit v2.3 and also a post on using this unit to create a weather display. Those examples and the little display unit were the basis for the code presented here. If you would like to make your own, you can buy a kit of parts for this display from my web shop here.
Basically this code needs to wake up every so often (or on a touch press), & try to connect to WiFi, creates an access point to add WiFi info if it can’t connect or, if it can connect, collect some data from the UK government data, display the information and then go back to sleep.
The above sentence makes it seem very simple, but my code knowledge is pretty low & I generally hack code until it works. I set myself this challenge in order to learn more.
Please see the previous two posts for getting the unit to display information and graphics and also for the wifi connectivity code.
Data source
First I needed to figure out where to obtain the data from and how to perform requests which return the data in a suitable format. This was available via an API which was reasonably well documented here:
https://coronavirus.data.gov.uk/details/developers-guide
As far as I understood it, this returns data in JSON format. I checked all these examples using a web browser (probably Chrome) and I could see the nice lists of data in JSON format. Great! So I’ll just be able to use this!
So the request to make for getting the overview data on a specific day (after lots of attempts at this) looked something like this:
https://api.coronavirus.data.gov.uk/v1/data?filters=areaType=overview;date=2020-12-23&structure={%22date%22:%22date%22,%22areaName%22:%22areaName%22,%22newCasesByPublishDate%22:%22newCasesByPublishDate%22,%22cumCasesByPublishDate%22:%22cumCasesByPublishDate%22,%22newDeaths28DaysByPublishDate%22:%22newDeaths28DaysByPublishDate%22,%22cumDeaths28DaysByPublishDate%22:%22cumDeaths28DaysByPublishDate%22}To get all the overview data from start of data set, we just don’t put the date in there. Like this:
https://api.coronavirus.data.gov.uk/v1/data?filters=areaType=overview&structure={%22date%22:%22date%22,%22areaName%22:%22areaName%22,%22newCasesByPublishDate%22:%22newCasesByPublishDate%22,%22cumCasesByPublishDate%22:%22cumCasesByPublishDate%22,%22newDeaths28DaysByPublishDate%22:%22newDeaths28DaysByPublishDate%22,%22cumDeaths28DaysByPublishDate%22:%22cumDeaths28DaysByPublishDate%22}The data for a local area needs a slightly different request than the data for a country area. This is shown here:
https://api.coronavirus.data.gov.uk/v1/data?filters=areaName=Nottingham;date=2020-11-20&structure={%22date%22:%22date%22,%22areaName%22:%22areaName%22,%22newCasesBySpecimenDate%22:%22newCasesBySpecimenDate%22,%22cumCasesBySpecimenDate%22:%22cumCasesBySpecimenDate%22,%22newDeathsByDeathDate%22:%22newDeathsByDeathDate%22,%22cumDeathsByDeathDate%22:%22cumDeathsByDeathDate%22}Coders probably get this straight away, but I needed quite a bit of time to figure out which bits are needed and how to format the URL string.
HTTPS requests
So the first this was that I was getting errors when I requested the data. I realised that I needed to use HTTPS requests, as HTTP requests would not be serviced. So I needed to use HTTPS and the WiFiClientSecure.h library.
I followed this tutorial to find the root CA certificate, but Firefox (used to find the certificate data) has changed slightly since this tutorial, but it only looks a bit different:
This gave me the rootCAcertificate for the website, which I copied and pasted into my code. It’s hard coded at the moment, but should have a few years validity.
Decompressing the data
Now I had the connection working and returning some data. I printed that data out to the serial port, but it was gibberish! I tried lots of things, as I just thought it was formatting or something else I had done wrong. In the end I read a bit more… the data is returned as a compressed file using gzip compression.
So I needed to figure out how to unpack the data. I found a nice library that performs this for me. I get the gzip data and save it to SPIFFS memory. I then used the library (ESP32-targz.h) to decompress the data. This worked first time without any fuss!
NTP server for date
I had done some work before to find the date and time via an NTP server. A very good overview and example code for this is available here:
In order to get past data I found that doing multiple calls for different dates was the best method for doing this. In order to do that I needed to take the time and date now, convert into UTC (in seconds since 1970) then subtract a days worth of seconds (86400) and then re-find the date as a string. This can then be used to get data for any range of dates I would like. This was more complex than I wanted it to be, but I could not figure out how to just return data between two dates from the API call – it was either all the data (too much!) or just one day. This part of my code needs work!
Display
At this point I had all the data downloaded from the web based source. I can then do the fun part which is make it display nicely and provide alerts. Please see the code for how I implemented this, but it was basically a mixture of text, graphics and some simple line drawing for the graphs.
Code repository
The code for this (and for other examples) is available in the github repository for this E-Paper unit. I’ve added it as an example code to go with the simple kit that I sell (here!).
https://github.com/re-innovation/TTGO_EPaper