VMGD API

I wanted weather data for Vanuatu but I wanted the data in a machine readable format which is not provided by the Vanuatu Meteorology Services. I was using DarkSky but they were bought out by Apple and my API key stopped working so I decided, obviously, to make my own weather API. Of course, I know it won’t ever be as easy as I think it should be but I know this ahead of time and just enjoy the adventure. You can visit the API at unofficial-vmgd-api.michaeltoohig.com.

I do wish to write a nice frontend for it one day.

Legally, I believe I must say my copy of the data from the VMGD is only for my personal use, and anyone wishing to use VMGD data should use the official website as my API is not intended for use by anyone for anything.

Tech Stack

The app is written in Python. The API is built with FastAPI and uses the underlying Starlette framework for the simple frontend. The web scraper is built using Anyio, HTTPX, Cerberus and BeautifulSoup. Data is stored on local disk with SQLite.

API

The API provides endpoints for querying forecasts, media releases and weather warnings and I’ve tried to include many query parameter options with defaults returning the latest available data. Plus, I included an endpoint to query the raw, unprocessed data scraped from the HTML pages so you can do what what you want with that or verify my aggregated data results. Lastly, I included an endpoint to track the scraping sessions themselves so I can alert when a particular session is failing.

For demonstration, a weather warning query such as the following…

http://localhost:8000/v1/warnings?date=2023-08-03&name=warning_marine

…returns a weather warning object.

{
  "meta": {
    "issued": "2023-08-02T23:14:00+00:00",
    "fetched": "2023-08-02T23:45:11.701411+00:00",
    "attribution": "The data provided was collected on the `fetched` date provided from the Vanuatu Meteorology & Geo-Hazards Department website at https://vmgd.gov.vu/. This service should not be used by anyone for anything; always get up-to-date and accurate data from the VMGD website directly."
  },
  "data": [
    {
      "date": "2023-08-02T13:00:00+00:00",
      "name": "warning_marine",
      "body": "Marine Strong Wind Warning for all Vanuatu open waters!! SE winds 26/30 knots with very rough seas to 3.0 meters over The Channel & Southern waters while SE winds of 21/25 knots with rough seas to 2.5 meters is expected elsewhere. Moderate to Heavy swells expected."
    }
  ]
}

The meta object contains both issued and fetched dates, the former for the date the VMGD published the data and the latter for the date the data was scraped. Using these values I figured a user could find both the most recently available data and check that it is up-to-date for their use case or search for historical data using the fetched date.