LogoLogo
OS Docs HomeOS NGDOS APIsOS Download ProductsMore than MapsOS Data Hub
  • Introduction to OS APIs
  • Core Concepts
    • Getting started with an API project
    • Authentication
    • Error codes
    • Rate-limiting policy
    • OS API branding
    • Service level agreements
    • Service availability dashboard
  • Accessing OS APIs
    • OS Maps API
      • Layers and styles
      • Getting started
        • ESRI ArcGIS Online
        • ESRI ArcGIS Pro
        • ESRI ArcMap
        • Cadcorp SIS
        • MapInfo Pro
        • QGIS
      • Technical specification
        • ZXY
        • WMTS
      • Code examples
    • OS Vector Tile API
      • What data is available?
      • Getting started
        • ESRI ArcGIS Pro
        • Cadcorp SIS
        • QGIS
        • Create a web application using the OS Vector Tile API
      • Technical specification
        • Service Metadata
        • Stylesheet
        • Tile request
      • Code examples
      • Stylesheets
    • OS NGD API – Tiles
    • OS NGD API – Features
    • OS Features API
      • OS Product Archive
      • What data is available?
      • Getting started
        • ArcGIS Online
        • ArcGIS Pro
        • ArcMap
        • Cadcorp SIS
        • MapInfo Pro
        • QGIS
      • Technical specification
        • getCapabilities
        • describeFeatureType
        • getFeature
        • Filtering
        • Paging
        • Empty values
      • Code examples
    • OS Names API
      • Getting started with example queries using Node.js
      • Technical specification
        • Find
        • Nearest
      • Code list
      • Code examples
    • OS Linked Identifiers API
      • What data is available?
      • Getting started with implementing a look-up application
      • Technical specification
        • Identifier
        • Identifier Types
        • Feature Types
        • Product Version Information
      • Code examples
    • OS Places API
      • Getting started with example queries using Node.js
      • Technical specification
        • Find
        • Postcode
        • UPRN
        • Nearest
        • BBOX
        • Radius
        • Polygon
      • Datasets
      • Code lists
      • Code examples
    • OS Match & Cleanse API
      • End of Life Information
      • Getting started with an example match query using Node.js
      • Technical specification
      • Datasets
      • Code lists
    • OS Downloads API
      • Getting started
        • Automating OS OpenData downloads
        • Automating OS Premium data downloads
      • Technical specification
        • OpenData products
        • OpenData product details
        • Download an OpenData product
        • OpenData product image
        • Data packages
        • Data package ID
        • Data package version
        • Data package version ID
        • Download a data package
    • OAuth 2 API
      • Getting started
      • Technical specification
  • Additional resources
    • OS API Wrappers
      • JavaScript
      • Python
      • R
  • Extra Links
    • Accessibility
    • Contact us
    • PSGA Product Summary
    • Terms and conditions
Powered by GitBook
On this page

Was this helpful?

  1. Accessing OS APIs
  2. OS Downloads API
  3. Getting started

Automating OS OpenData downloads

PreviousGetting startedNextAutomating OS Premium data downloads

Last updated 4 months ago

Was this helpful?

This guide takes you though the process of automating the download of OS OpenData downloads.

What you need

  • A basic understanding of automatic processing of JSON data and basic procedural (if-then-else) programming.

  • A text editor like Visual Studio Code

  • A working installation of and the popular module.

Please read the before working with data packages.

These are generic instructions intended to broadly outline the necessary steps. Please see below for one possible implementation.

1

Get a list of products

Get the list of products available from endpoint including the product id and version.

const axios = require('axios');

async function getList() {
    const productList = await axios('https://api.os.uk/downloads/v1/products');
    /* For explanation and debugging purposes we display the full response from the API in the console */
    console.log(productList.data)
    for (const product of productList.data) {
        /* This will print out the product ID which can be used in another request. This would be the ideal place
           to call another function which continues the processing or filters down the results to the set required. */
        console.log(product.id)
    }
}
getList() 

This is similar to a in that you first need to discover what is available. Be mindful that list items can change.

  • Each entry in the JSON response provides the Name, Description, Version, URL and ID.

  • You may at this point wish to create a loop that iterates through all available datasets or a subset that you are interested in based on common attributes (for example, vector data in a certain format).

  • You could store the product id and version. That way you can easily check periodically for new versions of the data being available, simply by comparing the version we provide with the one you hold.

2

Get product specific information

While the key parts of this are already provided in step 1, this shows another way of getting to specific detail.

const axios = require('axios');

/* This function will return the specific information on OS Open Greenspace */
async function getProductDetails() {
    const greenspaceDetails = await axios('https://api.os.uk/downloads/v1/products/OpenGreenspace');
    console.log(greenspaceDetails.data)
    /* At this point we could insert another function to process the results or act on them/download them */
}
getProductDetails()
3

Download data

Using the product id from the previous response make a call to the endpoint.

  • This will result in a list of all available download links. This may be a single link for some datasets or a set of links split by area covered and/or formats supplied.

  • Once you obtain the download link for the area of the dataset in the format you require you are ready to download the data, ready for automated processing by whichever means your system requires.

In this example we hard code the product and format we are interested in (OpenGreenspace in ESRI® Shapefile). This can also be provided dynamically into the function allowing the same function to be used for multiple products and formats.

const fs = require('fs');
const axios = require('axios');

/* ============================================================
Function: Uses Axios to download file as stream using Promise
============================================================ */
const download_file = (url, filename) =>
    axios({
        url,
        responseType: 'stream'
    }).then(
        response =>
            new Promise((resolve, reject) => {
                response.data
                    .pipe(fs.createWriteStream(filename))
                    .on('finish', () => resolve())
                    .on('error', e => reject(e));
            }
    )
);

/* ============================================================
Download Files in Order
============================================================ */
async function downloadFiles() {
    try {
        const downloadInfo = await axios.get('https://api.os.uk/downloads/v1/products/OpenGreenspace/downloads')
        for (const download of downloadInfo.data) {
            if(download.area !== 'GB' && download.format === 'ESRI® Shapefile') {
                let downloadFile = await download_file(download.url, `${download.area}.zip`);
                console.log(`Downloaded file ${download.area}`)
            }
        }
        console.log('Completed downloading files')
    } catch (error) {
        console.error(error);
    }
}

downloadFiles()
Node.js
axios
download support documentation
manual download
Sample implementation using Node.js
#products
#products-productid-downloads