> For the complete documentation index, see [llms.txt](https://docs.os.uk/os-apis/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.os.uk/os-apis/accessing-os-apis/os-downloads-api/getting-started/automating-os-opendata-downloads.md).

# Automating OS OpenData downloads

This guide takes you though the process of automating the download of OS OpenData downloads.

## What you need

* A basic understanding of automatic processing of JSON data and basic procedural (if-then-else) programming
* A text editor like Visual Studio Code
* A working installation of [Node.js](https://nodejs.org/) and the popular [axios](https://github.com/axios/axios) module

{% hint style="info" %}
Please read the [download support documentation](https://osdatahub.os.uk/support/faqs/downloads) before working with data packages.
{% endhint %}

These are generic instructions intended to broadly outline the necessary steps. Please see [Sample implementation using Node.js](#sample-implementation-using-node.js) below for one possible implementation.

{% stepper %}
{% step %}

#### Get a list of products

Get the list of products available from [Technical specification](/os-apis/accessing-os-apis/os-downloads-api/technical-specification.md#products) endpoint including the product `id` and `version`.

```javascript
const axios = require('axios');

async function getList() {
    const productList = await axios('https://api.os.uk/downloads/v1/products');
    /* For explanation and debugging purposes we display the full response from the API in the console */
    console.log(productList.data)
    for (const product of productList.data) {
        /* This will print out the product ID which can be used in another request. This would be the ideal place
           to call another function which continues the processing or filters down the results to the set required. */
        console.log(product.id)
    }
}
getList() 
```

{% hint style="warning" %}
This is similar to a [manual download](https://osdatahub.os.uk/data/downloads/open) in that you first need to discover what is available. Be mindful that list items can change.
{% endhint %}

* Each entry in the JSON response provides the `Name`, `Description`, `Version`, `URL` and `ID`.
* You may at this point wish to create a loop that iterates through all available datasets or a subset that you are interested in based on common attributes (for example, vector data in a certain format).
* You could store the product `id` and `version`. That way you can easily check periodically for new versions of the data being available, simply by comparing the version we provide with the one you hold.
  {% endstep %}

{% step %}

#### Get product specific information

While the key parts of this are already provided in step 1, this shows another way of getting to specific detail.

```javascript
const axios = require('axios');

/* This function will return the specific information on OS Open Greenspace */
async function getProductDetails() {
    const greenspaceDetails = await axios('https://api.os.uk/downloads/v1/products/OpenGreenspace');
    console.log(greenspaceDetails.data)
    /* At this point we could insert another function to process the results or act on them/download them */
}
getProductDetails()
```

{% endstep %}

{% step %}

#### Download data

Using the product `id` from the previous response, make a call to the [Technical specification](/os-apis/accessing-os-apis/os-downloads-api/technical-specification.md#products-productid-downloads) endpoint.

* This will result in a list of all available download links. This may be a single link for some datasets or a set of links split by area covered and / or formats supplied.
* Once you obtain the download link for the area of the dataset in the format you require you are ready to download the data, ready for automated processing by whichever means your system requires.

In this example, we hard code the product and format we are interested in (OS Open Greenspace in ESRI Shapefile). This can also be provided dynamically into the function, allowing the same function to be used for multiple products and formats.

```javascript
const fs = require('fs');
const axios = require('axios');

/* ============================================================
Function: Uses Axios to download file as stream using Promise
============================================================ */
const download_file = (url, filename) =>
    axios({
        url,
        responseType: 'stream'
    }).then(
        response =>
            new Promise((resolve, reject) => {
                response.data
                    .pipe(fs.createWriteStream(filename))
                    .on('finish', () => resolve())
                    .on('error', e => reject(e));
            }
    )
);

/* ============================================================
Download Files in Order
============================================================ */
async function downloadFiles() {
    try {
        const downloadInfo = await axios.get('https://api.os.uk/downloads/v1/products/OpenGreenspace/downloads')
        for (const download of downloadInfo.data) {
            if(download.area !== 'GB' && download.format === 'ESRI® Shapefile') {
                let downloadFile = await download_file(download.url, `${download.area}.zip`);
                console.log(`Downloaded file ${download.area}`)
            }
        }
        console.log('Completed downloading files')
    } catch (error) {
        console.error(error);
    }
}

downloadFiles()
```

{% endstep %}
{% endstepper %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.os.uk/os-apis/accessing-os-apis/os-downloads-api/getting-started/automating-os-opendata-downloads.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.