> For the complete documentation index, see [llms.txt](https://docs.os.uk/more-than-maps/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.os.uk/more-than-maps/os-downloads-api/using-the-os-downloads-api-with-python.md). # Using the OS Downloads API with Python ## Process Overview You will: 1. Set things up once (Python environment, database, schema). 2. Run Script 1 once to create the correct database schema. 3. Run Script 2 repeatedly (manually or on a schedule) to: * Check if OS has released a newer data package * Download it if needed * Load it safely into your database After setup, you will normally only touch Script 2. ## Prerequisites * A Windows machine (this guide uses Windows paths and Task Scheduler) * Python 3.9+ installed * Access to a PostgreSQL database (local or remote) * An OS Data Hub account with access to the Downloads API You will also need: * An API Key from OS Data Hub * The Package ID of the dataset you want to automate * CSV * Monthly or Annual FULL REFRESH ## Step 1: Create a working folder Create a new folder anywhere on your machine, for example: ```sql C:\os_downloads_automation ``` Inside this folder you will eventually have: * A Python virtual environment * Script 1 (schema setup) * Script 2 (download + upload) * A run.bat file (for automation) ## Step 2: Create and activate a Python virtual environment Open Command Prompt and run: ```plsql cd C:\os_downloads_automation python -m venv .venv ``` Activate it: ``` .venv\Scripts\activate ``` You should now see the virtual environment name in your prompt. ## Step 3: Install the required Python Packages With the virtual environment activated, install the dependencies: ``` pip install psycopg2-binary requests ``` The scripts also use standard Python libraries which do not need installing: * os * sys * zipfile * logging * datetime * time ## Step 4: Enable the Downloads API in OS Data Hub and initiate your data package 1. Log in to OS Data Hub 2. Open your API Project 3. Add Downloads API to the project 4. Copy your API Key – you will need this in Script 2 5. Create data package containing the data that you would like to upload. Select CSV and chose monthly or annual FULL REFRESH. 6. Create datapackage and note the datapackage number ## Step 5: Set up your environment file 1. Within your working folder create a new file called "**key.env**" 2. Open the file with a text editor 3. Input the parameters and corresponding values below as plain text i.e. no apostrophe's ``` OS_DOWNLOADS_API_KEY= DB_PASSWORD= DB_USER= DB_NAME= DB_HOST= DB_PORT= ``` {% file src="/files/4zT8xF4BpolM90yOJgX5" %} ## Step 6: Script 1 - Upload the database schema (RUN ONCE!) This script prepares your database so the data can be loaded correctly. #### When do I run this? * The first time you load a dataset * If OS releases a new schema and you update it You do not run this every day. #### What you need to edit In the first cell / config section of Script 1: * Database connection details * TARGET\_SCHEMA #### Add the schema SQL 1. Go to the OS NGD schema repository: [https://github\\.com/OrdnanceSurvey/osngd\\-resources/tree/main/database\\-resources/postgresql/ddl](https://github.com/OrdnanceSurvey/osngd-resources/tree/main/database-resources/postgresql/ddl)[osngd-resources/database-resources/postgresql/ddl at main · OrdnanceSurvey/osngd-resources · GitHub](https://github.com/OrdnanceSurvey/osngd-resources/tree/main/database-resources/postgresql/ddl) 2. Open the schema you need 3. Copy **the entire SQL file** 4. Paste it into the **schema upload section** of Script 1 #### Run Script 1 Save the file and run it once: ``` python 1_schema_setup.py ``` If this completes successfully, you do not need to touch Script 1 again unless the schema changes. {% file src="/files/4BxidX2vkWpBlTmilopc" %} ## Step 7: Download and upload data #### What Script 2 does * Checks OS for the latest version of the data package * Compares it to what is already installed * Downloads new data if required * Loads data into a staging schema * Safely replaces existing tables * Logs successes and failures #### What you need to edit At the top of Script 2, update: * Package ID * Download location * Staging schema name * Target schema name Save the file. #### Test Script 2 manually Run: ``` python 2_download_and_upload.py ``` Confirm that: * Data downloads correctly * Tables appear in the target schema * A log file is created {% file src="/files/Vh13pkYIgn4oxyTwjUZo" %} ## Step 8: Create a run.bat file This allows Windows Task Scheduler to run the script correctly. #### Create the file In your main folder (C:\os\_downloads\_automation), create a file called: ``` run.bat ``` #### Edit run.bat Open it in Notepad and paste: ``` cd C:\os_downloads_automation .venv\Scripts\python.exe 2_download_and_upload.py ``` Adjust the path if your folder is different. #### Test it Double‑click run.bat. ## Step 9: Automate with Windows Task Scheduler #### Create the task 1. Open **Task Scheduler** 2. Select **Create Basic Task** #### Task details * **Name**: OS Downloads API Refresh * **Description**: Automatically checks for new OS data releases and updates the database #### Trigger * Choose how often to run * Monthly is typical if your data package refreshes monthly #### Action * Select **Start a program** * Browse to and select run.bat #### Finish Save the task. ## Common troubleshooting tips * **Nothing downloads**: Check API key and package ID * **Database errors**: Check credentials and schema name * **Script runs but no data changes**: Data may already be the latest version * **Task Scheduler fails**: Re‑test run.bat manually first --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.os.uk/more-than-maps/os-downloads-api/using-the-os-downloads-api-with-python.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.