Accessing OS NGD FAQ's

🆕Getting started

This guide provides information and instructions to help you get started using the OS Emergency Services Gazetteer (OS ESG) data in your software.

Data Supply

The OS ESG can be downloaded from the Premium Downloads section of the OS Data Hub.

It comes in 2 geospatial data formats:

  • GeoPackage (GPKG)

  • Comma Separated Values (CSV)

The GPKG is formed of 3 tables: LPI, BLPU and Classification. This data does not need pre-processing before it can be used.

The CSV follows the AddressBase Premium data schema and contains rows of data with the following record identifiers: type 21, type 24 and type 32. This schema often requires pre-processing before it can be used. See below on how to do this.

Using the GPKG data

The GPKG format of OS ESG does not require splitting as a GPKG can contain multiple tables.

For OS ESG, the GPKG data supply contains 3 tables:

  • BLPU

  • LPI

  • Classification

It’s important to understand that only the BLPU table has a geometry field and this may dictate how you use it in your GIS software.

For general guidance on how to work with GeoPackage please read Getting Started with GeoPackage.

Accessing GeoPackage data via GIS software

The following sub-sections provide step-by-step instructions on how to access GeoPackage data via various GIS software packages, all current versions of these applications support GeoPackage natively.

ArcGIS Pro

Requirements:

  • ArcGIS Pro (version 1.1 or later)

  • OS ESG GeoPackage dataset, for example OSEmergencyServicesGazetteer_FULL_2024-04-03_001.gpkg

Note: Certain versions of ArcGIS Pro (for example, version 2.5) require a GeoPackage to have a spatial index added before the data can be viewed on the map. This can be done in the Catalog by opening the Feature Class Properties window within the Indexes page.

Instructions:

These instructions were created using ArcGIS Pro version 3.2.0. Versions from 1.1 onwards also support GeoPackage features.

  1. In ArcGIS Pro, open an existing project or create a new project:

    • To create a new project, select Map in the New Project section, enter a name, select a location for the project, and then click OK.

  3. In the Add Data dialog select the GeoPackage to be added into ArcGIS Pro and click Open. This opens the GeoPackage to reveal the individual layers.

  4. Select the layers you want to add and click OK. You can select more than one layer by holding down the Ctrl (control) key and clicking on multiple layers.

  5. The layers added into ArcGIS Pro will appear in the contents pane on the left-hand side of the project.

ArcMap

Requirements:

  • ArcMap (version 10.2.2 or later)

  • OS ESG GeoPackage dataset, for example OSEmergencyServicesGazetteer_FULL_2024-04-03_001.gpkg

Note: Certain versions of ArcMap (for example, version 10.8.1) require a GeoPackage to have a spatial index added before the data can be viewed on the map. This can be done in the Catalog by opening the Feature Class Properties window within the Indexes page.

Instructions:

These instructions were created using ArcMap version 10.5, but versions 10.2.2 onwards also support GeoPackage features.

  1. In ArcMap open an existing map or create a new one.

  3. In the Add Data dialog:

    • Select the GeoPackage file you want to add into ArcMap and then click Open. This opens the GeoPackage to reveal the individual layers.

    • Select the layers you want to upload into ArcMap. You can select more than one layer by holding down the Ctrl (control) key and clicking on multiple layers.

    • Click Add to add the layers into the map.

  4. The GeoPackage layers should now be viewable in the layers list in the Table Of Contents on the left-hand side of the workspace.

CadCorp SIS

Requirements:

  • CadCorp SIS (version SIS 9 or later)

  • OS ESG GeoPackage dataset, for example OSEmergencyServicesGazetteer_FULL_2024-04-03_001.gpkg

Instructions:

These instructions were created using CadCorp SIS 9.1 Desktop Express; however all versions of CadCorp SIS 9 or later support GeoPackage.

  1. In CadCorp SIS open an existing map or create a new one.

  2. Click Add Overlay in the top ribbon.

  3. In the Overlay Types dialog select Files > File and then click Next.

  4. In the File Browser select the GeoPackage file and then click Finish.

  5. The data should now appear on the map.

QGIS

Requirements:

  • QGIS (version 2.10.1 or later)

  • OS ESG GeoPackage dataset, for example OSEmergencyServicesGazetteer_FULL_2024-04-03_001.gpkg

Instructions:

These instructions were created using QGIS version 3.28. You can use any version of QGIS from version 2.10.1 onwards.

  1. In QGIS, open an existing project or create a new one.

  2. In the top navigation select Layer > Add Layer > Add Vector Layer.

  3. In the Data Source Manager | Vector dialog:

    • Source: Click ... (three dots) on the right of the Source field, select the GeoPackage file and click Open.

    • Click Add. This opens the Select Items to Add dialog.

  4. In the Select Items to Add dialog:

    • Select the the layers of the GeoPackage to be added to the map. You can select more than one layer by holding down the Ctrl (control) key and clicking on multiple layers.

    • Click Add Layers.

  5. The GeoPackage layers should now be viewable in the layers list on the left-hand side of the workspace.

Converting / Loading a GeoPackage to other data formats

It is possible to use Extract, Transform, Load (ETL) tools to convert the data into different formats and to load into databases.

The following sub-section provides step-by-step instructions for FME.

FME

FME is a data integration platform which can read the GeoPackage format and be used to convert and transform the data into other formats or databases. These instructions show you how to read a GeoPackage into a FME workbench.

Requirements:

  • FME Desktop

  • OS ESG GeoPackage dataset, for example OSEmergencyServicesGazetteer_FULL_2024-04-03_001.gpkg

Instructions:

These instructions were created using FME Workbench 2023.2.2.0.

  1. In FME, click Reader in the top ribbon.

  2. In the Add Reader dialog:

    • Format: Select OGC GeoPackage in the dropdown

    • Dataset: Click ... (three dots) to the right of the field, select the GeoPackage you want to read, and then click Open.

    • Coord. System: Select an appropriate coordinate system.

    • Parameters: Click this button to open the OGC GeoPackage Parameters dialog:

      • Tables: Click ... (three dots) to the right of the field to open the Select Tables dialog You can also use the Use Search Envelope option in this dialog to clip the GeoPackage to an extent.

        • In the Select Tables dialog select the layers you want to add into the workbench and click OK.

      • Click OK to close the OGC GeoPackage Parameters dialog.

    • Click OK to close the Add Reader dialog.

  3. An orange reader will display the name of the GeoPackage table that has been read in.

To use GDAL to load the GeoPackage datasets into a PostgreSQL database using the ETL tool please read and adapt in the generic Getting Started with GeoPackage guide > GDAL guide.

Using the CSV data

Pre-processing CSV data

This section guides you through the process of pre-processing the OS ESG CSV data as a stand-alone dataset. This means splitting the data by record identifiers and appending the Header files.

We have included instructions for two pre-processing options:

Gawk

To use gawk to split the OS ESG CSV files by their record identifier and append the Header files:

Note: Running the .bat file creates temporary files and requires extra space in the location in which you are creating your files. These files can be much larger than the original CSV files. They are deleted once the process has finished, but the space is still required.

  1. Add the OS ESG CSV files to an empty folder. It is very important to ensure that the folder does not contain any other CSV files. In addition, the folder name and the file directory path name must not contain any spaces, for example, C:\os_esg\.

  2. Download OSEmergencyServiceGazetteer-header-files.zip in the Formats overview > CSV section of the technical specification and extract the ZIP. You should have the following CSV files:

  3. Place the extracted OS ESG Header files into the folder containing the OS ESG CSV files (created in step 1).

  4. Go to the OS ESG/gawk GitHub repository (https://github.com/OrdnanceSurvey/OS-Emergency-Services-Gazetteer/tree/main/loading_scripts/gawk).

  5. Open the esg-gawk-split-script.bat file and copy the contents into a text editor, such as TextPad or NotePad++. Save this text file as a .bat file in the same folder as your OS ESG CSV data and Header files.

  6. In the OS ESG/gawk GitHub repository (location in Step 4), download and extract the ZIP file named gawk-4.0.2-bin.zip. This will extract a file called Gawk.exe. Place this file in the same folder as your OS ESG data and Header files.

  7. Double-click on esg-gawk-split-script.bat file and an MS-DOS window will open. Once the process is complete, the screen will either close automatically or you will have to press any key to continue.

  8. Running the batch file should create additional files with a similar naming convention to that of the Header files. The new files should have been generated in the same location as the original data and headers files. For example, there should now be an ID24_LPI_RECORDS as well as a Record_24_LPI_Header.csv in the folder.

Python

Note:

  • These instructions are based on Python 3.7.

  • You do not need to download the Header files if you are using Python to split the OS ESG.

To use Python (instead of gawk) to split the OS ESG CSV files by their record identifier and append the Header files:

  1. Add the OS ESG CSV files to an empty folder. It is very important to ensure that the folder does not contain any other CSV files. In addition, the folder name and the file directory path name must not contain any spaces, for example, C:\EmergencyServicesGazetteer\OS_ESG.

  2. Go to the ESG GitHub repository (https://github.com/OrdnanceSurvey/OS-Emergency-Services-Gazetteer/tree/main/loading_scripts/Python) and download the esg-record-splitter-v3_7.py file.

  3. Copy the file into the folder containing the CSVs (created in step 1).

  4. Double click the python script and it will automatically find the CSVs and split them into the relevant files.

  5. When the file splitting process is complete, the following message will be displayed: The program will close in 10 seconds.

  6. Once the CSV’s have been split out, they can be loaded into a GIS using a CSV import tool. There will be 3 main CSV’s:

    • BLPU

    • LPI

    • Classification.

Database loading

To load the data into a database you need to first create the tables and columns required for the data. These can be found at the following location and depend on the type of database you have: https://github.com/OrdnanceSurvey/OS-Emergency-Services-Gazetteer/tree/main/loading_scripts

Once you have created the tables, follow the guidance in Loading CSV files to load the data into your database.

Connecting Data

Each table in the OS ESG has a UPRN (Unique Property Reference Number). This is what links the rows of data together between tables.

Please follow guidance from your software provider on how to join datasets if using the CSV or GPKG directly (without a database load)

If you have the data in a database you can link it in the following way:

Select * from os_esg.blpu 
inner join os_esg.lpi on blpu.uprn = lpi.uprn 
inner join os_esg.classification on blpu.uprn = classification.uprn

Classification Scheme

Download the file below to understand the classification codes and their breakdown.

Related information

OS Emergency Services Gazetteer (OS ESG) Lightning Talk

OS Emergency Service Gazetteer (OS ESG) Technical Specification

PreviousTrailer (Type 99 Record)NextKnown data issues

Last updated