CSV file processing

As Code-Point Open data is supplied in separate .csv files by two-digit grid references, the files will require some processing in order to proceed.

The Great Britain .csv files should be combined with the Code-Point_Open_Column_Headers.csv. To do this:

1. Before processing the data, create the following folder:

   • Great Britain

Then, copy all the .csv files from the CSV folder for Great Britain into this folder (120 in total).

1. Add the Code-Point_Open_Column_Headers.csv (located in the Doc folder) into the Great Britain folder and prefix the name with: aa_

A screenshot showing CSV header file and Great Britain .csv files

Combining multiple CSV files

The following is an example of one way to combine all the individual .csv files into a single file by using a .bat batch file.

The .csv file that is created comes to approximately 150MB, and 1.7 million records. If this is opened in Microsoft Excel, only the first million records can be seen as the software cannot display more than this number of records.

To use the batch function:

1. Copy the following text and paste it into a new Notepad document: copy *.csv outputfile.csv

2. Save the Notepad document with the file extension .bat (for example, combine_csv.bat) in the Great Britain folder containing the 120 Great Britain .csv files.

3. Close the .bat file and navigate to the Great Britain folder, where it was saved. Double-click on the .bat file and an MS-DOS window will appear. Once the process is complete, the MS-DOS screen will close automatically.

4. A new CSV file with the name outputfile.csv has now been created within the Great Britain folder. All the .csv files apart from outputfile.csv can now be deleted as they have all been copied into the one ouputfile.csv.

The outputfile.csv will have two column headers at rows 1 and 2 – an abbreviated version and a full version.

The outputfile.csv in a text editor. There are two column headers at rows 1 and 2 – an abbreviated version (for example, PC)and a full version (for example, Postcode).

1. Open the .csv file in a text editor (not Excel) and delete the column header that you don’t want to use, ensuring that any resulting empty row is closed.

You will now have an output file ready for uploading into geospatial software (such as QGIS).