Preface: Geolocating Postcode Solution
I've found, so far, two legal sources of data which describes the geographical relationship of every postcode in the UK.- The Post Office: "PostZon" [ accuracy ~100m, cost ~£250 ]
The British Post office does provide a list of every postcode ( >1.5Million ) with the geographical relationship of every code. However, these locations are rounded to the nearest 100 metres which, for the exacting user is simply not good enough.
The reseller who seems to dominate associated adwords sells the data for £250 - not bad if you consider that's 60 postcodes for every penny!
- The Post Office PostZon File
- The Ordinance Survey: "Code-Point" [1m, £750+]
The Ordinance Survey ( in association with a number of other UK Gov. Depts. it seems ) provides an alternate database of 1.7Million postcodes with a 'resolution' of 1metre.
My understanding is that the figure given is the location of the property closest to the centre of the scatter of all the properties in the code, which seems about as good as it's going to get, short of giving you the locations of every one of the 26M properties. This also ensures that the code-point will be within the boundries of the scatter, whatever it's shape.
This is a little more pricey, but worth it for the extra resolution. There are also some interesting extras; as well as things like NHS wards and such you get a number of business versus residential versus PO-Box delivery points within the code. Licenses start at £750 and range to ~£5.5k for 101+ users.
- The Ordinance Survey Code-Point
The Catch! : Both provide "Northing" and "Easting"
The only catch is that both the OS and the PO provide their locations in what is known as Eastings and Northings. I shalln't go into a great deal of detail about these here as a number of sites/articles cited here explain it better than I can but in short this co-ordinating system stems from the UK Ordinance Survey maps of the country.
Use of Easting & Northing does not in of itself bring inaccuracy to mapping postcodes on something like Google Maps - these coordinates are exceedinly accurate - but translating between the two systems is not increadably simple.
There are two things you need to do here:
- Convert Easting & Northing to Longitude & Latitude
This is the process of translating OS Coordinates like "264046", "192487", which as a GridRef looks like SN26401924 to their Lat/Long equivalents : 051°50′40″N, 004°31′13″W or, in decimal 51.61430, -3.96382
There are a number of great online tools for demonstrating this; Chris Veness has published some javaScript here which shows how intricate just this little bit of maths is!
- Converting between Airy1830/OSGB36 and WGS84/GRS80
What? Ok, so you have your latitude & longitude but when you compare these to Google's estimation of where a postcode lies you'll realise that you're a few blocks out!
As I understand it it goes like this: The OS coordinates are based on a model of the shape of the Earth called the "Airy 1830 ellipsoid".
Google Maps ( and GPS systems etc ) are based on a different model of the curvature of the Earth, developed - to be fair to George Airy - some 150 years later. This model - or "Datum" is called called WGS84
It is this difference between the Datum used within the OS Coordinates and the Datum used by Google that causes the lats and longs to be out.
According to Wikipedia longitude lines are our byt around 70 metres in Cornwall, rising to ~120metres on the east coast of East Anglia.
Fitting the square peg through the round hole.
I have therefore been trawling around looking for a solution for the next problem: how do we convert our Eastings and Northings into Longitude and Latitude that work with Google Maps?
First we need to convert East/North to Lat/Long.
Solution 1: Geo::Coordinates::OSGB ( Perl )
This is a perl module written by a chap called Toby Thurston. I've just found a png on his site which describes the problem very well: here
Result for SA1 4LS
51.6143013958951, -3.96382366182886 ( Marker A )
Solution 2: Geography::NationalGrid::GB ( Perl )
Another perl module, this time written by a P Kent.
Now, solutions 1 & 2 both claim inaccuracies for mathematical and theoretical reasons. Interestingly though they do show exceedinly similar results ( a difference of around 1.5e-10 degrees ) which suggest to me that while written separately they have been written pretty exactingly to the same mathematical rules.
Result for SA1 4LS
51.614301395634, -3.9638236618435 ( Marker A )
Solution 3: PHPcoord, JScoord, Jcoord ( php, javaScript & java )
This is a PHP library written by a chap called Jonathan Stott.
This solution has a couple of distinct advantages over the first two: that for many people doing this in PHP or javaScript would make more sense.. and because it both converts to Lat/Long and converts into the WGS84 Datum.
It also calculates distances which could be very handy.
Result for SA1 4LS (Airy1830)
51.6143013965, -3.96382366167 ( Marker A )
Then we convert OSGB/Airy1830 to WGS84
Solution 1 : Geo::HelmertTransform ( perl )
A datum transformation module written by Chris Lightfoot.
This will take our Airy1830 Lat/Longs derived from our OS data and translate them to the WGS84 datum used by Google Maps
Result for SA1 4LS
51.614743995912299, -3.964993028066230 ( Marker B )
Solution 2 : PHPcoord, JScoord, Jcoord (php, javaScript & java)
Yup.. same libraries as above..
Result for SA1 4LS (Airy1830)
51.6147522043, -3.96497954404 ( Marker C )
Putting all this together
I think this is probably far more easily explained on a map! .. so here it is:
- A = Before transformation;
- B = After transformation;
- C = phpcoord, jcoord, jscoord's transformed answer ( which is so similar it's obscured by B );
- X = Google's opinion of "SA1 4LS"
So.. is that it?
Well... yes... that's it: some solutions for turning Easting/Northing into Lat/Longs that can be used with Google Maps.
.. I just spent a long time banging my head against this so I thought it might help someone else if it were all in one place... or in yet another place!
( Incidentally, if anyone can explain why all the methods above agree on the position but Google's own estimate is different I'd be interested to hear. It seems ( having looked up the postcode in question ) that Marker "C" is in exactly the right spot so I'd be interested in understanding the discrepancy. )
Excellent Writeup!
Excellent Writeup! From a quick read looks good.
When I was doing something similar a few years ago, I ended up converting some code from the OS site to do the Helmert Transform, to go with Geography::NationalGrid::GB**, as didn't find Geo::HelmertTransform* at the time. Unfortunately never got round to tidying it up to be able to release the code :(
As for Google's answer I believe they source the postcodes from Teleatlas, which uses its own datasource! (possibly based on PostZon, but certainly uses data from their street gazetteer)
- Barry
www.nearby.org.uk
More checking..
Checking, using your e/n numbers against Geography::NationalGrid::GB and my transformation:
51.614301 -3.963824 (OSGB36)
51.614745 -3.964993 (WGS84)
Matches pretty well with Geo::HelmertTransform - but still slightly different to PHPcoord.
Also a PHP solution converted from the OS website, matches:
51.614745270442,-3.9649930274528 (WGS84)
5 figure easting/northing
I am attempting to convert a 5 figure easting and northing into a long/lat reference to plot onto Google Maps. You have advised to add a zero to the end and use the converter, however this is only accurate within 100m.
This has worked pretty much spot on.
Thanks
Interestingly...
Google Maps now pinpoints that postcode as slightly to the right of B & C, actually on Victoria Terrace.
End results
Thanks for that
We just bought the 1m data - example application.
Just thought you may like to see what you can do with the right info :)
Also worth pointing out is that there's another (rather dated) legal dataset available from the govt ONS dept that costs summat like a tenner (not too sure cos we bought the entire set)
The problem with the cheap data is it's only England + Wales and it's from 2004 with a 100m resolution
Rough beta 100m test app at http://orchid.peardox.com/portfolio.php
I have checked with the ONS and - yep - we're legally allowed to use this lot if we buy them from the ONS, we just can't distribute them (which I'd love to do)
Stats...
Govt geog 1m dataset = 1,765,071 records when you tidy the rubbish out of it
ONS geog 100m dataset = 1,281,376 records - but beware that you'll be missing new postcodes and it ain't got Scotland (still, good for a tenner or so)
Scotland (GROS) do their own census but don't have a postcode db in their version :(
I bow down to your greatness!!
Thank you for providing an idots guide to a quite complex problem that would have completely stumped me otherwise.
Thanks
Andrew
PHP Script for Easting/Northing to Lat/Long
I wrote some code a couple of years ago that helps convert from Easting/Northing to Lat/Long and I thought I'd share the link here incase anything finds it useful. My Google ranking isn't very high so it seems no one has found the code burried in my blog.
Jim
@Andrew
Thanks Andrew, that looks great!
Ben
OS OpenData
The OS data is also available from their OpenData site (http://www.ordnancesurvey.co.uk/oswebsite/opendata/ - click on 'download and order' at the bottom). This has a very open license and no cost. There are some disadvantages - Jamie Thompson has a comparison between the open and licensed products on his blog at http://jamiethompson.co.uk/....ordnance-survey/ . I think he's been a bit harsh since the data for Northern Ireland is available separately (see comments on that blog post) and Jersey/Guernsey are technically under a different postcode scheme although their postcodes look the same as ours. I've found these missing from commercial products before as well, and from what I remember last time I used a PAF file they weren't in there either.
Phil
More Geocoding wrestling
I've been doing a fair bit of wrestling with geocoding recently too but slightly differently. Here is a writeup of what I've been up to: http://philsturgeon.co.uk/news/2011/02/geocoding-apis-compared
Trevor
Finally found a solution :)
Been trying to find a way to do this northing / easting thing for a while in PHP after discovering the OS data and finally with a combo of this blog and Andrew's blog post it's sorted.
One thing that might be helpful is that once you've got your eastling / northing converted into coordinates, and you're wanting to use PHPcoord to make it Google friendly, you just need to pass it through the LatLng() function.
For example say you've got your OS coordinates in $lat and $lng and included() phpcoord, then you just do:
$lat_long_fixed_obj = new LatLng($lat,$long);
echo "Lat: " . lat_long_fixed_obj->lat;
echo "
";
echo "Lat: " . lat_long_fixed_obj->lng;
Took me a while to figure out how to do the conversion with PHPcoord as it has so many functions in there, but seems to do this conversion automatically.
Thanks for all the heads up on how to sort this problem...
Trev
Hittesh
Good one!
Hi Jim,
Really good article .
kudos for the resources provided ( PHP ).
Juliana
Juliana
Finllay! This is just what I was looking for.