This vignette explains how to convert the National Public Transport Data Repository (NPTDR) archives into GTFS. NPTDR is the only practical source of a single, whole-of-Britain, all-modes historic snapshot, which makes it valuable - and slightly awkward - to work with.
Background: what is the NPTDR?
The National Public Transport Data Repository was an annual snapshot of every public transport journey in Great Britain - bus, coach, rail, tram, metro and ferry - taken during a selected week each October. It was produced between roughly 2004 and 2011, after which it was discontinued (its role was effectively taken over by the live Traveline and, later, Bus Open Data feeds).
Its distinguishing features:
- Comprehensive and multi-modal. One archive covers the whole of Great Britain across all modes in a consistent format - unusual, and the main reason to use it.
- Historic only. The newest snapshot is from around 2011, so NPTDR is for studying the past (network change over time, services that no longer exist, before/after analyses), not for current journey planning.
- A snapshot of one week. Each archive represents a single sample week, so the calendar it produces is short.
Why the format is fiddly
NPTDR distributes the timetable in the CIF format -
but a different dialect from the heavy-rail CIF covered in the
Heavy Rail CIF vignette. This NPTDR/ATCO-CIF
dialect uses record types beginning with Q (QS
service header, QO origin, QI intermediate,
QT terminating, QL/QB locations,
QE exceptions, QR repetitions). An NPTDR
archive is a nested set of zips:
- one CIF file per administrative area per mode,
grouped in
Admin_Areazips (so a single archive contains many CIF files); - a NaPTAN stops export for stop locations;
- various supporting files.
UK2GTFS unpacks all of this for you.
Where to get the data
The historic NPTDR archives are no longer served from a single
official portal. Copies are held at the UK Data Archive and various mirrors;
you download a per-year zip (for example October-2006.zip).
A small sample archive suitable for testing the package can be
downloaded with:
library(UK2GTFS)
dl_example_file(".", type = "nptdr")Converting NPTDR to GTFS
The single entry point is nptdr2gtfs():
library(UK2GTFS)
gtfs <- nptdr2gtfs(path = "October-2006.zip")
gtfs_write(gtfs, folder = "C:/GTFS", name = "gtfs_nptdr_2006")Arguments:
-
path- path to the NPTDR year zip. -
enhance_stops- ifTRUE(default) the current NaPTAN is downloaded and used to fill in any stops that are missing locations in the historic archive. See the note below. -
naptan- the NaPTAN stop locations, by default fetched withget_naptan(). Pass a pre-loaded copy to avoid re-downloading. -
n_files- a debugging aid: an integer vector selecting a subset of the CIF files (for example1:10) so you can test on a fraction of the archive before running the whole thing. -
silent- suppress progress messages.
Because an archive contains many CIF files covering the whole
country, a full conversion processes a lot of data - test with
n_files first.
Stop locations, old and new
The archive ships its own NaPTAN export, and the CIF files themselves
carry approximate coordinates (in British National Grid
easting/northing, which UK2GTFS reprojects to WGS84). These
historic locations are used first.
Some stops referenced in the timetable will still lack a location -
typically old, temporary or moveable stops. With
enhance_stops = TRUE the present-day
NaPTAN is consulted to fill gaps. Be aware of the trade-off: this
improves completeness but mixes a modern stop location into a historic
dataset, and stops that have since been removed simply cannot be
recovered. Any that remain unmatched trigger a warning; you can strip
them out afterwards with gtfs_clean():
gtfs <- gtfs_clean(gtfs)Things to watch for
-
Short calendars. Because each archive samples a
single week, the resulting GTFS
calendar/calendar_datescover a narrow date range. When you analyse the feed (see Working with GTFS files) make sure your analysis dates fall inside that week, or you will get empty results. -
Historic bank holidays. The conversion uses a
bundled
historic_bank_holidaysdataset to resolve operating profiles correctly for the year in question. -
Data quality. These are old files with occasional
malformed times and missing stops; the importer repairs the common cases
and warns about the rest. Expect to run
gtfs_clean()andgtfs_validate_internal()afterwards.
Next steps
Historic feeds are most useful for analysis and comparison. See the
Working with GTFS files vignette for counting
services (gtfs_stop_frequency(),
gtfs_trips_per_zone()), clipping to an area
(gtfs_clip()) and validating the result.
