vignettes/transxchange.Rmd
transxchange.RmdThis vignette explains how to convert TransXChange timetables into GTFS, the difference between the two main sources of TransXChange data (Traveline and the Bus Open Data Service), and the pitfalls to watch out for.
TransXChange (often abbreviated TXC) is the UK Department for Transport’s XML standard for describing bus, coach, tram, metro and ferry timetables. It is the format that operators must use to register services with the Traffic Commissioners, so almost every scheduled road-based service in Great Britain is described in TransXChange at some point in its life.
Because it was designed for registration rather than for journey planning it has some awkward properties:
GTFS, by contrast, is a flat set of CSV files designed to be consumed directly by journey planners such as OpenTripPlanner, R5 and Google Maps. Converting TransXChange to GTFS makes the data usable by that wide ecosystem of tools.
There are two national sources. They contain broadly the same underlying registrations but differ in packaging, freshness and coverage.
Traveline
publishes the Traveline National Dataset (TNDS). You must apply
for free access to their FTP server. The data arrives as zipped folders,
one per region of Great Britain (for example EA.zip for
East Anglia, S.zip for Scotland).
The Bus Open Data Service (BODS) is the DfT’s newer, statutory feed. Since 2020 bus operators in England are legally required to publish their timetables here, so it is the most complete and up-to-date source for English buses. You can download TransXChange directly, or download a ready-made national GTFS file produced by ITO World’s own converter.
UK2GTFS. The package adds extra data checks and supports
the non-bus modes that the ready-made BODS GTFS omits.The workhorse is transxchange2gtfs(). In its simplest
form:
library(UK2GTFS)
path_in <- "EA.zip" # a TNDS region zip, or a folder of .xml files
gtfs <- transxchange2gtfs(path_in = path_in, ncores = 3)Key arguments:
path_in - a single zip folder, or a character vector of
paths to .xml files.ncores - if > 1, multi-core processing
is used to speed things up. Always leave one core free for the operating
system.try_mode - if TRUE (the default) files
that fail to convert are skipped rather than aborting the whole run.
This is robust, but be aware the result may be missing some routes -
check the messages.cal / naptan - the bank-holiday calendar
and stop locations. By default these are fetched for you with
get_bank_holidays() and get_naptan(). If you
are converting many files it is much faster to fetch them once and pass
them in (see below).scotland - "auto", "yes" or
"no". Controls whether Scottish bank holidays are used.
"auto" treats a file ending in S.zip as
Scottish.filter_duplicate_files - see Duplicate and superseded
files.Once converted, save the GTFS to disk:
gtfs_write(gtfs, folder = "C:/GTFS", name = "gtfs_EA")This writes C:/GTFS/gtfs_EA.zip.
get_naptan() and get_bank_holidays()
download data from the internet. When looping over several regional
zips, fetch them once and reuse them:
library(UK2GTFS)
naptan <- get_naptan()
cal <- get_bank_holidays()
files <- list.files("path/to/tnds", pattern = "\\.zip$", full.names = TRUE)
for (f in files) {
message(f)
gtfs <- transxchange2gtfs(
path_in = f,
cal = cal,
naptan = naptan,
ncores = 3
)
name_out <- gsub(".zip", "", basename(f), fixed = TRUE)
gtfs_write(gtfs, folder = "path/to/output", name = name_out)
}transxchange2gtfs() is a convenience wrapper around
several lower-level steps, which you can also call individually:
transxchange_import() reads each XML file into R data
frames.transxchange_export() turns those into a GTFS
object.gtfs_merge() combines the per-file GTFS objects into
one.gtfs_write() saves the result.Each TransXChange file is converted into its own small GTFS object and these are then merged. Understanding this matters because the merge is where problems tend to surface.
TransXChange files reference stops only by their ATCO code; they do
not contain coordinates. UK2GTFS therefore
looks each stop up in the NaPTAN
(National Public Transport Access Nodes), the DfT’s database of every
public transport stop in Great Britain.
get_naptan() always downloads the latest copy. The
NaPTAN is not perfect - it misses some stops and has wrong locations for
others - so UK2GTFS ships two internal correction datasets,
naptan_missing (stops absent from the NaPTAN) and
naptan_replace (better coordinates for known-bad stops),
which are applied automatically. Contributions of new or corrected stops
are welcome via the UK2GTFS-data
repository.
Because each file is converted separately and then merged, the merge
can fail on small inconsistencies between files - most commonly when the
same operator is spelt two different ways (for example
"Yorkshire Coastliner Ltd" versus
"Yorkshire Coastliner"), which looks like two agencies
sharing one ID.
If transxchange2gtfs() cannot merge cleanly it returns a
list of GTFS objects instead of a single object. You
can then merge them manually, forcing past the conflict:
gtfs <- gtfs_merge(gtfs_list, force = TRUE)force = TRUE keeps the first instance of each duplicated
ID, which is almost always the correct behaviour. You can also set
force_merge = TRUE in the original
transxchange2gtfs() call to do this automatically.
This is the single most important thing to understand when working with bulk archives (especially the BODS change archives).
Every time an operator revises a timetable they upload a
new TransXChange file for the same
ServiceCode, but the older, superseded files usually remain
in the archive and often still declare an open-ended operating period.
If you convert all of them, the same physical bus journey appears once
per file version, so counting trips on a given date
over-estimates service - in one test archive a single route appeared
five times.
A normal single download of current data does not have this problem (each service appears once), so you only need to worry about this for accumulating archives.
Set filter_duplicate_files = TRUE to keep only the
operative version of each service:
gtfs <- transxchange2gtfs(
path_in = "bods_archive.zip",
filter_duplicate_files = TRUE,
filter_date = as.Date("2024-06-01"), # a date inside the period you care about
ncores = 3
)For historical analysis, set filter_date to a date
within the period you are studying - the filter keeps the version that
was operative on that date. Under the hood this calls
txc_filter_files(), which you can also run directly on a
vector of file paths; see ?txc_filter_files for the exact
rules and their one documented limitation (trip counts can still
double-count after the next scheduled timetable change).
UK2GTFS output has been validated against BODS’s own
(ITO World) GTFS. Where both feeds cover the same route, agreement is
high - around 80% of shared operator/line pairs had identical daily trip
counts and matching departure times. The remaining differences are
mostly about which input files are included (superseded
revisions, out-of-region services, and coach/ferry datasets that live
outside the TransXChange archive) rather than the conversion itself. In
short: both are reasonable, and having an independent open-source
converter is useful precisely because TransXChange-to-GTFS involves
interpretation.
Once you have a GTFS file you will usually want to clean and check
it. See the Working with GTFS files vignette for
gtfs_clean(), gtfs_validate_internal(),
gtfs_clip() and the analysis functions.