Given a set of TransXchange XML files, returns the subset that represents the operative timetable for each service, discarding superseded revisions of the same service.
txc_filter_files(files, date = Sys.Date(), ncores = 1, quiet = TRUE)character vector of paths to TransXchange XML files
Date, the reference date used to decide which file version is operative (default `Sys.Date()`). For historical analysis set this to a date within the period you are studying.
numeric, number of cores used to read the file headers (default 1)
logical, if FALSE a summary of removed files is printed
a character vector, the subset of `files` to convert
Archives of TransXchange data (such as the Bus Open Data Service change archive) often contain several versions of the same registered service: each time an operator updates a timetable a new file is uploaded for the same `ServiceCode`, but the superseded files remain in the archive and usually still declare an open-ended `OperatingPeriod`. If all versions are converted, the same physical bus journey appears once per file version, so counting trips on a given date over-estimates service levels.
This function reads only the header information of each file (`ServiceCode`, `OperatingPeriod` start date, `RevisionNumber`, and `ModificationDateTime`) and keeps, for each `ServiceCode`:
For each distinct operating-period start date, only the file with the highest `RevisionNumber` (ties broken by the most recent `ModificationDateTime`) - repeated uploads of the same timetable period are duplicates.
Of the start dates on or before `date`, only the most recent - this is the version operative on `date`; earlier versions have been superseded.
All files whose operating period starts after `date` - these are future timetables that have not yet come into effect.
Files whose `ServiceCode` cannot be read are always kept.
Note one limitation: when a future timetable (kept under rule 3) eventually starts, it supersedes the currently operative file, but both are retained here because the operative file usually declares an open-ended end date. Trip counts are therefore reliable around `date` but may double-count dates after the next timetable change. For analysis of a specific period, set `date` inside that period.
This filtering is not needed when converting a normal single download of current data (where each service appears once), only when converting archives that accumulate every uploaded version.