The business of railway engineering: was it worth it?

The project has been completed, hopefully on time and to budget. To have been approved by the investment panel, it must have had a good business case and so will undoubtedly benefit the local community. So, is that the end of the story?

Transport Scotland doesn’t think so. For years it has applied its Scottish Trunk Road Infrastructure Project Evaluation (STRIPE) to its road schemes. Amongst other things, STRIPE considers whether a project’s benefits are as predicted and if there are lessons to be learned for future projects. It takes account of HM Treasury’s Green and Magenta Books on project evaluation.

Having seen record investment in Scotland’s railways over the last 20 years, Transport Scotland is now applying a similar approach to rail projects. It is felt that these have traditionally been well developed at the objectives and appraisal stage prior to construction, but less so in respect of evaluation. As a result, guidelines for the evaluation of rail projects have been drawn up.

To test and refine this guidance, three pilot evaluations of recent rail projects were undertaken in 2013/14. These were Laurencekirk station, the Larkhall to Milngavie line, and the Airdrie to Bathgate line.

Laurencekirk’s new station

The old Laurencekirk station closed in 1967. In 2009, a new station and Park and Ride facility was provided at a cost of £3.5 million – £0.75 million under budget. The objectives of the project were to link rural commuters with employment and education, to encourage greater use of public transport through a shift from car to rail, and to improve road safety by a reduction in road traffic.

The 74-page evaluation report included a detailed analysis on the station’s impact on the town and surrounding area. It considered that, overall, Laurencekirk station had performed well against its objectives. In its first full year, 75,000 passengers used the station compared with the predicted 36,000. Of those commuting from the station, 56% would have driven all the way to work had the station not re-opened and, of the station users who had changed jobs, 62% stated that the station was the main or an important factor in doing so.

The report, however, did note that the project’s objectives were not measurable and noted that it was difficult to demonstrate a direct improvement in road safety.

Larkhall to Milngavie

This project opened in 2005 and, at a cost of £35 million, provided two new sections of track: the re-instatement of 4.7km of track to Larkhall complete with two new stations, and a 1.6km extension of Glasgow’s northern suburban line from Maryhill with one new station. Its objectives were to reconnect Larkhall with the rail network, to increase service frequencies by removing an operational bottleneck, and to improve attractiveness for inward investment and land development through increased social inclusion and a shift to public transport.

Following its opening, the frequency of trains to Milngavie and Hamilton has doubled to four trains an hour. Demand at the new stations in 2012/13 was 614,800, 26% higher than forecast, while 49% of commuters in the Larkhall area said they would have driven to work if new stations had not been provided. Overall, rail journey times have been reduced by 10 minutes for 60% of the Larkhall and Hamilton population.

The evaluation report considered both local impact and that on the wider economy. It concluded that the project has delivered value for money and has met its first three objectives – those that related to the rail network. However, it noted that, in the absence of quantitative targets, it was difficult to judge the achievement of other objectives which concerned social and economic benefit, although there was evidence of a positive impact such as an increased number of houses built at Larkhall.

Airdrie to Bathgate

Opened in 2011, the Airdrie to Bathgate project re-instated 15 miles of railway between the two towns to create a new rail link between Glasgow and Edinburgh. It also double-tracked parts of the existing railway and electrified the line throughout. It provided three new stations and cost £300 million.

Its objectives were to provide people along the line with direct access to jobs in Glasgow and Edinburgh, encourage inward investment, promote social inclusion, increase the use of public transport in Central Scotland, offer an alternative to the M8 motorway in order to reduce road congestion, and create an alternative to the main Edinburgh to Glasgow line, hence increasing capacity.

Since its opening, train frequency on the western end of the line has increased from four to six trains an hour and on the eastern end has been doubled to four trains an hour. For the four years up to 2013/14, there has been a 31% increase in passengers, compared with the national average growth of 14%, and an 8% increase in those travelling to work along the line.

A stage 1 ‘light touch review’ has been prepared which gives an early indication of its performance against objectives. Because it takes time for project impacts to work through, a further stage 2 report will be undertaken after the line has been open for five years.

The initial report concluded that the project was meeting most of its objectives, especially that of increasing use of public transport. However, in some cases, it was not possible to show the direct impact of the project. With the completion of the missing link between Airdrie and Bathgate, westbound journey times from Bathgate and eastbound times from Airdrie had been slashed. Evidence collected showed people without cars were using this link to make journeys they would not otherwise have done. Hence the new line is helping social inclusion and increasing access to jobs. However, there is no evidence that employment has increased as a result.

Since the line opened, West Lothian’s inward investment increased although North Lanarkshire’s remained static. The report acknowledged that it was too early to assess the project’s impact on inward investment. In respect of its last objective, it did, however, conclude that, with some commuters transferring to the Airdrie to Bathgate line, there was a slight reduction in overcrowding on the eastern section of the main Edinburgh to Glasgow line via Falkirk.

Cause and effect

These evaluations all showed that the three projects had been successful in terms of, for example, passenger numbers. However, it was difficult to demonstrate that social objectives had been met. A particular problem in this regard is assessing the extent to which a project has been responsible for a particular change, which requires an assessment of what would have happened had the project not taken place. This is not easy, as it requires an assessment of something that, by definition, cannot be observed. For example, would the houses that were built close to a new railway station have been built had there been no new station?

Transport Scotland’s guidance on the evaluation of rail projects explains how this problem can be addressed by, for example, using comparative data and, if possible, establishing a baseline before the project is announced. It describes what might have happened in the absence of a project as ‘counterfactual’ and recognises that establishing the counterfactual is not easy, since by definition it cannot be observed.

Lessons for future evaluations

In addition to assessing the three projects, these evaluations also tested the rail evaluation guidance that Transport Scotland has issued to see how it could be improved. Lessons from this included the need to set measurable objectives at the outset and to identify the data required afterwards by establishing monitoring plans, by archiving project documentation effectively and by funders and stakeholders collecting data after implementation. In practice, this requires a rigour and commitment for something that will be done perhaps ten years after a project’s initial feasibility study, at a time when project implementation is not certain.