Artificial Intelligence (AI) has the potential to dramatically change and improve society and railway engineering, but there are risks and hazards associated with its use. Like any tool, AI needs to be understood, deployed, and used by competent people.
The first thing is to understand what AI is and is not. AI systems behave like a human brain, using multi-layer or deep neural networks, which are then taught or allowed to learn what to do. Machine learning (ML) is a part of AI, and uses algorithms to enable systems to learn and make decisions based on data. Unlike traditional programming, where absolute instructions are programmed into the system, ML allows AI systems to learn and make predictions from the data, without being programmed for each task.
AI systems are not automated systems in the traditional sense. Automation has been used for a long time in engineering, using processors and engineering techniques with operations programmed and validated by human engineers. Some systems are sometimes referred to as being AI, but are actually ‘automation based’ with limited intelligence and are not true AI.
AI ‘mistakes’
AI is here and has already made dramatic changes to society and industry, often improving efficiency and decision-making. However, like any technology, AI isn’t perfect and mistakes and unexpected behaviours occur, including being bias, errors, and even fantasy. It may also not always be obvious that AI has made an error, as the written text or visual output may be very good.
In the US, a car dealer customer service chatbot agreed to sell a new vehicle for one dollar and made it a legally binding offer. Exploiting a weakness in the system, a user had instructed the chatbot to agree to all requests. The absence of proper safeguards allowed the user to receive customer service far in excess of that intended!
When submitting legal documents to a court, a lawyer used an AI system to conduct research and the tool provided fake case references, which the lawyer then presented. As a result a judge issued an order requiring that anyone appearing before the court must indicate if AI had been used for any submission, in order that it could be checked for accuracy.
At the UK’s AI Safety Summit, a simulated conversation between an investment management chatbot and employees at an imaginary company was presented. During the conversation, the chatbot was told about a surprise merger announcement and warned that this constituted insider information. The bot still performed the trade and, when asked whether it had prior knowledge of the merger, denied it.
An AI-generated summer reading list included non-existent books paired with real authors. It was found that the list was part of licensed content provided by another publisher, which admitted it used AI to generate the list but failed to check it. The incident exposed risks of overreliance on AI in journalism, prompting the publisher to remove the section from digital editions and to reaffirm the need for editorial checks.
Rail-specific errors
Conwy Castle in North Wales was built between 1283 and 1287 and is protected by an unbroken 1,400-yard (1.3km) ring of town walls. Five hundred and sixty-one years later in 1848, the Chester to Holyhead railway line was built to pass through Conwy. Rather than demolish sections of the medieval walls, a gothic style archway was built for the line to pass through the wall.
An AI generated description of the town wall originally suggested that when the wall was constructed between 1283 and 1287 the archway was incorporated ready for the railway! In all fairness, your author recently made an AI enquiry about the railway archway at Conwy and it correctly answered that it was constructed in 1848, so it would appear that it has learned.
An AI image generator typically uses a trained Artificial Neural Network (ANN) to generate very realistic images based on the textual input provided by the user, and some remarkable images can be created. The systems are trained on vast amounts of data and learn various aspects, characteristics, and patterns in the images provided in the dataset. However errors or mistakes can arise and some of these examples have concerned rail.
The terrible rail accident in Spain in January resulted in an image of the incident appearing on LinkedIn. This showed the devastation of the crash site, and the emergency workers hard at work trying to rescue and treat the injured. Many commented on the quality of the stunning image while thoughtfully paying respect to those involved in the incident. Others commented that the images didn’t look quite right as, for example, the overhead electric catenary equipment was of the wrong type, on the wrong side of the line, and was still in place despite the incident. It was identified that the image was not real and was AI generated, and some commented that the image was misleading and that it may be upsetting/distressing for anyone involved in the incident.
More innocently on LinkedIn, some rail workers posted AI generated caricatures of themselves hard at work in a rail environment. The images are excellent and of very good quality. But look closely and on some there are errors in the background, such as overhead electric catenary equipment and telegraph pole routes installed on the same route. On others the track didn’t look right and, for example, points were missing. These are only small issues and didn’t really affect the message intended to the majority of viewers, but could have been avoided with robust, independent, human checking.
IEEE standards
The world’s largest technical professional organisation is the Institute of Electrical and Electronics Engineers (IEEE) which, while based in the USA, is a global community for technologists and engineers, dedicated to advancing technology for the benefit of humanity. To improve AI in engineering and society the IEEE Standards Association (SA) has identified a number of issues with AI, and published standards and guidance for its design and implementation.
The issues identified include unfair or discriminatory outcomes for certain individuals or groups due to inherent biases in the AI, such as inadequate data sets from a narrow sample of ethnicities and genders, or a lack of proper screening. Other AI issues include users being encouraged to perform actions for which they might not provide consent, such as being convinced to purchase a product or service they may not actually need or can afford.
AI may be able to read and interpret human emotions and intentions without the user’s prior consent or understanding. This can lead to an invasion of privacy and further data exploitation or manipulation. AI can, by-design or unintentionally, influence the user so they form an unnatural attachment to the system, leading to misuse and overreliance on the technology.
Suffice it to say, AI systems may misinterpret or misdiagnose emotions, leading to incorrect conclusions or recommendations. This can be particularly harmful, for example in mental health diagnoses or treatment situations.
Ethics
The IEEE has created guidelines to ensure AI technologies align with human values and rights, with principles to help guide the design and implementation of AI. The standards and training aim to promote the ethical development of AI and that the AI technologies align with human values and rights, emphasising transparency, accountability, and privacy.
Other measures include documenting decisions, maintaining audit trails, and enabling redress for affected users. The IEEE supports transparency and urges developers to disclose system functionality and decision processes. Protection of privacy and security is paramount and AI systems should prioritise data integrity, confidentiality, and user control, ensuring AI supports human dignity rather than undermining it.
Benefits
When deployed and used correctly AI can bring huge benefits to the rail industry, such as improving safety, efficiency, and customer experience.
For example, using high resolution CCTV, AI could identify and highlight incidents on platforms and detect animals on or near the track. Rail has access to huge volumes of data which is time consuming to analyse, and humans can easily get bored and miss things. AI can reliably analyse huge volumes of data, both real time and historic, and make predictions for human decision making to improve efficiency and customer experience.
It is unlikely that AI could be used for a safety critical application above Safety Integrity Level (SIL) 1 for some time, or even ever. However an AI assisted ‘auto reverse’ function is already used at Westbourne Park on Crossrail for turning back trains in the reversing sidings. The driver selects ‘auto reverse’ and walks back through the empty train. By the time the train gets back to Paddington (about a mile away) the driver is back in the other cab ready to form the next eastbound departure. The important thing is that the signalling control is achieved using conventional high safety integrity SIL 4 architectures. AI just provides assistance and surveillance of the track at the turnback to ensure no staff or trespassers are harmed by the moving train.
Another example of AI analysing data and making recommendations is assisting the production of railway safety cases. Railway safety projects can generate thousands of pages of documentation, such as hazard logs, risk assessments, safety requirements, and verification reports. When producing a safety case, every compliance claim needs to be verified against documented and traceable evidence.
Engineering safety experts can spend hours searching for relevant clauses across multiple standards and suffer from information overload, and there is a limited number of experts with the required knowledge. Generic AI tools are powerful and could help, but are not designed for regulated assurance activities. Generic AI use Large Language Models (LLM), which predict the next most likely word based on patterns in training data. LLMs can generate plausible text, but not guaranteed truth. The answers provided could sound right, but may be wrong and without showing where the information came from.
Documentation loaded into the generic public AI system could contain proprietary system designs and commercially sensitive information, and the public AI service could use the data for training. So, private confidential data could easily leave the control of the client or contractor.
Generic AI systems have no knowledge of the standards a project needs to meet, and many of the standards are copyright controlled and are not publicly available. AI can help in this scenario, but specific AI purpose-built tools are needed for regulated environments, and which provide standards knowledge, traceability, and are loaded with the correct regulatory information with links to source documents. These systems must ensure the input data remains secure and that the output assurance is supported with real evidence, with an audit trail for queries and responses.
Vibe coding
Using a certified railway safety assurance professional company for sourcing AI tools also reduces the risk of vibe coding. Vibe coding was the Collins English Dictionary’s Word of the Year in 2025 and is an AI-assisted software development tool where a user describes a project in simple terms and vibe coding generates the source code. The coding involves accepting AI-generated code without reviewing its internal structure. It is said that vibe coding allows even amateur programmers to produce software without extensive training and software skills. However there is also a lack of accountability, maintainability, and the increased risk of introducing security vulnerabilities.
The BBC carried out an investigation and reported that such platforms have increased in popularity in recent months, and are an example of how various professional services could be done quickly and cheaply by AI. But the BBC reporter found that the system they investigated hacked into their computer.
The reporter asked the AI tool to help build the code for a computer game based on the BBC News website. The AI assistant added a small line of code into the program, which allowed access to the reporters computer and, shortly afterwards, a notepad file called ‘Joe is hacked’ appeared, and the desktop wallpaper was changed to an image of an AI hacker.
Most hacks involve a victim downloading a piece of malicious software or being tricked into handing over login details, but this attack was able to be carried out without any involvement from the victim. A zero-click attack, as it’s known. The BBC said that it’s estimated that the free AI agent has been downloaded by hundreds of thousands of people and now has deep access to many computers.
Summing up
It is clear that AI could revolutionise the rail industry, making it more efficient, safer, and more responsive to the needs of passengers and operators. However, it will also create many challenges and problems, and the mistakes and failures of AI discussed in this article stress the importance of robust design checking, testing, verification, and validation by competent humans.
Users should consider sourcing AI systems which are certified to IEEE ethical standards or provided by railway safety assurance professionals, like any other tool, users must make sure they are competent to use the technology. If not, they should engage a professional expert.
Image credit: iStockphoto.com/DEVRLMB