AI, Functional Safety and the Need to Get the Balance Right

As AI moves deeper into safety-critical systems, innovation is accelerating. The question is whether our ability to test, validate and assure those systems is keeping pace.

Artificial intelligence is advancing at an extraordinary pace.

Across aerospace, automotive, energy, transportation, manufacturing and other safety-critical industries, AI is increasingly being integrated into systems that have real-world consequences.

But as the technology advances, an important question needs to be asked:

Are we developing our safety capabilities quickly enough to keep pace with AI?

For those of us working in functional safety recruitment, this is becoming increasingly important.

Are We Moving Too Quickly?

AI has enormous potential. It can improve efficiency, automate complex processes, support decision-making and enable capabilities that were previously difficult or impossible to achieve.

But innovation should not come at the expense of safety.

Traditional safety engineering is built around understanding system behaviour, identifying hazards, assessing risk and demonstrating that risks have been reduced to an acceptable level.

AI makes some of these challenges considerably more difficult.

Systems can behave differently in situations that were not anticipated during development. Models can be difficult to interpret, data can change, and validating every possible operating scenario can be extremely challenging.

In safety-critical environments, we cannot simply assume that because a system performs well, it is safe.

When Safety Needs Time to Catch Up

I am not suggesting that AI development should stop.

But perhaps we need to slow down in certain areas, particularly where our ability to test, validate and assure a system is falling behind the speed at which it is being developed.

There needs to be greater consideration of:

• How AI systems are validated and verified
• How emerging hazards are identified
• How humans interact with autonomous decision-making
• How systems behave outside their expected operating conditions
• How changes to AI models are controlled and assured
• How responsibility is maintained when systems make increasingly complex decisions

Innovation needs safety alongside it, not behind it.

The Growing Importance of Functional Safety Professionals

This creates a significant challenge for organisations.

The demand for functional safety and systems safety professionals is likely to increase as AI becomes more deeply embedded within engineering systems.

We need people who can challenge assumptions, understand complex system behaviour and identify risks before technology reaches the operational environment.

Increasingly, organisations will also need professionals who can bridge the gap between traditional safety engineering, software, systems engineering and emerging AI technologies.

That makes recruitment particularly important.

Finding someone who simply understands a safety standard is no longer necessarily enough.

Organisations need people who can apply safety principles to increasingly complex and rapidly changing technologies.

Recruitment Needs to Evolve Too

As technology changes, so must the way we recruit safety professionals.

The strongest candidates may not always have the exact industry background or every keyword on a job description.

They may come from adjacent safety-critical sectors, bringing experience that can be transferred into new and emerging technologies.

Understanding that transferable capability requires a deeper understanding of engineering than simply matching a CV against a specification.

Knowing When to Accelerate and When to Slow Down

AI is not going away, and nor should innovation.

But we need to ensure that the development of safety assurance, regulation, testing and engineering capability keeps pace with technological advancement.

The organisations that succeed will not necessarily be those that move fastest.

They may be the ones that understand when to accelerate and when to slow down.

Because in safety-critical engineering, the ability to innovate is only as valuable as our ability to make that innovation safe.

Perhaps the future of AI isn’t about moving faster. Perhaps it’s about knowing when we need to stop, test, question and assure before moving forward.

If you’re building or scaling a functional safety or systems safety team across safety-critical engineering, reach out to Gen at gen@akkar.com to connect with specialist talent across the market.

Gen Richards - gen@akkar.com