AURA Software Tackles Uncertainty in Complex Systems

The software started as a way to evaluate the reliability of systems that look for anomalies in aircraft components and respond to them in flight. Today it is predicting how autonomous drones will behave and how drugs will move through the body, and its creators see no reason why it couldn’t be applied to any complex system: the stock market, the spread of an illness through a population, or the actuarial science of insurance pricing.

It’s software that predicts the future, or at least the probabilities of various outcomes. And it started at NASA’s Langley Research Center.

Called the Algorithms for Uncertainty Representation and Analysis (AURA) program, the software is based on polynomial chaos theory, a branch of mathematics so complex that it wasn’t much used until computing capabilities caught up to it in the 1990s.

By the mid-2000s, there was a lot of interest in developing advanced, intelligent, automated diagnostic and control systems that could detect and respond to aircraft damage and failures and thereby increase aircraft safety and autonomy. But no transition to new flight-safety algorithms is taken lightly.

“If you’re going to put a system on a plane that detects failure and responds, it is critical to understand that system’s performance so we can be certain it solves more problems than it creates,” says Alec Bateman, a principal research scientist at Barron Associates Inc., the Charlottesville, Virginia-based company that built the AURA software.

Such a system could miss a failure, give a false positive, or misidentify the component that failed, he explains. And even if the problem is identified correctly, the system could still take a corrective action that fails to solve the problem or even exacerbates it. Engineers want to know the probability and impact of any such glitch.

“Safety-critical resilient systems are developed to act when things go wrong, and a lot of things can go wrong in a lot of ways,” says Christine Belcastro, who, along with her identical twin Celeste Belcastro, worked in Langley’s Aviation Safety Program, where most of NASA’s work in this field was being done at the time. “It makes the problem very complex very quickly.”

Technology Transfer

Due to this complexity, at the same time that NASA was developing these autonomous systems, the Agency was also working on the means to validate them through analysis, simulation, and experimental testing, Belcastro says. This required new methods and tools, which is where AURA came in.

In 2007, Langley awarded Phase I and II Small Business Innovation Research (SBIR) contracts to Barron Associates to develop a software package that could measure the performance of integrated diagnostic and control systems—systems that the company had also been helping Langley develop. Belcastro oversaw the SBIR work. It built on a previous project that she and her sister had worked on with the company to adapt and improve a predictive technique known as Monte Carlo simulation, which relies on running millions or even billions of simulations with randomly altered variables.

This approach can take months to obtain meaningful results, is extremely expensive, and can miss worst-case combinations of uncertainties.

The 2007 SBIR contracts funded an effort to build on the earlier project and extend the uncertainty assessment capability to include probabilistic considerations. The idea was to come up with algorithms that could mathematically represent uncertainties and probabilities in a single calculation, using polynomial chaos methods. While this single calculation is very complex, it is much faster than running the huge number of simulations that would otherwise be needed. Polynomial chaos has been used in other areas where there are uncertainties in a system’s parameters, but it’s never been developed into a general-purpose tool, Bateman says.

With the SBIR funding, Barron Associates developed the software and successfully demonstrated it, testing it on a fault-detection and control system for Langley’s Generic Transport Model, a small-scale replica of a commercial aircraft.

The company built the program using the MATLAB computing environment, as well as the Simulink modeling and simulation package that integrates with MATLAB, both commonly used by engineers. Initially, though, AURA was difficult to use. “Until recently, it was hard to apply. Even a trained engineer would have a pretty steep learning curve,” says Michael DeVore, a senior research scientist at Barron Associates who helped develop the program.

As a result, few licenses had been sold by the end of 2018, and most commercial activity around AURA had come from the company using it to solve problems for Government agencies, primarily NASA and the military.

In 2019, however, the company rolled out a new, user-friendly version for sale. “We did a lot of work to integrate these tools with MATLAB in a way that would make them much more accessible to people familiar with that software,” Bateman says. The company also introduced a basic, free version of the program that’s available for download.

Benefits

For NASA and the military, the company has used AURA to continue verifying and validating advanced aviation safety systems. Work for the Navy has focused on autonomous vehicles like drones.

The Army hired the company to calculate probabilities of how a drug will be absorbed, metabolized, and excreted—a field known as pharmacokinetics. “There are a lot of uncertainties with the human body and how it’s going to react to a drug, and these are tied to different physiological parameters from person to person,” DeVore says.

The company has heard considerable interest from makers of autonomous vehicles like drones and self-driving cars. The technology for autonomy already largely exists or will in the near future, but it can’t be fully used until the risk of significant failure is proven to be acceptably low. “The benefit will be helping to move these systems toward reality,” Bateman says. “This technology is a key enabler for verification, validation, and regulatory approval.”

Belcastro retired from NASA in 2019 but is continuing her work as a distinguished research associate at Langley. She expects to incorporate AURA into work on resilience and safety assurance for multirotor drones, she says. This effort stems from research her twin started before dying of cancer in 2008, and Belcastro says she’s pleased to see it come to fruition and widespread application.

“This kind of validation tool opens the door to safety-critical resilient systems that will allow autonomous vehicles and autonomous aircraft,” Belcastro says. “Until you really solve the resilience problem and can evaluate the effectiveness of complex resilient systems when something goes wrong, you might as well not talk about having autonomous systems.”

Likewise, Bateman is excited at the healthcare possibilities that could arise from the ability to model the implications of physiological differences between patients.

“These are what I think of first, because that’s my background,” he says, noting that Barron Associates is atin the aerospace, aeronautics, and biomedical fields. The company has also discussed financial modeling with AURA, as well as validating the safety of automated systems at material handling facilities, Bateman says.

While there are other probabilistic analysis tools out there, he says, he doesn’t know of any that’s as easy to use and can be applied to virtually any complex system. “Hopefully, when we get the tool out in the world, people with other applications will use it for things we’re not familiar with.”