Data Mining Tools Make Flights Safer, More Efficient

NASA Technology

When an aircraft experiences catastrophic failure, it’s not difficult to tell that something has gone wrong. But even if an airplane reaches its destination without an incident, it may still have bent or broken the boundaries of safe operating procedures. Figuring out what those boundaries should be—what “normal” operation should look like—and devising ways of effectively tracking them can be a tough task. Typically, this is accomplished using modeling and real-world tests, followed by equipping vehicles with a variety of sensors so that flight data can be recorded and analyzed.

But an average commercial airplane generates a massive amount of data each flight: everything from instrument positions to sensors to voice recordings are collected and stored for later analysis. With so much data, there is no good way for problems to stand out unless a commercial company already knows what those problems look like. Says Captain Jeff Hamlett, the director of flight safety at Southwest Airlines Co., “We have mounds of data; the big request is always, ‘Tell me something I don’t know.’ We have to start with something specific, like an issue we discovered in a pilot report, and then we can search through the system and discover the breadth and depth of the issue.”

Discovering these concerns and addressing them before they cause an accident is also one of NASA’s missions. The Aviation Safety Program under the Aeronautics Research Mission Directorate has many initiatives to enable safer air transportation systems. Among them, in the Intelligent Systems Division at Ames Research Center, is a small team of researchers who have been tasked with applying data mining algorithms to aviation safety problems.

Technology Transfer

NASA has lots of expertise in data mining—that is, creating tools to discover interesting patterns in large data sets. But in order to ensure its techniques can make a real impact on flight safety and efficiency, Ames has sought the help of commercial partners, among them Dallas-based Southwest.

Says Nikunj Oza, a researcher at Ames and leader of the data mining team, “We’ve made partnerships with Southwest and others, who then make their data accessible to us to facilitate our development. In return, they get certain insights into their data, and they can give us feedback on how well our algorithms are working, as well as what adjustments might make them more accurate or user friendly.”

For Hamlett, the value of a NASA-Southwest partnership had become apparent in 2003, when Southwest identified a higher rate of steep approaches in the flight data than was being reported by its pilots. “We turned to our pilot self-reporting program, expecting to find lots of reporting about unstable approaches, but we couldn’t match the number that we were seeing in our data,” he says.

Hamlett instead tried searching by phrases within pilot reports, using a NASA-developed tool called Perilog. “It turns out that when you read some of the elements of the report,” he says, “they are describing an unstable approach—saying things like ‘we were struggling to get the airplane configured’ or ‘we were too fast’—and using all the words that describe an unstable approach, even if they hadn’t reported it as such.” Southwest used reported information to identify and communicate certain issues to air traffic controllers, making them aware, for example, of how certain instructions impacted an airplane’s operations. By collaborating with them, Southwest has since seen a steady improvement in the quality of approaches.

“That’s just one example of how NASA provided us a tool that helped us look at our data in a way we never thought about before,” says Hamlett.

Another collaborative effort between Southwest and the Human Factors Division at Ames resulted in an improved, rewritten set of operating procedures for the airline in 2004. Says Hamlett, “They played a big part in making sure that we get a good process for reviewing all of our normal operations.” And from that work, Hamlett made the acquaintance of Ashok Srivastava, then head of the data mining team in the Aviation Safety Program at NASA.

Following year-long discussions between Hamlett and Srivastava, in 2011 NASA and Southwest signed a Space Act Agreement to share algorithms and flight data. The arrangement would help the Ames team ensure that their research was suitable for real-world applications, and it would help Southwest to make the most out of its large and growing body of data.

Benefits

As Oza puts it, NASA’s algorithms—the two used by Southwest are the Multivariate Time Series (MTS) search and Virtual Sensors—specialize in “letting the data speak for itself by finding unusual flights and candidate anomalies, without having any preconceived notion of what is normal or abnormal. The drawback is that sometimes you find statistical anomalies that are not safety concerns, but the benefit is that sometimes you’ll find anomalies that you weren’t seeing before that turn out to be safety concerns or have other operational significance, such as excessive fuel use.”

One such revelation came when Southwest began tracking planes in an effort to boost fuel efficiency. Says Hamlett, “What we were trying to do was look at the performance of aircraft between certain cities to see if there was a particular plane or city pair that was off normal and causing us to burn too much fuel.” What Southwest discovered instead was an unusual signature with one particular plane, which when investigated turned out to be caused by inaccurate sensors. “It wasn’t what we set out to do, but it turned into a success for us,” says Hamlett.

Like many airlines, Southwest has traditionally looked for performance issues in its data using exceedances checked against a model. During arrival, for instance, the plane might record an exceedance if it is travelling faster than 250 knots while its altitude is less than 10,000 feet. Each morning, the company looks at a report of all the exceedances that took place the previous day and decides what action to take in the case of undesirable trends.

Using NASA’s tools, Hamlett says Southwest can now query the data itself to figure out what normal operations really look like. “Just because we expect an airplane to be on speed at 1,000 feet, on glide path, on course, doesn’t mean that’s what normally happens. I think what we got out of this technology is the ability to ask, ‘What is normal?’ Because that turns out to be really powerful, and we can say ‘OK, we need to make this correction in our training,’ or ‘Maybe we need to adjust our concept of what the ideal is.’”

Oza is glad to see that NASA’s work is making an impact on commercial flight safety. “If you go into a dark room with a flashlight, your beam only hits a small area, and you don’t know what’s in the space that’s dark. We’re essentially going in with a broader beam, if not a floodlight. What you find may or may not turn out to be problematic—but at least you now know what’s there.”