Astronaut Experience Inspires Single-Handed Drone Flight Controller

NASA Technology

Astronaut Scott Parazynski has flown airplanes, climbed Mount Everest, and scuba-dived deep in a volcanic lake—but he says floating in space was like nothing else he’s ever experienced. He loved “the freedom floating in space gives you, the ability to move perfectly in 3D space”—and the experience helped inspire and inform his latest invention: a joystick that puts all the primary drone fight controls into a single hand and will one day, he hopes, revolutionize robotic surgery.

Parazynski spent 17 years as an astronaut and conducted seven spacewalks, including four in a single space shuttle mission, becoming the second astronaut ever to accomplish that feat. In the final, unplanned spacewalk of the 2007 mission, he went farther from the airlock than any previous astronaut to repair a damaged solar array.

Moving through zero gravity allowed maneuvering that just isn’t possible when gravity is a factor, he recalls, describing how you can manipulate your direction not just moving back and forth, sideways, and up and down (the x, y, and z axes), but through rotations (pitch, yaw, and roll). “There’s a precision of flight you get when you’re moving in space. It’s really missing in all the ways we move here on Earth.”

In total, Parazynski spent more than 47 hours spacewalking in his more than eight weeks in orbit, and he spent many, many more hours and weeks training for the experience. “I flew the robotic arm aboard the space shuttle, I helped assemble the space station arm,” he says, “and I spent a lot of time training in a virtual trainer getting ready for flight, using the translational hand controller and rotational hand controller for the robotic arm and simulator.”

This is standard operating procedure, explains Eddie Paddock, who manages the virtual reality laboratory where astronauts train at Johnson Space Center. For example, there are both ground simulators and onboard simulators for all the different robotics on the space station.

Many of those are two-handed, as Parazynski noted, with one hand controlling translational movement (forward, sideways, and vertical) and the other controlling rotational movement. Astronauts spend around six months training on the ground for operations like vehicle capture, Paddock says, and then will brush up again on board before the actual attempt.

Likewise, before any EVA (extravehicular activity, the technical term for a spacewalk), every astronaut is also required to complete four to eight two-hour training sessions, as well as an on-orbit refresher, on the SAFER system, the spacesuit jetpack that activates in an emergency if the astronaut’s tether to the station were to snap.

In the SAFER system (SAFER stands for Simplified Aid for EVA Rescue), a single, small hand controller pops out of the suit, Paddock says. The controller “controls four degrees of freedom in each mode: x, y, z, and one axis of rotation (pitch) in one mode, and then with a push of a button, can switch to rotation mode, to control pitch, yaw, and roll, as well as x.”

Technology Transfer

Parazynski took all the experience he had with astronaut robotics controllers as well as aircraft joysticks into his next venture as an entrepreneur. But it was his experience as a physician that first gave him the idea.

After he retired from NASA, Parazynski, who also holds a medical degree and trained in emergency medicine and trauma, took a position as the chief technology officer at a hospital in Houston. When he started to look at robotic surgical tools, he was not impressed.

“Of course a master surgeon can have great outcomes,” he emphasizes, and indeed robotic surgery has led to many safer, less-invasive surgical procedures. But the tools are clunky, he noticed, and require a lot of hands-on training on fundamental maneuvers. “If a surgeon only does a robotic case every other week, outcomes are not nearly as good,” he says.

He saw an opportunity to use his space and piloting experience with hand controllers to build something better. He teamed up with George Guerrero and founded a company, Houston-based Fluidity Technologies Inc., to create a better controller for robotic surgery.

“What’s really interesting, the core tech goes back to the Mercury Program,” Parazynski says, referring to joysticks for rotational control. In the earliest designs for the Mercury spacecraft, yaw (rotation around a vertical axis) was controlled with foot pedals. However, because of high g-loads, the way spacesuits restricted leg movement, and general weight and space limitations, engineers needed instead to create a three-axis rotational hand controller for yaw, pitch, and roll.

“Fast-forward several decades and additional innovation, and we have the FT Aviator, our first product, which allows us to move in up to six degrees of motion with a single hand.”

Benefits

Fluidity Tech launched the FT Aviator with a Kickstarter campaign in November 2018, raising over $100,000 and generating hundreds of pre-orders. But as the name suggests, it’s a controller for aviation, specifically for flying drones, rather than robotic surgery.

Surgery is still the ultimate goal, Parazynski says, but the process of getting FDA approval is long and arduous, so in the meantime, to launch the business and start turning a profit, the company decided to apply its controller to the drone market. “It’s one of the most rapidly growing industries and has a much lower bar for regulatory hurdles,” he says.

The FT Aviator is currently compatible with a number of models from drone-maker DJI, which offers a software developer kit designed to enable third-party accessories. But after a high-profile appearance at popular electronics trade show CES in Las Vegas in January 2019, where it was named a CES Innovation Awards 2019 Honoree, Parazynski said he was fielding meetings with other companies interested in partnering with Fluidity Tech to integrate the new controller—including helicopter manufacturers and surgical and medical manufacturing companies.

The biggest advantage of the new controller, Parazynski says, is that it is far more intuitive than the industry-standard, two-thumb joystick models. With FT Aviator, the pilot grips the controller with the whole hand, resting the thumb at the top. That means the thumb tip movement correlates to the movement of the aircraft: “You lift up your thumb, you’re going to climb. If you push forward, you’ll move forward. And so on. Within a matter of moments, your surrogate drone is your thumb tip and you’re weaving through the sky like Luke Skywalker,” he says.

In other words, “You can fly like an astronaut.”

The other main benefit is that the single joystick frees up the other hand to do non-flight-related tasks. The FT Aviator makes that very easy by building in camera controls to the base of the unit, as well as a mount for a smartphone or tablet to view the camera feed or manipulate any other relevant app.

That means a single person can accomplish tasks that previously required two sets of hands, which could provide significant cost savings in businesses where drone imagery is becoming increasingly important, from cinematography to real estate, roofing, and utilities inspections.

The same benefits apply beyond drones, Parazynski emphasizes, and he sees his one-handed controller taking on new industries in the near future. “We can use our technology in a wide range of applications, but we wouldn’t necessarily want or need to create a new controller for every application. We can be a provider of a smart human-machine interface—the underpinnings of the controller that other providers could then build around.”