In Cloud Computing, Open Source Becomes Big Business

In 2010, open source software, once the playpen of renegade hackers and hobbyists, had already gone mainstream. Cloud computing was newer and less defined, but already commercially available. That year, a small team of engineers at NASA’s Ames Research Center in California’s Silicon Valley brought the two neatly together.

The result, a collaboration between NASA and Rackspace Inc., was called OpenStack. Since its release 10 years ago, open source cloud computing has steadily gained popularity, with many of the world’s largest companies making the switch.

“That original work has expanded across most industries, because it solved a general problem and opened that solution up to let people expand on it,” said Brian Gracely, senior director of product strategy at Red Hat Inc., based in Raleigh, North Carolina. Red Hat marketed its own version of OpenStack and subsequent open source cloud computing products and continues to be a player in that field.

Outsourcing to the Masses

The general problem the Ames team set out to solve was to find a way for computing power and storage to automatically scale up and down according to the needs of individual projects and users across a large number of virtual servers. This is the essential definition of cloud computing, or infrastructure as a service. They called the project Nebula.

Such systems already existed, including Amazon Web Services’ Elastic Compute Cloud, but they were proprietary, used on a privately owned cloud where subscribers bought space. The team at Ames wanted NASA to be able to host and manage its own computing and data.

“NASA didn’t have the staff to build the whole thing and maintain it long-term,” said Gracely. “It’s a problem a lot of companies had, too.”

NASA decided to take the open source route, developing the code online in the public eye, where any other programmers can contribute and no one owns the code.

After a few years of false starts and on-and-off work, the team decided to create its own “fabric controller,” the brains that would manage computing, storage, and networking. The result was called Nova.

“Launched Nova,” one of the Ames contractors wrote in a blog post, noting that the code was open source. “It's live, it's buggy, it's beta. Check it out.”

When engineers at the cloud computing company Rackspace did just that, they recognized a code that took the same approach they’d been working on. The two groups teamed up and, less than two months later, announced the release of OpenStack.

“The best, most flexible software gets built in the open source community,” said Gracely. “If I have a common problem, we can bring all our engineering talents together and solve it at a lower cost to everyone. If you have a unique aspect to your problem, you can tweak the code to your need, and now others with your need can use it.”

Red Hat’s free version of OpenStack, derived from the NASA code, launched in 2013 and has proven popular, especially among major nonprofits and research institutions. Perhaps the most famous use case is at CERN. The European particle physics laboratoryuses it to run calculations on the 100,000-core computer cluster for the Large Hadron Collider that made the first observations of a long-theorized particle called the Higgs boson.

Red Hat also sells an enterprise version of OpenStack with additional features and support for businesses.

Automation for Applications

During the initial development of OpenStack, the Ames team also made an attempt at creating a sort of second layer of code — an automated platform for building and deploying applications on a cloud infrastructure.

“It’s very hard for infrastructure people who don’t create applications to solve problems for an applications team,” Gracely said. The abandoned attempt signaled a need that the market could fill. Less than a year after OpenStack’s launch, Red Hat released its first version of OpenShift to help fill that niche.

OpenShift is a set of tools for developing and maintaining cloud applications. As such, its main job is to orchestrate application containers — standardized self-contained units that hold all the code and software an application needs to run in the cloud.

Gracely described OpenShift as “a sort of second generation” that arose to take better advantage of cloud software like OpenStack.

“Previously, a developer wrote an application and then had to think about how it talks to the network, how it talks to the storage system and database, how to make sure it can scale to more users, how to make it highly available, for example if a server dies,” he said. OpenShift automates these tasks. “Give it your application, tell it how you want it to run, and it takes care of all those complexities.”

OpenShift can be — and often is — run on an OpenStack cloud, but it can be used in any computing environment. It also builds on OpenStack in more direct ways. For example, while it doesn’t use the NASA-invented Nova component that still handles computing at the heart of OpenStack, it incorporates other OpenStack components that descended from Nova.

Back at You, NASA

NASA’s Langley Research Center in Hampton, Virginia, is now an OpenShift user, as the secondary benefits of Ames’ work a decade ago continue to reverberate through the space agency.

The Atmospheric Science Data Center, or ASDC, at Langley is working to restructure its applications and services to run on the OpenShift platform. “We’re trying to think about how we can be more ready to reap the benefits of cloud technology to be flexible and agile,” said Jeff Walter, systems architecture lead at ASDC.

The center houses five to six petabytes of data on local storage arrays and is preparing to migrate some of its data to another cloud service. But Walter says it’s not the volume of data that’s driving a proposed switch to a local OpenShift cloud platform, so much as a desire to improve efficiency and prepare for the future.

Currently, one project might have a server running at capacity while another server sits idle. “It would be nice for the first project to be able to use some of the other project’s hardware,” said Walter.

Virtual servers would be able to operate across physical servers or coexist on a single machine. OpenShift’s automation would eliminate work and human error. “It’s about having a centralized environment that allows developers to think about an application and how it’s structured from a business point of view and not worry about how and where this thing is going to run,” Walter said.

“By migrating to Openshift, we are preparing our applications to be highly portable and run in any number of possible environments, including on-premises or in any major commercial cloud with little or no rework,” he added. The group is also weighing whether it might run OpenShift on an OpenStack infrastructure.

Meanwhile, NASA’s Jet Propulsion Laboratory in Southern California has been using Red Hat’s OpenStack platform in its on-site cloud since 2016.

Today, the World

Red Hat’s free version of OpenStack now has well over 100,000 community members and is supported by about 700 companies in 185 countries. More than half of the company’s OpenStack customers use it in conjunction with OpenShift.

“We have OpenStack customers in every sector in the cloud computing industry,” said Nick Barcet, Red Hat’s senior director of technology strategy. “Telecom providers have the largest share of the deployments, but from healthcare to manufacturing, OpenStack is delivering value in a very wide variety of cases.”

OpenStack opened up cloud-building to the masses, and Red Hat is one of many companies that benefited from the shift, with most of its products now built around open source cloud computing. “At the time of that NASA work, it was really unique,” said Gracely. “We’re thankful they did it, and we’re thankful they’ve remained a partner and a customer today.”

NASA has a long history of transferring technology to the private sector. The agency’s Spinoff publication profiles NASA technologies that have transformed into commercial products and services, demonstrating the wider benefits of America’s investment in its space program. Spinoff is a publication of the Technology Transfer program in NASA’s Space Technology Mission Directorate. For more information on how NASA is bringing its technology down to Earth, visit: spinoff.nasa.gov