Leveraging OpenNebula for SMEs: Terradue’s Transition to DevOps Enabled Products and Services

OpenNebula is a major open-source cloud-computing software platform. The overall functions of this system are to manage the provisioning of virtual machines, within a data center delivering infrastructure-as-a-service. For a fast-growing SME like Terradue which needs to host and scale DevOps activities, this is an alternative to commercial Clouds and a very effective way of deploying a Private Cloud, that moreover allows us to contribute enhancements and build dedicated solutions for our customers.

Terradue’s DevOps team is supporting our corporate Services Portfolio for Earth Sciences Researchers: Developer Cloud Sandboxes that are supporting scalable scientific processors, Virtual Archives that are enabling federations of distributed data holdings, Data Challenge Platforms, aimed at automating the management of data mining contests, and Digital Marketplaces, that support publish & share features for reproducible scientific experiments.

We will give first a short overview of our infrastructure, that will lead the presentation to more specific points:

1. Terradue’s Cloud development environment

Terradue’s software development environment has a double purpose. First, as our laboratory, where we test the latest developments from OpenNebula. We will illustrate this with the ONe 4.0 release, that we installed and studied, especially for the integration of our own OpenNebula extensions. Technically, we will describe the hooks, drivers and plugins that we developed for OpenNebula, as well as our contribution to the open-source community (e.g. VMware drivers, OCCI…). Second as our developers facility to provision test servers, that are supporting both the engineering of project-based software developments.

2. Terradue’s Cloud production environment

This is our commercial and operational platform, where we host Customer Services (http://www.terradue.com/partners).
This platform leverages OpenNebula for the virtualization and management of hardware clusters, that we rent from commercial ’bare-metal’ providers.

Our flagship solution is the ”Developer Cloud Sandboxes” service, where scientists can provision their own Virtual Machines, plug their processors seamlessly in MapReduce frameworks (leveraging the Hadoop Streaming API), and test the scalability of their application workflows for large, high-performing clusters.

Developers also benefit from dedicated automation procedures and support tools embedded with the Sandbox service.
We will then describe how this platform is powered by OpenNebula with advanced contextualization tools.

3. Multi-Cloud bursting

Thanks to the collaboration with several stakeholders, within projects sponsored by the European Space Agency (ESA), the European Commission (EC FP7), or the Italian National Research Centre (CNR), we have developed specific drivers to provision virtual machines over public commercial clouds.

When their processor integration and validation phase concludes, our researcher users can seamlessly burst their applications at scale, leveraging OpenNebula drivers for on-demand processing tasks.

We will conclude with our future developments, aimed at extending the performances of the Terradue’s Cloud platform, like with engineering activities for porting algorithms to native MapReduce software code, or with additional data staging automation, e-collaboration tools and extended capacities to manage linked data-as-a-service. These new developments will be introduced as part of Terradue’s innovation prospective using OpenNebula.