Hybrid cloud has become an essential approach to high performance computing (HPC), helping enterprises increase their competitiveness and efficiency. Earlier this year, our Univa customer survey revealed hybrid cloud as the model of choice for 70% of surveyed enterprise customers who are already deployed or planning HPC deployments to the cloud. While there are many reasons hybrid cloud gets top billing, here we expound on six key motivations for adopting a hybrid cloud approach for your HPC workloads.
Public cloud providers offer specialized infrastructure on-demand, so you can access the latest technologies and pay for them only when needed, thereby shifting CAPEX to OPEX and avoiding the risk associated with capitalizing and depreciating assets. By accessing HPC infrastructure in the cloud you can potentially realize improved efficiencies, time-to-market advantages, and reduced IT expenditures. Cost advantages are further enhanced when you have the right software tools in place to manage hybrid clouds efficiently.
Hybrid cloud is especially advantageous for certain types of workloads, like Machine Learning (ML), where users require expensive GPUs for relatively short periods. For occasional needs, the cloud-resident clusters are usually cheaper than purchasing and maintaining a cluster on-premises. Sensibly managing your overall expenditures by deferring, reducing, or eliminating on-premise CAPEX becomes easier with hybrid cloud solutions such as Univa’s Navops Launch.
Because hybrid cloud lets you extend on-premises HPC clusters to the cloud as needed, you can instantly scale hardware and software resources based on job requirements. For example, you can scale compute-heavy workloads, such as TensorFlow ML models, without impacting other production workloads.
When integrated with on-premises infrastructure, hybrid cloud can be an effective way to dynamically extend your HPC capabilities. You can manage your ongoing business requirements while still accommodating growth, and IT agility is improved with faster deployment of applications and infrastructure.
If extreme scale is important to you, this is another reason to embrace the cloud. Univa deployed a million core cluster in just 2.5 hours, comprised of over 55,000 AWS instances in 3 availability zones, 16 different instance types, leveraging AWS Spot Fleet technology with Navops Launch. Navops Launch allows you to provision and manage both virtual and bare-metal environments, and dynamically create, scale, and tear-down cloud-based infrastructure based on changing workloads.
In our experience, most enterprises are using public cloud for some workloads while maintaining an on-premises infrastructure for others. Cloud bursting can be used to tune your enterprise’s capacity to demand, dramatically speeding up compute, and provide isolation between workloads. A good hybrid cloud solution can provide you with automated, workload-aware, policy-based bursting . Rapid provisioning and allocation of resources can occur in minutes and the deployment of clusters can be automated whether on-premise or in your choice of cloud provider.
The flexibility and ease of use of a smart hybrid cloud solution can reduce the need for user training and steep learning curves. End users should be able to interface with applications as they always have, and bursting should be automated and transparent. The best hybrid cloud solutions minimize disruption to existing applications and worklows.
Modern workload management tools can help make migration to the cloud seamless, enabling users to access cloud resources as if they are on-premise. For administrators, hybrid cloud software like Navops Launch enables them to bring their own images (BYOI) in the form of virtual machines or containers, and choose cloud providers with out-of-the box adapters.
New applications and technologies are being added to public clouds every day. Leading cloud providers all offer high-performance instances, the latest GPUs, and high-performance storage and networking. Some providers also offer bare-metal servers (or dedicated servers, which are the same thing under a different name). These are the most powerful infrastructure hosting option. Some providers may also offer parallel file systems to high-performance workloads with high storage bandwidth requirements.
A recent IT survey by Dimensional Research found that 93% of respondents have initiated Machine Learning projects in their companies, with 9 out of 10 expecting to use GPUs as part of their ML initiatives. Further, 80% intend to use hybrid cloud. Access to state-of-the-art GPUs has become a driving force in hybrid cloud adoption for many organizations.
Hybrid cloud models allow you to maintain full control over sensitive data by keeping it on-premise while less-sensitive workloads can reside in the cloud. For organizations bound by various legal, regulatory and privacy requirements such as HIPAA, HITECH, PCI, and GDPR hybrid cloud can help facilitate compliance. Hybrid environments can also help protect intellectual property, and accommodate applications with different licensing terms.
Continue following this blog for ongoing updates and new resources to help you along your HPC cloud journey. To learn more about Univa solutions for cloud computing or to speak with a Univa representative, contact us today.