Hybrid vs Cloud for an Enterprise Data Science Stack
Cloud-first strategy has led to an emphasis on cloud-only AI/ML solutions. However, this may be cost-prohibitive for payors.
A cloud-first strategy for enterprise data science stacks at large organizations makes sense. Why not utilize Platform as a Service and have all the heavy lifting done for you, to build, automate, monitor, and optimize your data pipelines and ML workflows? It’s so tantalizing and vendors promise the sky is the limit- why not? Vendors will even have your enterprise data science stack architected for you, and will often oversell what they can do for your organization with a cloud AI/ML stack. And all the while, data scientists and ML engineers alike may embrace this new cloud AI/ML ecosystem, as all of the prebuilt high level services that was previously so manual on prem, suddenly becomes more automated and requires little to no-code to build the pipelines and train/deploy ML models in the cloud.
Well, I can tell you that transferring your entire data science stack to the cloud is cost prohibitive. Imagine running all of your experiments and junk code in the cloud, with all of those data provisioning, GPU, and Spark cluster costs spinning/increasing at seemingly exponential rates.
A more cost effect enterprise data science stack involves a hybrid solution, including both an on prem platform, and an integrated cloud one. So this hybrid (on prem + cloud) workflow is a good base for a cloud bursting setup, where on prem resources are utilized first, then burst into cloud when needed.
Before going to the cloud and ditching your on prem data science stack, start to estimate the cloud costs with some current AI projects- there are volumetric and cost calculators with each major cloud AI platform. Whoever is paying for this cloud platform may be in for a shock over costs- GPUs and Spark clusters spinning in the cloud is a very expensive endeavour. The hybrid platform would be the most cost effective, as most of the exploration, junk code, and junk data artifacts would stay on prem first, then would burst into cloud as needed.
So rather than go with vendor zeal and go all in, consider scaling it back with a hybrid data science stack, to contain cloud AI/ML costs. This also speaks to the importance of in-house data scientists informing your enterprise data science stack, rather than relying exclusively on vendors and external consultants (this is a topic for future articles- stay tuned).
If you're looking for support from me, here are a few options:
Enterprise Data Science Consultancy: With my consult team comprised of a Senior Data Scientist/ML Engineer, Senior Data Engineer, and Senior Cloud Engineer, we will help you architect and build your Enterprise Data Science platform, and transfer knowledge to your IT team to maintain and optimize it. If you don’t have an MLOps team, I will help you build one. Please get in touch about this consultancy here
Coaching and Mentorship: I offer coaching and mentorship; book a coaching session here