Enterprise Data Science is Not About the ML Algorithms
If you want to have business impact, you can’t accomplish that if your models don’t see the light of day. Instead of ML algorithms, focus on building simple and complete ML systems.
Many junior colleagues and those outside of data science trying to break into the field focus on the ML models and algorithms. I get it- I still find it magical to train ML models and see how they perform magic by predicting on the test data, validated by the model metrics you apply for validity and accuracy.
But now you are an Enterprise Data Scientist at a large organization. You’re not at school any longer, having to defend your ML training approach and choice of algorithms to your graduate teaching assistant, and having to defend the model with the metrics and area under the curve, and your confusion matrix.
Your company is not paying you for school projects- they are paying you to help them solve their business problem, and they are trying to figure out your value and their return on investment in you. They are not paying you to learn the latest model algorithms and techniques- you are now helping them to solve problems and make money or save money.
So if your models stay on your laptop, local server or cloud instance, and if it is not deployed and consumed by some system or business domain expert, then why are they paying you all this money on your salary? To do school projects?
This is partly the fault of the current data science training systems which perpetuate this hyper-focus of new data science professionals on the ML algorithms. What you need to focus on is how you are going to deploy your model so it can be consumed by your key stakeholders, or is consumed by a system where the model outputs are making automated decisions.
Instead of focusing only on the ML algorithm, new people need to understand the components of a simple and complete ML system. You can start with the main components of the data science lifecycle, which can be broken down into 4 main components/modules:
Feature Engineering
Model Training
Model Testing
Model Inference
When you focus on these 4 components of the data science lifecycle when working on your Enterprise Data Science projects, you then start to get good at each of these 4 major components. This new focus will help you build simple and complete ML systems.
Of course as a junior data scientist, you will not be expected to build these complete ML systems for your projects on your own- a good supervisor will help you as you build your knowledge and skills at building complete ML systems.
So with the 4 components above, start practicing with your projects. For example:
Create a feature set from the raw data,
Train your ML model,
Apply performance metrics to your model,
Deploy your model (model inference).
With more projects, you will begin to understand how senior data scientists and ML engineers work collaboratively to build these end to end AI systems. As you get more experience building ML systems instead of ML models, you will get good at each component, and then you will see how these components can be reused as modules for other projects- think plug and play. And with model inference, learn from the ML engineers at how they automate this deployment process.
To be an Enterprise Data Scientist, you have to start building complete ML systems, not only ML algorithms. Start simple with the advice above, and pair up with an experienced data scientist and ML engineer. Soon you will lead your own team, building complete ML systems.
If you're looking for support from me, here are a few options:
Enterprise Data Science Consultancy: With my consult team comprised of a Senior Data Scientist/ML Engineer, Senior Data Engineer, and Senior Cloud Engineer, we will help you architect and build your Enterprise Data Science platform, and transfer knowledge to your IT team to maintain and optimize it. If you don’t have an MLOps team, I will help you build one. Please get in touch about this consultancy here
Coaching and Mentorship: I offer coaching and mentorship; book a coaching session here