Building Data Science Capacity at Your New Job as an Enterprise Data Scientist
So you got a job as a data scientist at a large organization. But most data scientists work solo, and many organizations are just beginning their AI journey. So how do you build data science capacity?
I began as an Enterprise Data Scientist a few years ago, where my prior data science experience was as a consultant and the primary data scientist for an AI startup. The challenges for an Enterprise Data Scientist are compounded by the fact that in many organizations, AI may represent only 1% of their total budget- your salary and some AI proof of concepts may be it! In addition, you may be working for a business, data, or a technology department where you may be the only data scientist, and your team (if any) is small. And then there are the expectations from management that their department can magically do data science projects!
We all know how this ends up- you were hired to build data science capacity in your department, but you are not given well defined business use cases to solve, so you end up building AI prototypes in search of a business problem. And being one of the few data scientists in your whole organization, they assume you can just utilize their antiquated analytics stack, and you find out quickly your prototypes don’t leave your desktop, as there is no way to deploy your ML models at scale. And forget about MLOps- you might as well be talking a foreign language to them.
And this is just it- as an Enterprise Data Scientist, you will feel like a fish out of water. At this point you have 3 choices:
Quit while you’re ahead, and find another job which addresses the above gaps/deficiencies,
Put your head down and just code, wrangle data and train models- let your manager deal with the politics, or
Get on with it, and start building data science capacity within your organization.
While I thought about choice #1, the problem is that the grass is not always greener on the other side, and then you have to explain the blemish on your resume about the short stint. I crossed that one off quickly with the image of the gruelling interview process and starting that all over again- no thank you.
To be honest, choice #2 is comfortable for most data scientists and the sweet spot. Who wouldn’t want to code and model all day, every day? I chose #2 for a little while, but became frustrated when most of my models never saw the light of day, and remained on my desktop. All that work and nothing to show for it.
So I arrived at choice #3 by exclusion- it’s never one of the first considerations, as it is the hardest to implement. And when you’re the only data scientist in your department, how do you build true data science capacity, beyond just prototyping on your desktop?
To build data science capacity, you have to realize this is not the early- to mid-2010s, where enterprise data science was just beginning to take off at large organizations, and they were hiring data scientist unicorns to work on the emerging data science stacks at the time. This is the 2020s, and our modern data science stack has evolved and matured to the point where you need a whole data science team working Agile in an MLOps framework to deploy AI/ML models at scale.
So no, you can’t build data science capacity on your own- you will need help, and a team. I started out solo on a small analytics team in the technology department, focused on reports and provisioning data. Therefore, I had to network outside my department to find other data scientists.
You can start networking by finding the data organization. This is where the CDO usually lives, and the CDO office can be a resource. However, if the CDO is even focused on AI (many are not), they may not have a senior data scientist on staff, and they will not be a help to you. They might be focused on data governance, but they may not even have AI/ML model governance, so the CDO may not be helpful. The data organization may however hire data scientists to work as analysts, but none may be in the more senior roles you need. But it’s worth networking with all data scientists available, as you can start forming alliances to discuss and strategize about shared struggles and pain points as data scientist users.
If the data organization does not have the senior data scientist resources, then try the business organization. This is where most of the senior data scientists will be hired, as they are in close proximity to the business domain experts, use cases, and data. This is where I found the senior data scientists I needed for mentorship and networking.
With my new informal data science network, I proceeded to start formalizing it by starting a Data Science Lunch and Learn series, where I had my new data scientist work colleagues present their work to all of us data scientists at the company. What started out as a handful of attendees in a conference room pre-pandemic, exploded into about 100 attendees monthly in a virtual setting post-pandemic.
From this ever-growing informal data science network, we decided to meet outside of the Lunch and Learns and form an informal data science community of practice. We soon had our first chance to showcase our abilities by the group being hired officially by the execs to work on solving a critical analytics project. Although the project only lasted a brief time and the data science capabilities of our small matrix group (at the time) were no match with the operational data mining and reporting groups to solve the critical business problem, it did show us data scientists that although there is interest in AI, we still had a lot of work to do in order to realize business impact when using AI solutions to solve business problems.
Undeterred, I decided to continue to build data science capacity, so when I found out that a new AI Centre of Excellence (AICoE) was being formed, I jumped at the opportunity and contacted the hiring manager. Immediately, I was hired to fill the gap they had for an AI/ML technical lead, and now I co-lead this AICoE with my manager. In my new role, I have since been sanctioned to bring a data science framework for my whole department, and this has executive support.
I am just beginning this new role as co-lead for my organization’s AICoE, and there will be lots of opportunities for continuing to build data science capacity. I will keep you updated on my journey to build data science capacity, as I switch from a senior technical/tactical role, to a more strategic one.
Take home message- building data science capacity begins with networking with other data scientists at your company. The opportunities for building capacity and promotions will come if you are successful at networking and going outside your siloes.
If you're looking for support from me, here are a few options:
Enterprise Data Science Consultancy: With my consult team comprised of a Senior Data Scientist/ML Engineer, Senior Data Engineer, and Senior Cloud Engineer, we will help you architect and build your Enterprise Data Science platform, and transfer knowledge to your IT team to maintain and optimize it. If you don’t have an MLOps team, I will help you build one. Please get in touch about this consultancy here
Coaching and Mentorship: I offer coaching and mentorship; book a coaching session here