Organizing data science for impact: Machine learning as software engineering

Data scientists are in heavy demand. LinkedIn’s August 2018 report on the workforce suggests there is a national shortage of more than 150,000 people with data science skills.

And yet, for all this enthusiasm and desire to make use of data science, many businesses will hire people with such skills then fail to see any business impact at all.

In my seven years of working as a data scientist I haven’t seen a single model I developed go into production, much less have appreciable business impact. That certainly reflects some of my own failures, but more crucially it can be traced to poor organizational choices and a misunderstanding of how to get maximum value from data science.

Andrew Moore, head of Google’s Cloud AI business, disputes the idea that AI is some kind of “magic dust” you sprinkle on an organization so that it becomes smarter:

When people come and say “I’ve got this massive amount of data—surely there’s some value I can get out of it,” I sit them down and have a strong talk with them.

What you really need to be doing is working with a problem your customers have or your workers have. Just write down the solution you’d like to have; then work backwards and figure out what kind of automation might support this goal; then work back to whether there’s the data you need, and how you collect it.

Here’s a good overview of how Uber has scaled their machine learning efforts, reinforcing Moore’s assertion. Uber’s experience underscores the importance of making product engineering teams the owners of machine learning model development and deployment:

We found that it works best if the product engineering teams own the models they build and deploy in production. For example, our Map Services team owns the models that predict Uber’s ETAs. Product teams are typically staffed with the full set of skills they need to build and deploy models using Uber’s ML platforms. When they need additional expertise, they get assistance from the research and/or specialist teams.

Too often, data scientists are placed outside the product development workflow. For example, they might be positioned as an offshoot of a business intelligence team, and asked to “generate insight” from masses of old data. This is not how data science achieves impact. ML models need to be built with attention to what features users and staff need to make ongoing decisions more optimally.  Their development needs to be prioritized against development of non-ML based product features.

Business intelligence and machine learning efforts do not belong together, even if some of the data engineering skills they require do.

Alternatively, data scientists are often put into an innovation and research team where they may develop deployable machine learning models, but without the discipline that product development processes provide. In this organizational setting, data scientists are likely to waste resources on models that will never be used in a production system.

One mistake I have seen time and again is that a company will hire one data scientist and place her outside product development processes in an R&D type role, then wonder what went wrong when AI/ML has little to no effect on improving business outcomes.

Uber’s experience suggests you need all of the following to incorporate ML at scale into your product engineering efforts:

  • Machine learning design and development capabilities within product teams
  • Specialists with knowledge of advanced topics like NLP, computer vision, recommender systems, econometric forecasting that product teams can consult with
  • Machine learning platform teams to provide general purpose ML support to all the product teams

Uber also mentions a research team–but to me this seems the least necessary for achieving short- and medium-term impact with data science. Unfortunately for companies and the data scientists they hire, it’s often the place company’s start with the AI efforts. You could also probably do without the specialist team to start, and just hire consultants if you determine they’re needed. The low-hanging fruit of the ML possibility tree will often require only the most basic of approaches–an understanding of how to build a simple classification or regression model and an ability to put together a labeled data set for training. You don’t need a specialist team for that.

Uber’s ML platform team thinks of machine learning as software engineering. This is a critical switch from thinking of it as a data analytics activity that produces insight or as a research and development function aimed at long-term innovation only.