How Will Data Science Evolve Over The Next Decade?
How do you think data science will change over the next 10 years?originally appeared on Quora: the place to gain and share knowledge, empowering people to learn from others and better understand the world.
There are many aspects to data science and being a data scientist that are constantly changing and will change over the next ten years. I’m personally going to limit this answer to the career fundamentals of data science.
To be clear, I also think it’s wonderful that data science prepares you well for a variety of ways to add value in tech, and I also think you’re seeing the lines blur between data science and product management and other related roles at a lot of companies. (At Quora, for example, roughly half the product management team has data science in their background, and our Head of Product used to be the Head of Data Science, though I think we’re a bit of an outlier in how data-driven our product processes are.) But I think the commitment I and other data science leaders make to career paths within data science as its own discipline are also very real, and it’ll be nice for folks to have more role models and belief in that reality as time passes.
- Roles that are just running basic, reactive SQL queries 100% of the time so someone else can make a decision
- Roles that are focused on research and paper writing with limited product impact
- Data engineering and data infrastructure roles
- Machine learning roles (anything from research to building production models to building prototype models that someone else implements to purely feature generation and parameter tuning)
- Full stack data science roles like we have at Quora, which focus on product analytics and include experimentation, metric design, modeling, proactive exploration, etc. and are embedded within product teams
- 100 other things
all being called “Data Scientist” right now. Part of the challenge of companies who do make a big investment in data science as a function is that it’s hard to distinguish yourself from companies who think of data science as a ‘back office’ thing—and I don’t really expect that to go away—but there’s also a ton of horizontal diversity all with this one title, where different people just want to do different things, and so hiring processes are pretty inefficient, and in the worst case, you have folks not only interviewing at but actually joining companies who don’t have the role they actually want. Whereas at this point, if you look at engineering for example, most companies can pretty clearly distinguish an infrastructure engineer from a product engineer at the very beginning of the hiring process (i.e. on their careers page).
This isn’t something I expect to happen overnight—you don’t want to put your team at risk by giving them strange titles that can hurt their professional growth, so there can be this tendency toward uniformity—but I think as more companies with prominent data teams grow in size, you’ll see them try more different things to help make the hiring matching process more efficient, and some of them will stick and proliferate, and you’ll eventually see a diversity of job titles that more closely map to the work and expertise. A slightly different take we’re currently exploring is to try and find data science candidates withand signal the demand for that work directly up front in the hiring process.