Who’s Hiring And Paying The Best For Data Roles?

In good times, it doesn’t really matter where you work. In a tough economy, where you work is everything. We’re not in bad times, but there’s an unevenness to this economy. In times like these, it’s essential to be working in the right places, but not for career survival. Companies set up to do well provide the best opportunities for advancement, interesting projects, and stability.

This article is a variation of what I post quarterly about the jobs outlook in the data field. It’s not just who’s hiring that is important this time. It’s where you’ll find the best compensation and what’s changing in the capabilities companies are looking for.

The lowballers come out in force whenever the labor market story gets choppy. In this cycle, there are many more of them than I have seen in the past. Lowballers hope people feel desperate after a layoff and will take a below-market offer.

When enough of these roles hit the market, their salaries suddenly become the “market rate.” It should be classified as a type of collusion but is rarely called out. It's critical to ignore the market rate noise and focus on roles with top 25% compensation ranges. That’s a feasible strategy for most data roles.

This is a reskilling year for many jobs, and the next 2 years will continue to reward data professionals with in-demand capabilities. Skate to where jobs are being created, or demand is shifting.  You’ll be at the front of the wave with very little competition for top-tier salaries.

Data Roles Under Pressure – Data Analysts

I’m seeing 2 roles experiencing legitimate salary declines: mid-level data leadership (Senior Manager to entry-level VP) and data analyst. They aren’t being hit by the low-baller trend but are in a legitimate down cycle.

Companies are moving away from data analysts in favor of data engineers, analyst engineers, product analysts, and business translators. It’s not a collapse in demand, and if you’re a data analyst, this isn’t a call to panic. Salaries are falling for traditional data analyst roles, so it’s time to reskill into a safer haven role.

Most data analysts already lean in one of the hybrid directions. They face the business, engineering, or products. They have begun picking up the capabilities for new roles and need a few courses or training programs to fill in the gaps. Demand isn’t collapsing.

The role is maturing, and that’s normal for any job with an initially broad definition. The analyst side of the field has always been too broadly defined. This will result in more opportunities and structured career paths. Engineering, product, and business strategy tracks are formalizing, but that’s not exactly a new development.

About a third of my course participants come from analyst roles. Their reasons are similar. The business asks them to take on new responsibilities, and they need new skills. No one has come in with the fear of being laid off. That’s why I think this change is more of an opportunity than a threat.

Data Roles Under Threat – Mid-Level Data Leaders

The second role, mid-level data leaders, is under threat. Demand is shifting as more businesses gravitate towards a flatter organizational structure. It’s having an interesting impact on the job market. Demand for managers and executive leaders is growing, but it’s quite the opposite for middle management roles.

My clients and many senior leaders I talk to can’t find managers and leads. Data professionals at the individual contributor level tell me they don’t want those roles because they fear losing their skills. The trend toward less management is obvious; no one wants to be caught in the down cycle. A move to manager comes with risks that most aren’t willing to take.

Other options are emerging for technical individual contributors to extend their careers. Leadership isn’t the only track up after we reach senior and staff level roles. The only option is for salaries to rise enough to pay people for taking the leadership risk. So far, I only see that in a limited number of companies.

Demand for C-level roles appears strong at the other end of the spectrum. However, there’s a lot of conflicting evidence available. Recruiters are seeing a massive drop in openings for CDO roles. I can’t explain that. In my seminars and workshops, I see strong evidence that CEOs are hiring or planning to hire executive and C-level data leaders.

There’s a strong bias since most people attending my seminars and workshops are on the monetization path. That’s the trend across industries. AI strategy, monetization, and products are on the agenda across businesses. I may be seeing a small slice of the overall market, but it doesn’t make sense, given the macro trends.

It’s also hard to confirm with data. Most C-level roles are never published on job boards. What you see represented there isn’t indicative of demand or a lack of demand. The only data available is the announcement after hiring has happened. In this case, the data can tell you if the trend has already taken hold but not where the job market is going.

Data Engineers – Signs Of Leveling Off, But Still Hot

Lowballer insanity is on full display in the data engineering role. Robert Half has an opening with a range of $100K-$120K, right above an opening from Vertisystem with a range that starts at $145K for the same qualifications. Just under that role is Netflix crushing the salary range game with numbers I won’t even mention. ByteDance is offering recent graduates $112K to start.

It’s that kind of market because supply is catching up with demand. Big Tech companies are more selective, and you need to interview well to get the best openings. There are a lot of talented data engineers interviewing for the highest-paying roles.

Tier 2 companies are usually a big step down, but that’s not the case for data engineers. Salaries are lower, but there are quality openings in startups and large incumbents. GSK and BlueCross BlueShield are good examples of incumbents that provide solid compensation.

Senior and Staff-level roles pay well across companies. Mid-level roles are where salaries fall off a cliff outside of Big Tech. It’s a great time to be at the seniormost end of the talent spectrum, but it’s becoming increasingly challenging at the beginning and middle. Supply is finally catching up with demand at some levels.

High-paying roles look for people who can scale and handle streaming data more often than in the past. Other than that, the requirements haven’t changed very much in the last few years. Across the board, data engineering roles are stabilizing and moving towards standardization. The top end of the salary range might level off or decline slightly in the next couple of years, but prospects are still good.

We will probably see some new skills required as data engineering migrates towards knowledge management. That’s also a couple of years away from hitting the mainstream but look for high-end opportunities to emerge for data engineers with information sciences capabilities. Leadership roles ask for knowledge management capabilities, which will soon filter down to the execution level.

AI Product Managers “The Must Have Role”

There are 7,800 openings on LinkedIn. In the last 3 months, the range has swung from 6,900 to the level it’s at right now. Many of these have been open since early last year. Companies are struggling to hire. I hear that from clients and see that in the job openings data. Big 5 consulting companies are starting to call AI product managers “The Must Have Role.”

We can be salary setters in this market. Look for roles with the top half of the salary range in the low to mid-$200Ks. For principal level roles, shoot for mid to high-$200Ks. Total compensation should include bonuses and stock that takes those numbers into the $300K range. Roughly 25% of the openings are paid at that level—Hunt through the openings to find the best compensation range.

Netflix made headlines with its range-topping $500K, but that number isn’t typical. NVIDIA, Microsoft, Intel, ServiceNow, Snowflake, Asana, Grammarly, and Tinder have great compensation ranges. It’s not difficult to find a quality job opening. What’s going on?

Intuit, The Walt Disney Co., and many others have had AI product manager jobs open for 6-9 months. There are plenty of applicants, but few meet the requirements. The role is experiencing rising demand, but supply hasn’t kept up because getting experience or training is tough.

Companies are looking for people who understand AI monetization and product paradigms. Most technical product managers have experience with digital and cloud technologies but have had no success with data and AI products. It shows in the interview process. The biggest talent pool doesn’t make a smooth transition into AI product manager roles.

Most experienced AI product managers work in Big Tech. Those companies pay the best and have the best products to work on, so few want to move. The supply problem is a massive bottleneck.

Companies Need Internal Training Programs To Meet Their Talent Requirements

Companies rely on external training and upskilling programs to pump out enough people to meet demand. For over a decade, that’s been a failed strategy for data talent. I enjoy the current demand for my courses, so it will sound strange for me to advocate for competing programs to develop internally. But that’s what I’m going to do, and here’s why.

I am one of the few who offers an AI product management certification course. We’re not upskilling people fast enough to meet demand. I taught 212 AI product managers last year in 8 public and 5 private cohorts. We need 40 programs like mine to keep up with demand at the current level, and we’re nowhere close to that.

I know of 3 programs taught by someone with over 5 years of hands-on experience with data and AI products and a technical background in the field. The need for practitioners to also be instructors is where the problems start.

Most programs are academic and prepare people for entry-level roles. Unfortunately, that’s not what we need, and graduates struggle to land a role. It’s a catch-22. The best teachers are also practitioners. If you have a day job, there’s not much time for teaching.

The sensible solution is for businesses to let experienced people spend 3-4 hours a week teaching internally. This would help solve shortages across roles, from data engineering to applied research to AI product management. It’s one of the first things I build for new clients.

Unfortunately, no matter how many success stories emerge for these programs, few businesses put them in place. This talent shortage will span a few years. Be selective and cash in on the rising demand.

Data And AI Strategists Are Gaining Traction

At the end of 2022, I saw the start of the AI product manager upswing. The role was no longer confined to Big Tech and Fortune 100 companies. Openings increased from ~2000 to ~4000 in 9 months. At the end of 2023, the role took off.

Let me reward everyone who reads this post to the end with one of my secrets to predicting demand. New skills show up in leadership roles first, so it’s a great leading demand indicator. Leaders are hired to create the function and hire people into it. 6-12 months later, demand picks up for those roles. You can follow this trend back to data engineering in 2017-2019 and MLOps a year earlier than that.

Today, data and AI strategists are in the same trend that AI product managers were between October 2022 and July 2023. Data and AI strategy used to be primarily part of leadership roles. The balance is shifting, and we’re seeing the ramping phase now.

Salaries are all over the place. Supply is nonexistent. Many roles have less than 20 applicants on LinkedIn, which is a rarity in the data field. Booz Allen Hamilton has an opening with 19 applicants after being up for over a week. Amazon has one with 13 applicants.

Many roles are still in leadership, and this appears to be one of the few leadership safe havens in the data field for directors and VPs. The best roles have salaries in the high $200K to mid $300K range, with little competition for each opening. Those salaries aren’t concentrated in Big Tech, either.

If you’re looking for a promising career path, this is a quality candidate. The only caveat is that demand will not reach the same levels as AI product managers or data engineers. Data and AI strategists are following the data scientist and AI researcher demand levels.

Higher salaries and a high-end career path make these roles attractive, but most companies only need one strategic leader and one data and AI strategist. For roles like data engineer and AI product manager, each company will have multiple openings.

Wrapping Up – A Note About Data Scientists

You’ll notice the gap in this article: the data scientist. Demand is really volatile, and making a reliable prediction is impossible. I see demand for researcher roles growing, but it’s too soon to call it a trend. Will businesses realize they don’t need AI researchers for Generative AI? Most don’t, and that could quickly reverse the trajectory of demand.

Generative AI isn’t the only growth area. Robotics and federated learning are spiking. Autonomous systems capabilities also seem to be on an upward trendline. Data scientists who can work on scientific research teams (pharma, chemicals, energy, and aging) are also seeing growth. I have no confidence in calling any of these a trend with legs.

These areas will eventually take off, but is this the start of that trend or another false start? Time will tell, and I don’t have quality insights to share.