14 Comments

> Big screen TVs occupy a lot of warehouse space and have very low margins. ... Discounting TVs to free up space for higher margin or faster selling products can significantly impact the bottom line.

That's fascinating. Very entertaining dive into the complexity of seemingly ordinary stuff. Thanks for sharing.

> I cannot tell you how much of a pain building that data pipeline was.

Did the project survive after all? Or did it slowly die on crutches? :D

Expand full comment

It survived and is still running. As an outside consultant, I have the luxury of saying and doing things that will get an internal employee fired. I also had the US COO and CTO as backers.

Expand full comment

I'm curious if you pulled off building a clean solution or if all the complexity created a pile of tech debt. Meaning the pipeline is running, yet it's expensive to maintain it.

Expand full comment

The first pipeline was messy and we needed to rebuilt it a year later once we had developed a reference implementation. That took several iterations.

Expand full comment

> It will be interesting to see how the next 18 months treat our currently oversized data tools landscape. There's not enough space for all the players we now have.

I'm not sure I get why there's not enough space

Expand full comment

There is so much duplication (5-10 tools that do the same thing) and too many tools that don't fit a data science life cycle. The latter is filled with train/test/deploy tools that are easily replaced with self-service solutions.

Expand full comment

How closely do you follow the data stack landscape as a strategist? What do you look for? Where do you stop?

Expand full comment

I follow it closely because I need to advise clients about infrastructure and help data teams develop the infrastructure roadmap. I stop at the point of being a data engineer or the person maintaining the infrastructure.

Expand full comment

I was hoping to see a different answer. :D Is there a general tooling baseline that you could share? I'm lost on how to start monitoring and evaluating everything to confidently advise something. It feels like you need to be sure your suggestion is the best for the specific need, not simply okay. Potential maturity progressions add even more complexity, as I can see it.

Expand full comment

Learn the ecosystems and you'll have a better grasp on how each solution meets current needs and can scale to meet future needs. IBM, SAP, Salesforce, and Microsoft (obviously more) are the core players to start with. Those companies all provide someone to help guide you through the decision process and educate you on their solutions. Lean on them for a deeper dive into ecosystem fit and supporting resources to make a final recommendation.

Expand full comment

So the suggestion is to start a conversation with some big vendors and their offers to a particular company you work with. Did I get it right?

Expand full comment

For the final decision, yes. To learn, start with the large vendors and read through their solution ecosystems. Focus on implementations and content from people who have deployed those ecosystems.

Expand full comment

This helps. By this time, it clicked that I had overthought it and gotten to it with the wrong perception of the need to give a recommendation on the spot. It still requires research for the final decision. The difference will lie in the ability to conclude quicker. However, prior learning couldn't hurt. Thanks!

Offtopic: I just stumbled upon an example of a book cover related to data, which delivers a message visually, just for inspiration. https://www.linkedin.com/posts/allendowney_we-have-a-cover-what-do-you-think-https-activity-7051569151059849218-hUPo?utm_source=share&utm_medium=member_desktop

Expand full comment

This was a challenging read for me. I'd handle it better if it was in 2 pieces.

It got more comfortable when I looked at the graphics, though: https://www.sap.com/dam/application/shared/graphics/sap-datasphere-marketecture-graphic.svg

I'll drop a couple of questions separately to make answering them easier

Expand full comment