How To Successfully Answer Technical Data Science Questions From Non-Technical Executive Leaders

“I got a spicy text last night” is the best way to start a post that loses 90% of my subscriber base. Except, in this case, it was a spicy data science question from a semi-technical founder. It’s an excellent opportunity for a short post about relationship building and working with non-technical executives.

Formal presentations to non-technical audiences are well-covered ground, but most of our interactions are in one-off requests for information. These questions require a different communications framework.

I have taught this capability for almost 10 years as a leadership coach and mentor. It’s part of communicating for impact, which requires us to structure communications that drive an outcome.

Handling ad hoc information requests is part of an unstated job function. We are trusted technical advisors for audiences across the business. Filling this role well builds relationships that are critical for our long-term success. Each one-off request must be structured to achieve that objective. We are providing the requested information, and it must be structured to build our relationship.

The two sides of communications for impact are more apparent when we frame it that way. I am delivering information and want to encourage more questions in the future. Executive leaders should be excited about data science and enjoy learning more. It’s an interesting field that has begun to capture non-technical people’s curiosity. Cultivating that is powerful. Why?

We need C-level buy-in for advanced projects. Groups throughout the enterprise will make changes and go through training to support internal projects. Product managers and customer-facing organizations will train customers to prepare them for model-supported features and products. The relationship and trust built in small exchanges like these are a big step towards moving those pieces forward.

This subtle tradecraft gets data scientists into the room where decisions are made about our work. Once we are trusted technical advisors, people realize we can add value to meetings, so we get invited to more. Including data scientists in project prioritization and estimation meetings seems obvious, but we are frequently excluded.

A quick email or messenger exchange feels inconsequential, but they are critical. My process:

1. Assess their needs and knowledge.

2. Consider knowns and unknowns.

3. Evaluate intent and what they will do with my answers.

4. Set up to answer the question more thoroughly later.

5. Provide a brief explanation to meet their immediate curiosity.

6. Field technical questions with concise answers.

U Up? Send LLMs.

Anon Founder 9:44 pm -> “Could I use text prompts generated from {text data source} and an LLM to text classify via inference without needing tons of pre-training data?”

My wife was sitting next to me and saw the text roll in. She just laughed and said, “That’s the worst pickup line I have ever seen!”

There’s danger in this question and not just from a jealous significant other. Incomplete information and poorly elaborated requirements come with the job. In a face-to-face meeting, I would be diving in with follow-up questions. I cannot respond to this message with a long list of questions. Why?

The person I am texting will not have the answers. All questions will do is frustrate them, and the next time they have questions, someone else will get the text. This is all the background information that I will be getting.

I want to cultivate their curiosity. My answer must be positive and supportive. There are risks I must manage, and the best way to succeed with that is to avoid getting into those areas in my answer.

• I have knowns and unknowns to consider. Unknowns are risks:

• What kind of classification are they doing?

• How many classes (K)?

• How much training data do they have for each class (shots)? Do some classes have none?

• How reliable do the predictions need to be?

• Will inference be served to another app or users?

• Why is this on your mind at 9:44 pm?

There are so many questions to be answered and follow-ups right behind them. I am not engineering a solution. I’m satisfying a curiosity. What I know defines the areas I can provide answers around.

• The data source will have potato-quality text prompts.

• An LLM is like using a nuclear weapon to take out a spider.

• This founder is considering building this solo, is semi-technical enough to do it, and is semi-technical enough to create a solution that seems to work but doesn’t.

• They just finished reading a Forbes or Towards Data Science post about one or more LLMs.

• This is likely for a presentation or proof of concept.

Formulating An Answer

This is the scene in every horror movie where the audience yells at the main character, “Don’t go in that room!” Remember, if I don’t answer this question, someone less qualified will. Replying isn’t a choice. It’s a lack of options.

Technical people generally struggle to build relationships in cases like this. Our day job is to reduce uncertainty down to the ground floor. We are intentionally cautious, and that frustrates people outside of our field. Building relationships starts with the other person and their needs. I must meet people halfway in every communication interaction.

“I will send you a detailed write-up with options and tradeoffs in the morning. Quick answer…”

The follow-up is critical because I ask them to meet me halfway. I can get into some depth and ask clarifying questions later. This first sentence is my entry point to answering this question the right way. I also need to satisfy their immediate curiosity. Part 2 is me meeting them where they are to satisfy their need.

The solution is self-service machine learning. OpenAI has an API connection to GPT-3 that takes all the data science knowledge out of working with LLMs. It’s easy to sign up for and gives some free credits to play around with. The API documentation is better than most, and training the model for specific tasks is intuitive.

Untrained, the accuracy will not meet any production use case. Accuracy improves with a few examples for each class to support a proof of concept. That’s what the founder is looking for. They will probably use this early implementation to show investors that their ideas are feasible and gain credibility.

“As long as you don’t get into detailed classification tasks, GPT3 can handle that. You can get an API key from OpenAI, and there’s minimal code required. You’ll need to label a few examples of each category to train, and then it should be good.”

I only listed a single constraint and chose the most important one. Listing several caveats is counterproductive, but they need to know there are considerations. I am trusting my audience, and that comes through in this message. They will never trust me if it sounds like I am talking down to them. Listing a ton of pitfalls right now will come off as condescending.

I am speaking to their technical level and respecting their capabilities. This is another trust-building element. It doesn’t sound like I am dumbing it down. I am reducing the level of effort but assuming they can handle a level of technical detail.

Fielding A Technical Question

“Is your sense that GPT3 is miles above BLOOM? Also, this is called zero-shot classification, correct?”

My answer would be completely different if this came from a data scientist or student. Fielding a technical question from someone outside the field requires a different approach. Again, I am starting with their individual needs.

The motivation behind the first question is the need for a justification. Asking for it now means they are thinking about fielding and answering this question themselves. I just got a good piece of information. I have not built a relationship as a trusted partner yet. They come to me for advice but don’t trust me enough to bring me into the meeting.

In every communication, there is obvious and implied information. The implication is how far I have built the relationship. In my follow-up email, I should offer to help them pitch this or be there to answer any technical questions that arise. I need to establish credibility and talk about times I have done this successfully in the past. They don’t trust me yet, but this is an opportunity to show them another way I can add value.

If they fly solo and follow my recommendations, they must back up their decision without me. Their need is an answer that fits into a bullet point or 2. If I give them too much detail, they could enter unfamiliar territory in a presentation. Sounding technical invites more technical questions.

“GPT3 has been used more, so it’s more reliable, in my opinion.”

That answer feeds the bullet point. ‘I chose GPT3 because it is better adopted and more reliable.’ They don’t want details.

For the second question:

“Few-shot if you give it a few examples to improve the accuracy.”

The answer is also feeding a slide deck. Having a term like that improves their technical credibility.

Few executive leaders are looking to become data science experts. They want model literacy more than anything else. It’s very use case focused knowledge.

In this example, the founder understands that LLMs can be used for text classification. They want a layer deeper understanding. I am providing bullet points about the implementation in both responses. Their question is the top-level line in a deck, and my answers are the supporting points below them.

I have great success building relationships through small interactions like these. My follow-up will have more detail and continue the conversation. I want them to know I am interested in their needs and can meet them.

I am opening my door to them; hopefully, they will walk through it again.