I saw this question on Reddit; it provides an excellent structure for exploring causal methods. Surveys are complex, but few explore what it takes to implement a high-quality survey. An entire field is dedicated to survey design and methods with experts for different domains.
The struggle with surveys is the opportunity for bias and confounding to creep in undetected. The phrasing of questions, the framework for answering, the order of questions, the sampling methodology, the respondent pool, and so many other elements can introduce significant errors to surveys. Most businesses use surveys, but few use them correctly.
In the scientific research community, surveys can be tightly controlled and implemented. In data science, that level of rigor is rarely possible. The data I get from surveys will be biased and imperfect. As I’ve said in other posts, that’s OK for business use, even if it’s frowned upon by the scientific community.
However, I need a process to manage the data issues and frameworks to understand the survey’s utility. I can apply causal methods to answer business questions and prescribe actions to impact business metrics.
The survey must be built with a causal analysis in mind from the beginning. Let’s first explore what that looks like.
Survey Design For Causal Methods
The most important concept to apply is the relationship between variables. Surveys must be designed around business questions that are rephrased to fit the causal paradigm.