Many years ago I taught a stats class for which one of the topics was hypothesis testing. Many of the students had a hard time remembering what situation each test was designed for, so I made a flowchart to help piece together the wild world of hypothesis tests. While the flowchart isn’t pretty (if I made it today, it would be much more attractive), I feel like it might be useful for others, so here it is:
Suppose, as many do, that we want to estimate the effect of an action (or treatment) on an outcome. As an example, we might be interested in estimating the effect of receiving a drug vs not receiving a drug on the incidence of heart disease. In an ideal futuristic world, we would take each individual in our population and split them into two identical humans: one who receives the treatment and the other who doesn’t.
In my previous post, I introduced causal inference as a field interested in estimating the unobservable causal effects of a treatment: i.e. the difference between some measured outcome when the individual is assigned a treatment and the same outcome when the individual is not assigned the treatment. If you’d like to quickly brush up on your causal inference, the fundamental issue associated with making causal inferences, and in particular, the troubles that arise in the presence of confounding, I suggest you read my previous post on this topic.