Causal models for qualitative and mixed methods inference

Exercise 3: Process tracing

Author

Macartan Humphreys and Alan Jacobs

1 Part 1: Process Tracing with a Causal Model

(40 minutes)

In this exercise, you will work through “by hand” the logic of process tracing with a causal model. We are providing you with a causal model, drawn from Chapter 12 of the book, that relates the existence of a free press to the removal of governments from office. The model represents a set of beliefs about when and through what mechanism(s) the existence of a free press threatens incumbents’ hold on office.

1.1 The model

1.1.1 Variable definitions

  • X: A free press exists (1) in the country or does not exist (0)
  • S: Government is sensitive (1) or insensitive (0) to public opinion
  • C: The government is corrupt (1) or non-corrupt (0)
  • R: The media reports (1) or does not report (0) on any governmental corruption that may be occurring
  • Y: The government is replaced (1) or survives (0)

1.1.2 DAG

Figure 1: Causal model in which \(S\) and \(X\) are stochastic \(C\) and \(R\) have only two possible nodal types each \(Y: 1\) if and only if \(C\) and \(R\) are both 1.

1.1.3 Beliefs about causal relationships

Specifically, we go into the study believing the following about how each node depends on its parent node(s):

  • X: in 50% of the population of cases, there is a free press; in 50% there is not
  • S: 50% of governments in the population are sensitive to public opinion; 50% are not
  • C: 50% of governments in the population will act corruptly regardless of prior conditions; 50% will engage in corruption unless both there exists a free press (X = 1) and that government is sensitive to public opinion (S = 1) (\(C\) is either 1111 or 1110).
  • R: in 50% of cases, the press will never report on corruption, regardless of antecedent conditions; in 50% of cases, the press will report on corruption if and only if it is free (X = 1) and corruption is present (C = 1) (\(R\) is either 0000 or 0001).
  • Y: in all cases, the government will survive in office unless there is both corruption (C = 1) and reporting on corruption (R = 1) (\(Y\) is always 0001).

We now want you to “do” process tracing on a hypothetical case using this model.

Prior case observation: Y = 0 (the government was not replaced)

Query: What is the probability that a free press would have caused government replacement in this case?

Now, imagine different process-tracing strategies you might pursue—that is, different combinations of other nodes you might choose to observe. We want to think through what inference you will draw on the query depending on what you see.

1.2 Your task

Figure out what you would conclude—what is the probability that X = 1 would cause Y = 1 in this case—if you observe each of the following:

Prior (no additional observations)

  • Nothing: before you do the process tracing

Strategy: Observe S only

  • S = 0
  • S = 1

Strategy: Observe C only

  • C = 0
  • C = 1

Strategy: Observe R only

  • R = 0
  • R = 1
    Bonus: Why is the second of these a trick question? Why would it not make sense to observe R in this case?

Strategy: Observe S and X

  • X = 0, S = 0
  • X = 1, S = 0
    Bonus: If you first observed X = 1, why should you not bother measuring S?

Strategy: Observe S and C

  • S = 1, C = 0
  • S = 1, C = 1
  • S = 0, C = 1
    Bonus: If you first observed S = 0, why should you not bother measuring C?

Strategy: Observe S, X, and C

  • S = 1, X = 1, C = 0
  • S = 0, X = 0, C = 1
  • S = 1, X = 0, C = 1
  • S = 0, X = 1, C = 1
  • S = 1, X = 1, C = 1

Final question: How would you think about which of the above process-tracing strategies are most and least worth pursuing?

See solutions

(15 minutes) Feedback

2 Part 2: Process trace your own DAG

(30 minutes)

Make a table showing what you would conclude about your query given each pattern of data you might see.

Query Possible observations Inference
“X causes Y” “If I see X = 1, Y = 1, M = 1” The query is true with probability …
“X causes Y” “If I see X = 1, Y = 1, M = 0” The query is true with probability …
“X causes Y through M” … …

3 Discussion

4 Additional (if you have extra time!)

Look at our shiny app and work through Part 1 again using the app for the cases with complete data.