Session 06 goals

By the end of today, you will be able to:

  1. Describe how a conjoint allocation game turns a choice into a measure of partisan animosity.
  2. Compare behavioral and attitudinal measures on what each can and cannot support.
  3. Explain, with potential outcomes, why only one outcome per person is observed and why random assignment still recovers the average causal effect.
  4. Distinguish internal from external validity, and assess both for a given design.
  5. Explain why coalition partnership changes the pattern of animosity in European party systems.

Today

  1. Halfway
    • The course so far
  2. From saying to doing
    • What counts as behavior?
  3. Two designs
    • Where they sit, what they share, where they differ
  4. Primer: potential outcomes
    • Why randomization recovers a cause
  5. Stoetzer et al. (2023) and Hahm, Hilpert and König (2024)
  6. Internal and external validity
  7. Wrap-up, and an open round on AI tools

Halfway

The course so far

Figure 1: The seminar schedule for sessions 2 to 6, the concepts block and the measurement block. Source: the course website, Schedule page.
  • 02 An identity is a demand for recognition, not a preference, and that makes it political
  • 03 Polarization is a family of concepts, and many disputes are about which one is meant
  • 04 AP is warmth toward one’s own side and dislike of the other, rooted in social identity
  • 05 An AP score is a construction: in-group rule, distance, weights and target all move it
  • 06 Today: what people do

From saying to doing

The grid again

Figure 2: Two dimensions of measuring affective polarization: the target, parties and elites (vertical) or ordinary partisans (horizontal), and the mode, what people say (attitudinal) or what people do (behavioral). Last session covered the left column, today covers the right one.

Last week: the left column. Today: the right one.

What is behavior?

What makes something a behavior rather than an attitude?

Is answering a survey question not itself a behavior?

Where do today’s readings sit?

Place the readings

Figure 3: The empty grid of target (vertical, horizontal) and mode (attitudinal, behavioral), left open for placing today’s two readings.

Where do the two readings go?

Both in the bottom right

Figure 4: The grid with today’s readings. Both sit in the horizontal behavioral cell, because their profiles are ordinary people and the task is a choice. The arrow shows that the same conjoint design with a candidate profile and a vote question would sit in the vertical behavioral cell.

The design travels: change the target and you change the row.

Two designs side by side

What they share

  • A conjoint design: profiles with several attributes, each randomly drawn
  • Party is one attribute among nonpolitical ones
  • The profile is an ordinary person, not a party
  • Several tasks per respondent, hypothetical stakes
  • Online panels in several countries

Where they differ

Same logic, different choices about task, outcome and party.

Table 1: The two designs compared. Both randomize the profile attributes, but they differ in the task, the outcome, and how partisanship is shown and coded.
Stoetzer et al. (2023) Hahm et al. (2024)
Task Choose one of two patients for intensive care Give 0 to 10 tokens to one player
Outcome Binary: chosen or not Integer: tokens given
Party shown One named left or one named right party per country, or none A named party, drawn at random from the country’s parties, or none
Co- vs out-partisan Built from respondents’ ratings of the two parties Coded afterwards from respondents’ party identification
Estimate Marginal means, conditional effects Average marginal component effects
Scope 5 countries, 2020, N = 6,415 25 countries, 2019, N = 29,827

Primer: potential outcomes

The causal question

Does learning that player 2 supports another party cause you to give fewer tokens?

  • A cause compares two worlds for the same person
  • World 1: player 2 is an out-partisan
  • World 0: the same player 2 is a co-partisan

Potential outcomes

  • \(T_i = 1\): respondent \(i\) meets an out-partisan (treatment)
  • \(T_i = 0\): respondent \(i\) meets a co-partisan (control)
  • \(Y_i(1)\), \(Y_i(0)\): tokens \(i\) would give in each case

\[ \tau_i = Y_i(1) - Y_i(0) \tag{1}\]

\[ \tau = \frac{1}{N} \sum_{i=1}^{N} \left[ Y_i(1) - Y_i(0) \right] \tag{2}\]

Note

How to read it. \(\tau_i\) is how many tokens fewer person \(i\) gives because player 2 is an out-partisan. \(\tau\), the sample average treatment effect (SATE), is that difference averaged over everyone.

The complete data table

Suppose we could see both worlds for everyone.

Table 2: The complete data table for four fictional respondents: the tokens each would give an out-partisan, \(Y_i(1)\), and a co-partisan, \(Y_i(0)\), and the individual effect \(\tau_i\). The sample average treatment effect is \(-2\) tokens.
\(i\) \(Y_i(1)\) \(Y_i(0)\) \(\tau_i\)
1 2 5 \(-3\)
2 4 4 \(0\)
3 1 3 \(-2\)
4 5 8 \(-3\)
Average 3 5 \(\mathbf{-2}\)

With both columns, the causal effect is simple arithmetic.

The fundamental problem

In reality, each respondent meets one player 2.

Table 3: The same four respondents as observed. Respondents 1 and 3 met an out-partisan, 2 and 4 a co-partisan. For each person one potential outcome is seen and the other is missing, so no individual effect can be computed.
\(i\) \(Y_i(1)\) \(Y_i(0)\) \(\tau_i\) \(T_i\) \(Y_i\)
1 2 ? ? 1 2
2 ? 4 ? 0 4
3 1 ? ? 1 1
4 ? 8 ? 0 8

The observed outcome is the potential outcome of the world that happened:

\[ Y_i = Y_i(1)\,T_i + Y_i(0)\,(1 - T_i) \tag{3}\]

Important

The fundamental problem of causal inference: we never see both potential outcomes of the same person. So no individual effect \(\tau_i\) is ever observed.

Comparing groups

Compare the groups instead, with the difference in means:

\[ \hat\tau = \bar{Y}_{T} - \bar{Y}_{C} \tag{4}\]

  • Respondents 1 and 3 treated: \(\frac{2+1}{2} - \frac{4+8}{2} = 1.5 - 6 = -4.5\)
  • If only the generous ones, 2 and 4, had met out-partisans: \(\frac{4+5}{2} - \frac{5+3}{2} = 4.5 - 4 = +0.5\)

Warning

When who is treated depends on the potential outcomes, the comparison is biased. Here selection even flips the sign of the effect.

Why randomization works

  • A coin decides \(T_i\), so treatment is unrelated to \(Y_i(1)\) and \(Y_i(0)\)
  • The treated are a random draw from the \(Y_i(1)\) column, the controls from the \(Y_i(0)\) column
  • So \(\bar{Y}_T\) estimates the average of \(Y_i(1)\), and \(\bar{Y}_C\) that of \(Y_i(0)\)
Table 4: The difference in means under each of the six ways to treat two of the four respondents. Single estimates scatter, but their average equals the true effect of \(-2\).
Treated 1, 2 1, 3 1, 4 2, 3 2, 4 3, 4 Average
\(\hat\tau\) \(-2.5\) \(-4.5\) \(0\) \(-4\) \(+0.5\) \(-1.5\) \(\mathbf{-2}\)

Any single draw is noisy. On average, randomization gets it right.

From one coin to a conjoint

  • A conjoint flips one coin per attribute: party, religion, class, age, …
  • Each attribute gets its own treatment and control comparison
  • AMCE: the difference in means for one attribute, averaged over all the others (Hainmueller et al. 2014)
  • Hahm et al.: an out-partisan player 2 gets \(\hat\tau = -0.99\) tokens

Important

Randomization identifies the effect of what was randomized, and nothing else. Anything measured rather than assigned is back to a comparison of groups.

Stoetzer et al. (2023)

Introduction

  • Does partisan dislike go so far as to deny fundamental rights?
  • AP reaches dating, hiring and health, but extreme consequences are hard to observe
  • COVID-19 made triage a public question in every country
  • A least likely case: guidelines say only medical criteria count

Theory

  • Partisan identity triggers in-group favouritism and out-group hostility (Iyengar et al. 2012)
  • That logic can extend beyond politics
  • Expectation: the patient’s party matters, most of all to the affectively polarized
  • Hypotheses preregistered

Where is the theory section?

Research design

Table 5: An illustrative profile pair in the German version, drawn up for this slide, not taken from the paper. Respondents chose which patient should get intensive care.
Patient A Patient B
Age 42 76
Gender Female Male
Children Yes No
Occupation Nurse Cook
Arrival First Second
Survival chance 50% 80%
Partisanship AfD None
  • Brazil, Germany, Italy, Poland, US, August 2020
  • N = 6,415, four pairs each
  • Party: a named left or right party, or none
  • Who is polarized: the gap in ratings of the two parties

Results: triage criteria

Figure 5: Marginal means of choosing a patient for intensive care, by attribute level, pooled and for each of the five countries. Survival chance, age, children and arrival move choices most, partisanship less. Source: Stoetzer et al. (2023), Figure 1.
  • Choices mostly follow the guidelines: survival, age, children
  • Partisanship matters less on average
  • No party: 53%, left: 50%, right: 46%
  • AfD patients: 43% in Germany

Results: who discriminates

Figure 6: Effect of a patient’s party on being chosen, compared with a patient with no party, for respondents who are left-polarized, in the middle, or right-polarized by their party ratings. Left-polarized respondents penalize right-party patients most. Source: Stoetzer et al. (2023), Figure 2.
  • Left-polarized: right-party patients \(-16\) pp
  • Right-polarized: left-party \(-5\) pp, right-party \(+5\) pp
  • Hostility and favouritism, in different camps

Why might the left penalize more?

Saying and doing

  • Asked openly what should matter, only 1 to 2.5% name partisanship
  • Yet the choices move with the patient’s party

Which of the two is the respondent’s real view?

Conclusion

  • Partisanship reaches a life and death choice, in most of the five countries
  • The party cue may carry beliefs about compliance with COVID rules
  • No claim about real hospitals: this is about moral judgement

Hahm, Hilpert and König (2024)

Introduction

  1. How strong is the partisan divide next to class, religion and nationality?
  2. Is it driven by in-group favouritism or out-group derogation?
  3. Does coalition partnership soften it?
  • Partyism is a US finding, Europe has old social cleavages
  • Observational data cannot separate party from the identities it proxies
  • Thermometers ask about parties, not fellow citizens

Theory

One hypothesis for each part of the question.

Table 6: The three hypotheses of Hahm et al. (2024) and the mechanism behind each.
Expectation Mechanism
H1 Party outweighs other divides Parties built on long-standing social cleavages
H2 Not mainly out-group derogation In-group love without hate, consensus politics
H3 Coalition partners are treated better A shared, superordinate identity

Research design

Figure 7: An example profile of player 2, with nationality, age, party, gender, religion and social class, and the token allocation box. Source: Hahm et al. (2024), Figure 1.
  • Dictator and trust games, 10 hypothetical tokens
  • 3 + 3 rounds, a new random profile each
  • Six attributes randomized
  • A control with no party shown
  • 25 countries, 2019, N = 29,827

Results: party first

Figure 8: Effect of each randomized attribute on the tokens given to player 2, out-group against in-group, in dictator and trust games, pooled over 25 countries. Partisanship has by far the largest effect. Source: Hahm et al. (2024), Figure 2.
  • Out-partisans: about one token less, a quarter of the baseline
  • Religion second, at half that
  • Class, nationality small, gender none
  • In all 25 countries

Results: love or hate?

Figure 9: Tokens given to a co-partisan and to an out-partisan, against a control with no party information (top), and the absolute size of the two effects (bottom), in dictator and trust games. In-group favouritism is at least as large as out-group derogation. Source: Hahm et al. (2024), Figure 4.
  • Co-partisans: \(+0.5\) tokens, out-partisans: \(-0.5\)
  • Favouritism at least as large as derogation
  • In the US, Iyengar and Westwood (2015) found the reverse

Country, party system, or design?

Results: coalition partners

Figure 10: Predicted tokens for co-partisans and out-partisans, by whether the parties are currently in a coalition together, in dictator and trust games. Coalition partnership lowers the co-partisan bonus and raises what out-partisans receive. Source: Hahm et al. (2024), Figure 6.

  • Coalition partners get more (\(3.43 \rightarrow 3.72\)), co-partisans less (\(4.67 \rightarrow 4.48\))
  • Institutions such as district magnitude or the number of parties: no effect

Conclusion

  • The primacy of partyism holds across Europe
  • European AP is favouritism and derogation in equal parts
  • How parties treat each other matters, institutions do not
  • Caveat: no ethnicity attribute

Which attribute would you add?

Internal and external validity

Back to the primer

  • Randomization makes \(\hat\tau\) a good estimate of \(\tau\)
  • For what was randomized, for these respondents, in this task
  • Does that hold here? Internal validity
  • Is this \(\tau\) the one we care about? External validity

Two kinds of validity

Internal validity: does the design identify the causal effect, for these people, in this task?

External validity: does the effect hold for other people, real stakes, other places and times?

Note

The two often pull apart. A clean, simple task makes the comparison credible. The same simplicity can take it far from the situations we care about.

Your turn

In pairs, three minutes.

Table 7: Strengths and threats to fill in for each paper.
Internal validity External validity
Stoetzer et al. (2023)
Hahm et al. (2024)
  • What is randomized, and what is only measured?
  • Whom, which stakes, and which years do the results speak to?

Open questions

  • Is a hypothetical token a behavior?
  • Would you trust 25 countries of hypothetical tokens, or one lab with real money?
  • If a thermometer and a game disagree, which one is AP?

Wrap-up

Recap: takeaways

  • Behavioral measures ask people to choose, with a trade-off, and target ordinary people
  • Both readings randomize profiles, but differ in task, outcome and how party is coded
  • Only one potential outcome per person is observed, randomization still recovers the average effect
  • Across 25 countries party beats every other divide, and favouritism matches derogation
  • What is randomized has strong internal validity, what is only measured does not

Recap: learning goals

Goal 1. A conjoint allocation game

  • Randomize the profile, let people give tokens, and read the effect of party off the difference in means

Goal 2. Behavioral and attitudinal measures

  • Behavior adds trade-offs and a horizontal target, but hypothetical choices remain cheap

Goal 3. Potential outcomes

  • We see \(Y_i(1)\) or \(Y_i(0)\), never both, but random groups estimate each column’s average

Goal 4. Internal and external validity

  • Internal: is the comparison clean? External: does it travel? Assess both, separately

Goal 5. Coalition partnership

  • Partners share a superordinate identity, so their supporters favour their own less and the partner more

Open round

How do you use AI tools?

  • Which tools do you use?
  • For what: finding literature, reading, writing, code?
  • Where has one misled you?
  • What should the rule for the term paper say?

Next session

  • Session 07: Beyond party: social sorting and non-partisan identities
  • Today party beat religion and class. Next: what happens when they line up?
  • Core reading: Harteveld (2021) and Hobolt et al. (2020)

References

Hahm, Hyeonho, David Hilpert, and Thomas König. 2024. “Divided We Unite: The Nature of Partyism and the Role of Coalition Partnership in Europe.” American Political Science Review 118 (1): 69–87. https://doi.org/10.1017/S0003055423000266.
Hainmueller, Jens, Daniel J. Hopkins, and Teppei Yamamoto. 2014. “Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments.” Political Analysis 22 (1): 1–30. https://doi.org/10.1093/pan/mpt024.
Harteveld, Eelco. 2021. “Ticking All the Boxes? A Comparative Study of Social Sorting and Affective Polarization.” Electoral Studies 72: 102337. https://doi.org/10.1016/j.electstud.2021.102337.
Hobolt, Sara B., Thomas J. Leeper, and James Tilley. 2020. “Divided by the Vote: Affective Polarization in the Wake of the Brexit Referendum.” British Journal of Political Science 51 (4): 1476–93. https://doi.org/10.1017/S0007123420000125.
Iyengar, Shanto, Gaurav Sood, and Yphtach Lelkes. 2012. “Affect, Not Ideology: A Social Identity Perspective on Polarization.” Public Opinion Quarterly 76 (3): 405–31. https://doi.org/10.1093/poq/nfs038.
Iyengar, Shanto, and Sean J. Westwood. 2015. “Fear and Loathing Across Party Lines: New Evidence on Group Polarization.” American Journal of Political Science 59 (3): 690–707. https://doi.org/10.1111/ajps.12152.
Stoetzer, Lukas F., Simon Munzert, Will Lowe, et al. 2023. “Affective Partisan Polarization and Moral Dilemmas During the COVID-19 Pandemic.” Political Science Research and Methods 11 (2): 429–36. https://doi.org/10.1017/psrm.2022.13.