# Choice-Level Analysis

## Why Choice-Level Analysis?

💡 Choice-level analysis is simpler, easier, and more powerful than profile-level analysis.

- 🧠 Conjoint designs began in market research and psychology, where respondents rated two profiles (e.g., products). Each rating was treated as an independent observation, creating the profile-level design with `2 × n` rows for `n` respondents.
- 🔍 Social scientists later used conjoint designs for choices instead of ratings but kept the same structure with a single binary outcome. This shift introduced statistical and conceptual issues.

### ⚠️ Problems with Profile-Level Analysis

🚫 Profile-level analysis forces researchers to correct a dependence that they created themselves.

- 🔁 Redundant structure: Each choice task generates two rows per task for each respondent even though there is only one independent choice.
- 🔗 Mechanical dependence: Selecting one profile necessarily implies rejecting the other, violating the independence assumption.
- 🧩 Artificial complexity: Analysts must correct for this dependence using complicated statistical adjustments, even though it arises solely from how the data were organized—not from respondents’ behavior.

### ✅ Advantages of Choice-Level Analysis

In contrast, choice-level analysis organizes data by respondent decisions rather than profiles.

- 📋 Each unit of observation represents a choice for a given task per respondent.
- ⚡ The outcome variable reflects which profile was chosen in that task.
- 🧠 This data structure allows researchers to model the choice conditional on the full comparison.

🎯 Choice-level analysis directly models the respondent’s decision between two (or more) alternatives,
capturing the true structure of the conjoint task.

## Key Issues and Applications

- Profile-level estimands like AMCEs assume that each profile is generated independently and ignores how respondents evaluate one profile relative to another. This limits the types of questions researchers can ask.

- Choice-level analysis allows researchers to explore questions that explicitly depend on the comparison between profiles, such as:

Examples of Choice-Level Research Questions

- 🗳️ Do voters choose a white candidate over a non-white candidate?
  (The levels—white vs. Asian, Black, Hispanic—always differ between profiles.)

- 🌐 Do Asian Democrat respondents prefer an Asian Republican over a white Democrat?
  (Profiles are intentionally designed with multiple correlated attributes.)

- 📊 Do voters care about electability?
  (The two percentages representing win probability must sum to 100.)

- ⚖️ Do voters prefer the status quo over a policy proposal?
  (One profile is fixed while the other varies across tasks.)

- 🧭 How much do voters prefer extreme left-leaning or extreme right-leaning policies?
  (Attributes are consistently positioned on the ideological spectrum.)

Furthermore, when individuals compare profiles side-by-side, their evaluations are often psychologically influenced by the alternative, such as through assimilation or contrast effects
(see Horiuchi and Johnson 2025).

## Why Move to Choice-Level Analysis?

🔍 Choice-level analysis models the decision between two profiles, not the evaluation of a single profile.

This structure more closely mirrors:

- 🧠 Real-world decision-making, where people choose among competing alternatives.
- 🔄 Comparative cognition, where evaluations depend on the context of available options.
- 🎛️ Tradeoff reasoning, where respondents weigh attribute combinations jointly.

Hence, rather than estimating the probability of selecting an isolated profile, choice-level analysis estimates the probability of choosing one profile over another, conditional on all attributes involved.

✅ Mirrors real-world behavior
✅ Captures comparative judgment and psychological context
✅ Reveals authentic tradeoffs and priorities

## Summary

Profile-Level Analysis | Choice-Level Analysis
--- | ---
Treats profiles as independent | Models the decision between profiles
Ignores comparative context | Captures mutual influence of options
May blur or bias tradeoffs | Highlights actual tradeoffs
Can misstate uncertainty | Produces more interpretable estimates
Requires complex correction methods | Works with simple and transparent models

## Key Takeaway

🚀 If your conjoint design presents respondents with two or more profiles for comparison,
then choice-level analysis is essential for valid, interpretable, and psychologically realistic inference.

It provides:

- Deeper insights into human decision-making
- Cleaner estimation procedures
- Closer correspondence to real-world behavior

## 📚 References

- Clayton, Horiuchi, Kaufman, King, & Komisarchik (Forthcoming).
  Correcting Measurement Error Bias in Conjoint Survey Experiments.
  Forthcoming, American Journal of Political Science.
  Preprint available

- Horiuchi & Johnson (2025).
  Advancing Conjoint Analysis: Delving Further into Profile Comparisons.
  Work in progress.