Social Science Methods Beyond Causal Inference

Existing scholarship on multi-method research dedicates significant focus to the tandem use of qualitative and quantitative methods toward the goal of causal inference. I expand upon this scholarship to show the utility of multi-method research in other aspects of social scientific research. I provide innovative methodological solutions for scholars researching developing countries or information-constrained contexts where some data are prohibitively expensive, unreliable, or impossible to collect. These points are core contributions in the three papers of my dissertation, Formalizing Tools for Meta-Analysis and Construct Validity in the Social Sciences.

“Bayesian Integrative Meta-Analysis”

• Invited presenter, Southwest Workshop on Mixed-Method Research 2024

Meta-analysis is a sophisticated and essential tool for cumulative knowledge production. However, traditional approaches to meta-analysis systematically exclude qualitative scholarship. This project proposes a systematic framework to deal with this shortcoming of meta-analysis: Bayesian Integrative Meta-Analysis (BIMA). Drawing on existing literature in Bayesian elicitation, I show how information from qualitative manuscripts can be systematically converted into Bayesian prior distribution information via conversion elicitation. This converted qualitative information can be combined and integrated into a Bayesian meta-analysis and the resulting posterior distribution of the standardized effect size results from a qualitatively informed prior and a likelihood distribution composed of the effect sizes from quantitative studies. I demonstrate the framework for BIMA using a toy case with simulated data and an applied example related to the effect of competitive wages on bureaucratic corruption.

“Costly Concepts, (Mis)Aligned Proxies”

• Invited presenter, Emerging Methodologists Workshop, APSA 2023

Practical challenges to measurement may frustrate scholars’ attempts to measure their concepts of interest directly. Researchers may be compelled to use proxy variables that substitute values for the true concept measurement instead, often only assuming that the chosen proxy is good enough. I propose an intermediate-range solution for discovering and reporting the potential measurement divergence between misaligned proxies and the true measures they intend to capture. I suggest obtaining a validation sample for which true measurements are obtained for the concept of interest is always preferable to assuming that a chosen proxy is sufficient. I outline three useful metrics to estimate, report, and potentially incorporate into additional statistical inference strategies. I use a simulation study to demonstrate which sampling procedures are most effective for obtaining these quantities. For more complex population structures, random or traditional stratified samples are informative in estimating metrics related to proxy-costly concept divergence. However, other stratified sampling approaches that consider the potential likelihood of measurement divergence across a population can also be informative of measurement divergence. This guide to validating proxies and reporting measurement divergence provides scholars with a practical and straightforward tool to account for the limitations of measurement choices.

“Concept Measurement: A Unified Framework on the Challenges in Translation and Practice”

• Invited presenter, Emerging Methodologists Workshop - Virtual 2024

This paper develops a novel categorization of the distinct practical challenges that social scientists may encounter when attempting to measure concepts: resource constraints, ethical obligations, confidence in indicators, and unit divergence. I illustrate these practical challenges of concept measurement using various concepts throughout political science and discuss how these challenges may intersect with more theoretical considerations related to translating concepts into operational dimensions. I develop a cost-benefit roadmap for devising alternative concept measurements when researchers encounter insurmountable challenges to pursuing a preferred, theoretically-driven measure. Though this paper’s primary contribution is to the scholarship on concept measurement, it also has implications for transparency in social science. The roadmap I develop provides scholars with a conceptual vocabulary to justify their concept measurement choices, allowing scholars to be more explicit about how different logistical costs of research contour methodological and design choices.

The Legacies of Civil War and Comparative Political Economy

“Subnational Inequality After Armed Conflict in Colombia: Mapping a Research Agenda” with Ana Arjona and Silvia Otero Bahamón

• White paper prepared for World Bank background report “Trajectories: Prosperity and Poverty Reduction in the Colombian Territory”

This study offers one of the first explorations of the association between prior exposure to conflict dynamics beyond violence and subnational inequality in human development. Using detailed data from a random sample of Colombian communities, we assess whether localities previously exposed to armed group presence, the breakdown of order, violence, and governance by guerrillas and paramilitaries display different levels of human development compared to non-exposed areas. Our analysis provides suggestive evidence of negative, positive, and null associations across various development outcomes. Notably, we find that armed group presence is associated with lower access to technology, educational attainment, institutionalized dispute resolution, and enjoyment of rights and freedoms. Additionally, certain components of armed group governance are linked to differences in household assets, educational mobility, obstacles to market entry, access to subsidies, and some indicators of social capital. Our results open new avenues of research and illustrate the importance of advancing our understanding of the consequences of civil war for policy design and implementation.

“The Legacies of Civil War on Citizens’ Preferences for the Rule of Law” with Ana Arjona, Juan Camilo Cárdenas, Ana María Ibáñez, and Patricia Justino

• Under review, Revise and Resubmit

Scholars and policymakers often assume that civil war decreases citizen support for the rule of law (RoL). Yet, this premise has been assumed rather than investigated. To rigorously examine this proposition, we integrate insights from various literatures, deriving hypotheses on specific dynamics of civil war that may decrease support for the RoL. We then test these hypotheses using novel data on Colombia. Contrary to expectations, individuals in former conflict zones and those exposed to violence, disorder, rebel-provided security, or repression within conflict zones do not exhibit decreased support for the RoL. These findings challenge prevailing beliefs about war-affected communities and open new avenues of research on the political legacies of civil war, political behavior, and the RoL. Our findings also have implications for post-conflict interventions aimed at promoting the RoL in post-conflict societies.

“States and Cooperation Capital” with Ana Arjona

Most empirical work on state capacity focuses on the state’s attributes, such as its bureaucratic quality, its inputs, like financial resources, or its outputs, such as public goods provision. However, these approaches tend to overlook a critical element: the ability of the state to elicit various types of cooperation from its subjects, which is integral to its capacity. This paper introduces a new, subject-centered approach to state capacity, focusing on what we term ‘cooperation capital’: the state’s latent ability to elicit cooperative behaviors from its subjects. We propose that measuring behavioral precursors of cooperation — namely, beliefs, emotions, and preferences — provides a valuable proxy of a state’s level of cooperation capital at a given time and location. We illustrate this approach using original qualitative and quantitative evidence on Colombia, demonstrating vast subnational variation in a unique dimension not captured by existing measures. In the future, we plan to extend this approach to the U.S. context. This new empirical approach emphasizes the importance of subjects’ engagement with the state, complementing existing measures that focus on more tangible dimensions of state capacity.

“Challenges of Relying on Individual Testimonies to Describe Community Phenomena: A Research Agenda” with Ana Arjona and Kiran Stallone

A growing literature has demonstrated that gender identity influences how people experience and respond to civil war violence. We ignore, however, whether the data we gather about community-level violence by interviewing people also vary by gender. Moreover, research has not yet addressed whether differences in reporting about other wartime dynamics beyond violence can also vary across people of different genders. If such systematic reporting differences do exist, they could impact the quality of our descriptions of conflict zones and hinder our ability to explain wartime phenomena. This paper explores whether there are systematic differences between how men and women report on community events during war. Relying on original survey data on two radically different civil wars — in Colombia and Kosovo — we compare the responses of men and women regarding not only violent events in their communities but also other behaviors and practices of combatants (especially those related to rebel governance), and about civilian behavior vis-à-vis armed groups. We find that men and women do systematically report some local dynamics differently, especially when it comes to the behavior of community members towards armed groups. These differences, however, are likely to be context specific. We argue that these disparities have profound implications on how we describe and explain the phenomena that occur in war zones and conclude by proposing a research agenda on gender and data collection in civil war studies.