Quantitative methods for policy impact evaluation

Aims of the course

The course is intended for students who want to learn about a range of tools and techniques for public policy evaluation. Many pressing problems and challenges in contemporary public policy are based on the analysis of larger amounts of data and necessitates the use of causal inference tools. The course offers a broad and practical introduction to program and policy evaluation techniques for decision making based on database modeling, data mining, machine learning, data visualization, relational databases using software tools such as R Studio and network analysis, pattern recognition analysis and unstructured data analysis. Upon course completion, students will be equipped with the modern tools of data-driven analysis of public policy for small and larger datasets and will be able to apply them accordingly.

Course syllabus

1. Introduction to data analysis
a. Sources of data
b. Data quality and preparation
c. Short review of linear regression

2. Causal inference and program evaluation
a. Causality and potential outcomes
b. Counterfactuals

3. Randomized experiments
a. Identification through randomized assignment
b. Estimation and inference in randomized studies
i. Sampling-based inference from frequencies
ii. Randomization inference through sharp null hypothesis
iii. Permutation methods
c. High dimensional inference based on large datasets

4. Conditioning on observables
a. Identification by conditional independence
b. Regression adjustment
c. Matching estimators
i. Matching on covariates and nearest neighbors
ii. Matching on propensity scores
d. Inverse probability weighing
e. Imputation and projection methods
f. Hybrid models
g. Comparison and choice of estimators
h. Recent developments and further topics

5. Difference-in-differences and synthetic control analysis
a. Difference-in-differences with homogeneous and heterogeneous treatment
b. Difference-in-differences with staggered treatment assignment
c. Synthetic controls
i. Estimation and inference
ii. Placebo analyses
iii. Analysis with penalization
iv. Matrix completion methods and recent developments

6. Instrumental variables
a. Framework
b. Local average treatment effects
c. Identification and estimation
d. Marginal treatment effects
e. Diagnostic and identification tests

7. Regression discontinuity design
a. Estimation and inference
b. Falsification and validation
c. Extensions and additional topics

8. Matching
a. Artificial Comparison Groups
b. Propensity Score Matching

Course director(s)