Activity 3: Randomized Experiment Balance and Estimation in Gerber et al. 2008

Activity 3: Randomized Experiment Balance and Estimation in Gerber et al. 2008#

2026-09-22

# imports we'll need
import numpy as np
import pandas as pd

First, let’s load in the data at ~/COMSC-341CD/data/gerber2008.csv and take a look at the first few rows via DataFrame.head().

# TODO uncomment to load data
#vote_df = pd.read_csv("~/COMSC-341CD/data/gerber2008.csv")
#vote_df.head()

# TODO examining the top and bottom of the dataframe

# TODO giving a summary of the dataframe

# TODO get the column names

# TODO get the number of rows and columns

# TODO return the data types of each column

The columns we have are:

  • sex: the respondent’s sex

  • yob: the respondent’s year of birth

  • treatment: the treatment the respondent received

  • voted: our outcome: whether the respondent voted in the 2006 primary election

  • hh_size: the size of the respondent’s household

  • g2000, g2002, g2004: whether the respondent voted in the 2000, 2002, and 2004 general election

  • p2000, p2002, p2004: whether the respondent voted in the 2000, 2002, and 2004 primary election

1. Column generation#

Let’s exmaine the treatment column by running .unique() on it:

# TODO show the unique values of the treatment column

These two options correspond to the \(T=0\) and \(T=1\) interventions we discussed earlier:

\[\begin{split} T = \begin{cases} 1 & \text{mail stating that their household turnout would be publicized to their neighbors} \\ 0 & \text{mail reminding them that voting is a civic duty} \end{cases} \end{split}\]

Let’s generate an additional column that corresponds to the respondent’s age in 2006.

# TODO generate the 'age' column

2. Balance check#

One way we can check whether exchangeability is plausible is to check whether the distribution of data characteristics are the same between the treatment and control groups.

Let’s now check the balance of the various characteristics between the \(T=1\) ('Neighbors') and \(T=0\) ('Civic Duty') groups.

To do this, we can call the groupby(column) method on the dataframe and then use the mean() method:

# example usage of groupby and mean
df.groupby('treat_column')['continuous_column'].mean()

For the continuous variables of age and hh_size, let’s compute the mean for each group of the treatment column.

# TODO your code for the grouped mean ofage

# TODO your code for the grouped mean of hh_size

Let’s also check the balance of the categorical variables across the two treatment groups.

For categoricalvariables, we can use the value_counts() method with normalize=True along with .groupby() to compute the proportions for each individual characteristic:

# example usage of value_counts and groupby
df.groupby('treat_column')['categorical_column'].value_counts(normalize=True)

For the each of the categorical variables of ['sex', 'g2000', 'g2002', 'g2004', 'p2000', 'p2002', 'p2004'], let’s compute and print the proportion of respondents who have the characteristic for each treatment group:

categorical_vars = ['sex', 'g2000', 'g2002', 'g2004', 'p2000', 'p2002', 'p2004']

# loop over all the categorical variables
for column in categorical_vars:
    # TODO your code for grouped value counts
    pass

Are there any differences between the \(T=1\) and \(T=0\) groups in terms of all of these characteristics, or do they seem approximatelycomparable?

Your response: TODO

3. Causal effect estimation#

Finally, let’s estimate the ATE for the voted outcome. From what we looked at in class, because this is a randomized experiment, we can estimate the ATE by computing the difference in means between the \(T=1\) and \(T=0\) groups for the voted outcome. For this particular dataset, that means:

\[\begin{split} \begin{align*} \text{ATE} = E[Y(1) - Y(0)] \xrightarrow[]{\text{Identification}} \; &E[Y | T=1] - E[Y | T=0]\\ E[Y | T=1] - E[Y | T=0] \; \xrightarrow[]{\text{Estimation}} \; &\hat{E}[Y | T=1] - \hat{E}[Y | T=0]\\ =&\hat{P}(\text{voted='Yes'} \mid \text{treatment='Neighbors'}) - \hat{P}(\text{voted='Yes'} \mid \text{treatment='Civic Duty'}) \end{align*} \end{split}\]

Tip

Try computing the two proportions separately and then subtracting them.

We can compute the proportion of respondents who voted ‘Yes’ in each treatment group by using the .value_counts() method with normalize=True and indexing on the 'Yes' category.

# TODO your code for estimating the ATE
estimated_ATE = 0

Write down an interpretation of this estimate as a causal quantity.

Your response: TODO

References#

Contents