Observing Behavior: Digital Trace (Big) Data

MACS 30000: Perspectives on Computational Analysis
University of Chicago

Agenda

I will release next week’s module on Friday.
Plan: Assign Sourati & Evans (2023) (the paper you read for the 1st week of CSS workshop!) and try to crack it.
Goal: Understand a CSS paper without fully understanding the math.

On Tuesday

Chan et al. 2021 “Can psychological traits explain mobility behavior during the COVID-19 pandemic?”

What is the primary research question?
What data do the authors use to answer this question?
How do they attempt to bridge the gap between the theoretical constructs in their research question and the data they use in the paper? Assess the construct validity of this study.
How do the authors answer their research question? What methods do they use?
What are the authors’ main findings?
What are some strengths and limitations of this research (design)? Consider both the data and the methods used. (In other words, why did this paper get published?/What is its main contribution (special ingridient)? What is left unanswered?)

Note that only one question is on the main findings.
The primary focus of this class is on research design. We focus less on understanding statistical analyses.
But, you can understand the main findings and contributions of most CSS papers if you can crack their main figures.
Understanding figures in CSS papers requires some background training in statistics. But as you get used to it, you can obtain a lot of information by just looking at some main figures.

Don’t you find these figures wierd?

Openness has a positive effect on mobility change? And agreeableness has a negative effect?

{width = “30%” fig-align=“center”}

What is going on?

In-Class On Thursday

Shi et al. 2017 “Millions of online book co-purchases reveal partisan differences in the consumption of science”

What is the primary research question?
What data do the authors use to answer this question?
How do they attempt to bridge the gap between the theoretical constructs in their research question and the data they use in the paper? Assess the construct validity of this study.
How do the authors answer their research question? What methods do they use?
What are the authors’ main findings?
What are some strengths and limitations of this research (design)? Consider both the data and the methods used.

[~25 minutes: 15 min. in groups, 10 min. full class; write answers on board]

I had you read one such paper for today’s class that used counting as the prime metric for evaluation – Shi et al. 2017.

Let’s dissect this study using some of the principles we’ve been talking about this week with regard to the strengths and weaknesses of “Big Data” as well counting/measuring.

Reading Notes

Studies political polarization in the context of science
- Is science apolitical or nonpartisan?
Disconnect between survey analysis and observational data
- Asking people is not always the best strategy
- Misleading/incorrect answers
- Don’t remember
- Implicit bias/people are not actually aware of their polarization or true preferences on science
Uses book co-purchase data to reveal the network of consumption of science
- Novel research strategy
- Leverages digital trace data without requiring individual-level purchases
Counting strategy
- Count co-purchase links between scientific books and political books to identify the probability of a linkage given info on the nodes
- Calculates probabilities for a Binomial distribution
- Uses Bayes’ Rule(!) to calculate these values
Nonreactive
- Studying people in their natural state
- Free to purchase and consume the science they actually care about and are invested in
- Doesn’t require interference in the system
Inaccessibility
- Amazon and Barnes and Noble have no incentive to provide individual-level purchase data
- Instead, authors just scrape publicly accessible data from their websites
- Overcomes access issues
- Also limits the types of questions that can be answered
  - Can only describe the state of the network/co-purchase links
  - Undirected network analysis
  - Cannot assert causality
    - Does purchasing conservative books cause individuals to purchase specific books in the sciences?
    - Does purchasing scientific books which are more central lead to moderation in political ideology?
    - None of these questions are answerable with the existing data
Algorithmic confounding
- Should we trust the co-purchase data? It is derived from Amazon’s algorithms which are intended to sell more books
- Authors argue that is not a problem here because the algorithm’s intended purpose coincides with the research project - recommended books should be things individuals are already likely to purchase
- Do you buy this argument?
Open access data
- Site hosting code and data
- Not all source data is publicly available
- Analysis code and supplementary tables open for inspection and further exploration

Mental Exercise

Group Assignment

	Week 2 Discussion Groups
0	Andy / Abbey / Kuang
1	Jiazheng / Cosmo / Kexin
2	Pritam Rashtrapal / Agnes / Tianle
3	Huanrui / Tian / Max
4	Daniela / Ertong / Zhuojun
5	Emma / Lorena / Yue
6	Anny / Yuhan / Thomas

Read Stephens-Davidowitz (2014) and understand how the author used Google’s search records to measure racial animus in the United States. Critically evaluate the strengths and weaknesses of the measure.

Play around with Google Trends to see what functionalities it offers and try to figure out Google’s methodology by checking out information that Google publicly releases (such as its FAQ page).

Then, consider the following scenario:

You were asked to conduct a study on how the U.S. public’s attitude toward mask-wearing varies across regions during the COVID-19 pandemic. You thought maybe you could obtain some good measurements by making use of Google’s search records. How would you construct your measure(s)? What search queries (or combination of search queries) would you use? What research question(s) does your measure best answer? (i.e. what public attitude/interest does your query exactly measure?) What does it NOT measure? What are possible measurement errors and biases?