Digital Enhancements to Traditional Survey Design, Day 2
MACS 30000: Perspectives on Computational Analysis
Ecological Momentary Assessment
Take long surveys, chop them up, and incorporate into lives
Major benefits of EMA
Collection of data in real-world environments
Assesments that focus on individuals’ current or very recent state or behavior
Assessments that may be event-based, time-based, or randomly prompted
Completion of multiple assessments over time
Involves taking traditional surveys, chopping them up into pieces, and sprinkling them into the lives of participants (in contemporary world where we all use smartphones and other wearable devices, lots of potential uses)
Major benefits of EMA
Collection of data in real-world environments
Assesments that focus on individuals’ current or very recent state or behavior
Assessments that may be event-based, time-based, or randomly prompted
Completion of multiple assessments over time
Compared to traditional survey methods, all of this is really useful!
Typically ask questions months after the fact
Respondents have bad memories
Respondents drop out of the study
Cornwell and Cagney (2017)
RQ: How much does residential neighborhood define older adults’ activities? A lot of previous studies assume that getting older implies that people are more anchored to their neighborhood – operationalized as census tracts (and this is thus the primary factor shaping health and well-being). To what extent is this actually true?
Method: gave iPhones to 60 older adults in 4 NYC neighborhoods
Captured GPS location every 5 minutes for a week + administers 17 EMAs at different times over 4 days to assess activities they were performing
Find that they’re much more mobile than typically assumed – activity spaces are much bigger than the residential (census) tracts they’re normally confined to from an analytical perspective
some socioeconomic variation, though: e.g. respondents with less education and lower incomes spent more time outside of their residential tracts than high-status respondents + low poverty in residential tract seems a good proxy for overall exposure to poverty outside
activities inside tract: housework, personal care, leisure, eating
outside tract in >40% observations involving socializing, shopping, exercising
Cornwell and Cagney 2017 “Aging in Activity Space”
Summarize the research design
What is the RQ?
What are the theoretical constructs of interest?
How did the authors operationalize their measure?
How does the research design leverage computational methods?
What are there major findings?
What are the strengths and limitations of their research design?
Enriched asking
Combine digital trace with survey data (1:1)
Record linkage \(\rightarrow\) messy
Digital trace has a core measure of interest and the survey data builds the necessary context around it (complements digital trace data in a useful way)
Here, we’re mapping 1:1 – survey response matched to digital trace data (rather than extrapolating out – as we saw with amplified asking)
General problem is that this method is messy - linking records across databases can be extremely complicated
Will have an opportunity to see this for yourself in MACS 112/122
Kosinski et al. 2013 “Predicting traits from Facebook likes”
Summarize the research design
In particular, what is the role of of enriched asking (e.g. why can’t we just use observational data alone?)
How did the authors gather their data?
Critique: strengths and weaknesses
Ethical concerns
Good example of this in the Kosinski et al. paper you read for today…
[Discuss in groups: 5 min in groups, 10 min as a class; can either solicit responses as a full class or as groups depending on time]
Uses enriched asking to enhance a predictive study of private traits and attributes based on Facebook likes
Need to conduct a survey in order to merge with observed Likes to evaluate the predictive accuracy of the model
Great example of enriched asking
Obtained survey data via an app created for the study - opt-in for participants (not just taking data from Facebook without asking)
Some attributes can be tested strictly using observational data from Facebook
Age
Gender
Relationship status
Sexual orientation
Partisanship
Others cannot be directly revealed with digital trace
Personality - Five Factor Model (OCEAN)
Openness
Conscientiousness
Extraversion
Agreeableness
Neuroticism (emotional stability)
Intelligence
Satisfaction with life
Ethical concerns – thoughts on ethics of this sort of data linking approach?
Can predict private details of people who didn’t even sign off on the survey!
Shortly after publication, Facebook switched Likes to be private by default (previously were public)
Mental Exercise: Failure of US presidential election polls (2016)
Discussion Groups
0
Zhuojun / Yue / Tian
1
Andy / Ertong / Max
2
Anny / Abbey / Kuang
3
Daniela / Jiazheng / Yuhan
4
Huanrui / Pritam / Kexin
5
Lorena / Agnes / Tianle
6
Emma / Cosmo / Thomas
Digital Enhancements to Traditional Survey Design, Day 2 MACS 30000: Perspectives on Computational Analysis University of Chicago