Principles of Ethical Digital Research

MACS 30000: Perspectives on Computational Analysis
University of Chicago

Ethics in Digital Research

Why consider ethics?

Science can have severe physical, mental, and or emotional repercussions on human subjects
- Historical example: Tuskegee Syphilis Study
- Subsequent government regulations and human subject research guidelines:
  - Belmont Report (1978)
  - Menlo Report (2010)
Major concerns in a digital age
- Informed consent
- Unanticipated consequences of data collection
- Algorithmic bias
- Data privacy
Not always clear-cut in the digital age, though…

Science can have severe physical, mental, and or emotional repercussions on human subjects
- Tuskegee Syphilis Study
  - In 1932 government researchers enrolled 400 black men around Tuskegee, Alabama infected with syphilis in a study to monitor the effects of the disease
  - Participants were deceived about the nature of the study
  - No treatment was provided for their disease, even after safe and effective treatments were discovered
  - For over 40 years, these men were on a death watch
  - Even with dozens of researchers and a complete ethical review of the study, researchers continued to withhold treatment
  - Eventually public outcry led to the termination of the study
- Directly led to the creation of government regulations still in place today governing human subjects research
  - Belmont Report (1978)
  - Menlo Report (2010)
Most contemporary ethical dilemmas do not rise to this level of outcry, but still raise important concerns about the propriety of specific methods of research
- Informed consent
- Unanticipated consequences of data collection
- Algorithmic bias
- Privacy and the use of intimate data
Not always clear-cut, though – many of common ethical guidelines obstruct or hinder important social research only possible now in the digital age
- Need to balance ethical concerns of interference with ethical concerns of non-interference
- We don’t want to shut off all research, but we want to ensure research is conducted ethically with equitable outcomes for all involved parties

Four Principles of Ethical Research

Belmont Report (1978)
- Respect for Persons
- Beneficence
- Justice
Menlo Report (2010)
- Respect for Law and Public Interest

Respect for persons

Individuals should be treated as autonomous
Individuals with diminished autonomy should be entitled to additional protections

Beneficence

Obligation researchers have to participants
- Do not harm
- Maximize possible benefits and minimize possible harms
What happens if they are in conflict?
- Need to balance these concerns
- Conduct a risk/benefit analysis
- Make a decision whether the benefits balance the risk;
- Discussion: success-breeds-success study (van de Rijt et al. 2012)

Justice

Distribute the burdens and benefits of research
- Is it reasonable to assign or not assign treatment to one random subset of participants?
Initially meant protecting vulnerable people from researchers
- Example: Studying prisoners due to convenience sampling
- Example: Kosinski et al. (2013)’s study of predicting personal traits using public data from Facebook

Distribute the burdens and benefits of research
- not only in terms of knowledge – think about algorithmic biases that might have real-world consequences for certain groups of people
Initially meant protecting vulnerable people from researchers
- Example - studies of prisoners
- Prisoners have diminished capacity to make free decisions while incarcerated
- Cannot study them as a mere population of convenience
- Requires lengthy IRB approval to ensure prisoners are safeguarded
Also emphasizes that minority groups should be proactively sought to include in study
- Need to be studied in order to benefit
- Example - clinical trials of medicine
  - Ensure not just that children, women, and ethnic minorities are protected but also that they are included in the research so that they can benefit from the findings too
- In a digital age - not all minority groups are represented online

Justice

Also emphasizes that minority groups should be proactively sought to include in study
- Example: Facebook vote mobilization study (Bont et al. 2012)

Respect for law and public interest

Make sure you don’t break the law when conducting research
Compliance
- Terms of Service
- Web scraping
  - HiQ labs v. LinkedIn
- Aaron Swartz
When is non-compliance ethical?
Transparency

Although, implicit in “beneficence,” make sure you don’t break the law when conducting research – focus of beneficence tends to be on participants, whereas this principle from the later Menlo Report emphasizes on bringing the law into consideration
Compliance
- Identify and obey relevant laws, contracts, and terms of services
- Example - web scraping
  - Used frequently to gather data from web sites via machine
  - Usually prohibited by the terms of service
  - Illegal under U.S. law to violate the terms of service
  - However: LinkedIn scraping case
    - 9th Circuit Court (reaffirmed April 2022): “gates up” vs. “gates down” analogy – if site is publicly accessible without login, can reasonably collect data without permission, programmatically or otherwise
    - But: LinkedIn and others will likely continue to fight this decision, so tread carefully…
- Aaron Swartz
  - Co-founder of Reddit
  - Major contributor to Markdown
  - Internet activist
  - Used his access to JSTOR via MIT to download millions of academic journal articles, intending to share them freely with the public
  - Arrested for violating federal law - later committed suicide while under indictment
  - Didn’t comply with the law, you can be prosecuted
- Sometimes non-compliance may be ethical if the TOS are unfair or unnecessarily restrictive, but opens one up to legal risk
Transparency
- Be open about your methods and process
- If you have to make debatable ethical decisions, talk about it and explain how you reached your decision

Common ethical considerations in digital research

Informational risk

Data leakage and anonymization
Massachusetts health data

Informational risk: Massachusetts health data

Informational risk

Need to balance concerns and develop data protection plans
Differential privacy

Privacy

Privacy should be respected because people should be respected.

But what is “public” vs. “private” in the digital world?

If information is publicly accessible, then it can be used by researchers without concerns about violating people’s privacy.

Can it?

Institutional Review Board (IRB)

Who needs to submit to IRB review

Research projects involving human subjects require review and approval by an Institutional Review Board.

Research

A systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge

Human subject

A living individual about whom an investigator (whether professional or student) conducting research obtains (1) data through intervention or interaction with the individual, or (2) identifiable private information.

About whom

To be considered research with “human subjects,” the data received from a living individual must be about the person.

Intervention

Includes both physical procedures by which data are gathered and manipulations of the subject or the subject’s environment that are performed for research purposes.

Interaction

Includes communication or interpersonal contact between investigator and subject.

Private information

Includes information about behavior that occurs in a context in which an individual can reasonably expect that no observation or recording is taking place, and information which has been provided for specific purposes by an individual and which the individual can reasonably expect will not be made public

When are studies not research with human subjects?

Data collection for internal purposes
Internal class projects not designed to be generalizable or used beyond the class
Some examples that do not meet the definition of human subjects (according to the SBS IRB):
- Data available to the general public
- Analysis of data/specimens from deceased individuals
- Observations in a public park
- Thoughts?

Data collection for internal purposes
- E.g. teaching evaluations
Class projects whose purpose is to provide training in research methods and not designed to contribute to generalizable knowledge
- So if you want to conduct human subjects research in the spring term for your research paper, you do not need to get IRB approval
- However, if you intend to then use that data or the same methodology for your master’s thesis, then you do need to get IRB approval
- Cannot obtain IRB approval after the data is already collected - so if you need to reuse the data for generalizable research, get IRB approval before writing your paper in the spring
Some examples that do not meet the definition of human subjects (according to the SBS IRB):
- Data available to the general public
- Analysis of data/specimens from deceased individuals
- Observations in a public park
- Thoughts? [Discuss] Knowing what you know about research involving big digital trace data, can you think of situations where the IRB might allow your research but it might not be ethical by the standards of the four principles?

The IRB is a floor, not a ceiling!

General IRB Considerations

Minimal risk
- “[T]he probability and magnitude of harm or discomfort anticipated in the research are not greater in and of themselves than those ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests.”
- Are research subjects part of a vulnerable population?
- Types of potential risks: physical, psychological, social, economic, legal, etc.
Informed consent
Data confidentiality and provisions to protect privacy of research subjects

Human subjects protection training

Complete human subjects protection training
- “Social and Behavioral Sciences IRB Human Subjects Protection Training Course”
- Most Important Elective Modules for CSS:
  - “Human Subjects Considerations and Big Data Research”
  - “Internet-Based Research”
Must be done before submitting an IRB proposal (and required for this class as a part of your final proposal grade!)

If in doubt about whether your project requires IRB approval, ask!

In MACSS: Sabrina Biggus (Student Affairs Administrator) is our IRB liason who will guide your project through the IRB process

Source: “Navigating the IRB Process” slides from SBS IRB

Generally, though, what this will look like…

Principal investigators are researchers who have primary responsibility for the conduct of a research project
- Typically these are faculty members at a university - students cannot serve as the PI
- If you need to get IRB approval, you will need a faculty advisor to serve as the PI
  - You would then be a co-PI
Basic review takes 2 to 3 weeks
- Only applies to human subjects research with minimal risk or less to the human subjects
  - For full list of exemptions see SBS slides on Canvas – but for instance surveys and interviews and observation of public behavior that don’t involve personally identifiable information are generally exempt. If in doubt, talk with Sabrina Biggus and she can direct you through the IRB process.
- Would most likely be for any research project you want to run
Full board review could take 2 months or more
Get your proposal in early if you are on a timeline

Next class: Evaluating the Ethics of Digital Research

Read Kramer et al. (2014)’s Facebook contagion study and Watts (2014)’s defense and answer: should this research be allowed?
Discuss the ethics of the collection/release of the “Tastes, Ties, and Time” dataset (Zimmer 2010) in light of the 4 principles of ethical research.
Mental Exercise: Debate the ethics of Milkman et al.(2012)’s research design
- Before class: read the Canvas prompt and think about the questions
- In class: Will randomly split class into two groups (so think about both positions as you read!)
  - Group 1: “Balanced across the 4 principles of ethical research, this is an ethical study”
  - Group 2: “Balanced across the 4 principles of ethical research, this is an unethical study”

Principles of Ethical Digital Research

Ethics in Digital Research

Why consider ethics?

Four Principles of Ethical Research

Respect for persons

Beneficence

Justice

Justice

Respect for law and public interest

Common ethical considerations in digital research

Informed consent

Informational risk

Informational risk: Massachusetts health data

Informational risk

Privacy

But what is “public” vs. “private” in the digital world?

Institutional Review Board (IRB)

Who needs to submit to IRB review

Research

Human subject

About whom

Intervention

Interaction

Private information

When are studies not research with human subjects?

General IRB Considerations

Human subjects protection training

Next class: Evaluating the Ethics of Digital Research