Research methods and evidence

Attributional ambiguity is uncertainty about why someone gave feedback or treated a person in a particular way. In research on stigma, it often concerns whether treatment reflects personal qualities or group-based prejudice.

Bayesian updating is the process of revising probabilities with new evidence, using Bayes’ theorem. A posterior probability combines a prior probability with the likelihood of the evidence under each hypothesis. The result is normalized across the alternatives considered.

Belief updating is revising a belief when new information becomes available. It can involve changing a probability estimate, accepting a different explanation, or adjusting confidence. The term does not imply that the revision is accurate or biased.

A between subjects design, also called an independent groups or between-groups design, compares different participants in different conditions. One group experiences one condition; another group experiences another. The outcomes are then compared.

These study designs differ in whether comparisons involve different people or repeated observations of the same people.

Causal inference asks whether one thing caused another, rather than whether they simply happened together. It uses comparisons and assumptions to estimate what would have happened without the treatment, policy, or event.

Collider bias occurs when a statistical analysis conditions on a shared consequence of other variables, potentially creating or distorting an association. The relevant causes need not be the treatment and outcome themselves; they may also be causes of those variables.

Confounding occurs when a third variable influences both the independent and dependent variables, distorting the association used to estimate a causal effect. Confounding can occur alongside a genuine causal effect. It is an important threat to valid causal inference from observational data.

Construct operationalization means deciding how to measure or manipulate a concept in a study. For confidence, for example, researchers need to specify which questions, scores, or observations will represent it.

Construct validity concerns the evidence for interpreting a measure as an assessment of a particular theoretical concept. A measure behaving as the theory predicts is useful evidence, but one successful prediction does not settle what it measures.

Content validity concerns whether a measure’s content adequately represents the construct it is intended to assess for a particular population and purpose. It asks whether relevant aspects are included, important aspects are missing, and the items make sense to the intended respondents.

Cultural bias is systematic distortion in judgment or measurement arising when culturally specific meanings, assumptions, or procedures are treated as equally appropriate across groups.

Debiasing means trying to reduce systematic errors in judgment or choice. It needs a standard for what counts as better. Changing a person's behavior is not enough to show that an error was corrected.

Digital phenotyping uses measurements from digital devices to study behavior and its relationship with health. Smartphone sensors, device-use records, and surveys can give researchers repeated observations of aspects of daily life. These records capture parts of a person’s activity, not the whole person.

Ecological validity usually refers to how well research represents or generalizes to the everyday conditions we care about. The important question is which conditions.

An effect size describes how large a difference or association is. It helps answer the practical question: how much changed?

Experimenter bias occurs when a researcher’s expectations influence how a study is conducted, observed, or interpreted. One important form is the experimenter expectancy effect: expectations influence interactions with participants or animals and may affect the measured response.

External validity is the extent to which study findings can be generalized to other populations, settings, times, and conditions beyond those specifically examined in the study.

An extraneous variable is a factor outside the focal independent variable that can influence the measured outcome. Examples might include prior experience, fatigue or testing conditions, depending on the study.

Factor analysis models relationships among observed variables using a smaller set of unobserved factors. In a common-factor model, variation shared across measures is separated from variation specific to each measure. Researchers use it to investigate the structure of questionnaires and other measurements.

HARKing means Hypothesizing After the Results are Known: developing a hypothesis after seeing the data, then presenting it as a prediction made beforehand. It makes an explanation fitted to a result look like a successful test of a prior idea.

Internal validity is the extent to which a study can establish a causal relationship between the treatment and the outcome, ruling out alternative explanations.

In a matched-pairs design, researchers pair participants on characteristics relevant to the outcome, such as baseline symptoms or prior achievement. In a randomized experiment, one member of each pair is assigned to one condition and the other to the comparison condition. The pair is split between conditions; it is not assigned as a unit.

Mathematical modeling, also spelled mathematical modelling, expresses a proposed account of a system in equations. In behavioral science, a model might describe how people learn from feedback or choose between options. Making the account precise lets researchers derive predictions and compare them with data.

Measurement error is the difference between a measured value and the variable’s true value. Errors can add noise or systematically push results in one direction.

Mediation describes a proposed pathway through which one variable affects another via an intermediate variable. The mediator represents part of the explanation of how an effect arises.

Mediation analysis estimates a proposed indirect pathway between a treatment or exposure and an outcome through an intermediate variable. It can distinguish direct and indirect effects under a specified model and set of assumptions.

Moderation occurs when the relationship between two variables differs across levels of another variable. A moderator describes when, where, or for whom an association or treatment effect varies.

Moderation analysis tests whether a relationship or treatment effect changes across levels of another variable, often using an interaction term in a regression model.

A natural experiment arises when an event or process outside researchers’ control produces differences in exposure between groups. Researchers can use those differences to study an intervention’s effects. Examples include policy changes, eligibility rules, or lotteries run by public authorities.

Naturalistic observation is systematic recording of behavior in its usual setting without deliberately manipulating the variables of interest. Researchers aim to minimize their influence, but their presence or recording methods can still change behavior.

P-hacking is outcome-guided selection among research choices that makes a reported p-value look more convincing than the full process warrants. Transparent exploration and planned sequential testing are different.

A placebo is an intervention used as a comparison or treatment context without the specific active component under study. A placebo effect is a change caused by aspects of that context, such as expectations or learned associations.

Pre-registration is the practice of recording a study’s hypotheses, methods, and analysis plan before observing the relevant results. It creates a time-stamped record that helps distinguish planned tests from data-informed exploration.

Publication bias is the selective availability of studies according to their results. Missing studies can make an intervention look more effective or dependable than the full evidence supports.

A quasi-experiment is a study that resembles a randomized experiment in structure but lacks random assignment to conditions. It uses naturally occurring groups, policy changes, or pre-existing differences to approximate an experimental comparison.

A randomized controlled trial (RCT) is an experimental design that randomly assigns participants to treatment and control conditions to isolate the causal effect of an intervention. Proper randomization protects against systematic selection into treatment groups, making it a strong basis for causal inference.

Recall bias is systematic error in reports of past events caused by how those events are remembered or reconstructed.

Reliability is the consistency or reproducibility of measurement scores under specified conditions, for a particular population and purpose.

In a repeated measures design, the same participants are measured more than once or take part in more than one condition. Comparing each person’s responses helps researchers study change while accounting for stable differences between people.

The replication crisis is the ongoing discovery that many published research findings in psychology, medicine, and other sciences cannot be reproduced when studies are conducted again by independent researchers. It has prompted scrutiny of how studies are designed, analyzed, and published.

A representative sample adequately reflects a defined population for the characteristics and estimates of interest. Representativeness is not a single property guaranteed by sample size or matching a few demographics.

In questionnaire research, response bias is a systematic influence on answers that is separate from the specific characteristic an item is intended to measure. It can distort survey scores and relationships between measures without requiring deliberate dishonesty.

Sampling bias happens when the people or cases in a study differ in relevant ways from the population its findings are meant to describe.

Self-report bias is systematic error in reports of one’s own behavior, characteristics, beliefs, or experiences.

Statistical power is the probability that a planned analysis will reject a specified null hypothesis when a particular alternative effect is true.