Research Methods in Psychology and Behavioral Science: Choosing the Right Design

A team wants more people to attend its training sessions. Someone suggests a survey. Someone else wants an A/B test. Both may be useful, but they answer different questions. A survey can help you learn why people say they are missing sessions. A randomized test can help you estimate whether a particular change increases attendance. Neither is useful until you know what you need to find out.

Psychology research methods include observation, interviews, surveys, experiments, longitudinal studies and evidence synthesis. Applied behavioral science uses these methods to understand what people do, why patterns might occur, and what happens when something changes. The method should follow the question. Choosing a familiar technique first and finding a question for it afterward is a poor way to learn.

First decide what kind of answer you need

Start by separating four tasks. Description asks what is happening: how many people attend, who leaves early, or where a process breaks. Understanding experience asks how people interpret the situation and what matters to them. Prediction asks whether information available now helps anticipate a later outcome. Causal evaluation asks what would happen if you changed one thing rather than another.

A predictor need not be a cause. People who ask the most questions before a course may be more likely to complete it. That does not show that making everyone ask extra questions will improve completion. They may already have more interest, time or prior knowledge. The practical question is whether a proposed action changes the outcome, not whether an existing characteristic accompanies it.

Before choosing an intervention, also question the behavior you are trying to encourage. If the goal is useful learning, attending a two-hour evening session is only one possible route. A shorter daytime session, supervised practice or a different task may fit people better. The behavioral-design overview explains why that choice belongs before the work of optimizing an invitation.

The main research methods and what they can answer

Observation and behavioral records

Observation records what happens. A researcher might watch someone book a session, or inspect attendance records across a term. Structured observation uses defined categories so that observers count the same things. More open observation can reveal unanticipated problems: perhaps the room is hard to find or the booking form fails on a phone.

Records give you behavior in context, but their meaning needs checking. A page view is not a completed task. A missing check-in can mean absence, a scanner failure or a different entrance. Decide which events count, inspect exceptions and make the observation period explicit. If you are watching people directly, their awareness of being observed can also affect what happens.

Interviews, focus groups and qualitative research

Qualitative methods examine experiences, meanings and processes through material such as interviews, observations or documents. An interview can explore how work schedules interfere with attendance. A focus group can reveal how participants discuss a shared problem, although agreement in a group may reflect its social dynamics as well as people's private views.

Ask about specific episodes rather than inviting a general theory of behavior: “Walk me through the last time you planned to attend and did not.” Look for differences between people and cases that challenge the first explanation. Document recruitment, questions, analysis and how interpretations were checked. Six interviews can expose an obstacle worth investigating; they do not establish what percentage of all students face it. J-PAL's qualitative-methods guide explains how to choose and conduct interviews and group discussions within an evaluation.

Surveys and questionnaires

A survey collects reports about experiences, attitudes, intentions or behavior. It can describe how common a reported obstacle is if the sampling and measurement support that conclusion. Who can be reached, who responds and who is missing matter as much as the response count.

“How helpful and convenient was the course?” asks about two things at once. Separate them. A course can be useful but scheduled at an impossible time. Ask neutral questions, test whether people understand them, and distinguish reported attendance from attendance records. AAPOR's survey guidance addresses sampling, question wording and whether a survey is the right tool in the first place.

A questionnaire is a measurement method, not a complete study design. It can be used in an observational study or to measure outcomes in a randomized experiment. The questionnaire alone does not create a causal comparison.

Cross-sectional and longitudinal studies

A cross-sectional study compares observations from one period. It can describe differences between students who attend regularly and those who do not, but it may be unclear which difference came first. A longitudinal study follows observations over time, allowing you to examine trajectories, persistence and temporal order.

Following the same people is useful if the question concerns change within a person. It still does not automatically remove confounding. A student may attend more and report less stress after their work schedule improves; the earlier change in attendance is not sufficient to establish that attendance caused the later reduction in stress. Attrition matters too: a cheerful end-of-course survey among only the remaining students can conceal the experience of those who left.

Experiments and randomized controlled trials

An experiment deliberately changes a condition. In a randomized controlled trial, chance determines assignment to the compared conditions. This makes pre-existing characteristics comparable in expectation. Chance imbalance, missing outcomes, spillovers and poor delivery can still complicate the result.

A laboratory experiment can isolate a process under controlled conditions. A field experiment tests an intervention during ordinary activity. A digital A/B test is often a randomized experiment, but the label does not prove that assignment or measurement was sound. Specify whether the unit is a person, team, classroom or time period, and analyze the data accordingly.

Quasi-experiments and observational causal methods

Sometimes random assignment is impractical or inappropriate. A policy threshold, staged rollout or change affecting one group can offer a useful comparison. A regression-discontinuity design examines outcomes near a treatment cutoff. Difference-in-differences compares changes across treated and comparison groups. Each needs a specific argument about why the comparison identifies the desired effect.

A simple before-and-after improvement is weaker. If attendance rises after a schedule change at the same time a new instructor starts, the comparison does not separate those causes. More data about the same before-and-after contrast will not fix that ambiguity. The causal-inference entry explains the assumptions of several nonrandomized approaches; Hernán and Robins's methods text develops the underlying counterfactual question.

Systematic reviews and meta-analysis

Before collecting new data, inspect what is already known. A systematic review uses explicit methods to find and assess research addressing a question. A meta-analysis statistically combines compatible estimates. Neither turns a weak or selectively reported literature into certain knowledge. Use the evidence to refine the new study's question, comparison and plausible effect sizes.

Work through the attendance problem

Here is a hypothetical design decision. An adult education provider has reliable attendance records showing many missed sessions. It is considering a reminder sent 24 hours before a booked session. The relevant question is not whether people like reminders. It is whether assigning the reminder increases attendance at the session enough to justify its cost and inconvenience.

First, inspect the booking and check-in process. A missing record must mean the same thing in both groups. Then interview attenders and nonattenders about specific missed sessions. Suppose this exploratory work identifies two plausible obstacles: forgetting and incompatible schedules. Do not turn that discovery into an estimate of how many people have each obstacle. A suitable survey could investigate prevalence, while testing the reminder would answer a different question about its effect.

Possible design	What it contributes	What remains unresolved
Interviews with people who missed sessions	Detailed accounts of barriers and candidate alternatives	How common each barrier is and whether a remedy causes improvement
Survey of eligible students	Reported barriers and preferences, subject to sampling and measurement quality	Whether people would attend more if sent a reminder
Attendance before and after sending reminders to everyone	A description of change over time	Other changes in schedules, instructors and student composition
Randomized reminder versus usual communication	An estimate of the effect of assignment on attendance during the study	Longer-term learning, effects elsewhere and benefits of untested alternatives

If the provider can implement random assignment, protect participants and measure both groups consistently, the randomized comparison fits the immediate decision. Interviews remain useful for interpreting delivery problems and developing alternatives. They do not need to pretend to be an experiment to earn their place.

Write a research-design brief

A short brief makes the choice reviewable before the study begins. This filled example is a proposal, not a report of completed research.

Question: Among eligible adult students booked for a session, does an additional reminder 24 hours beforehand increase attendance compared with usual communication?
Population and recruitment: Adults with a valid contact number who book during the next term. Use the first eligible booking per person. Record who is excluded, including those without a reachable number; the estimate will not directly cover them.
Intervention and comparison: A single factual reminder versus the existing booking confirmation. Keep access, cancellation rules and other communications the same.
Assignment: Randomly assign eligible people before the reminder window, within course and session-date groups where feasible. Save the assignment record and prevent manual reassignment based on engagement.
Primary outcome and period: Attendance at the booked session, measured through the same verified check-in process. Define how cancellations, rescheduling and missing check-ins will be classified before assignment.
Secondary outcomes: Record delivery failures, complaints, cancellations and staff time. Study later course completion separately; this short test cannot establish durable learning.
Effect and sample-size rationale: Report the attendance-rate difference in percentage points, its confidence interval and the relative difference with its baseline. Choose the sample from a justified smallest worthwhile effect and power calculation before the study; do not infer an adequate number from how many students happen to be available.
Analysis: Compare assigned groups, accounting for the assignment design. State how missing outcomes, multiple secondary tests and any prespecified subgroup comparison will be handled. Do not remove people simply because the reminder was not opened.
Main limitation: This tests one reminder among reachable, already-booked students. It does not identify the best class time, fix schedule conflicts or establish effectiveness for people who never booked.
Decision: Compare the estimated gain and its uncertainty with delivery cost, unwanted effects and a realistic alternative. Continue, revise or investigate another behavior based on those results—not merely on whether p is below .05.

For your own brief, replace each answer with the details of your setting. An unanswered item is useful: it identifies something to resolve before collecting data. An individual making a small, reversible change may need far less formality; Try, observe, and adapt explains what a personal trial can teach. The brief is especially useful when other people will rely on your conclusion or a costly program depends on it.

Resolve these choices before the first result

Check feasibility as carefully as statistical elegance. Can the provider send the correct message at the correct time? Can it link a booking to attendance without exposing unnecessary personal information? Could classmates forward the reminder to one another? Would withholding it remove an established entitlement or create a foreseeable harm? Resolve consent, privacy, oversight and stopping requirements appropriate to the study before launch.

Then record the main plan. Make exploratory work visible without treating it as a failure. A pilot may reveal that the attendance scanner misses late arrivals; repairing the measure is progress, even if it delays the comparison. Keep that pilot separate from a later confirmatory test where needed.

The output of good methods selection is a defensible answer to a specific question. Once results arrive, Read the evidence before the promise helps you interpret them. A randomized reminder test might show that a reminder helps a little. It would still leave open whether changing the time, format or activity helps much more.