Field experiment

What is a field experiment?

A field experiment tests an intervention in a setting where the behavior normally occurs, such as a workplace, school, shop or public service. Researchers change something and compare outcomes. Many field experiments use random assignment, which helps separate the intervention's effect from pre-existing differences between the groups. A real-world observation without an intervention or defensible comparison is not automatically an experiment.

Test a process where people actually use it

Consider a hypothetical company that wants fewer customer requests to go unanswered. It is considering a handoff form that requires the next responsible person to be named. A field experiment could assign eligible service teams to the new form or the existing process for eight weeks, then measure the proportion of requests still unanswered after two working days.

This measures the process during ordinary work: actual deadlines, staff absences and competing tasks. That is useful because these are conditions the form must survive. The primary outcome is unanswered requests, not merely whether staff completed the form. Record extra time per request and mistaken assignments too; faster completion of the procedure would be little help if it sent work to the wrong person.

Choose the right unit of assignment

Assigning each employee separately could cause contamination. Colleagues share requests and may adopt each other's handoff habits, so the comparison group could partly receive the new process. Assigning whole teams can reduce that problem, although teams that collaborate can still affect each other. These are spillovers: one unit's treatment changes another unit's outcome.

J-PAL's randomization guidance treats delivery and spillovers as reasons to choose the assignment unit carefully. The analysis and sample-size planning must reflect that unit. A trial with many requests from only a few assigned teams does not gain the information of an equally large number of independently assigned requests.

What the field setting does not solve

Randomization still needs to be implemented correctly. Records can go missing. Teams can abandon the assigned process. A manager can unknowingly give one group more help. Define the comparison, follow-up and analysis before examining which version wins, and record what was actually delivered.

Consent, privacy and research oversight depend on the setting and activity. The fact that a change resembles normal operations does not remove those responsibilities. J-PAL's evaluation overview explains why ethical, practical and scientific design questions belong together.

The result would inform this process in these teams over eight weeks. It would not automatically establish a permanent improvement or an effect in a company with different roles and software. That further question is external validity. Use the methods-selection guide to decide whether a field experiment is the evidence your decision needs.