Methods

How economists know things.

Since the 1990s economics has moved from theory first to evidence first. Knowing which method produced a claim tells you how far to trust it.

Rule of thumb

A randomised trial or a clean natural experiment beats a correlation, which beats an untested model. But only a model can tell you why.

Theory and models

Simplify to understand

Emerged 1838 · Augustin Cournot's Recherches used mathematical functions to model demand and competition between two rival producers.

SourceHistory of Economic Thought website: Augustin Cournot (opens in a new tab)

A model strips a problem down to a few assumptions and works out what follows. Introductory maths for economists is mostly the toolkit for this: constrained optimisation, derivatives as “marginal” anything.

Strength

Forces clear thinking and shows mechanisms. Dani Rodrik's Economics Rules (2015) compares models to maps: the skill is picking the right one.

Weak spot

Results can depend entirely on assumptions. Paul Romer (2015) coined “mathiness” for maths used to dress up weak arguments.

Econometrics on observational data

Find patterns in existing data

Emerged 1930 · The new Econometric Society set out to join economic theory with statistics and mathematics, and in 1933 launched Econometrica to publish the work.

SourceThe Econometric Society: About and history (opens in a new tab)

Regression and related tools estimate relationships from data the world already produced.

Strength

Uses large, real datasets at low cost.

Weak spot

Correlation is not causation. Countries with more police may have more crime because crime attracts police.

Natural experiments

The credibility revolution

Emerged 1990 · David Card's article on the Mariel boatlift used the sudden arrival of Cuban migrants in Miami in 1980 as a natural experiment on how immigration affects local wages.

SourceNobelPrize.org: Prize in Economic Sciences 2021, scientific background (opens in a new tab)

Find situations where chance or policy split people into comparable groups: a border, a cut-off score, a lottery. Methods include difference-in-differences, instrumental variables and regression discontinuity.

Strength

Much more convincing causal answers. Card, Angrist and Imbens won the 2021 Nobel; Angrist and Pischke named it the “credibility revolution” in 2010.

Weak spot

Answers can be very local. A result from New Jersey in 1992 may not apply to another country today.

Randomised field experiments

Run the trial yourself

Emerged 1968 · The New Jersey income maintenance experiment randomly assigned low-income families to different negative income tax plans, the first large-scale test of a policy by random assignment.

SourceAlicia H. Munnell, 'Lessons from the Income Maintenance Experiments: An Overview', Federal Reserve Bank of Boston conference series 30 (opens in a new tab)

Randomly assign a policy, like a drug trial, and compare outcomes.

Strength

The cleanest test of cause and effect. Banerjee, Duflo and Kremer won the 2019 Nobel for using them against poverty.

Weak spot

Angus Deaton (Nobel 2015) and Nancy Cartwright argue trial results don't automatically transfer to new places. Some questions, such as monetary policy, cannot be randomised.

Lab experiments

Test behaviour in controlled games

Emerged 1948 · Edward Chamberlin's article “An Experimental Imperfect Market” had students bargain as buyers and sellers in classroom markets to test the theory of perfect competition.

SourceNobelPrize.org: Prize in Economic Sciences 2002, scientific background (opens in a new tab)

Participants play games for real money to test theories of choice.

Strength

The ultimatum game (Güth et al., 1982) showed people reject unfair offers even at a cost to themselves, against the narrow self-interest prediction.

Weak spot

Participants are often students from rich countries. In 2016, 11 of 18 economics lab studies replicated.

Structural and DSGE models

Estimate the machine, then simulate policy

Emerged 1982 · Finn Kydland and Edward Prescott's “Time to Build” paper built business cycles from the choices of households and firms, giving macroeconomic models microeconomic foundations.

SourceNobelPrize.org: Prize in Economic Sciences 2004, popular information (opens in a new tab)

Estimate the deep parameters of a full model of the economy, then run “what if” policy experiments.

Strength

The only way to ask about policies never tried. Used by central banks for forecasting.

Weak spot

Assumptions drive the answers. Romer's 2016 paper “The Trouble with Macroeconomics” is a sharp critique from inside.

Big data and machine learning

New data, new questions

Emerged 2014 · In Science, Liran Einav and Jonathan Levin described how administrative and private-sector data covering whole populations were replacing small surveys in economic research.

SourceLiran Einav and Jonathan Levin, 'Economics in the age of big data', Science 346 (2014) (opens in a new tab)

Tax records, card transactions, satellite images and text, analysed with modern computing.

Strength

Chetty and Hendren (2018) used tax data to show that the neighbourhood a child grows up in causally affects adult income.

Weak spot

Prediction is not explanation, and privacy is a real constraint.

Agent-based and computational models

Simulate many interacting agents

Emerged 1971 · Thomas Schelling's “Dynamic Models of Segregation” showed that weak individual preferences about neighbours can add up to sharp segregation.

SourceNobelPrize.org: Prize in Economic Sciences 2005, scientific background (reference list) (opens in a new tab)

Program thousands of agents with simple rules and watch patterns emerge.

Strength

Captures crashes, networks and herding that equilibrium models miss.

Weak spot

Many free parameters; hard to validate against data.

History and case studies

Archives, narratives, comparisons

Emerged 1843 · Wilhelm Roscher's outline of lectures on the historical method argued that economic laws depend on their historical, social and institutional context, and must be found by studying history.

SourceHistory of Economic Thought website: Wilhelm Roscher (opens in a new tab)

Detailed study of particular episodes, firms or countries.

Strength

Friedman and Schwartz's Monetary History (1963) changed how economists saw the Depression.

Weak spot

Hard to generalise from a few cases.