The History of Operant Conditioning

Who discovered operant conditioning: Thorndike or Skinner?

Thorndike was the first to show experimentally, with learning curves rather than anecdotes, that the consequences of a response change its future probability. Skinner was the first to treat that fact as the foundation of a science: he defined the operant as a class of behavior controlled by its consequences, distinguished it from Pavlov's reflexes, built the free-operant chamber and cumulative recorder, and discovered the phenomena — schedules, shaping, extinction, stimulus control — that make up the subject today. Thorndike's own term, instrumental learning, survives as "instrumental conditioning," a synonym still used in the discrete-trial tradition.

Timeline: the history of operant conditioning

• c. 350 BCE

  Aristotle on habit

  In the Nicomachean Ethics Aristotle argues that character is built by repetition — “we become just by doing just acts” — and that the pleasure or pain attached to an act is what decides whether it is repeated. It is not the law of effect: there is no experiment, no measurement of response frequency, and no functional definition of a reinforcer. But the intuition that consequences shape conduct is about as old as written philosophy, and Thorndike’s contribution was to make it testable rather than to think of it.^[48]

• 1855–1882

  Bain and Romanes

  The Scottish philosopher Alexander Bain describes how spontaneous movements that happen to produce pleasure are retained and those producing pain dropped — a law of effect before the name. George Romanes collects anecdotes of animal cleverness in Animal Intelligence (1882), a method soon discredited.^[1][2]

• 1894

  Morgan's canon

  C. Lloyd Morgan rules that no animal behavior should be explained by a higher mental faculty if a lower process will do. Comparative psychology adopts the rule, and the search for simple learning processes begins.^[3]

• 1898

  Thorndike's puzzle boxes

  For his Columbia dissertation, Edward Thorndike puts hungry cats in latched boxes with food outside and times their escapes. Escape times fall gradually, with no sudden insight; he proposes that satisfying consequences "stamp in" the connection between situation and response.^[4]

• 1911–1932

  The law of effect, stated and then truncated

  Animal Intelligence (1911) names the law of effect. Two decades later, on the basis of human learning experiments, Thorndike concludes that "annoyers" do not weaken connections the way "satisfiers" strengthen them, and drops the negative half of the law — the first evidence that punishment and reinforcement are not mirror images.^[5][6]

• 1913

  Watson's behaviorist manifesto

  John B. Watson declares that psychology's subject matter is behavior, not consciousness, and its goal prediction and control. Behaviorism becomes a movement — though its early learning theory rests on Pavlov's reflex rather than Thorndike's law.^[7]

• 1927

  Pavlov in English

  Ivan Pavlov's Conditioned Reflexes is translated, giving American psychology the terms "reinforcement," "extinction," "generalization," and "discrimination" — all of which Skinner will borrow for a different kind of learning.^[8]

• 1928

  Konorski and Miller's "type II" reflex

  Two Polish physiologists, Jerzy Konorski and Stefan Miller, passively flex a dog's leg and then feed it; the dog begins flexing on its own. They call this a second type of conditioned reflex, distinct from Pavlov's, and later argue the point with Skinner in print.^[9][10]

• 1930–1938

  Skinner: the chamber, the operant, the book

  At Harvard, B. F. Skinner builds the operant chamber and cumulative recorder and defines behavior by its function rather than its form (1935); at Minnesota, where he takes his first faculty post in 1936, he coins "operant" and "respondent" (1937) and publishes The Behavior of Organisms (1938).^[10][11]

• 1943–1944

  Hull's system; Estes on punishment

  Clark Hull's Principles of Behavior offers a rival, mathematical behaviorism in which reinforcement works by reducing a biological drive. Skinner's student W. K. Estes shows that punishment suppresses lever-pressing only temporarily, evidence Skinner will cite against punishment for the rest of his life.^[12][13]

• 1948

  Cognitive maps and superstitious pigeons

  Edward Tolman argues from latent-learning and maze studies that rats form "cognitive maps," not chains of responses. The same year Skinner shows that pigeons fed on a timer develop idiosyncratic rituals — accidental contingencies shape behavior with no understanding required.^[14][15]

• 1950

  A textbook and a manifesto

  Fred Keller and William Schoenfeld's Principles of Psychology teaches introductory psychology entirely from reinforcement principles. Skinner's "Are Theories of Learning Necessary?" argues for describing functional relations rather than inventing internal mechanisms.^[16][17]

• 1957–1958

  Schedules; a journal

  Ferster and Skinner publish Schedules of Reinforcement. The Journal of the Experimental Analysis of Behavior is founded in 1958, giving the field its own outlet and its own methods: single organisms, steady states, within-subject replication.^[18][19]

• 1959–1961

  Premack, Herrnstein, the Brelands, Chomsky

  David Premack shows that a more probable behavior can reinforce a less probable one. Richard Herrnstein finds that pigeons match response ratios to reinforcement ratios — the matching law. The Brelands report trained animals drifting toward instinct — "instinctive drift." Noam Chomsky's review of Verbal Behavior opens the cognitive revolution.^{[20][21][22][23]}

• 1968

  Applied behavior analysis is born

  Teodoro Ayllon and Nathan Azrin publish The Token Economy, from their work on a psychiatric ward. The Journal of Applied Behavior Analysis launches, and Baer, Wolf, and Risley's paper in its first issue defines what makes an analysis "applied."^[24][25]

• 1972

  The Rescorla–Wagner model

  A model of classical conditioning in which learning is driven by surprise — the gap between what was predicted and what occurred. Though about Pavlovian learning, it reshapes all of learning theory and, decades later, turns out to describe what dopamine neurons compute.^[26]

• 1982–1987

  Functional analysis, momentum, and Lovaas

  Brian Iwata and colleagues show that self-injury can be experimentally traced to its reinforcer, making assessment of function the first step in treatment. John Nevin's behavioral momentum research shows that persistence depends on the rate of reinforcement in a context. Ivar Lovaas reports that intensive early intervention brought nine of nineteen autistic children to normal educational functioning.^[27][28][29]

• 1997–1998

  Dopamine, reinforcement learning, and a credential

  Schultz, Dayan, and Montague show that midbrain dopamine neurons signal a reward prediction error. Sutton and Barto publish Reinforcement Learning: An Introduction. The Behavior Analyst Certification Board is founded, and behavior analysis becomes a credentialed profession.^[30][31]

• 2015–2025

  Deep reinforcement learning

  Reinforcement learning combined with deep neural networks masters Atari games from pixels and then Go; the same family of methods is used to fine-tune large language models from human feedback. In 2025 Sutton and Barto are awarded the ACM Turing Award (for 2024) for founding the field.^[32][33]

From the law of effect to the operant: what Skinner changed

It is tempting to read Skinner as Thorndike with better equipment. The differences are deeper, and they explain why the field is called the experimental analysis of behavior rather than the study of trial-and-error learning.

Add the distinction from Pavlovian conditioning and you have the framework every page on this site uses. Operant vs. classical conditioning ›

The cognitive revolution: what it overturned and what it didn't

Between roughly 1956 and 1970 psychology's center of gravity moved from behavior to mind. George Miller's paper on the limits of short-term memory (1956), Chomsky's review of Verbal Behavior (1959), and Ulric Neisser's Cognitive Psychology (1967) are the usual markers.^[23][36] Within learning research itself, a series of findings showed that consequences were not the whole story:

• Learning without performance. Tolman's rats explored mazes without reward and then, once food appeared, ran them almost immediately — they had learned the layout all along.^[14]
• Learning by observation. Albert Bandura's children imitated an adult's aggression toward an inflatable doll without ever being reinforced for it.^[37]
• Biological constraints. The Brelands' raccoons "washed" the coins they were supposed to deposit; John Garcia's rats associated taste with nausea over hours but could not associate a light with nausea at all. Organisms come prepared to learn some things and not others.^[22][38]
• Contingency, not contiguity. Robert Rescorla showed that pairing alone does not produce conditioning; what matters is whether one event predicts another. The Rescorla–Wagner model made this quantitative.^[26]

Behavior analysis today

The experimental tradition became a profession in the decades after 1968. Applied behavior analysis now has a certifying body (the BACB, founded in 1998), a credential (the Board Certified Behavior Analyst), licensure in most U.S. states, insurance coverage for autism services in every state, and a research literature spanning developmental disabilities, education, organizational management, addiction treatment, brain-injury rehabilitation, and animal welfare.^[39] Its methods — reinforcement-based teaching, functional assessment before treatment, single-subject designs with continuous measurement — descend directly from the laboratory. How ABA is practiced ›

The neurodiversity critique

An honest history has to note that ABA is also contested. Many autistic adults and advocates object to its early history (Lovaas's original 1960s program used aversives, including electric shock, a practice long since abandoned), to goals that aimed at making autistic children "indistinguishable" from peers, to suppressing harmless self-stimulatory behavior, and to intensive programs delivered without the child's assent.^[40] Practitioners have responded that modern ABA is reinforcement-based, individualized, and increasingly assent-driven, and the field's own journals now publish on these concerns.^[40] The debate is about goals, consent, and history, not about whether reinforcement changes behavior — which everyone in it agrees it does.

Beyond the clinic

Outside ABA, the operant tradition underlies reinforcement-based animal training, contingency management in addiction medicine, school-wide positive behavior supports, organizational behavior management, and the analysis of choice that became behavioral economics. Herrnstein's matching law and George Ainslie's hyperbolic discounting — the finding that we overvalue immediate consequences along a predictable curve — gave economists a behavioral account of impulsiveness years before it was fashionable.^[21][41]

The neuroscience of reinforcement

The last chapter of the history is being written by neuroscience. In 1953 James Olds and Peter Milner found that a rat would press a lever for hours to deliver a pulse of current to its own brain — the first demonstration that reinforcement had an identifiable neural substrate.^[42] In 1997 Wolfram Schultz, Peter Dayan, and Read Montague showed that midbrain dopamine neurons report a prediction error — firing to unexpected reward, falling silent to predicted reward, dipping when a predicted reward fails — precisely the quantity in the Rescorla–Wagner model and in temporal-difference learning.^[30][43] Kent Berridge and Terry Robinson then separated "wanting" from "liking": animals without dopamine still enjoy sugar but no longer work for it.^[44] The operant laboratory's strangest findings — the grip of the variable-ratio schedule, the fading of a fully predictable reinforcer — suddenly had a mechanism. What happens in the brain, in depth ›

Operant conditioning in artificial intelligence

Reinforcement learning is the branch of machine learning in which an agent learns by acting in an environment and receiving a numerical reward. Its founders were explicit about the lineage: Richard Sutton and Andrew Barto's textbook opens with Thorndike's law of effect, and their early work with Charles Anderson on "neuronlike adaptive elements" was an attempt to build a learning rule that behaved like an animal under reinforcement.^[31][45] Sutton's 1988 temporal-difference learning algorithm updates predictions from the difference between successive predictions — the computational cousin of Rescorla–Wagner — and it was TD error that Montague, Dayan, and Sejnowski proposed in 1996 as the thing dopamine neurons encode, a year before Schultz's data confirmed it.^[43][46]

The vocabulary crossed over intact. AI researchers speak of reward, policy, exploration and exploitation, and reward shaping — the practice of adding intermediate rewards to guide an agent toward a hard-to-reach goal, named for Skinner's procedure.^[47] Deep reinforcement learning, which pairs these algorithms with neural networks, learned Atari games from raw pixels in 2015 and beat the world's best Go players in 2016; reinforcement learning from human feedback is now a standard stage in training large language models, with human preferences standing in for the food hopper.^[32][33]

The differences matter too. An RL agent's reward is written by an engineer, not discovered functionally; the agent can explore millions of episodes an animal never could; and the algorithms include explicit value estimates that Skinner would have called explanatory fictions. But the central claim — that a system with no built-in knowledge of a task can acquire complex, purposeful behavior purely from the consequences of its actions — is Thorndike's and Skinner's, vindicated at a scale neither imagined.

Key takeaways

• Two names answer the question of discovery. Thorndike showed experimentally, with learning curves, that the consequences of a response change its future probability; Skinner treated that fact as the foundation of a science, defining the operant and inventing the free-operant methods that revealed schedules, shaping, extinction, and stimulus control.
• Skinner was not Thorndike with better equipment. He replaced time to escape with rate of response, discrete trials with the free operant, a specific movement with an operant defined by its consequence, theories of why reinforcement works with a functional definition, and group comparisons with single organisms studied to steady state.
• Thorndike himself dropped the negative half of the law of effect in the 1930s, the first evidence that punishment and reinforcement are not mirror images. Estes's 1944 finding that punishment suppresses responding only temporarily became Skinner's lifelong argument against it.
• The cognitive revolution bounded the law of effect rather than overturning it. Latent learning, observational learning, biological constraints, and contingency over contiguity showed that reinforcement is one powerful process among several; its findings still replicate, and behavior analysts kept publishing throughout.
• The tradition became a credentialed profession, applied behavior analysis, whose current debate is about goals, consent, and history rather than whether reinforcement works. It found a mechanism in the dopamine prediction-error signal and was formalized as reinforcement learning, whose founders cite the law of effect directly.

Check yourself

Thorndike or Skinner: who discovered operant conditioning? Make the case for each.

Thorndike discovered the principle: his 1898 puzzle-box experiments showed, with learning curves rather than anecdotes, that consequences change the future probability of a response, and he named it the law of effect. Skinner named the process in 1937, defined the operant as a class of behavior controlled by its consequences, built the chamber and cumulative recorder, and discovered schedules, shaping, extinction, and stimulus control. Thorndike found the law; Skinner turned it into a field.

Behaviorism began with Watson's 1913 manifesto, so it seems safe to say early behaviorist learning theory was built on Thorndike's law of effect. Is that right?

No. Watson's early learning theory rested on Pavlov's reflex, not on Thorndike's law. It was Skinner, in the 1930s, who borrowed Pavlov's vocabulary of reinforcement, extinction, generalization, and discrimination and applied it to a different kind of learning, behavior controlled by its consequences rather than elicited by a prior stimulus.

A textbook states that Chomsky's 1959 review and the cognitive revolution killed behaviorism. What does the historical record show?

Mainstream psychology's attention did shift to mental processes, and findings such as latent learning, observational learning, and biological constraints showed that consequences are not the whole story. But those findings bounded the law of effect rather than overturning it: reinforcement, schedule effects, extinction bursts, shaping, and stimulus control still replicate, and behavior analysts kept publishing in their own journals, founding an applied journal in 1968 and a credentialing body in 1998. The "death of behaviorism" was mostly a change in who wrote the introductory textbook.

Sutton and Barto open their reinforcement learning textbook with Thorndike. What does a reinforcement learning agent share with a cat in a puzzle box, and where does the comparison break down?

Both acquire complex, purposeful behavior with no built-in knowledge of the task, purely from the consequences of their actions, and the vocabulary of reward, exploration, and reward shaping crossed over intact. The differences: an agent's reward is written by an engineer rather than discovered functionally, the agent can explore millions of episodes an animal never could, and its algorithms carry explicit value estimates that Skinner would have called explanatory fictions.

Explain it to a friend. Explain why the cognitive revolution did not overturn the law of effect, without using the words "behaviorism" or "cognitive."