Primary and Secondary Reinforcers: What Is Built In, What Is Learned, and Why Both Fail

Primary reinforcers: unlearned, but not unconditional

A primary reinforcer works the first time. A rat that has never seen a lever will eat the pellet that arrives after it presses, and pressing will increase; no pairing is needed, because the reinforcing effect of food is part of what a rat is. The standard list is short: food, water, warmth when cold and cooling when hot, sleep, sexual contact, and the removal of pain or other aversive stimulation, which is the negative reinforcement side of the same list.^[1] For some species physical contact belongs on it too.

Unlearned does not mean always effective. Food does not reinforce a rat that has just eaten. Skinner treated the rat's hunger as an experimental variable in its own right, set by hours of food deprivation, and reported that the rate of lever pressing rose and fell with it.^[2] A primary reinforcer's power is a property of the stimulus and the organism's current state together, and a trainer who works after dinner with kibble has changed the second without noticing.

The modern name for that state is the motivating operation. Jack Michael defined an establishing operation as an event that does two things at once: it raises the effectiveness of some stimulus as a reinforcer, and it raises the frequency of whatever behavior has produced that stimulus in the past.^[3] Deprivation is the establishing operation for food; satiation is its opposite, an abolishing operation; the two were later grouped under the term motivating operation.^[4]

Secondary reinforcers: how a click comes to count

A secondary reinforcer starts as a stimulus that does nothing. The sound of the food magazine in a Skinner box means nothing to a naive rat. In The Behavior of Organisms Skinner described what happens once the sound has been followed by food a number of times: the rat comes to the tray at the click, and the click itself, delivered after a lever press with no food behind it, is enough to strengthen pressing, for a while. As clicks keep arriving without food, the effect fades.^[2] That fading is the signature of a conditioned reinforcer: its function is borrowed, and it is withdrawn when the relation that produced it is broken.

The procedure here is pairing: a neutral stimulus is followed by a reinforcer. The process is the change in what the stimulus does, and it is the only evidence that a conditioned reinforcer exists. Trainers who "charge" a clicker are running the procedure; the dog's head snapping toward them at the sound is the first sign of the process. Magazine training is step one in the virtual Skinner box.

Paul Bersh measured how much pairing it takes. Rats received a light followed by food, with the number of pairings and the light–food interval varied across groups; the light was then tested by making it the only consequence of lever pressing. More pairings produced a stronger conditioned reinforcer, up to a limit, and a light that came on shortly before the food worked far better than one that came on long before it.^[5] The order is the one classical conditioning requires. How the two kinds of conditioning combine ›

Kelleher and Gollub's review sorted the evidence by method: maintaining an old response in extinction with the stimulus still following it, or teaching a new response with the stimulus as its only consequence. Both methods demonstrated conditioned reinforcement, and both showed it to be short-lived when the stimulus was never paired again, because the test itself extinguishes the pairing.^[6] Zimmerman's answer was to establish the stimulus intermittently: a buzzer followed by water only some of the time, for thirsty rats, went on to reinforce a new bar-pressing response that persisted far longer than earlier methods had achieved.^[7] The reason is familiar from schedules of reinforcement: a relation that was never reliable is slow to be detected as broken.

Pairing is not enough: prediction, information, and dopamine

Pairing is the procedure, but not the variable that matters. Egger and Miller gave rats a food pellet after a sequence of two stimuli. For one group the first stimulus reliably predicted the food, so the second added nothing; for another the first stimulus was unreliable, so the second carried the news. The second stimulus had been paired with food equally often in both groups, and it was a far stronger reinforcer when it was the informative one.^[8] A stimulus becomes a conditioned reinforcer to the extent that it predicts a reinforcer, not merely accompanies one.

Edmund Fantino's delay-reduction hypothesis makes the point quantitative: a stimulus reinforces to the extent that its onset signals a reduction in the time to the next primary reinforcer, relative to the average wait in that situation. A stimulus that means "food soon" when food is usually far off is a strong conditioned reinforcer; one that means "no food" is not a reinforcer at all, however informative. In the observing-response experiments he reviewed, pigeons work to produce a stimulus that tells them which schedule is in effect only when it can announce the better outcome.^[9] Good news reinforces; news does not. Ben Williams's review reached the same place: conditioned reinforcement is a sound and necessary concept, but it is a statement about predictive relations.^[10]

The brain agrees, up to a point

Schultz, Dayan, and Montague recorded midbrain dopamine neurons in monkeys while a cue came to predict a squirt of juice. Early in training the neurons fired at the juice; once the cue reliably predicted it, the burst moved to the cue, and the fully predicted juice evoked nothing.^[11] A predictive cue inherits the signal, which is close to a physiological description of a conditioned reinforcer. Olds and Milner's rats, which pressed a lever for nothing but a brief pulse of electrical stimulation to the septal area, mark the other boundary: a reinforcer that needs no learning history and serves no known biological need.^[12] The full account is on the neuroscience page.

Generalized reinforcers: money, tokens, and attention

A conditioned reinforcer backed by one primary reinforcer inherits that reinforcer's motivating operations: a click backed only by food is worth little to a full dog. Skinner's explanation of why money, attention, and approval work almost all the time was the generalized reinforcer, a conditioned reinforcer paired with many different reinforcers, so that at any moment at least one of the deprivations behind it is likely to be in force. Money buys food, warmth, shelter, and company; attention is the precondition for everything another person might provide. Skinner added that a generalized reinforcer can eventually work even when none of the reinforcers behind it is currently relevant, which is as close as behavior analysis comes to explaining the miser.^[1]

The laboratory version is the token. John Wolfe taught chimpanzees to operate a weighted lever to earn poker chips, which they could insert into a vending machine, the "Chimp-O-Mat," for grapes. The chimps worked for chips, preferred a chip that bought two grapes to one that bought one, learned to ignore a chip that bought nothing, and kept working when the chips could not be cashed in until later, though longer delays weakened the effect.^[13] John Cowles then showed that tokens alone, exchanged for food only after a run of trials, could sustain the learning of new discriminations.^[14] A token is a conditioned reinforcer that can be carried, counted, and saved, which is most of what money is.

Timothy Hackenberg's review draws the modern picture: a token system is three schedules at once, one for earning tokens, one for reaching the exchange period, and one for exchanging, and the tokens' power depends on all three, above all on how reliably and how soon exchange happens.^[15] A token economy in a classroom or a ward is the same machinery, and it fails the same way: when the exchange stops, the tokens become paper, and the behavior they were maintaining goes with them.

Three cases that strain the categories

The primary/secondary distinction is clean in the laboratory and leaky everywhere else. Three cases show the seams.

Social reinforcers and Harlow's caution

Praise, attention, a smile, being listened to: social reinforcers are usually filed under "conditioned," on the theory that a caregiver was paired with food and warmth from birth. Harry Harlow's surrogate-mother experiments showed that this cannot be the whole story. Infant rhesus monkeys raised with two artificial mothers, one of bare wire that held the milk bottle and one covered in soft cloth that gave no milk, spent most of their time clinging to the cloth mother and ran to her when frightened, whichever mother had fed them.^[16] Contact comfort reinforced on its own, not as a by-product of feeding. The category of a reinforcer is an empirical question: some social reinforcers are primary for some species, some are conditioned, and some are not reinforcers at all for a particular person.

The clicker

A clicker is a conditioned reinforcer built on purpose. Karen Pryor, who learned the method training dolphins with a whistle, describes charging the clicker by pairing it with food and then using the click to mark the exact instant of the behavior, with the food following a moment later.^[17] Feng, Howell, and Bennett compared the three explanations on offer, that the click reinforces, that it marks, and that it bridges, and concluded that the published studies cannot yet tell them apart; direct comparisons of clicker-plus-food against food alone are few and mixed.^[18] What is not in doubt is how the click acquires its function. Marker training in practice ›

Activity reinforcers

David Premack's work sits across the distinction. In his account a reinforcer is not a stimulus but a behavior, eating rather than food, and any behavior can reinforce a less probable one.^[19] Running needs no pairing to reinforce drinking in a rat that has been kept from running, so it behaves like a primary reinforcer without being a biological necessity. The Premack principle page covers the experiments; here it is a reminder that the primary/secondary sorting was built for stimuli, and a good deal of what reinforces is not a stimulus.

Types of reinforcers and how each fails

When a reinforcer stops working, its type tells you where to look; the last column is the practical one.

"Social" and "activity" describe what form a reinforcer takes; "primary," "secondary," and "generalized" describe where its function comes from. A smile can be any of the latter three.

How to build a conditioned reinforcer

The procedure is the same for a rat, a dog, a first-grader, or you, and the same variables govern it: order, interval, number of pairings, and prediction.

1. Choose a stimulus that is brief, distinct, and otherwise meaningless. A click, a short word you do not use in conversation, a specific check mark, a chip; not a word the learner already hears forty times a day for free.
2. Confirm that the backing reinforcer works right now. A conditioned reinforcer can only be as strong as what stands behind it at the moment of pairing. Pair before the meal, not after it, and use something the learner would actually choose.
3. Stimulus first, then reinforcer, within about a second. A light that came on shortly before food became a stronger conditioned reinforcer than one that came on long before, and more pairings made a stronger one up to a limit.^[5] A few dozen pairings across two or three short sessions is a reasonable start; it has worked when the learner orients to the stimulus before the reinforcer appears.
4. Make it predictive, not merely frequent. No backing reinforcer without the stimulus, no stimulus without the backing reinforcer. A redundant stimulus, one whose reinforcer was already predicted by something else, acquires little function no matter how often it is paired.^[8]
5. Use it to mark, then pay. The stimulus goes at the instant the behavior occurs; the backing reinforcer follows. A conditioned reinforcer can be delivered with a precision the primary reinforcer cannot match, which is what makes shaping possible.
6. Keep it backed. Every click earns a treat while a behavior is being taught; tokens are exchanged on a schedule the learner can count on. Intermittent pairing makes a conditioned reinforcer more durable, but nothing makes it permanent.^[7] If the learner stops orienting to the stimulus, recharge it.

Common mistakes

• Assuming praise is reinforcing for everyone. Praise is a conditioned reinforcer, which means it was conditioned somewhere, for some learners. For a teenager in front of friends it may be a punisher; for a child whose praise has always come bundled with a demand it may be a warning. The only test is what the behavior does afterward.
• Letting a token lose its backing. A sticker chart with nothing at the end, classroom points that never buy anything, a bonus scheme that stops paying out: each is a conditioned reinforcer on extinction, and it takes the behavior down with it.
• Confusing a reward with a reinforcer. A reward is something the giver thinks is nice; a reinforcer is something shown to increase the behavior it follows. A "secondary reward" that changes no behavior is not a secondary reinforcer, or any other kind. The functional definition, in full ›
• Paying without marking, or marking without paying. Treats without the click make the click redundant; clicks without treats extinguish it. Either way the stimulus stops predicting, and prediction is the whole mechanism.
• Hearing "secondary" as "weaker." Money outcompetes a sandwich for most adults most of the time, precisely because it is backed by everything.

What the evidence does not show

The literature on conditioned reinforcement is large, and several claims made in its name go beyond it.

• It does not show that every conditioned reinforcer traces back to food. Harlow's infants preferred the cloth mother that never fed them.^[16] How much human social reinforcement is primary is not settled.
• It does not show that a conditioned reinforcer can be made permanent. Zimmerman's buzzer was durable, not permanent, and reviewers from Kelleher and Gollub onward have noted that conditioned reinforcers maintained for long periods in the laboratory are almost always still being paired, however thinly, within chained schedules.^[7][6][10]
• It does not show that the number of pairings is what matters. More pairings helped in Bersh's study, but an equally paired stimulus was much weaker in Egger and Miller's when it was redundant.^[5][8]
• It does not show that a clicker beats a word, or that a marker beats food alone. The comparisons are few and inconsistent.^[18] The clicker's advantages are practical: it is fast, consistent, and free of the tone of voice a word carries.
• It does not show that dopamine is the reinforcer, or that it is pleasure. The transfer of the dopamine burst to a predictive cue fits the predictive account, but the signal is a prediction error, not a reward in itself.^[11]

Key takeaways

• A primary reinforcer strengthens behavior without any learning history; a secondary reinforcer strengthens behavior because it has come to predict another reinforcer. Both are reinforcers only if the behavior actually increases.
• "Unlearned" does not mean "always effective." A primary reinforcer's power depends on the motivating operation in force, and satiation switches it off as surely as deprivation switches it on.
• A conditioned reinforcer is established by pairing but governed by prediction: stimulus first, reinforcer within about a second, and no reinforcer arriving without it. It fades by extinction when the pairing stops.
• Generalized reinforcers such as money, tokens, and attention are backed by many reinforcers, so they work almost regardless of the person's state; they fail when the backing is withdrawn.
• The categories leak: contact comfort is a primary social reinforcer for infant monkeys, an activity can reinforce without being paired with anything, and the clicker's mechanism is settled while its superiority over alternatives is not.

Check yourself

A teacher hands out points every day for on-task behavior. For the first month, points buy ten minutes of free time on Friday; then the free-time period is dropped but the points continue. On-task behavior falls back to where it started. What happened to the points?

The points were a conditioned reinforcer backed by free time. When the exchange stopped, the pairing stopped, and the points' function extinguished; they went on being delivered but predicted nothing. This is the standard failure mode of a token system, and the remedy is to restore the backing, not to hand out more points.

A trainer charges a clicker with treats for two sessions, then starts using it, but also keeps tossing the dog treats "to keep him motivated" whether or not a click has sounded. Within a week the dog barely reacts to the click. Why?

The click stopped predicting anything. Treats that arrive without a click make the click redundant, and Egger and Miller showed that a redundant stimulus acquires little reinforcing function even when it has been paired with food just as often as an informative one. Restore the rule that treats follow clicks and only clicks, and recharge.

Is a warm blanket a primary or a secondary reinforcer?

Warmth is a primary reinforcer: it strengthens behavior in a cold organism with no learning history. But whether the blanket reinforces anything right now depends on the motivating operation, and in a warm room it will not. "Primary" describes where the function comes from, not whether it is currently in force.

Want more? The 20-question quiz covers every page on the site with instant explanations.