Psy 342 Learning

“Of several responses made to the same situation, those which are accompanied or closely followed by satisfaction to the animal will, other things being equal, be more firmly connected to the situation, so that, when it (the situation) recurs, they will be more likely to recur.” Thorndike called this the Law ___ ________.
The environmental stimuli present when the response occurs come to control the escape response. These stimuli are called _______________ and signal when or where reinforcement is available.

Trial and error or insight?

According to Thorndike, learning involves establishing a connection between the discriminative stimulus and the instrumental response (___-___ connection).

instrumental learning—learning the connection between a behavior and its consequence

Responsible for an enormous increase of interest in instrumental/operant conditioning in the 1940s and 1950s.

reinforcement—the contingency that results when the consequence of a behavior causes the future probability of that behavior to INCREASE

punishment—the contingency that results when the consequence of a behavior causes the future rate of the behavior to decrease…

positive reinforcement—when reinforcement involves the presentation of a stimulus

negative reinforcement—when the consequence in a reinforcement contingency is the removal of a stimulus

negative punishment—when the consequence in a punishment situation is the removal of a stimulus

One effective treatment for alcohol abuse is the community reinforcement approach (CRA) (Azrin, Sisson, Meyers, & Godley, 1982; Hunt & Azrin, 1973).

The Lovaas treatment program (established in 1960's) uses operant conditioning principles to decrease problem behavior of autistic children and to increase the frequency of appropriate behaviors (e.g., language and social skills).

(1) We define consequences and contingencies by their effects on behavior, not by what we EXPECT their effects to be.

(2) If behavior is followed by a reinforcer, it is the behavior that is reinforced, not the organism.

(3) The apparent contingent relationship between behavior and outcomes can lead to behaviors that do not appear to make sense. Skinner's classic article

Measuring Operant Responses—usually rate at which organisms emit response is the dependent variable in most operant conditioning studies

Cumulative recorder creates a record of __________________ as a function of time.

Reinforcement of behaviors that are closer and closer approximations of the target response Shaping in Skinner Box

Generally, responding occurs faster and becomes more accurate as we increase the amount of a reinforcer delivered after each response.

**Results: After 20 trials, the running speeds of the groups corresponded directly to the number of pellets received.

Complicating factor—Some researchers have disagreed as to the definition of reinforcer magnitude.

Several studies have shown that if an experimenter gives two groups the same amount of food for each response, but for one group the food is simply broken into more pieces, the group receiving the most pieces will respond faster.

We can define quality by assessing how vigorously an organism consumes a reward when it is presented.

High quality reinforcers are consumed quickly by organisms, whereas low quality reinforcers are consumed less quickly.

Generally, the longer that reinforcement is delayed after a response, the poorer the performance of that response.

***One reason for this difference might be ____________________? (Why would a 10 second delay matter? What could possibly go on during the 10 s delay?)

C. Contrast Effects—Effects of quantity, quality, and delay of reinforcement on instrumental responding can vary depending on an organism’s past experiences with a particular reinforcer.

Crespi (1942)—Group 1 trained to traverse a runway for large amount of food, Group 2 for a small amount of food

After several trials, the rats receiving the large reinforcer were running consistently faster than the small-reinforcement group.

Then switched half of the large-reinf. animals to the small reinforcer being given to the other rats (large-small).

And switched half the small-reinforcer rats to the large-reinforcer magnitude (small-large).

D. Drive
motivational need or desire for a given reinforcer...How might we manipulate the hunger and thirst drives of rats? What impact would we expect this manipulation to have on behavior (e.g., maze running)?

Hull (1949)--Instrumental performance is determined by level of conditioning AND motivational factors...Instrumental conditioning was represented by Habit Strength (H), which was influenced by # of reinforced training trials and delay of reinforcement. Motivation was represented as Drive (D) (e.g., # hours of food deprivation) and Incentive (K) (e.g., reinforcer amount and quality).
Response strength = H x D x K

E Schedules of Reinforcement
Intermittent Reinforcement—Often, we can't reinforce each time an appropriate response is emitted.

It is clear that instrumental learning is not dependent on continuous reinforcement. In most species, instrumental responding develops quite efficiently even when reinforcement occurs only intermittently.

Interval schedules—deliver reinforcer only for the first response that occurs after a period of time has elapsed.

Self-Control--capacity to inhibit immediate gratification in exchange for a larger reward in the long run

Primary Reinforcer—events that are capable of producing behavior changes naturally without the benefit of any prior learning.

Secondary Reinforcer—events that function as reinforcers because of their consistent association with one or more primary reinforcers.

Generally, theories explaining why reinforcers work fall into one of the following two categories:

1) Stimulus-based theories--something about the particular stimulus makes it reinforcing

--Hull's Drive-Reduction Hypothesis--stimuli that reduce drives are capable of reinforcing emitted behavior

Learning is adaptive & biological in nature
An internal deficiency (need) causes an energized motivation state (a drive, e.g., hunger, thirst). The drive then activates a series of actions (responses) in order to attain a goal (such as food or water). Then when the drive is satisfied, the organism should be relatively inactive until another drive is activated by a need.

Conclusion--Events that reduce drives will function as reinforcers, but an event need not reduce a drive in order to function as a reinforcer.

--Incentive motivation--reinforcers increase drive, reinforcers pull the organism toward them and elicit certain behaviors

2) Response-Based Theories of Reinforcement --reinforcement depends on the response made possible by some reinforcer

Premack Principle (1965, 1971)--Any kind of emitted response can serve to reinforce operant behavior. Specifically, any response that is preferred by an organism can serve to reinforce the performance of a less preferred response....Reinforcement is relative...

F. Constraints on Response Learning
So as long as we immediately reinforce desired behavior, we should be able to condition any organism to do any behavior, right?

Breland & Breland--"The misbehavior of organisms"

Amount of Reinforcement—In escape learning, the amount of reinforcement corresponds to the degree to which the aversive stimulation is reduced after a successful response.

Campbell & Kraeling (1953)—exposed all rats to the same shock intensity in the runway and reduced this shock by varying degrees in the safe box.

That is, all animals received shock reduction after responding, but most animals still received SOME shock in the safe box.

The speed of the escape response was a direct function of the degree to which shock was reduced.

**Escape learning depends more on the amount of negative reinforcement (degree to which aversive stimulation is reduced) than on the intensity of the aversive stimulus per se.

Delay of Reinforcement—time between the escape response and the reduction of the aversive stimulation.

Fowler & Trapold (1962)—exposed rats to shock in an alleyway and required the animals to run to a safe box to escape the shock.

B. Avoidance Learning—A stimulus signals the presentation of the aversive event. The organism must respond in the presence of the signal in order to avoid the aversive event.

The characteristics of both of these stimuli are important determinants of avoidance learning.

Intensity of Aversive Stimulus--If the intensity of the aversive stimulus (e.g., shock) is too strong, avoidance conditioning is actually slowed

Signal-Aversive Stimulus Interval--Avoidance learning appears to occur most efficiently when the signal and the aversive stimulus overlap, as is the case in a delayed-conditioning procedure.

Duration of the signal before onset of aversive event--In general, signals of longer duration tend to facilitate the learning of the avoidance response (Low & Low, 1962).

Termination of the Signal--Even when the organism’s response results in avoidance of the aversive stimulus, the rate of learning depends on whether the response also leads to the termination of the signal.

Kamin (1957) conducted a two-way avoidance experiment using rats. All rats were able to avoid shock by moving to the safe chamber during the signal-shock interval. For one group, the signal terminated as soon as the avoidance response occurred (o-second delay). In the other three groups, the signal ended either 2.5, 5, or 10 seconds after the avoidance response.

Classical Conditioning and Instrumental Conditioning in Avoidance Learning

Mowrer (1947) proposed that avoidance learning involved two processes--(1) classical conditioning and (2) instrumental conditioning.

(Part 1) Dangerous, painful, aversive stimuli (US) cause an innate fear response (UR). Other stimuli present at the time get associated with fear through classical conditioning. When these other stimuli (CSs) are encountered again, they evoke a fear response (CR).

(Part 2) The presence of fear and all of its visceral effects is aversive. Any response that removes these fear-evoking stimuli will be negatively reinforced. The avoidance response, therefore, is reinforced through instrumental conditioning.

It was easy to understand how the escape behavior persisted. The termination of the aversive stimulus would be rewarding. Escape behavior was maintained through negative reinforcement.

But how was the avoidance response maintained? The animal continues to respond even though it no longer receives aversive stimulation, but why?

Since the electric shock (US) was painful and produced an innate fear (UR) response (increased heart rate, sweating, etc.), the dark side became a feared stimulus (CS) that led to innate fear (CR) (increased heart rate, sweating, etc.). These visceral responses are unpleasant so any response that caused their reduction or elimination (i.e., escape or avoidance) would be reinforced. Mowrer argued that the termination or reduction of fear stimuli negatively reinforced the avoidance response.

Two-process theory predicts that the avoidance responding will be learned only to the extent that the warning signal terminates when a response is made.

The figure shows that a significant amount of avoidance responding occurred in the first group only (response terminates signal and enables animal to avoid shock).

As predicted by two-factor theory, avoidance responding was poor in the group that was able to avoid shock but could not terminate the signal.

We know that delaying the onset of reinforcement reduces the effectiveness of reward. So it should be possible to reduce the level of reinforcement by introducing a delay between the avoidance response and termination of the feared stimulus.

As predicted, the animals in the zero-delay condition successfully avoided shock on over 80% of the trials. Animals in the 10-second delay condition avoided shock on fewer than 10% of the trials.

I. Early Experiments

Classical Conditioning and Instrumental Conditioning in Avoidance Learning