ETBD | popa

Learning as Evolution

McDowell's evolutionary theory of behavior dynamics (ETBD; McDowell, 2004) instantiates the Skinnerian analogy between positive reinforcement and natural selection (Skinner, 1981).

In a nutshell, behavior is viewed as a tool that operates on the environment, with more or less success. Behavioral strategies that prove successful tend to be repeated; those that do not, tend to be abandoned.

ETBD arranges a range of integers, that represent all behaviors that can be exhibited by an agent, its behavioral repertoire. The range can be separated into target classes, groups of integers that are occasionally reinforced.

POINTS

Button 1

Button 2

0 ... 471 ... 511 512 ... 552 ... 4000

Target Class 1

Target Class 2

Continuous-choice environment used to examine preference patterns in humans (top) and in ETBD (bottom; Popa, 2013). Two sources of reinforcement, Button 1 and Button 2, were programmed to deliver points according to certain rules (i.e., schedules of reinforcement).

ETBD functionality

A typical experiment begins with a random population (generation 1), chosen from the available range (with replacement). From the population of behaviors available "now", one is randomly chosen and emitted.

The emission initiates local rules of selection, recombination, and mutation that transform the existing population into a new one (generation 2). From it, one behavior is randomly emitted, a third generation is created, and so on. The sequence of responses that emerges from the reiteration of Darwinian rules is analyzed as if they were produced by a biological organism.

Generation n

EMISSION

DARWINIAN CYCLE

Generation n+1

time

ETBD

Student

516

511

548

533

500

480

475

476

499

522

SELECTION

Parents for the next generation are always chosen with replacement from the existing population. If the emission was not reinforced, parents are chosen at random. If it was reinforced, a probability density function is used to assign greater chances of becoming parents to behaviors that are closer to the reinforced emission.

Selection and reinforcer magnitude

The strength of selection events depends on the mean of the function (µ) and was shown to be computationally equivalent to the reinforcer magnitude (McDowell et. al., 2008; Popa & McDowell, 2016) , with strong selection events corresponding to high reinforcer magnitude.

ETBD functionality overview - selection.

RECOMBINATION

Two parents produce a child behavior by recombining their "genotypes", the corresponding binary form of their integer. The method used here is called bit-string recombination: each bit in the child’s binary string has a 50% chance of coming from one parent or the other. Children produced this way tend to resemble their parents.

MUTATION

Introduces a small amount of spontaneous variation in the population, by altering the genotypes of some child-behaviors. If a behavior is affected, one bit in its binary string is chosen at random and flipped - from 0 to 1 or from 1 to 0.

Mutation rate and the

Default Mode Network (DMN)

The probability to be affected by mutation is referred to as mutation rate. This value is set in the beginning of the experiment and can take values between 0% (no mutants) and 100% (all mutants). The rate of mutation was hypothesized to correspond to the magnitude of spontaneous fluctuations of the brain's DMN (Popa & McDowell, 2016). This network of interconnected regions shows strong spontaneous activation at rest (Buckner, Andrews-Hanna, & Schacter, 2008; Raichle et al., 2001) and failure to suppress it during tasks that require sustained attention or response inhibition was associated with increased levels of behavioral variability (Weissman, Roberts, Visscher, & Woldorff, 2006; Feige, Biscaldi, Saville, Kluckert, & Bender, 2013).

References

Buckner, R. L., Andrews-Hanna, J. R., & Schacter, D. L. (2008). The brain’s default network: anatomy, function

and relevance to disease. Annals of the New York Academy of Sciences, 1124, 1–38. doi: 10.1196/annals.1440.011

McDowell, J. J (2004). A computational model of selection by consequences. Journal of the Experimental Analysis of Behavior, 81, 297–317. doi: 10.1901/jeab.2004.81-297

Popa, A., & McDowell, J, J. (2016). Behavioral Variability in an Evolutionary Theory of Behavior Dynamics. The Journal of the Experimental Analysis of Behavior, 105, 270-290.

Skinner, B. F. (1981). Selection by consequences. Science, 213, 501–504.

Skinner, B. F. (1984). Selection by consequences. Behavioral and Brain Sciences, 7, 477–510.

Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., & Shulman, G. L. (2001). A default mode of brain function. Proceedings of the National Academy of Sciences, 98, 676–682. doi: 10.1073/pnas.98.2.676

Waslick, B., & Greenhill, L. L. . (2004). Attention-deficit/hyperactivity disorder. In J. M. Wiener & M. K. Dulcan (Eds.), The American psychiatric publishing textbook of child and adolescent psychiatry (3 ed., pp. 485-507). Washington DC: American Psychiatric Publishing Inc.