Skip to main content

Tutorial 1: Learning to Predict

Difficulty level

This tutorial presents how to estimate state-value functions in a classical conditioning paradigm using Temporal Difference (TD) learning and examine TD-errors at the presentation of the conditioned and unconditioned stimulus (CS and US) under different CS-US contingencies. These exercises will provide you with an understanding of both how reward prediction errors (RPEs) behave in classical conditioning and what we should expect to see if dopamine represents a "canonical" model-free RPE.

Topics covered in this lesson
  • Using the standard tapped delay line conditioning model
  • How reward prediction errors (RPEs) move to conditional stimulus (CS)
  • How variability in reward size affects RPEs
  • How differences in conditioned and unconditioned stimulus (US-CS) timing affect RPEs

Experience with Python Programming Language

Back to the course