Sign in Sign up
Last updated 2 min read

Brain is a Prediction Machine

#Cognition

There is a pretty curious hypothesis on how the brain works. According to it, the brain (at least a part of it) is a prediction machine, and Dopamine is a universal reinforcement learning reward for good predictions.

It's often said that the brain is trying to predict the sources of the rewards and the actions that lead to rewards, which looks like an absolute truth at the high level of the reinforcement learning actor.

However, I like the low-level idea representation suggesting that the brain doesn't care much about what to predict: it simply predicts the next step in a sequence of input signals. What's more important is that the brain doesn't make predictions from time to time, but permanently analyses sensory input as an infinite sequence, makes predictions, and adjusts itself.

Predictions are made at various levels of processing, from low-level sensory features to high-level concepts.

  • Good predictions are backed by a dopamine reward system.
  • Bad ones are penalized by depressed dopamine neurons' activity and also by the unpleasant consequences. Such as fear, pain, and stress often caused by [[Cortisol]] release after getting into trouble due to bad predictions.
  • Not making any predictions is also punished by gradually increasing anxiety and feelings of insecurity. Probably also related to [[Cortisol]]

It forces the brain to constantly observe the surroundings, make predictions, and crave rewards.

How it works:

  • The brain takes the sensory input (sight, hearing, touch, etc).
  • Builds up a model of the surroundings
  • Predicts its future state
  • In the next moment it compares the prediction with the new input
  • Adjusts the model
  • Repeats

This is how the brain adjusts the outer world model and reinforces it relying on good predictions.

Dopamine is considered to be a "teaching signal" that drives reward-based learning by signaling Temporal Difference Model prediction errors.

The dopamine reward is defined by the prediction error:

Most dopamine neurons in the midbrain signal a prediction error; they are activated by a better outcome than predicted (positive prediction error), remain at baseline activity for fully predicted outcomes, and show depressed activity with less outcome than predicted (negative prediction error).

For me, it seems rather counterintuitive, that "better outcomes than predicted" are rewarded while it's still an error: The Surprise. Why better outcome than predicted is rewarded.

The brain certainly remembers past experiences and rewards. If they become a part of the observed (and predicted) reality it would mean that the brain also tries to predict the expected reward (that is an equivalent of its error rate): Brain Predicts Its Error Rate

A declining error rate encourages us to learn, and seek novelty(new data), and gets us bored when the learning process stops.

References

Backlinks

The Surprise. Why better outcome than predicted is rewarded

Dopamine is not a reward for your achievements

Further Questions on "Brain is a Prediction Machine"

Comments