What appears to be a single task actually depends on multiple cognitive abilities. We face a similar problem in AI research, where the complexity of a task can often make it difficult to tease apart the individual skills required for an agent to be successful. But understanding an agent’s specific cognitive skill set may prove useful for improving its overall performance.

To address this problem in humans, psychologists have spent the last 150 years designing rigorously controlled experiments aimed at isolating one specific cognitive faculty at a time. For example, they might analyse the supermarket scenario using two separate tests – a “visual search” test that requires the subject to locate a specific shape in a pattern could be used to probe attention, while they might ask a person to recall items from a studied list to test their memory.

We believe it is possible to use similar experimental methods to better understand the behaviours of artificial agents. That is why we developed Psychlab, a platform built on top of DeepMind Lab, which allows us to directly apply methods from fields like cognitive psychology to study behaviours of artificial agents in a controlled environment. Today, we are also open-sourcing this platform for others to use.

Psychlab recreates the set-up typically used in human psychology experiments inside the virtual DeepMind Lab environment. This usually consists of a participant sitting in front of a computer monitor using a mouse to respond to the onscreen task. Similarly, our environment allows a virtual subject to perform tasks on a virtual computer monitor, using the direction of its gaze to respond. This allows humans and artificial agents to both take the same tests, minimising experimental differences. It also makes it easier to connect with the existing literature in cognitive psychology and draw insights from it.

Along with the open-source release of Psychlab we have built a series of classic experimental tasks to run on the virtual computer monitor, and it has a flexible and easy-to-learn API, enabling others to build their own tasks.

Each of these tasks have been validated to show that our human results mirror standard results in the cognitive psychology literature.

Take the ‘visual search’ task for example. The ability to locate an object among a complex array of stimuli, like one item on a supermarket shelf, has been studied as a way of understanding selective attention in humans.

Source: https://deepmind.com/blog/article/open-sourcing-psychlab