A simple neural network module for relational reasoning

Authors: Adam Santoro, David Raposo, David Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, Timothy Lillicrap

“We demonstrate the use of a simple, plug-and-play neural network module for solving tasks that demand complex relational reasoning. This module, called a Relation Network, can receive unstructured inputs – say, images or stories – and implicitly reason about the relations contained within.”  Read more on the blog.

• Read the paper
• Listen to the spotlight talk in Hall A from 1525-1530
• Check out the poster at Pacific Ballroom #129 from 1830-2230

Simple and scalable predictive uncertainty estimation using deep ensembles

Authors: Balaji Lakshminarayanan, Alexander Pritzel, Charles Blundell

Quantifying predictive uncertainty in neural networks (NNs) is a challenging and yet unsolved problem. The majority of work is focused on Bayesian solutions, however these are computationally intensive and require significant modifications to the training pipeline. We propose an alternative to Bayesian NNs that is simple to implement, readily parallelisable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates. Through a series of experiments on classification and regression benchmarks, we demonstrate that our method produces well-calibrated uncertainty estimates which are as good or better than approximate Bayesian NNs.

• Read the paper
• Listen to the spotlight talk in Hall A from 1545-1550
• Check out the poster at Pacific Ballroom #133 from 1830-2230

Natural value approximators: learning when to trust past estimates

Authors: Zhongwen Xu, Joseph Modayil, Hado van Hasselt, Andre Barreto, David Silver, Tom Schaul

We revisit the structure of value approximators for RL, based on the observation that typical approximators smoothly change as a function of input, but the true value changes abruptly when a reward arrives. Our proposed method is designed to fit such asymmetric discontinuities using interpolation with a projected value estimate.

• Read the paper
• Listen to the spotlight talk in Hall A from 1725-1730
• Check out the poster at Pacific Ballroom #6 from 1830-2230

Successor features for transfer in reinforcement learning

Authors: Andre Barreto, Will Dabney, Remi Munos, Jonathan Hunt, Tom Schaul, David Silver, Hado van Hasselt.

We propose a transfer framework for reinforcement learning. Our approach rests on two key ideas: “successor features”, a value function representation that decouples the dynamics of the environment from the rewards, and “generalised policy improvement”, a generalisation of dynamic programming’s policy improvement step that considers a set of policies rather than a single one. Put together, the two ideas lead to an approach that integrates seamlessly within the reinforcement learning framework and allows transfer to take place between tasks without any restriction.

• Read the paper
• Listen to the spotlight talk in Hall A from 1740-1745
• Check out the poster at Pacific Ballroom #9 from 1830-2230

Deep reinforcement learning from human preferences

Authors:  Paul Christiano (Open AI), Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, Dario Amodei (Open AI)

“A central question in technical AI safety is how to tell an algorithm what we want it to do. Working with OpenAI, we demonstrate a novel system that allows a human with no technical experience to teach an AI how to perform a complex task, such as manipulating a simulated robotic arm.” Read more on the blog.

• Read the paper
• Check out the poster at Pacific Ballroom #1 from 1830-2230

Source: https://deepmind.com/blog/article/deepmind-papers-nips-2017