Continual Learning

Literatures of Continual Learning (持续学习, also called Lifelong / Incremental / Cumulative Learning).

Other Awesome Lists

• ContinualAI Wiki
• optimass/continual_learning_papers
• xialeiliu/Awesome-Incremental-Learning
• prprbr/awesome-lifelong-continual-learning
• floodsung/Lifelong-Learning-Paper-List

Courses & Tutorials

• Life Long Learning. Hung-yi Lee (李宏毅). [Slide] [Video]

• CVPR 2020 Tutorial - Towards Annotation-Efficient Learning: Few-Shot, Self-Supervised, and Incremental Learning Approaches

Surveys

• Continual Lifelong Learning with Neural Networks: A Review. German I. Parisi, et al. Neural Networks 2019. [Paper]

  A survey on different approaches (regularization / dynamic architectures / rehearsal) for continual learning.

• Three Scenarios for Continual Learning. Gido M. van de Ven and Andreas S. Tolias. arXiv 2019. [Paper] [Code]

  A survey on three different scenarios (task / domain / class) for continual learning.

• Continual Learning: A Comparative Study on How to Defy Forgetting in Classification Tasks. Matthias De Lange, et al. arXiv 2019. [Paper]

• Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challenges. Timothée Lesort, et al. Inf. Fusion 2020. [Paper]

Theses

• Continual Learning with Deep Architectures. Vincenzo Lomonaco. University of Bologna, 2019. [Thesis]

• Continual Learning with Neural Networks. Sayna Ebrahimi. UC Berkeley, 2020. [Thesis]

• Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes. Timothée Lesort. ENSTA-ParisTech, 2020. [Thesis]

Blogs & Communities

• Continuous Learning Strategies (CORe50 Dataset)

• Why Continual Learning is the key towards Machine Intelligence. Vincenzo Lomonaco. Medium, 2017.

• ContinualAI

Approaches

Regularization

Impose constraints on the update of the neural weights.

• Learning without Forgetting. Zhizhong Li and Derek Hoiem. ECCV 2016. [Paper] [Code] (LwF)

• Overcoming Catastrophic Forgetting in Neural Networks. James Kirkpatrick, et al. PNAS 2017. [Paper] (Elastic Weight Consolidation, EWC)

• Continual Learning Through Synaptic Intelligence. Friedemann Zenke, et al. ICML 2017. [Paper] [Code] (Intelligent Synapses, IS)

• Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting. Hippolyt Ritter, et al. NIPS 2018. [Paper]

• Improving and Understanding Variational Continual Learning. Siddharth Swaroop, et al. NIPS 2018 Continual Learning Workshop. [Paper] [Code]

• Memory Aware Synapses: Learning What (Not) to Forget. Rahaf Aljundi, et al. ECCV 2018. [Paper] [Code]

• Task Agnostic Continual Learning Using Online Variational Bayes. Chen Zeno, et al. arXiv 2018. [Paper] [Code]

• Task-Free Continual Learning. Rahaf Aljundi, et al. CVPR 2019. [Paper]

• Online Continual Learning with Maximally Interfered Retrieval. Rahaf Aljundi, et al. NIPS 2019. [Paper]

• Uncertainty-based Continual Learning with Adaptive Regularization. Hongjoon Ahn, et al. NIPS 2019. [Paper] [Code]

• Efficient continual learning in neural networks with embedding regularization. Jary Pomponi, et al. Neurocomputing 2020. [Paper]

• Uncertainty-guided Ccontinual Learning with Bayesian Neural Networks. Sayna Ebrahimi, et al. ICLR 2020. [Paper] [Code]

• Continual Learning with Node-Importance based Adaptive Group Sparse Regularization. Sangwon Jung, et al. CVPR 2020 Workshop on Continual Learning in Computer Vision. [Paper]

• SOLA: Continual Learning with Second-Order Loss Approximation. Dong Yin, et al. arXiv 2020. [Paper]

• CPR: Classifier-Projection Regularization for Continual Learning. Sungmin Cha, et al. ICLR 2021. [Paper] [Code]

Rehearsal

Extra Memory

Use extra memory to store data from previous tasks.

• Gradient Episodic Memory for Continual Learning. David Lopez-Paz and Marc'Aurelio Ranzato. NIPS 2017. [Paper] [Code]

• iCaRL: Incremental Classifier and Representation Learning. Sylvestre-Alvise Rebuffi, et al. CVPR 2017. [Paper] [Code]

• Variational Continual Learning. Cuong V. Nguyen, et al. ICLR 2018. [Paper] [Code] (VCL)

• Experience Replay for Continual Learning. David Rolnick, et al. NIPS 2019. [Paper]

• Continual Learning with Bayesian Neural Networks for Non-Stationary Data. Richard Kurle, et al. ICLR 2020. [Paper]

• Graph-Based Continual Learning. Binh Tang and David S. Matteson. ICLR 2021. [Paper]

• Gradient Projection Memory for Continual Learning. Gobinda Saha, et al. ICLR 2021. [Paper] [Code]

Generative Replay

Mimic past data by generative models (GAN, VAE, etc).

• Continual Learning with Deep Generative Replay. Hanul Shin, et al. NIPS 2017. [Paper]

• FearNet: Brain-Inspired Model for Incremental Learning. Ronald Kemker and Christopher Kanan. ICLR 2018. [Code]

• Generative Models from the perspective of Continual Learning. Timothée Lesort, et al. IJCNN 2019. [Paper] [Code]

• Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning. Oleksiy Ostapenko, et al. CVPR 2019. [Paper] [Code]

• Continual Unsupervised Representation Learning. Dushyant Rao, et al. NIPS 2019. [Paper] [Code]

  Continual learning without task boundaries via dynamic expansion and generative replay (VAE).

Dynamic Expansion

Increase in network capacity that handles new tasks without affecting learned networks.

• Net2Net: Accelerating Learning via Knowledge Transfer. Tianqi Chen, et al. ICLR 2016. [Paper]

• Progressive Neural Networks. Andrei A. Rusu, et al. arXiv 2016. [Paper]

• Expert Gate: Lifelong Learning with a Network of Experts. Rahaf Aljundi, et al. CVPR 2017. [Paper] [Re-implementation]

• Random Path Selection for Continual Learning. Jathushan Rajasegaran, et al. NIPS 2019. [Paper] [Code]

• Compacting, Picking and Growing for Unforgetting Continual Learning. Steven C. Y. Hung, et al. NIPS 2019. [Paper] [Code]

  Gradual model pruning (compacting) → Train a 0-1 mask to pick weights of previous tasks (picking) → Dynamic expansion (growing)

• Continual Unsupervised Representation Learning. Dushyant Rao, et al. NIPS 2019. [Paper] [Code]

  Continual learning without task boundaries via dynamic expansion and generative replay (VAE).

• Continual Learning with Adaptive Weights (CLAW). Tameem Adel, et al. ICLR 2020. [Paper]

• A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning. Soochan Lee, et al. ICLR 2020. [Paper] [Code]

Task Free

• Task Agnostic Continual Learning Using Online Variational Bayes. Chen Zeno, et al. arXiv 2018. [Paper] [Code]

• Task-Free Continual Learning. Rahaf Aljundi, et al. CVPR 2019. [Paper]

• Continual Unsupervised Representation Learning. Dushyant Rao, et al. NIPS 2019. [Paper] [Code]

  Continual learning without task boundaries via dynamic expansion (Dirichlet process) and generative replay (VAE).

• Reconciling Meta-Learning and Continual Learning with Online Mixtures of Tasks. Ghassen Jerfel, et al. NIPS 2019. [Paper]

• Continuous Meta-Learning without Tasks. James Harrison, et al. arXiv 2019. [Paper] [Code] [OpenReview]

  Integrate Bayesian online changepoint detection algorithm with existing meta-learning approaches to enable meta-learning in task-unsegmented settings.

• A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning. Soochan Lee, et al. ICLR 2020. [Paper] [Code]

• Task Agnostic Continual Learning via Meta Learning. Xu He, et al. ICML 2020 LifelongML Workshop. [Paper]

+ Meta Learning

Here is also a list of literatures for Meta Learning.

• Meta Continual Learning. Risto Vuori, et al. arXiv 2018. [Paper]

  Train a RNN as optimizer, and the optimizer leverages information of both current and previous tasks to learn to preserve previous parameters.

• Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference. Matthew Riemer, et al. ICLR 2019. [Paper] [Code] (Meta-Experience Replay, MER)

  Combine Reptile (a meta-learning algorithm) with experience replay for rapidly learning the current and future experience and preserving past knowledge.

• Meta-Learning Representations for Continual Learning. Khurram Javed and Martha White. NIPS 2019. [Paper] [Code] [Poster]

• Reconciling Meta-Learning and Continual Learning with Online Mixtures of Tasks. Ghassen Jerfel, et al. NIPS 2019. [Paper]

• Continuous Meta-Learning without Tasks. James Harrison, et al. arXiv 2019. [Paper] [Code] [OpenReview]

  Integrate Bayesian online changepoint detection algorithm with existing meta-learning approaches to enable meta-learning in task-unsegmented settings.

• Task Agnostic Continual Learning via Meta Learning. Xu He, et al. ICML 2020 LifelongML Workshop. [Paper]

• La-MAML: Look-ahead Meta Learning for Continual Learning. Gunshi Gupta, et al. NIPS 2020. [Paper] [Code]

+ Reinforcement Learning

• Reinforced Continual Learning. Ju Xu and Zhanxing Zhu. NIPS 2018. [Paper] [Code]

• Experience Replay for Continual Learning. David Rolnick, et al. NIPS 2019. [Paper]

+ Generative Modeling

• Lifelong GAN: Continual Learning for Conditional Image Generation. Mengyao Zhai, et al. ICCV 2019. [Paper]

• Generative Models from the perspective of Continual Learning. Timothée Lesort, et al. IJCNN 2019. [Paper] [Code]

Bayesian

• Variational Continual Learning. Cuong V. Nguyen, et al. ICLR 2018. [Paper] [Code] (VCL)

• Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting. Hippolyt Ritter, et al. NIPS 2018. [Paper]

• Improving and Understanding Variational Continual Learning. Siddharth Swaroop, et al. NIPS 2018 Continual Learning Workshop. [Paper] [Code]

• Task Agnostic Continual Learning Using Online Variational Bayes. Chen Zeno, et al. arXiv 2018. [Paper] [Code]

• Continual Unsupervised Representation Learning. Dushyant Rao, et al. NIPS 2019. [Paper] [Code]

  Continual learning without task boundaries via dynamic expansion and generative replay (VAE).

• Reconciling Meta-Learning and Continual Learning with Online Mixtures of Tasks. Ghassen Jerfel, et al. NIPS 2019. [Paper]

• Uncertainty-based Continual Learning with Adaptive Regularization. Hongjoon Ahn, et al. NIPS 2019. [Paper] [Code]

• Continuous Meta-Learning without Tasks. James Harrison, et al. arXiv 2019. [Paper] [Code] [OpenReview]

  Integrate Bayesian online changepoint detection algorithm with existing meta-learning approaches to enable meta-learning in task-unsegmented settings.

• Task Agnostic Continual Learning via Meta Learning. Xu He, et al. ICML 2020 LifelongML Workshop. [Paper]

• A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning. Soochan Lee, et al. ICLR 2020. [Paper] [Code]

• Continual Learning with Bayesian Neural Networks for Non-Stationary Data. Richard Kurle, et al. ICLR 2020. [Paper]

• Uncertainty-guided Ccontinual Learning with Bayesian Neural Networks. Sayna Ebrahimi, et al. ICLR 2020. [Paper] [Code]

New Settings

• Online Fast Adaptation and Knowledge Accumulation (OSAKA): a New Approach to Continual Learning. Massimo Caccia, et al. arXiv 2020. [Paper] [Code]

• Wandering Within a World: Online Contextualized Few-Shot Learning. Mengye Ren, et al. arXiv 2020. [Paper]

• Compositional Language Continual Learning. Yuanpeng Li, et al. ICLR 2020. [Paper] [Code]

  Continual learning in NLP for seq2seq style tasks.

Workshops

• Continual Learning and Deep Networks Workshop, NIPS 2016
• Continuous and Open-Set Learning Workshop, CVPR 2017
• First Lifelong Machine Learning Workshop, ICML 2017
• Second Lifelong Machine Learning Workshop, ICML 2018
• Continual learning Workshop, NeurIPS 2018
• Third Lifelong Machine Learning Workshop, RLDM 2019
• Workshop on Continual Learning, ICML 2020
• 4th Lifelong Machine Learning Workshop, ICML 2020