Intro

Lifelong Embodied Agents

LoadingHyperFrames composition

Why lifelong learning?

An embodied agent must continually adapt to a dynamic physical world.

Open-ended demand keeps shifting, while every agent can hold only a bounded set of capabilities.

01The physical world keeps changing.
02An agent can only hold limited capabilities at a time.
03No fixed capability set covers every future demand. ^[5]

Key messageTherefore, the best agent never stops adapting.

[5]David Abel, et al. "A Definition of Continual Reinforcement Learning." Advances in Neural Information Processing Systems 36, 2023.

7

Skill Memory for Lifelong Embodied Agents · Submission · Part 1 · Intro.4 · Definition of CRLLifelong Embodied Agents

Current 77 / 38

1TitleSkill Memory for Lifelong Embodied Agents

2Part 1Lifelong Embodied AgentsLifelong Embodied Agents

Intro

3Research IntroductionAbout Me
41.1Lifelong AgentsLifelong Embodied Agents
51.2Skill DefinitionLifelong Embodied Agents
61.3Long HorizonLifelong Embodied Agents
71.4Definition of CRLLifelong Embodied Agents
81.5After DeploymentLifelong Embodied Agents

Part 1-1

9LocalityLocality
101-1.1IsCiLLocality
111-1.2IsCiL IILocality
121-1.3IsCiL ProblemLocality
131-1.4MethodSkill Memory
141-1.5IsCiL Fig. ISkill Memory
151-1.6IsCiL Fig. IIISkill Memory
161-1.7CompleteTask-wise Selective Adaptation
171-1.8Semi-completeTask-wise Selective Adaptation
181-1.9IncompleteTask-wise Selective Adaptation
191-1.10Training OverviewTask-wise Selective Adaptation
201-1.11UnlearningTask-wise Selective Adaptation
211-1.12Summary MessageTask-wise Selective Adaptation

Part 1-2

22CompatibilityCompatibility
231-2.1SIL-CSkill-Policy Compatibility
241-2.2The ProblemSkill-Policy Compatibility
251-2.3SIL-C Fig. ISkill-Policy Compatibility
261-2.4MethodBilateral Lazy Learning Interface
271-2.5The MechanismBilateral Lazy Learning Interface
281-2.6SIL-C Fig. IIIBilateral Lazy Learning Interface
291-2.7SIL-C Fig. IIForward & Backward Skill Compatibility
301-2.8Emergent SILForward & Backward Skill Compatibility
311-2.9Explicit SILForward & Backward Skill Compatibility
321-2.10Across SettingsForward & Backward Skill Compatibility
331-2.11Interface AblationForward & Backward Skill Compatibility
341-2.12Summary MessageForward & Backward Skill Compatibility

35Part 2DiscussionPart 2 · Discussion

Section 1

36Real-Time AdaptationReal-Time Adaptation for Real-World Robots

Section 2

37Q&AQuestions & Answers

Section 3

38ReferencesReferences