Continual learning is widely talked about these days. As far as I can tell, there is no consensus on a single definition. The clearest articulation I’ve come across is Section 3 of this 2021 benchmark paper on continual RL.
It’s a good definition to stress test in-context learning against.