In A lifelong companion, we described an agent that becomes more useful through years of shared experience. Such a companion needs to continually learn: your preferences, which procedures work in your environment, and what to do differently after something goes wrong. It also needs to adapt as your circumstances and expectations change. For a personal agent, improvement depends in part on who it is working with and what that person is trying to accomplish.
Consider the email example from that post. You repeatedly shorten drafts, remove formal language, and add a clear next step. Those corrections should improve future drafts. The same principle applies when an agent discovers a reliable way to test your software or learns which information your team needs before making a decision. Over time, the experiences should make it better at the work you do together.
Making that happen is a systems-level problem. The system has to identify useful lessons in what happened, preserve them, and apply them in the right situations. It must recognize when a lesson no longer holds, and judge whether a change actually improves behavior. How quickly and economically it can do this matters too, because learning is something a lifelong companion would keep doing throughout its use.
Our memory and consolidation system already handles parts of this process. In this post, we want to look at the computational cost of carrying those lessons forward, and how learning could fit more closely into the operation of the agent itself.
The cost of applying what has been learned
Explicit memories and instructions can personalize an agent and improve how it performs a task. For frequently used preferences and procedures, however, the agent may repeatedly receive and interpret much the same guidance. Even a well-selected set of instructions consumes input tokens, context space, and computation each time it is used.
This is the main economic motivation for internalizing those behaviors. If a model can reliably apply your usual email style without having it described in every request, more of its context and computation can go toward the content of the email. Across many recurring skills and preferences, we want to reduce the work of repeatedly explaining how to do the task, leaving more capacity for doing it.
One way to achieve this is through parametric learning: training changes some of the model’s parameters so that the learned behavior persists across requests. That could involve a small, personal adapter while keeping the underlying model fixed. The aim would be to generalize from your corrections to future tasks, rather than simply memorize the last edited email.
Today, our consolidation system can save an instruction such as “keep emails short and informal”; with a personal adapter, we would test whether the model can learn to write that way without needing the instruction in each request. Its economic benefit depends on how often the learned behavior is used, how reliably it works, and how much it costs to acquire and maintain. Reducing recurring context is useful, but the training needed to do so also has to be affordable.
Learning where the data already is
I have spent a large part of my career building variations of online-learning systems in the telecom industry. These run in production, in infrastructure through which roughly 30% of the world’s internet traffic passes. At that scale, sending every observation into an offline training pipeline would consume substantial I/O and compute capacity that is needed to serve traffic and do the work the system was built for. Learning has to fit within those constraints.
That experience becomes particularly relevant if we imagine every person on the planet having their own personalized agent. Internalizing each user’s skills and preferences would become a continuous workload across billions of agents. That learning needs to be extremely fast and efficient, so that agents can keep adapting without spending an ever-growing share of their capacity moving data around and processing the same experiences again.
Online learning updates a model as observations arrive during operation. For parametric personalization, we are particularly interested in co-locating training and inference. Producing an answer involves a forward pass through the model, which creates intermediate values called activations. Training uses these in a backward pass to calculate parameter updates. If the system retains the necessary activations and a suitable learning signal arrives, it can update a controlled part of the model on the same machine, without repeating the forward computation.
The potential saving extends beyond the model computation. An offline pipeline needs CPU capacity to serialize and prepare examples, network bandwidth to transfer them, and storage I/O to write and read them again. These activities consume resources that could otherwise support the work the system was built for. Learning where the data is already being processed can avoid parts of that path, as well as the repeated forward pass.
Retaining training activations occupies GPU memory that could otherwise serve more requests. Feedback must also arrive soon enough and support an objective compatible with the retained computation. An email correction arriving ten minutes later, for example, may require a new training example and new computation.
This also fits naturally with the encryption design described in How MidBrain encrypts agent memory. We want to minimize the time-of-flight of plaintext memories. Learning during that processing window means the data need not be stored or sent elsewhere for later training: the system can learn from it and then discard the plaintext. An offline pipeline would instead need retained or renewed access to those examples.
So how much could we actually expect to save under this paradigm?
Let F represent the computation cost of one forward pass. A common approximation puts the backward pass at twice that cost, so training on an example takes roughly 3F. Serving it first and training on it separately therefore takes F + F + 2F = 4F; reusing the serving forward pass reduces that to 3F.
Offline cost = 4F + P {R = 0, P = [0, F, 2F]} = [4F, 5F, 6F]
Online cost = 3F + R {R = 0} = 3F
Savings = [25%, 40%, 50%]P is the additional pipeline cost: serializing and transferring examples, writing and reading storage, preparing or re-tokenizing the data, and loading it into the training cluster.
R is the additional overhead of online learning. All terms are expressed in the same cost units.
If the required state fits in GPU memory without reducing serving throughput, and update coordination adds little overhead, we can approximate R as small relative to P.
This means that we might expect savings in the 25–50% range under these assumptions.
Obviously, this is a very rough estimate, but it gives an indication of the savings we could expect where the original forward pass can be reused. How much we gain beyond that depends on the overhead of the offline pipeline.
Learning while preserving what already works
These potential efficiency and privacy benefits depend on being able to make useful, controlled updates. Suppose your agent learns to write shorter emails, but also starts omitting important details from technical explanations or loses a preference you had spent months establishing. Learning new behavior can damage earlier capabilities, sometimes abruptly. This is known as catastrophic forgetting, and it is one of the central difficulties in continual learning.
People also change their minds, so preserving everything indefinitely would be a problem of its own. The system needs to distinguish a lasting change in preference from a correction that applies to one situation. Shortening an email might reflect your usual style, or it might just mean you were in a hurry that afternoon. Reacting too strongly to each observation could make the agent’s behavior oscillate; reacting too weakly could leave it repeating the same mistakes. Small updates alone do not guarantee stability over a long sequence of interactions.
There is a further difficulty: the model can gradually lose its ability to learn at all. Research on loss of plasticity shows how continued training can make networks less responsive to new learning. For a companion expected to adapt over years, retaining earlier capabilities and remaining able to acquire new ones are both important.
Nor can we simply assume that enough updates will converge to better behavior. There are theoretical guarantees for online learning, but they depend on assumptions that do not transfer directly to deep language models adapting to changing users and tasks. They often concern performance relative to a benchmark, rather than convergence to a fixed model. What counts as good behavior is itself changing, and the feedback available to the model can be incomplete or contradictory.
These are practical reasons to retain control over learning. Offline training allows us to inspect examples, repeat experiments, and evaluate a fixed model before deploying it. With online learning, those controls have to operate alongside a changing model. The system needs to detect regressions, limit harmful updates, and recover when something goes wrong. It also needs to distinguish feedback from the user from instructions encountered in external content.
Adapters, regularization, and replay offer ways to constrain updates and preserve earlier capabilities. Methods such as continual backpropagation address plasticity, while checkpoints provide recovery points. Each helps with part of the problem and brings costs of its own. Replay, in particular, requires renewed access to earlier examples, so its use must respect the user’s encryption and access boundaries.
Discarding the plaintext does not erase what the model learned. Personal parameters can contain sensitive information, just as consolidated memories can, and need protection of their own. Deleting a source memory also does not automatically remove its influence from the model. Learning, replay, and deletion therefore need to be designed together, within the access boundaries the user has chosen.
Continual learning in practice
For a lifelong companion, the aim is to become better at working with its user throughout their shared experience. Internalizing frequent skills and preferences could make that behavior less expensive to apply. Learning alongside inference could make it less expensive to acquire, while fitting an architecture that limits how long raw memories remain readable. Establishing those benefits requires measuring the total cost alongside the quality of the learned behavior and what the model retains over time.
The same considerations apply beyond personal agents. Recommendation systems, industrial monitoring, robotics, and edge devices all encounter changing conditions during operation. Some already use online learning successfully; others face much harder stability and control problems. Making it practical for more capable models would allow more of these systems to improve as they operate, with less infrastructure devoted to exporting experience and processing it again later.
That is the direction we want to pursue at MidBrain. A companion should keep learning what works for you, and doing so should leave it with more capacity for the work you actually need it to do.
