In our last update, a person and an experimental assistant shared one local Linux desktop. This week we asked a harder question: when you hand the assistant a task, how much does it actually get done, and how do you know it got it right?
Measuring instead of guessing
We built a small harness that runs the same everyday tasks again and again on a fresh local desktop: changing a field on a form and saving it, resizing an image to a requested size, packing files into an archive. In one task a person steps in partway through and touches up a picture by hand, and the assistant has to carry on from that saved edit instead of redoing its own version.
For each run, the harness records where the time went: waiting on the assistant, the apps doing their work, and the checks afterward. It also inspects the result itself. Before any real run, we rehearse the whole path with a fixed script, so a broken measurement can’t pass for an assistant mistake.
The mistakes are the useful part
In one run, the assistant read the requested size correctly and still saved the image at the wrong size. Our product only checked that the file was a valid image, so it marked the task complete. The harness’s own size check caught it.
Two changes followed. In our measurements, a task now counts as a success only when the result passes those checks too. And we’re moving the arithmetic itself out of the assistant and into ordinary code, where it can’t be misread. That second change hasn’t been tested with the assistant yet.
Another run ended with the assistant reporting that it couldn’t confirm a step, because the page gave it no way to see the result. That was the right call. The fault was in our task, so we rewrote it. We’d rather the assistant stop and say so than claim work it can’t verify.
We also noticed some summaries coming back in a different language from the request. The work itself was fine, but a summary you can’t read isn’t much help, so that’s on our list.
What this is, and what it isn’t
These are early development experiments with a small number of runs, on a desktop running on our own machine. They aren’t a feature you can use, and they don’t tell us how a real Linux Cloud PC will behave. Early access is still closed.
The next tasks we measure should be ones you’d really hand off. If there’s a chore on your computer you’d happily never do again, that’s exactly the kind of thing people share in #feedback on our Discord, and it helps decide what we test next.
Where we are: measuring an experimental assistant on everyday tasks in local development, and fixing what the checks catch. No real Cloud PC has been validated yet, and early access is closed.