Artificial intelligence currently advances by spending more: more parameters, more memory, more compute, more energy. Symbioz explores the other direction — getting the same for far less. Not as a finishing touch, but as the whole of our research.
For a decade, progress in artificial intelligence has followed a single recipe: make it bigger. More parameters, more data, more silicon. It works — it has produced remarkable systems. But it also carries a cost that rarely gets examined.
A model that is twice as frugal delivers the same service for half the resources — on every single request, forever. At the scale AI is used today, a gain in efficiency is worth more than one more point of benchmark performance.
That is the direction Symbioz works in, and the only one. We are not trying to beat anyone on a capability leaderboard. We are trying to fit the same thing into far less.
Every byte a model does not have to keep is memory that is never manufactured and power that is never drawn — on every request, indefinitely.
Training and serving a very large model is the preserve of a handful of organizations. What becomes efficient becomes reachable again for small teams.
A model that fits on an ordinary machine can run at home, offline, without handing your data to anyone.
The same question every time: can we get the same result for less? What follows is what we are after and why — the architectural details stay in the lab.
To answer, a model keeps a trace of every token it has read. That memory grows without end, and it — not the parameter count — is what sets the real cost and the context length you can afford.
A model that "accepts" 128,000 tokens does not thereby use them. Advertising a window costs nothing; actually using it costs a great deal. We measure the reach you pay for.
Making a model think currently costs thousands of tokens, written and then thrown away. What if it formed a picture of what comes next before writing anything at all?
A model computes with far more decimal places than it needs, and each one costs memory and energy. We step down one level at a time, measuring what is lost at each.
In a trained model, not everything earns its keep. Some parameters no longer contribute anything yet cost exactly as much as the rest. We want to count them, then stop paying for them.
Everything we publish is trained and measured on consumer GPUs, one at a time. This is not a limitation we endure: it is the constraint that forces us to find the saving.
Not in the parameter count, the number everyone puts on the box. In the bytes a model must keep for every token it reads. In four steps.
Every token read leaves a trace the model has to carry for the rest of its answer.
The longer the text, the larger that memory grows — token after token, never emptying.
Once the available memory runs out, that trace sets the real limit: it caps the context and drives the bill.
Compressing that trace to 90 bytes per token — then checking, text by text, that nothing was lost on the way.
Chasing efficiency makes it unusually easy to fool yourself: the gains you are hunting are small, and a tiny difference looks a lot like noise. So we took the opposite approach to intuition — acceptance criteria are written down, dated and frozen before each experiment starts.
What we measure afterwards is not open to interpretation. An appealing idea that lands in the grey zone is dropped — even if we liked it, even if it cost us weeks. Most of our leads end that way, and that is simply what a lab looks like from the inside.
A saving only counts as a gain if quality has not moved. It is the least spectacular half of our work, and by far the longest.
Acceptance thresholds, stopping rules and analysis strata are fixed before a single experiment is launched.
Two architectures are compared on identical texts, with a confidence interval, and the result is broken down by type of passage. An average gain that hides a local loss is not a gain.
Abandoned leads are documented as carefully as the successes. Knowing which saving does not work is worth something.
Better said plainly than left to be guessed.
The questions we ask, the reasons we chose them, the savings we measure and the results we get — including the ones that prove us wrong. Formal publications will follow once our large model is finished; until then, this site serves as the record.
Architectural detail, training recipes and internal mechanisms stay in the lab. We communicate the what and the why, not the how.
No in-house leaderboard cut to flatter us, no demonstration picked after the fact, no saving announced without stating what it cost in quality.
Symbioz is an independent research lab based in France. We have nothing to sell: this site exists to report on our work, not to turn a visitor into a customer.
We work on deliberately compact models. Not for want of anything better, but because it is the right scale at which to study efficiency: an idea that adds nothing shows up immediately, where a very large model would absorb it unnoticed.
Results obtained at that scale are then re-examined before any generalization. We would rather have a narrow, solid conclusion than a broad promise.
Getting the same for less. Anything that does not serve that question leaves the research programme.
No outside funding to satisfy, and therefore no reason to force a result or announce it too early.
Every published result points back to a dated experiment, its protocol and its reference data, all archived.
We read everything that touches model efficiency: inference-memory compression, quantization, honest evaluation. A remark, an objection, a result that contradicts ours — write to us.