The model doesn't need training. Not yet.

Training a model sounds like the serious version of building an AI product.

The first version uses a general model. The ambitious version gets its own brain, learns from experience, and becomes harder for anyone else to copy. That was the direction I started exploring for LeagueLoop, a post-match coaching product: what if it trained its own model to coach Talon players?

Then a more useful question appeared. If a real expert reviewed the coaching, how much could the product improve without training a model at all?

Potentially, a lot. More importantly, that work would show whether the model was actually the part holding the product back.

Fluent output was not the same as coaching judgment

LeagueLoop already had a bounded generation pipeline. Deterministic code extracted match evidence, produced candidate moments, retrieved relevant knowledge, and rejected unsupported claims. The model's job was narrower: decide which moments deserved attention and explain what the player could try next.

A controlled four-matchup run produced valid reviews. Every result passed the existing evidence, candidate, temporal, and reference checks. The final batch used four model calls with no retries and cost about twenty cents at the rates recorded for that experiment.

The output was good enough for a private prototype. It was not expert coaching.

Some advice repeated the same synthesis pattern. Situation-specific knowledge was incomplete. One earlier packet sent five near-identical takedown-conversion candidates among twelve options and contained only two broad Talon knowledge claims. Asking the model to produce a more distinctive answer from that material would have treated an upstream product problem as a model-intelligence problem.

The next result improved after the candidate set became more diverse and the retrieved knowledge became more relevant. The model had not changed. The material it was asked to reason over had.

That is an important distinction because "the output is generic" does not tell you where the defect lives.

Fine-tuning the second column would not make those missing inputs appear. It could make the system more confidently repeat them.

An expert should leave more than edited prose

The obvious expert workflow is to show a coach a generated report and ask for a rewrite. That can make one report better. It does not automatically make the next report better.

The reusable contribution is the judgment behind the correction:

• Was this moment worth coaching at all?
• How important was it compared with the other available moments?
• What should the player have done instead?
• Which conditions make that action valid?
• When would the same advice be wrong?
• Is the principle matchup-specific, rank-specific, patch-specific, or general?
• What information would require replay evidence rather than match metadata?

Those answers can become versioned coaching cards and approved examples. When a similar situation appears, LeagueLoop can retrieve a small relevant set and give it to the general model alongside the match evidence. Candidate scoring can also learn which moments experts repeatedly reject or prioritize.

1. 01Generate evidence-backed candidates
2. 02Record expert approval, rejection, or correction
3. 03Store the reason and its conditions
4. 04Retrieve relevant coaching cards
5. 05Generate the next bounded review
6. 06Measure the change against the same evaluation

No model weight changes in that loop. The product still learns.

It gains a better knowledge base, better examples, better ranking signals, and a clearer evaluation set. Those assets are inspectable. A questionable rule can be traced to its source, limited to the patch where it applies, corrected without another training run, or removed when experts disagree.

Training too early hides the most valuable uncertainty

Fine-tuning may eventually be useful. It could internalize stable prioritization patterns, reduce prompt size, improve consistency, or make a smaller model economically viable. But none of those benefits matter until the product can identify the behavior it wants to preserve.

Without an expert-reviewed corpus, training would force several unresolved decisions into the weights:

• Which advice was actually correct rather than merely persuasive?
• Which moment mattered most?
• Which pattern should generalize to another matchup?
• Which correction reflected one player's preference rather than expert judgment?
• Which fact will become stale after a game patch?

That is not only a technical risk. It is a product and investment risk. Expert time, data preparation, training runs, evaluation, and eventual model hosting all cost something. If the first useful expert review shows that candidate selection or knowledge coverage is the real limitation, spending that budget on weights would improve the wrong layer.

There is a clear limit to this conclusion. LeagueLoop does not yet have that expert corpus, so it has not measured how much retrieval or reviewed examples will improve coaching. It has not proved that fine-tuning is unnecessary. A general model may still become the bottleneck after the surrounding system gets better.

That is exactly why "not yet" matters.

The next useful asset is not a custom model. It is a correction loop that can establish what good coaching looks like, where the current workflow misses it, and which lessons deserve to become reusable. If that evidence later points to the weights, training will have a concrete target and a credible benchmark.

Owning a model can become an advantage. Owning a reliable way to decide what it should learn comes first.