Why this matters to a client
Why this matters to a client
If you're evaluating any vendor's AI accuracy claim — a scheduling model, a demand forecast, a risk score — the question that matters isn't "how good is it," it's "how would they know if it stopped being good, and would they tell you." I built a prediction pipeline for a domain with real recurring outcomes specifically to test my own discipline on that question. Twice, the honest answer changed the number I was allowed to show anyone.
What it does
What it does
TruLine forecasts outcomes for a series of recurring scheduled events, using historical and live data to produce a probability for each one. Every prediction is checked against what actually happened, on a fixed schedule, and the system is built to make that check impossible to skip or fudge.
Product screenshots
Placeholder — not a real screenshot. TruLine product imagery to be added before launch.
How it's built
How it's built
Technical detailFor readers who want the engineering — skip ahead if you just want the outcome.
Python/FastAPI backend on Railway, PostgreSQL, LightGBM models trained per event-type, a probability calibration layer (isotonic/Platt/sigmoid), and a React/TypeScript dashboard on Vercel. New or revised models don't go live directly — they run in an append-only "shadow" table alongside the live model, scored against real outcomes with zero effect on anything, until a pre-registered accuracy gate either clears them or doesn't. A config flag, not a deploy, turns any model's output on or off, so suppressing a model is a one-line change, not a rollback.
The interesting decision
The interesting decision
A model ran for a season and looked strong. Then a defect turned up in how outcomes were being matched back to their predictions — the scoring was pairing some results to the wrong prediction. I fixed the pairing logic and re-scored the entire season honestly. The corrected number inverted: the model had been losing, not winning. I turned it off and left it off, and rebuilt grading to read from data frozen at prediction time so the same class of bug can't recur.
Separately, a second model was held in shadow mode — computing predictions, touching nothing — until it had scored against 125 real outcomes. It came in just under the accuracy threshold it needed to clear. The threshold had been set before the result was known, so there was no room to argue with it. It stayed suppressed.
The pattern behind both: keep the model computing, stop it from mattering, and decide with a number set in advance rather than a number that looks good in hindsight.
Status
Status
Running in production for my own use, not commercialized. One model is live for its event category; two others have been evaluated in shadow mode and are currently suppressed by config rather than removed from the codebase, so they can be re-evaluated without a rewrite if the underlying data changes.