Pitcher strikeouts
It passes the backtest. The live forward test is still running: 164 settled starts so far (minimum 200) over 26 dates (minimum 7).
Promising: Real skill in the backtest, but one check is still open: calibration spread, or the live forward test. Tier 1: Lead market. Promoted to Validated when the evidence says so.
The backtest
engine v14 · every graded start · walk-forwardSkill is how much lower the mean squared error of the served number is than always guessing the league-wide rate known at the time. Every date is predicted using only earlier dates, with calibration curves refit each date the way the site serves them.
Model better
Compared with the pitcher's own start-by-start average, shrunk toward the league. Difference in mean squared error, model minus player: -0.549 (95% interval -0.735 to -0.367). Negative means the model is more accurate.
| Line | Skill | AUC | Predicted | Happened |
|---|---|---|---|---|
| Over 5.5 | 16.4% | .742 | 35.5% | 34.7% |
| Over 6.5 | 15.2% | .759 | 23.2% | 22.4% |
| Over 7.5 | 13.2% | .776 | 14.2% | 13.0% |
164 settled starts (200 needed) · 26 dates (7 needed)
Every settled start the engine projected counts here. 142 of the 164 also had a book line and a de-vigged price; those are the starts the at-line gate below grades.
Counted from the ledgers at this build: predictions frozen before first pitch, under the engine now serving, with the game final and the result in. A start with no strikeout count is void and left out. The count restarts with every engine version. A status moves only when the model card is rebuilt and re-graded, which also checks live skill.
The last engine with a finished live sample (engine v8, 312 results over 12 dates) showed skill of 24.8% (14.0% to 35.6%).
Book more accurate
Log loss, model minus the no-vig closing price: +0.087 (95% interval +0.056 to +0.122; across dates, +0.053 to +0.135) on 275 predictions over 12 dates, engines v10–v14. Positive means the book was more accurate. This comparison changes no status; it answers whether the number is a reason to bet.
Not yet beating a coin · 125 starts with a book line (100 needed) · 26 dates (7 needed)
engine v14: Brier .257 for the model, .249 for the book, .250 for a coin flip, on 125 settled pre-game starts over 26 dates. The model has not yet beaten a coin flip at the book's own line, because the model's chance at the book's own line has not beaten a coin flip (Brier 0.2570 against the coin's 0.2500 over 125 starts).
The strikeout model passes this test only once it beats a coin flip at the book's own line, under the engine serving, on 100 or more pre-game starts with a book line over 7 or more dates, with the 95% interval clear of zero. Re-graded with every build. This gate is separate from the live test above, which counts every settled start whether or not a book posted a line.
17 settled starts are left out of this grade: the book's opening line in the ledger is not the line that day's page showed, so grading them "at the book's line" would grade a line no reader saw. The ledger keeps them as logged.
Every engine pooled, for context: Brier .257 for the model, .245 for the book, .250 for a coin flip, on 275 starts over 34 dates (engine v11, engine v12, engine v13, engine v14). The pool does not open the gate.
Read the research →