How We Measure Success: Our Pre-Declared Live Performance Criteria
Every backtested system looks good on paper. The parameters were chosen because they worked on historical data. The events were selected because the system detected them. The presentation was designed to be persuasive.
We know this, and so should you.
That is why we are publishing our success criteria before the first live signal fires. We want our future performance to be judged against standards we declared in advance, not standards we reverse-engineered after the fact.
What Counts as a Hit
A live ALERT signal is classified as a HIT if the S&P 500 experiences a peak-to-trough drawdown of 7% or greater within 60 trading days of the system first entering the ALERT directional state.
This is the same criterion used in our 28-year backtest. We are not changing it for the live period.
What Counts as a Miss
A live ALERT signal is classified as a MISS if the S&P 500 does not experience a 7%+ drawdown within 60 trading days of ALERT entry. If the market declines 4%, 5%, or 6% and recovers, that is a miss by our definition. We will report it as such.
What Counts as Resolved
If the system enters DETERIORATING but never escalates to ALERT before returning to CLEAR, the episode is classified as RESOLVED. This is neither a hit nor a miss. The system detected early structural change that did not develop into a full regime deterioration. We track and report these separately.
How We Measure Lead Time
Lead time is measured in calendar days from the date the system first enters ALERT to the date of the subsequent drawdown trough (if a hit) or the end of the 60-day observation window (if a miss). We report both median and individual lead times.
Our Backtest Baseline
These are the numbers the live track record will be compared against:
What We Expect Going Forward
We do not expect perfect replication of backtest results. An 84% hit rate with a 95% confidence interval of 69–93% means we expect some misses. A system that claims 100% forward accuracy is either lying or has not been tested long enough.
We expect the live hit rate to fall within or near the confidence interval of the backtest. If it does, the system is performing as designed. If it falls significantly below, we will investigate parameter stability and publish our findings.
We will report every ALERT episode within 60 trading days of its resolution, including full analysis of hits, misses, and resolved episodes.
Why We Are Publishing This
Three reasons.
First, intellectual honesty. Declaring criteria in advance eliminates the temptation to rationalize outcomes after the fact.
Second, accountability. Our subscribers deserve to know exactly how we define success and failure.
Third, credibility. Anyone can publish a backtest. Very few signal providers publish pre-declared live criteria and then report against them transparently. We intend to be one of those few.
The Live Record Starts Now
March 19, 2026
Beginning March 19th, every signal is live. Our daily state is computed after market close, recorded in our database, published on our public history page, and posted to our Discord and social channels. These timestamps are verifiable and immutable.
We will maintain a separate live performance page that tracks every ALERT episode from this date forward, scored against the criteria above. Backtest results and live results will always be presented separately, never blended.
Our backtest gave us confidence to build this product. The live record will determine whether that confidence was justified.
