Run history & trials

Every run is recorded permanently as a trial. What that means for the significance bar, and why refining costs something.

Every run is recorded permanently, pass or fail, as a numbered trial against your account. That record is what stops the oldest trick in backtesting: running an idea fifty times and reporting only the run that cleared.

What a trial is

When you submit a run, the platform writes an immutable record: the hash of the exact spec, the floors it resolved, the timestamp, the tier, and the outcome. Nothing about it can be edited afterwards — not the floors, not the result. If you want the strategy judged against different floors, that is a new run, and it is a new trial.

Your Run history lists them all: trial number, outcome, observed Sharpe, trade count, tier, and a link to the stored scorecard where one exists. Trials recorded before durable scorecard storage existed show as “not stored” but still count.

The deflated-Sharpe accounting

Every scorecard shows a line like Trial 7 in your history · deflated-Sharpe-adjusted significance = 0.41. The deflated Sharpe takes your observed Sharpe and adjusts it downward for the number of trials you have spent, the non-normality of the returns, and the length of the sample. Run the same idea enough times and one version will look significant by luck; this is the number that accounts for that, across your whole history rather than the current run alone.

“Grade until it passes” is blocked by design
Pre-registered floors are fixed before a run and cannot move afterwards, and every attempt is counted whether it passed or failed. There is no sequence of retries that quietly improves your apparent significance — each one raises the bar instead.

What does and does not count

  • The first run of a strategy — a trial. Creating the strategy and running it are the same act.
  • Every refinement — a trial. See Getting started on why the Refine tab keeps the change and the run as two separate steps.
  • Registering for forward confirmation — not a trial. It records a hash and a date and computes nothing, so it cannot move the accounting. See Running & forward testing.

History is append-only. You cannot delete a trial — a record you could erase would let you keep only the runs that went well, which is the exact effect the accounting exists to defeat.