Backtesting & simulation
vectorbt for breadth, an event-driven simulator for realism, and four levels of simulation before real capital.
Two engines, two jobs: vectorbt explores many strategy variants fast, while an event-driven simulator models the messy reality vectorized backtests miss.
The two engines
| Engine | Role |
|---|---|
| vectorbt | Rapid parameter exploration, vectorized simulations, large experiment grids, portfolio-level analysis. |
| Event-driven simulator (Backtrader / custom) | Order-book execution, partial fills, latency, slippage, low liquidity, outages, failed orders. |
For new listings a custom simulator may eventually be required — generic engines do not model launch conditions accurately.
Four levels of simulation
| Level | What it does |
|---|---|
| Historical backtest | Previously recorded market data |
| Shadow mode | Observes the live market, submits no orders |
| Paper trading | Simulated orders on live prices with modeled execution |
| Small-capital production | Real orders with restricted position sizes |
Execution realism (modeled in every backtest)
- fees
- spread
- slippage
- latency
- partial fills
- missing liquidity
- rejected orders
- order limits
- API delays
- gas costs
- price impact
No look-ahead bias
A decision at time t may only use data available at or before t. The signal is generated after the candle closes and executed at the next open (or via an explicit fill model) — never at the same close it was computed from.
Wrong: decide on candle t close, assume fill before that close
Right: decide after candle t closes, execute at candle t+1 openFill simulation models
| Model | How a fill is priced |
|---|---|
| Next-open | ExecutionPrice = Open(t+1) — simple, optimistic on volatile tokens |
| Spread-adjusted | Ref × (1 + HalfSpread + Slippage) for a buy |
| Order-book replay | Walk historical asks/bids, compute fillable VWAP and partial fills |
| Latency | ExecTimestamp = Signal + ProcessingLatency + NetworkLatency |
Walk-forward, grids & robustness
Parameters are never fit on the full dataset. Walk-forward rolls train → validate → test windows forward to measure stability and expose overfitting; a grid enumerates candidates; sensitivity perturbs the winner.
breakout_periods = [10, 20, 30, 50]
volume_thresholds = [1.5, 2.0, 2.5, 3.0]
atr_multipliers = [1.0, 1.5, 2.0, 2.5]
grid = [
{"breakout_period": b, "volume_threshold": v, "atr_multiplier": a}
for b in breakout_periods for v in volume_thresholds for a in atr_multipliers
]A robust parameter stays acceptable when nudged. A sharp, isolated performance peak is a sign of overfitting, not edge.
Monte Carlo reshuffles or resamples trades to estimate the drawdown distribution, the worst losing streak and the probability of exceeding a loss threshold — it quantifies uncertainty, it does not create certainty.