Strategy reality check: baseline, decay and position-size audit
Before trusting any backtest you saw on X, have your agent compare it with buy-and-hold, check whether the edge faded, and stress-test the risk per trade.
What they did
Four posts this week showed how backtests mislead. BacktestingMax ran the same 456 RSI trades on the S&P 500 at 1% and 10% risk per trade. SPXVIX tested a popular VIX buy rule against simply holding SPY. Trading Time Machine checked whether a famous TLT month-end rule still works. BacktestingMax also compared a gold candlestick pattern with a no-pattern baseline.
How it went
The posters' claims: at 10% risk, $10,000 went to $68,742 and then down to $38, while 1% risk ended at $11,304 (26 backtests profitable at 1%, 21 of them lost money at 10%). The VIX 30/45/14 rule turned $1 into $7.78 since 1993, versus $32.09 for SPY, and sat in cash 48.1% of days. Larry Williams' TLT rule earned 0.11% a trade versus 0.10% for any 6-day window, and the last-day-of-month edge shrank to about 3 basis points since 2018. Gold rose 3 days after a bullish engulfing 53% of the time, the same as on all other days.
Sources:
Worth knowing
- Always compare with a baseline: buy-and-hold, or 'any random day'. Many 'edges' are just the market going up.
- Edges decay once they're published. Check the most recent years separately.
- Position size can turn a winning system into a wipe-out. Test at least two risk levels.
- This is an education prompt: it audits a strategy, and it doesn't trade.
The prompt
You are my backtest skeptic. I'll give you a strategy and its claimed results: [paste rules + claims].
Do NOT place any trades. Using [my backtesting tool / Python with daily data], report:
1. Baseline: the same period's buy-and-hold result for [the traded asset / SPY], and the average return of a random entry with the same holding period. Does the strategy beat both after costs of [$0.01/share or 0.05%]?
2. Decay: split results into [before 2018] and [2018–today]. Is the edge still there recently?
3. Sizing: rerun at [1]% and [10]% risk per trade. Show the ending equity, max drawdown and longest losing streak for each.
4. Sample size: the number of trades, and whether any single year or a handful of trades drives most of the profit.
5. Time in market: the % of days invested, versus the return achieved.
End with a plain-English verdict: 'looks robust', 'needs more testing' or 'probably noise', with reasons.
SAFETY DEFAULTS (keep unless you deliberately change them):
- Analysis only: never place orders while running this audit.
- Before every order, show me: symbol, side, size, order type, limit price, stop, and the rule that triggered it. Then WAIT for my "approve".
- Hard limits: max [$500] per trade, max [3] open positions, stop all trading for the day after a loss of [$300].
- Log every signal, decision and fill to [a running log / spreadsheet] with timestamps.To go fully autonomous: (1) run on paper for at least [4–8 weeks] or [30+ trades]; (2) delete the 'WAIT for my approve' line and keep the hard limits; (3) switch to live keys or account. On Robinhood, turn off trade approvals in your agent settings, but note that some orders can still require in-app approval or acknowledging a pre-trade alert, so unattended execution isn't guaranteed for every order. IBKR's AI connector always asks for approval; use its regular API for unattended trading. Never remove the daily loss limit.