26-Week Drawdown Prediction: Findings

· @u_gS85FJSu6MpPror4Z7ilAA

Conclusion

The 149-feature weekly dataset contains no usable signal for predicting a 26-week drawdown in an individual stock. What variation there is belongs to market regime rather than to the individual company — which says where the variance sits, not that it can be forecast: predicting the regime rate directly failed out of sample too.

Three independent methods — an exhaustive single-threshold rule scan, gradient boosting, and a converged Bayesian posterior — were run against the same purged walk-forward protocol and reached the same answer. This is reported as a negative result, not a work-in-progress: the failure modes that would normally suggest "try harder" were tested for and ruled out.

Evidence: three methods, all below the detection floor

A coin flip is AUC 0.500. All figures are out-of-sample across 14 six-month walk-forward folds, each trained only on rows whose 26-week label window closes before the test window opens.

MethodPooled AUCMean fold AUC
Single-threshold rule scan—best rules: 1.2–1.9× lift on 1–4% of rows
Gradient boosting (histogram GBM, 300 trees)0.5290.554
Bayesian logistic regression, NUTS0.5590.548

The Bayesian run converged everywhere — split R-hat 1.003–1.008 across all 14 folds, zero divergences, zero tree-depth-cap hits — so its agreement with the others is not an artifact of a sampler that failed to explore.

What "at chance" means here. The fourteen fold AUCs average 0.548 with a period-to-period sd of 0.046: a t-test puts that 3.9 standard errors above 0.500 (p = 0.002), while a distribution-free sign test on the same folds gives p = 0.18, the gap carried by the two folds covering the 2020 crash. The sample's own detection floor settles which to believe — at 80% power, after correcting for the 149 factors searched, this design could not resolve an edge below +0.054 AUC, against the +0.048 measured. The result sits below the floor of its own test: consistent with a small real edge and with none, and too small either way to clear costs. power.cpp computes that floor, and the number of out-of-sample periods a blind test would need to settle a claim of a given size.

Why more modelling will not fix it

The three usual explanations for a weak result were each tested and each failed.

It is not overfitting. A heavily regularised GBM — shallow trees, 500-sample leaves, half the features per split, 10× L2 — scored 0.478, worse than the unrestrained model's 0.529. Restraint helps when signal is being over-fit. There was none to over-fit.

It is not feature dilution. Selecting the strongest 60 of 149 features inside each fold, ranked on training rows only, gave pooled AUC 0.516 and mean fold AUC 0.553 against 0.554. The mean per-fold difference was −0.0002, against fold-to-fold noise of ±0.05. The two configurations are indistinguishable; which one wins a given fold is noise larger than the entire effect being measured.

No individual feature carries weight. The best feature by permutation importance moves AUC by 0.008. The importance curve is flat and near zero — there is no strong subset hiding in the 149 to concentrate into.

There is no sequence identity to look up. Every symbol's pe_ttm series, scaled ×100 to integers, was submitted to the OEIS search. Of 681 symbols (8 skipped for having fewer than six observations), 14 matched something at three terms (2.1%) and none matched at six terms or beyond. A control arm of 95 symbols, given length-matched integers drawn at random from the same column's pooled values, matched nothing at either length. The three-term hits are small rising integers that happen to fall inside some monotone sequence's data, not a rule the series follows, and the ×100 scaling is arbitrary — it changes every digit, so a match would not transfer. oeissweep.cpp runs the sweep and caches each query; its 1,552 result rows are in oeissweep_dir/oeis_hits.csv, and seqdump.cpp writes one symbol's prefixes and controls to seqdump_dir/.

Why the data cannot support the question

The effective sample is in the hundreds, not 64,203. A 26-week forward label sampled weekly overlaps the next week's label in 25 of 26 weeks, so nine years yields roughly 18 genuinely separate outcomes per stock. The ~700 names then share one market factor, so they are not 700 independent bets either.

The base rate swinging from 6% to 46% across six-month windows is itself the proof: averaging thousands of independent outcomes cannot move that far. That 8× regime swing dwarfs the 1.2–1.9× available from any stock-level feature.

Coefficients are precise within a regime and unstable across them. With every fold converged, 18 of 134 features are statistically significant in both directions in different folds. The clearest case is z_op_mar_ttm:

WindowCoefficientz
2019 H2−0.230−4.5
2021 H2+0.508+20.8
2022 H2+0.518+21.4
2025 Q3+0.255+17.4

A z of +17 means 57,739 rows pin the parameter down tightly given that period. It does not mean the relationship is a fact about markets. Only 15 of 134 features hold one sign across all 14 folds, and most of those are universe- or industry-breadth measures — regime indicators, not stock-specific signals.

What is feasible instead

The constraint is a shortage of independent observations, so the workable questions are the ones that do not need many. Not which stock, but:

Ruled out: regime exposure. Modelling the factor that moves the base rate 8× directly, instead of letting it be absorbed as noise, was the route this memo had named as most promising. Aggregating to one observation per week — the realised 26-week drawdown rate across the bullish universe — and predicting it from the eleven universe and market features under the same purged walk-forward gives an out-of-sample R² of −0.75 against predicting the training mean, with correlation −0.20. In sample those features look mildly encouraging, rsa_univ_avg at +0.20, which is how the suggestion came to be made; out of sample the sign flips.

The 26-week label overlap leaves roughly 18 independent observations across 476 weekly dates, so this refutes the simple linear form of the idea rather than regime forecasting in general. Tool: regime.cpp, output in regime_dir/.

A caveat worth carrying forward: the label is only ever 1 when pgr_bull = 1, so it is defined inside the bullish universe. All figures above are computed within that universe (64,203 rows, 17.4% base rate); outside it the target is identically zero.

Method and reproduction

Data. AI Master Data 2017.2026 Weekly All.csv — 238,522 rows × 154 columns, 696 symbols, 2017-01-06 to 2026-03-13. Target Target 26 dd 10 is already forward-looking and is used as-is; an earlier version shifted it a further 26 weeks, which was wrong.

Protocol. 14 six-month test windows from 2019 H1. Each fold trains only on rows whose 26-week label window closes before the test window opens (a purge gap, so no training label overlaps the test period). Test rows after max_date − 26w are dropped because their labels are truncated.

Tools, all dependency-free C++ (g++ -O2 -std=c++17), each writing to its own output directory:

ToolPurpose
predict26single-threshold rule scan, full universe
predict26_bullsame scan restricted to pgr_bull = 1, with precision/recall/lift
predict26_gbmhistogram gradient boosting + purged walk-forward (GBM_TOPK=n for nested feature selection)
predict26_hmcBayesian logistic regression sampled by NUTS
colcorrpairwise feature correlations; generates the near-duplicate drop list
foldcmpper-fold AUC comparison between two runs
coefpickone feature's per-fold posterior, out of the by-fold coefficient file
powerdetection floor and required periods, from the spread of the fold results

The first two were verified byte-identical against the original Python. The GBM is an independent reimplementation of sklearn's HistGradientBoostingClassifier and agrees with it to within ±0.015 AUC per fold.

Sampler. NUTS with recursive doubling, Stan-style windowed mass-matrix adaptation, dual-averaged step size (target 0.8), split R-hat. 2 chains × (300 warmup + 600 draws) per fold. Momentum is drawn by Box–Muller from uniform_real_distribution, whose draw sequence the standard fixes, rather than from std::normal_distribution, whose sequence is implementation-defined. Fold 0 now reproduces to the last digit under both g++ 13.3.0 and g++ 15.2.0, where before it did not, and no fold's AUC moved by more than 0.002 in the switch. 15 columns with |r| ≥ 0.95 against another column were dropped for the Bayesian run only — collinear predictors create posterior ridges that a diagonal mass matrix cannot precondition. Trees are indifferent to collinearity, so the GBM keeps all 149.

Full sweep: 8,691 s on four worker threads. The thread count is part of the result — gradients are summed one block per thread, so changing it changes the summation order and the last digits. Per-fold posteriors in predict26_hmc_dir/predict26_hmc_coefficients_by_fold.csv (every feature × fold with mean, sd, z, CI and that fold's R-hat).