ML-Trading
Machine Learning Enhanced Trading Strategy on Tencent Daily Stock Data (Index Enhancement)
We build an end-to-end pipeline including data preprocessing, factor mining, machine learning forecasting, and an enhanced trading strategy with risk control. The goal is to achieve excess return over a baseline strategy while keeping maximum drawdown as low as possible.
Training: 2018–2023 | Backtest: 2024-01-01 to 2025-04-24 | Initial Capital: 100,000 CNY
Assignment Requirements
- Data preprocessing: clean the provided Tencent daily stock data and mine informative factors.
- Machine learning forecasting: build a model with good predictive performance for stock prices/returns.
- Trading strategy: generate buy/sell timing signals from predictions to achieve index enhancement (excess return).
- Risk control: maximize profit while keeping maximum drawdown as low as possible.
- Visualization & evaluation: visualize final investment results and evaluate against actual data.
- Report: clearly describe algorithms, training procedure, feature engineering, preprocessing, and advantages of ML for this task.
Raw Stock Data
We use Tencent daily OHLCV-style data. The dataset spans 2018-01-02 to
2025-04-24 with 1,797 trading-day records and the following fields:
timestamps, open, high, low, close, volume, amount.
- Training set: 2018-01-02 to 2023-12-31
- Backtest set: 2024-01-01 to 2025-04-24
Full data file: static/data.csv
Data Preview (first 6 rows)
| timestamps | open | high | low | close | volume | amount |
|---|---|---|---|---|---|---|
| 2018-01-02 | 364.39 | 374.39 | 363.99 | 374.19 | 240,805 | 10.01B |
| 2018-01-03 | 380.39 | 383.19 | 375.39 | 378.59 | 227,802 | 9.64B |
| 2018-01-04 | 383.39 | 388.79 | 380.59 | 388.19 | 279,045 | 11.97B |
| 2018-01-05 | 392.79 | 392.79 | 384.59 | 389.59 | 199,584 | 8.61B |
| 2018-01-08 | 392.59 | 395.59 | 390.19 | 394.99 | 179,948 | 7.86B |
| 2018-01-09 | 397.39 | 402.79 | 395.39 | 400.19 | 244,765 | 10.85B |
Data Preprocessing
- Cleaning: handle missing values, non-trading days, and abnormal entries.
- Adjustments: ensure consistent price series (if adjusted prices are available/needed).
- Labeling: define prediction targets (e.g., next-day return, next-day close, direction).
- Normalization: apply scaling/standardization where appropriate (especially for neural nets).
- Walk-forward split: avoid leakage by respecting time order during training/validation.
Tip: In the final report, explicitly state target definition, feature window sizes, and how you prevent look-ahead bias.
Data preprocessing workflow used in this project.
Factor Mining Results
We mine predictive factors from price and volume information (momentum, volatility, candlestick shapes, volume/flow). Factor quality is evaluated by Information Coefficient (IC) and rank-based IC.
Top Factors by |Rank-IC|
Ranking factors by absolute Spearman rank-IC.
Pearson vs Spearman IC
Comparing linear IC (Pearson) and rank IC (Spearman).
Top-10 Factors (from IC ranking)
| factor | pearson_ic | spearman_ic | abs_rank_ic |
|---|---|---|---|
| price_vwap_diff | -0.1040 | -0.1233 | 0.1233 |
| ma_20 | -0.0926 | -0.0963 | 0.0963 |
| ma_5 | -0.0894 | -0.0906 | 0.0906 |
| volatility_10 | 0.1167 | 0.0836 | 0.0836 |
| volatility_20 | 0.1184 | 0.0775 | 0.0775 |
| upper_shadow | 0.1070 | 0.0638 | 0.0638 |
| hl_vol | 0.1254 | 0.0620 | 0.0620 |
| lower_ratio | -0.0692 | -0.0600 | 0.0600 |
| lower_shadow | -0.0571 | -0.0561 | 0.0561 |
| vol_roc_5 | 0.0458 | 0.0537 | 0.0537 |
Factors are computed on the training period and evaluated by IC against future returns.
Prediction Model and Forecasting Results
We train an XGBoost-based forecasting model using factors mined from price and volume. The model is trained on 2018–2023 and evaluated with time-aware splits (train/valid/test).
Train Forecast (Close)
Train Forecast (Volume)
Valid Forecast (Close)
Valid Forecast (Volume)
Test Forecast (Close)
Test Forecast (Volume)
Model Evaluation Summary
| split | mae | rmse | spearman_ic | direction_acc |
|---|---|---|---|---|
| train | 0.0181 | 0.0257 | 1.0000 | 98.86% |
| valid | 6.3936 | 8.6984 | -0.0474 | 48.76% |
| test | 7.8201 | 10.6094 | -0.0055 | 48.13% |
We use the model score mainly for relative ranking and trading signal generation rather than pure price-level accuracy.
Strategy Comparison
Part I — Comparing Different Strategies
We benchmark our pipeline against several alternative strategies implemented in Backtrader. All strategies use the same data window (2024-01-01 to 2025-04-24) and initial cash (100,000 CNY).
Strategies included
- Benchmark (Buy & Hold): fully invested throughout the backtest.
- StrategyH_HighAlphaOverlay: score-ranked overlay around beta=1 (more aggressive).
- StrategyB_BalancedEnhanced: quantile-based 0.7 / 1.0 / 1.3 exposure (balanced).
- StrategyD_DefensiveMeanRev: defensive mean-reversion with smaller exposure.
- StrategyHPP_QlibStyle: Qlib-style aggressive enhancement with risk controls.
- StrategyBPP_QlibStyle: Qlib-style balanced enhancement with stricter risk controls.
Backtest Summary (Alternative Strategies)
| Strategy | Final Value (CNY) | Total Return | Annual Return | Annual Vol | Sharpe | Max Drawdown |
|---|---|---|---|---|---|---|
| Benchmark_BuyAndHold | 163,127.2 | 63.13% | 47.02% | 34.32% | 1.37 | -23.49% |
| StrategyH_HighAlphaOverlay | 151,876.7 | 51.88% | 38.97% | 24.51% | 1.59 | -12.93% |
| StrategyB_BalancedEnhanced | 151,774.2 | 51.77% | 38.90% | 28.27% | 1.38 | -16.25% |
| StrategyD_DefensiveMeanRev | 123,608.0 | 23.61% | 18.16% | 18.16% | 1.00 | -12.92% |
| StrategyHPP_QlibStyle | 146,811.1 | 46.81% | 35.31% | 24.74% | 1.43 | -12.93% |
| StrategyBPP_QlibStyle | 158,576.4 | 58.58% | 43.78% | 30.63% | 1.43 | -19.57% |
Why can these strategies underperform Buy & Hold?
- Beta reduction / time-out-of-market: most alternative strategies deliberately spend time below 100% exposure to control risk. In a strong uptrend, reducing exposure can lower returns.
- Noisy out-of-sample signals: if the prediction/factor score is weak (low IC / ~50% direction accuracy), timing trades may miss rallies and create whipsaws, hurting total return.
- Risk controls trade return for stability: trend/volatility/drawdown filters help avoid deep losses, but they also cut participation during fast rebounds.
Advantages of these strategies
- Better drawdown control: e.g., StrategyH_HighAlphaOverlay and StrategyHPP_QlibStyle reduce max drawdown to ~-12.9% vs -23.5% for Buy & Hold.
- Lower volatility / higher risk-adjusted performance: StrategyH_HighAlphaOverlay achieves a higher Sharpe (1.59) with much lower volatility (24.5%).
- Different “styles” for different preferences: DefensiveMeanRev is designed for stability, while Qlib-style strategies add systematic risk controls (trend, vol, drawdown guards, position smoothing).
Part II — Our Final Strategy and Horizon Variants
Final Strategy: Conviction_Filter_Strategy
Our final submission is a conviction-based long-only strategy. Each day we compute a blended score
from (i) the standardized ML predicted return score and (ii) a composite alpha-factor score:
score_t = w_pred · z(r_pred)_t + w_alpha · alpha_score_t.
We then compute rolling quantile thresholds from recent score history (lookback window) and map score into discrete exposure: 0 / 0.5 / 1.0. This design improves robustness and reduces turnover compared to continuous leverage scaling.
Ablation-selected best parameters
- Best weights:
w_alpha = 0.22,w_pred = 0.78(found by grid ablation). - Best thresholds:
q_exit = 0.09,q_half = 0.12(found by grid ablation with constraintq_exit < q_half).
Final Strategy Performance (Overall)
| Strategy | Final Value (CNY) | Total Return | Annual Return | Annual Vol | Sharpe | Max Drawdown |
|---|---|---|---|---|---|---|
| Benchmark_BuyAndHold | 163,127.2 | 63.13% | 47.02% | 34.32% | 1.37 | -23.49% |
| Conviction_Filter_Strategy (Final) | 224,877.5 | 124.88% | 89.30% | 27.00% | 3.31 | -12.93% |
Horizon Variants (pre1 / pre3 / pre5 / 10-day→1-day)
To test robustness, we run the same strategy logic under four forecasting settings:
- pre1: predict the next-day return and trade daily.
- pre3: predict 3-day ahead (smoother signal, potentially lower noise but more lag).
- pre5: predict 5-day ahead (even smoother, but may react slower to regime changes).
- 10-day window → 1-day prediction: use the previous 10 trading days as input features to predict next-day return (more context for short-horizon prediction).
Below is the table template. Replace the TODO cells with the values from your
backtest_results_longonly.csv (pre1/pre3/pre5/10→1). If you paste that CSV here, I can fill the table for you.
| Setting | Final Value (CNY) | Total Return | Annual Return | Annual Vol | Sharpe | Max Drawdown | Interpretation |
|---|---|---|---|---|---|---|---|
| pre1 (1-day ahead) | 142,650.4 | 42.65% | 32.16% | 28.37% | 1.13 | -24.88% | Most responsive, but most sensitive to noise. |
| pre3 (3-day ahead) | 190,351.4 | 90.35% | 65.75% | 28.78% | 2.28 | -15.07% | Smoother signal; may reduce whipsaws but can lag turning points. |
| pre5 (5-day ahead) | 170,123.4 | 70.12% | 51.76% | 29.59% | 1.75 | -20.44% | Even smoother; can miss fast rebounds but improves stability. |
| 10-day window → 1-day pred | 213,918.9 | 113.92% | 82.00% | 26.84% | 3.05 | -12.93% | Uses richer short-term context; often improves robustness vs pure 1-day inputs. |
Weight & Threshold Selection (Ablation Study)
Our final strategy maps a blended conviction score into discrete exposure
(0, 0.5, 1.0). We tune two key components:
(1) the blending weights between ML prediction and alpha-factor signals, and
(2) the rolling quantile thresholds that control exit / half exposure.
Best weights
w_alpha = 0.22, w_pred = 0.78
Found by grid ablation under fixed thresholds (lookback=120, q_exit=0.05, q_half=0.11).
Best thresholds
q_exit = 0.09, q_half = 0.12
Found by grid ablation with constraint q_exit < q_half under fixed weights.
A) Weight Selection
We define the daily conviction score as:
score_t = w_pred · z(r_pred)_t + w_alpha · alpha_score_t.
- Constraint:
w_pred + w_alpha = 1 - Search:
w_alpha ∈ [0, 1]with step0.02,w_pred = 1 - w_alpha - Backtest: 2024-01-01 to 2025-04-24, initial cash 100,000 CNY
- Fixed during this stage:
lookback=120,q_exit=0.05,q_half=0.11 - Objective: maximize total return
Best result: w_alpha=0.22, w_pred=0.78.
This indicates the ML prediction dominates the score, while alpha factors provide complementary stabilization.
Note: this weight ablation is performed before the final q tuning; the final reported performance
uses both the optimal weights and thresholds.
Log-scale visualization of total return across the grid of w_alpha / w_pred ratios.
The optimum appears near w_alpha=0.22, w_pred=0.78.
B) Quantile Threshold Selection
Using score history over the most recent lookback=120 days, we compute rolling quantile cutoffs:
thr_exit(t) = Quantile(score_hist, q_exit),
thr_half(t) = Quantile(score_hist, q_half).
Today’s score is mapped to exposure:
0 if score < thr_exit,
0.5 if thr_exit ≤ score < thr_half,
else 1.
- Fixed weights:
w_pred=0.78,w_alpha=0.22 - Search ranges:
q_exit ∈ [0.00, 0.20],q_half ∈ [0.00, 0.40](step0.01) - Constraint:
q_exit < q_half - Objective: maximize total return
Best result: q_exit=0.09, q_half=0.12.
Quantile thresholds are robust because they adapt automatically when the score distribution shifts.
3D visualization of total return over (q_exit, q_half).
The best region lies around q_exit=0.09, q_half=0.12 under the constraint q_exit < q_half.
Final Strategy Results
We select Conviction_Filter_Strategy as the final submission. The backtest runs from 2024-01-01 to 2025-04-24 with initial capital 100,000 CNY. Below we visualize return, risk, exposure, and signal effectiveness. All figures are rendered as high-resolution PNGs.
Final Value
224,877.5
CNY
Total Return
124.88%
vs 63.13% Buy & Hold
Sharpe
3.31
vs 1.37 Buy & Hold
Max Drawdown
-12.93%
vs -23.49% Buy & Hold
Equity Curve (NAV)
Stronger growth with smoother trajectory. Our NAV stays above the benchmark for most of the backtest, confirming consistent excess return from the ML + factor conviction signal.
Drawdown Curve
Downside risk is controlled. The final strategy cuts max drawdown to -12.93% (vs -23.49% Buy & Hold), showing the quantile-based de-risking rule effectively limits large losses.
Position / Exposure Over Time
Stable, interpretable decisions (0 / 0.5 / 1). Instead of noisy continuous leverage, the strategy maps conviction into discrete exposure, reducing whipsaws and turnover while staying invested when the signal is strong.
Return by Score Quantile
Clear ranking power. Higher conviction quantiles correspond to higher realized returns, validating that the blended score (ML prediction + alpha factors) contains actionable information for timing.
Score & Rolling Thresholds
Adaptive risk control. Rolling quantile thresholds (q_exit, q_half)
adapt to distribution shifts in the score, helping the strategy reduce exposure during weak-conviction regimes
and re-risk when conviction improves.
Performance & Risk Metrics
| Metric | Final Strategy | Buy & Hold |
|---|---|---|
| Final Value (CNY) | 224,877.5 | 163,127.2 |
| Total Return | 124.88% | 63.13% |
| Annualized Return | 89.30% | 47.02% |
| Annualized Volatility | 27.00% | 34.32% |
| Sharpe Ratio | 3.31 | 1.37 |
| Maximum Drawdown | -12.93% | -23.49% |
| Calmar Ratio | 6.91 | 2.00 |
Evaluation & Key Advantages
We evaluate the final submission (Conviction_Filter_Strategy) on the out-of-sample backtest period 2024-01-01 to 2025-04-24 with 100,000 CNY initial capital. The strategy is designed to achieve index enhancement (excess return over Buy & Hold) while keeping maximum drawdown as low as possible.
1) Performance vs. Benchmark (Buy & Hold)
| Metric | Final Strategy | Buy & Hold | Improvement |
|---|---|---|---|
| Final Value (CNY) | 224,877.5 | 163,127.2 | +61,750.3 |
| Total Return | 124.88% | 63.13% | +61.75 pp |
| Annualized Return | 89.30% | 47.02% | +42.28 pp |
| Annualized Volatility | 27.00% | 34.32% | -7.32 pp |
| Sharpe Ratio | 3.31 | 1.37 | +1.94 |
| Maximum Drawdown | -12.93% | -23.49% | +10.56 pp (less drawdown) |
| Calmar Ratio | 6.91 | 2.00 | +4.91 |
Summary: The final strategy more than doubles the benchmark’s total return while cutting the maximum drawdown by nearly half, delivering substantially stronger risk-adjusted performance.
2) Why the Strategy Works (Strengths)
-
Signal fusion (ML + factors): we combine a standardized ML prediction score with a composite
alpha-factor score using learned weights (
w_pred,w_alpha). This reduces reliance on a single noisy signal source and improves robustness across market conditions. -
Distribution-aware decision rule: instead of fixed numeric thresholds, we use rolling
quantile thresholds (
q_exit,q_half) computed from recent score history. This adapts automatically when the score distribution shifts (e.g., volatility regime changes). - Discrete exposure (0 / 0.5 / 1): mapping scores into a small number of stable position levels prevents over-trading and reduces whipsaw risk compared to continuous scaling.
- Ablation-driven parameter selection: key parameters are selected via systematic grid-search hinting (weight tuning and threshold tuning), ensuring the final configuration is supported by empirical evidence rather than manual trial-and-error.
- Better downside control: the strategy reduces deep drawdowns by exiting or de-risking when the conviction score falls below the lower quantile, which is especially important for index enhancement tasks requiring controlled risk.