Benchmark Results¶
Note
We will continue to update the following benchmark results. If you have proposed new (classic or data-driven) OLPS models, you can send us your paper/code link via Contact Us or raise a pull request. We will add them to this repository and update the leaderboard as soon as possible.
Benchmark Results on Profitability¶
See also
The benchmark results on profitability and the analysis of the results are available in the paper.
Benchmark Results on Risk Resilience¶
Besides the benchmark results of profitability,
we further evaluate 2 vital risk metrics: Volatility Risk (VR) and Maximum Drawdown (MDD) on FinOL. Table below reveals that:
The risk profiles vary markedly across different OLPS methods. Methods like SSPO and PPT exhibit higher volatility due to their aggressive betting behaviors. In contrast, methods such as UP and ANTI produce lower risk.
For Pattern-matching methods, AICTR and KTPT undertake more risk exposures for pursuing higher returns.
Most Meta-learning methods exhibit moderate risk profiles comparable to bas. This validates their strength in balancing between return and risk by combining multiple sub-methods.
Model |
NYSE(O) |
NYSE(N) |
DJIA |
SP500 |
TSE |
SSE |
HSI |
CMEG |
CRYPTO |
|||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
VR |
MDD |
VR |
MDD |
VR |
MDD |
VR |
MDD |
VR |
MDD |
VR |
MDD |
VR |
MDD |
VR |
MDD |
VR |
MDD |
|
Classic OLPS: Benchmark baseline |
||||||||||||||||||
Market |
0.15 |
0.21 |
0.26 |
0.55 |
0.28 |
0.18 |
0.23 |
0.28 |
0.17 |
0.26 |
0.18 |
0.29 |
0.18 |
0.36 |
0.16 |
0.24 |
0.41 |
0.36 |
Best |
0.19 |
0.17 |
1.01 |
0.96 |
0.45 |
0.15 |
0.48 |
0.16 |
0.55 |
0.38 |
0.49 |
0.35 |
0.53 |
0.60 |
0.45 |
0.52 |
0.54 |
0.36 |
UCRP |
0.15 |
0.20 |
0.27 |
0.54 |
0.28 |
0.18 |
0.24 |
0.29 |
0.17 |
0.26 |
0.17 |
0.23 |
0.18 |
0.36 |
0.13 |
0.15 |
0.40 |
0.34 |
BCRP |
0.18 |
0.15 |
0.66 |
0.77 |
0.44 |
0.16 |
0.45 |
0.17 |
0.55 |
0.38 |
0.49 |
0.35 |
0.34 |
0.37 |
0.45 |
0.51 |
0.50 |
0.30 |
Classic OLPS: Follow-the-winner |
||||||||||||||||||
UP |
0.15 |
0.20 |
0.27 |
0.54 |
0.28 |
0.18 |
0.24 |
0.29 |
0.17 |
0.26 |
0.17 |
0.23 |
0.18 |
0.36 |
0.13 |
0.15 |
0.40 |
0.34 |
EG |
0.15 |
0.20 |
0.27 |
0.54 |
0.28 |
0.18 |
0.24 |
0.29 |
0.17 |
0.26 |
0.17 |
0.23 |
0.18 |
0.36 |
0.13 |
0.15 |
0.40 |
0.34 |
SCRP |
0.27 |
0.58 |
0.72 |
0.92 |
0.38 |
0.23 |
0.47 |
0.40 |
0.56 |
0.70 |
0.48 |
0.64 |
0.55 |
0.63 |
0.52 |
0.70 |
1.04 |
0.89 |
PPT |
0.36 |
0.39 |
1.04 |
0.84 |
0.44 |
0.12 |
1.55 |
0.60 |
0.94 |
0.22 |
0.41 |
0.53 |
0.77 |
0.77 |
0.49 |
0.61 |
0.92 |
0.85 |
SSPO |
0.36 |
0.39 |
0.92 |
0.85 |
0.44 |
0.12 |
1.52 |
0.60 |
0.95 |
0.23 |
0.42 |
0.57 |
0.78 |
0.75 |
0.52 |
0.68 |
0.92 |
0.85 |
Classic OLPS: Follow-the-loser |
||||||||||||||||||
ANTI1 |
0.21 |
0.20 |
0.60 |
0.62 |
0.31 |
0.20 |
0.74 |
0.31 |
0.45 |
0.17 |
0.24 |
0.24 |
0.49 |
0.65 |
0.28 |
0.37 |
0.43 |
0.28 |
ANTI2 |
0.24 |
0.21 |
0.70 |
0.62 |
0.34 |
0.20 |
0.90 |
0.42 |
0.59 |
0.16 |
0.26 |
0.23 |
0.62 |
0.72 |
0.33 |
0.46 |
0.44 |
0.25 |
PAMR |
0.31 |
0.70 |
0.58 |
0.58 |
0.40 |
0.19 |
0.78 |
0.48 |
0.66 |
0.25 |
0.32 |
0.56 |
0.46 |
0.69 |
0.36 |
0.52 |
0.70 |
0.86 |
CWMR-Var |
0.31 |
0.70 |
0.62 |
0.65 |
0.40 |
0.19 |
0.85 |
0.49 |
0.66 |
0.26 |
0.31 |
0.56 |
0.47 |
0.72 |
0.37 |
0.51 |
0.67 |
0.79 |
CWMR-Stdev |
0.31 |
0.70 |
0.63 |
0.67 |
0.40 |
0.19 |
0.85 |
0.49 |
0.66 |
0.26 |
0.31 |
0.56 |
0.47 |
0.72 |
0.37 |
0.51 |
0.67 |
0.79 |
OLMAR-S |
0.35 |
0.46 |
0.94 |
0.85 |
0.46 |
0.16 |
1.17 |
0.71 |
0.71 |
0.20 |
0.41 |
0.58 |
0.54 |
0.81 |
0.46 |
0.60 |
0.83 |
0.75 |
OLMAR-E |
0.35 |
0.64 |
1.00 |
0.86 |
0.46 |
0.23 |
1.24 |
0.85 |
0.75 |
0.28 |
0.41 |
0.67 |
0.55 |
0.81 |
0.44 |
0.64 |
0.72 |
0.89 |
RMR |
0.35 |
0.54 |
0.90 |
0.84 |
0.46 |
0.21 |
1.16 |
0.76 |
0.72 |
0.19 |
0.40 |
0.55 |
0.56 |
0.79 |
0.44 |
0.57 |
0.82 |
0.74 |
RPRT |
0.35 |
0.60 |
1.08 |
0.87 |
0.47 |
0.29 |
1.65 |
0.80 |
0.95 |
0.30 |
0.44 |
0.58 |
0.56 |
0.76 |
0.47 |
0.59 |
0.72 |
0.89 |
Classic OLPS: Pattern-matching |
||||||||||||||||||
AICTR |
0.36 |
0.52 |
1.05 |
0.93 |
0.47 |
0.20 |
1.35 |
0.75 |
0.93 |
0.31 |
0.45 |
0.71 |
0.56 |
0.79 |
0.46 |
0.62 |
0.90 |
0.65 |
KTPT |
0.36 |
0.64 |
0.98 |
0.68 |
0.43 |
0.12 |
1.32 |
0.64 |
0.88 |
0.24 |
0.38 |
0.68 |
0.58 |
0.78 |
0.50 |
0.73 |
0.79 |
0.94 |
Classic OLPS: Meta-learning |
||||||||||||||||||
SP |
0.15 |
0.20 |
0.27 |
0.54 |
0.28 |
0.18 |
0.24 |
0.29 |
0.17 |
0.26 |
0.17 |
0.23 |
0.18 |
0.36 |
0.13 |
0.15 |
0.40 |
0.34 |
ONS |
0.19 |
0.24 |
0.77 |
0.89 |
0.36 |
0.23 |
1.22 |
0.72 |
0.37 |
0.28 |
0.21 |
0.24 |
0.60 |
0.71 |
0.07 |
0.14 |
0.43 |
0.31 |
GRW |
0.15 |
0.20 |
0.26 |
0.52 |
0.27 |
0.15 |
0.28 |
0.27 |
0.31 |
0.34 |
0.22 |
0.27 |
0.20 |
0.37 |
0.15 |
0.15 |
0.42 |
0.46 |
WAAS |
0.15 |
0.20 |
0.27 |
0.54 |
0.28 |
0.18 |
0.24 |
0.29 |
0.17 |
0.26 |
0.17 |
0.23 |
0.18 |
0.35 |
0.13 |
0.16 |
0.40 |
0.34 |
CW-OGD |
0.15 |
0.21 |
0.37 |
0.66 |
0.24 |
0.16 |
0.26 |
0.37 |
0.21 |
0.25 |
0.24 |
0.33 |
0.23 |
0.37 |
0.24 |
0.29 |
0.42 |
0.34 |
The analysis reveals that while certain OLPS methods can gain higher profitability, their improvement often accompanies greater risk exposures. Hence, an ideal OLPS method should maximize returns while controlling the downside risk. The benchmark results motivate the development of innovative mechanisms to manage risk in the data-driven OLPS field, such as adaptive position sizing, robust rebalancing rules, and risk-sensitive loss functions.
Benchmark Results on Practicality¶
We also examine 2 practical OLPS metrics: Average Turnover (ATO) and Running Time (RT).
Table below presents the ATO results on FinOL.
Several key observations can be summarized:
Methods like UP and EG generate relatively low ATOs by stable rebalancing.
However, some methods that fall into the Follow-the-loser classification (e.g., CWMR and OLMAR) exhibit high ATOs due to their frequent portfolio adjustment, which may raise practical concerns about high transaction costs.
Most Meta-learning methods maintain ATO levels comparable to UP, which are lower than methods like PAMR with higher turnover rates. This demonstrates Meta-learning methods can effectively balance adaptability and trading frequencies.
Model |
NYSE(O) |
NYSE(N) |
DJIA |
SP500 |
TSE |
SSE |
HSI |
CMEG |
CRYPTO |
|---|---|---|---|---|---|---|---|---|---|
Classic OLPS: Benchmark baseline |
|||||||||
Market |
– |
– |
– |
– |
– |
– |
– |
– |
– |
Best |
– |
– |
– |
– |
– |
– |
– |
– |
– |
UCRP |
0.57% |
0.75% |
1.24% |
1.01% |
1.06% |
1.98% |
1.94% |
1.52% |
0.96% |
BCRP |
0.20% |
1.15% |
0.70% |
0.41% |
0.30% |
0.37% |
2.59% |
0.44% |
1.15% |
Classic OLPS: Follow-the-winner |
|||||||||
UP |
1.37% |
0.75% |
1.97% |
1.01% |
2.02% |
2.50% |
2.66% |
2.10% |
1.88% |
EG |
0.55% |
0.71% |
1.20% |
0.96% |
1.02% |
1.90% |
1.86% |
1.47% |
0.92% |
SCRP |
7.20% |
2.83% |
19.36% |
12.31% |
12.42% |
15.45% |
14.01% |
20.43% |
12.63% |
PPT |
55.43% |
54.27% |
53.66% |
52.18% |
55.37% |
54.74% |
50.88% |
51.21% |
49.88% |
SSPO |
52.40% |
53.74% |
53.46% |
52.43% |
52.97% |
52.55% |
48.35% |
45.92% |
47.11% |
Classic OLPS: Follow-the-loser |
|||||||||
ANTI1 |
18.16% |
22.99% |
13.82% |
21.42% |
20.47% |
16.49% |
17.08% |
14.90% |
16.78% |
ANTI2 |
27.21% |
27.40% |
20.65% |
24.09% |
25.51% |
25.93% |
19.19% |
21.25% |
25.79% |
PAMR |
84.99% |
67.16% |
86.00% |
66.67% |
78.49% |
66.67% |
68.28% |
74.41% |
65.21% |
CWMR-Var |
84.06% |
65.77% |
84.85% |
65.69% |
77.52% |
64.42% |
67.38% |
73.44% |
63.70% |
CWMR-Stdev |
84.06% |
65.55% |
84.85% |
65.59% |
77.51% |
64.34% |
67.32% |
73.50% |
63.71% |
OLMAR-S |
64.43% |
58.68% |
60.99% |
56.49% |
63.65% |
57.63% |
60.71% |
56.13% |
55.64% |
OLMAR-E |
75.44% |
73.34% |
71.47% |
72.21% |
72.83% |
69.81% |
70.84% |
67.00% |
62.46% |
RMR |
66.00% |
54.58% |
63.00% |
54.73% |
67.49% |
57.97% |
58.47% |
57.75% |
54.69% |
RPRT |
72.71% |
69.78% |
65.47% |
68.52% |
68.70% |
66.52% |
66.28% |
65.33% |
58.06% |
Classic OLPS: Pattern-matching |
|||||||||
AICTR |
63.30% |
69.44% |
58.47% |
69.34% |
66.41% |
63.36% |
69.31% |
61.37% |
59.14% |
KTPT |
87.05% |
81.88% |
79.46% |
72.07% |
76.36% |
70.96% |
70.94% |
63.17% |
69.87% |
Classic OLPS: Meta-learning |
|||||||||
SP |
0.57% |
0.75% |
1.24% |
1.01% |
1.06% |
1.98% |
1.94% |
1.52% |
0.96% |
ONS |
4.25% |
4.66% |
9.17% |
9.27% |
9.53% |
7.99% |
10.83% |
5.12% |
6.01% |
GRW |
0.57% |
6.39% |
13.24% |
14.76% |
12.99% |
11.55% |
11.69% |
8.23% |
6.07% |
WAAS |
0.59% |
0.76% |
1.63% |
1.12% |
1.22% |
2.14% |
2.11% |
1.76% |
1.01% |
CW-OGD |
0.57% |
2.07% |
1.96% |
6.29% |
1.87% |
3.59% |
4.69% |
2.21% |
1.21% |
Note
“–” indicates that the method incurs (almost) no transaction costs.
The results reveal a trade-off between profitability and practicality. Some methods gain high returns at the cost of heavy portfolio adjustments. Hence, an ideal data-driven OLPS should optimize returns under real-world constraints. The benchmark motivates developing mechanisms to lower rebalancing frequencies and reduce turnover.
In addition to the evaluation of ATO, we also examine the Running Time (RT) metric as a practical measure of OLPS performance.
Table below presents the RT results on the FinOL.
Several key observations can be summarized:
Some OLPS methods have poor scalability with respect to the number of assets, particularly SCRP and SSPO. This is mainly because their past development relied heavily on the
OLPSdatabase, which fortunately does not involve any large-scale datasets.The majority of OLPS methods perform well in terms of running time.
Model |
NYSE(O) |
NYSE(N) |
DJIA |
SP500 |
TSE |
SSE |
HSI |
CMEG |
CRYPTO |
|---|---|---|---|---|---|---|---|---|---|
Classic OLPS: Benchmark baseline |
|||||||||
Market |
– |
– |
– |
– |
– |
– |
– |
– |
– |
Best |
– |
– |
– |
– |
– |
– |
– |
– |
– |
UCRP |
0.0041322 |
0.0059764 |
0.0003916 |
0.001057 |
0.000611 |
0.0002481 |
0.0002688 |
0.000459 |
0.0008224 |
BCRP |
0.0038118 |
0.0067816 |
0.0002166 |
0.001065 |
0.000996 |
0.000306 |
0.0002986 |
0.000275 |
0.0008464 |
Classic OLPS: Follow-the-winner |
|||||||||
UP |
1.8864056 |
0.0719632 |
0.0254685 |
0.01317 |
0.091344 |
0.0412119 |
0.0512968 |
0.037949 |
0.4567986 |
EG |
0.0209176 |
0.7080021 |
0.0007138 |
0.021471 |
0.0023 |
0.0009838 |
0.0014511 |
0.001174 |
0.0048522 |
SCRP |
21.3714095 |
3563.216028 |
2.1666272 |
751.7104 |
5.162115 |
1.3369282 |
4.1774117 |
1.326479 |
10.0444827 |
PPT |
0.0220539 |
1.4248651 |
0.000765 |
0.067611 |
0.002196 |
0.0011361 |
0.0016884 |
0.000959 |
0.0038207 |
SSPO |
10.365433 |
1593.101725 |
4.6482987 |
285.346 |
11.55126 |
5.7671978 |
6.5936851 |
2.656706 |
19.6497393 |
Classic OLPS: Follow-the-loser |
|||||||||
ANTI1 |
2.412872 |
942.9277071 |
0.9028023 |
89.10436 |
4.825674 |
1.5139913 |
3.4485262 |
1.232613 |
6.3589876 |
ANTI2 |
5.3055469 |
1019.015005 |
0.7316441 |
67.97805 |
6.081596 |
1.8371354 |
5.6760943 |
2.697906 |
11.1037912 |
PAMR |
0.0281713 |
0.6058916 |
0.0016882 |
0.016369 |
0.003871 |
0.0021293 |
0.0024729 |
0.002359 |
0.0094379 |
CWMR-Var |
0.0486561 |
5.0129114 |
0.0021516 |
0.384601 |
0.006705 |
0.0036254 |
0.0053173 |
0.002194 |
0.0118123 |
CWMR-Stdev |
0.0554241 |
7.6208348 |
0.0030268 |
0.89136 |
0.009707 |
0.0057918 |
0.0093459 |
0.00356 |
0.0203114 |
OLMAR-S |
0.0388321 |
0.6172465 |
0.0016541 |
0.012853 |
0.003063 |
0.0014739 |
0.0017771 |
0.001449 |
0.0065995 |
OLMAR-E |
0.0398049 |
0.6097762 |
0.0012705 |
0.012054 |
0.002877 |
0.0018873 |
0.0021814 |
0.001494 |
0.0063896 |
RMR |
0.1065905 |
0.4709969 |
0.005274 |
0.060043 |
0.012206 |
0.0084572 |
0.0093933 |
0.009332 |
0.0335466 |
RPRT |
0.0483275 |
2.3619424 |
0.0022602 |
0.089113 |
0.003629 |
0.0017498 |
0.0022784 |
0.001699 |
0.0074966 |
Classic OLPS: Pattern-matching |
|||||||||
AICTR |
0.1552033 |
10.3116284 |
0.0036974 |
0.441138 |
0.009328 |
0.0039644 |
0.0060411 |
0.003575 |
0.0364883 |
KTPT |
1.33071 |
5.1376433 |
0.0563649 |
0.506718 |
0.498936 |
0.6813439 |
0.5995139 |
0.704583 |
1.9678092 |
Classic OLPS: Meta-learning |
|||||||||
SP |
0.0786008 |
1.8743168 |
0.0124197 |
0.336125 |
0.016422 |
0.0049131 |
0.0063926 |
0.016176 |
0.0561672 |
ONS |
1.5128513 |
701.8432832 |
0.1277076 |
13.5781 |
0.250247 |
0.0999844 |
0.1304165 |
0.094656 |
0.5413568 |
GRW |
0.7999147 |
44.0762048 |
0.0620426 |
8.250292 |
0.374519 |
0.0961017 |
0.3086175 |
0.085338 |
0.8238871 |
WAAS |
1.0277107 |
295.7563543 |
0.1262029 |
45.77576 |
0.626888 |
0.425847 |
0.8555393 |
0.197126 |
1.9221044 |
CW-OGD |
0.0373499 |
1.8004252 |
0.0027351 |
0.210646 |
0.007928 |
0.0042008 |
0.0057244 |
0.003265 |
0.0165171 |
Note
All time are reported in seconds.
RT is an important consideration that is often overlooked in backtesting but critical for real-world OLPS task.
To address this challenge, FinOL offers researchers access to two large datasets: NYSE(N) and SP500.
These datasets provide researchers with opportunities to develop OLPS methods that are not only profitable but also practical.