return selection criterion that the RC and the SPA-test for 19 out of 51 indices lead to different conclusions. The SPA-test finds for more than half of the indices that the best strategy does beat the buy-and-hold significantly after correction for data snooping and the inclusion of bad strategies. Thus the biased RC misguides the researcher in several cases by not rejecting the null. However, if as little as 0.25% costs per trade are implemented, then both tests lead for almost all indices to the same conclusion: the best technical trading strategy selected by the mean return criterion is not capable of beating the buy-and-hold benchmark after correcting for the specification search that is used to find the best strategy. In contrast, for the Sharpe ratio selection criterion we find totally different results. The SPA-test rejects the null hypothesis for 35 indices in the case of zero transaction costs, while the RC rejects the null hypothesis for 24 indices. If costs are increased further to even 1% per trade, then for approximately a quarter of the indices analyzed, the SPA-tests rejects the null of no superior predictive ability at the 10% significance level, while the RC rejects the null for only one index. We find for the Sharpe ratio selection criterion large differences between the two testing procedures. Thus the inclusion of poor performing strategies, for which is corrected in the SPA-test, can indeed influence the inferences about the predictive ability of technical trading rules.
Next we apply a recursive optimizing and testing method to test whether the best strategy found in a specification search during a training period also shows forecasting power during a testing period thereafter. For example, every month the best strategy from the last 6 months is selected to generate trading signals during that month. In total we examine 28 different training and testing period combinations. In the case of zero transaction costs, the best recursive optimizing and testing procedure yields on average an excess mean return over the buy-and-hold of 37.72% yearly, if the best strategy in the training period is selected by the mean return criterion. Thus the best strategy found in the past continues to generate good results in the future. If transaction costs increase, then the excess mean returns on average decline. In the presence of 1% transaction costs the excess mean return over the buy-and-hold benchmark is on average 4.47% yearly. For both selection criteria, mainly profitable results are found for the Asian, Latin American, Middle East and Russian stock market indices.