apply White's (2000) Reality Check (RC) and Hansen's (2001) Superior Predictive Ability (SPA) test. Because Hansen (2001) showed that White's RC is biased in the direction of one, p-values are computed for both tests to investigate whether these tests lead in some cases to different inferences.
In the case of 0 and 0.25% transaction costs table 5.8 shows for trading case 3 the nominal, RC and SPA-test p-values, if the best strategy is selected by the mean return criterion5. Table 5.10 summarizes the results for all transaction cost cases by showing the number of indices for which the corresponding p-value is smaller than 0.10. That is, the number of data series for which the null hypothesis is rejected at the 10% significance level.
The nominal p-value, also called data mined p-value, tests the null hypothesis that the best strategy is not superior to the buy-and-hold benchmark, but does not correct for data snooping. From the tables it can be seen that this null hypothesis is rejected for all indices in all cost cases at the 10% significance level. However, if we correct for data snooping, then in the case of zero transaction costs we find for only 8 of the stock market indices that the null hypothesis that the best strategy is not superior to the benchmark after correcting for data snooping is rejected by the RC, while for 27 indices the null hypothesis that none of the alternative strategies is superior to the buy-and-hold benchmark after correcting for data snooping is rejected by the SPA-test. The two data snooping tests thus give contradictory results for 19 indices. Thus the RC misguides the researcher in several cases by not rejecting the null. The number of contradictory results decreases to 9 if 0.10% costs per trade are implemented and to 4, 2, 1 and 1 if 0.25, 0.50, 0.75 and 1% costs per trade are implemented. In the 0.10% costs per trade case, the SPA-test rejects for 15 indices its null hypothesis, but this number declines to 2 in the
Trading case 3 costs pn pW pH 0% 51 8 27 0.10% 51 6 15 0.25% 51 2 6 0.50% 51 0 2 0.75% 51 0 1 1% 51 0 1
Table 5.10: Summary: Testing for predictive ability, mean return criterion. For each transaction cost case, the table shows the number of indices for which the nominal (pn), White's (2000) Reality Check (pW) or Hansen's (2001) Superior Predictive Ability test (pH) p-value is smaller than 0.10. Note that the number of indices analyzed is equal to 51.
198