I am interested in other peoples ideas on avoiding curve fitting. I will post my own answer below as well.
TM
Curve-fitting (aka: Over-Optimization) is by definition a process of constructing a curve to fit a set of data points. Over-Optimization can be accomplished very easily by making the system fit your available data.
When developing a trading strategy, backtesting is the only means to test our ideas. After all we only have the past to test future. While positive hypothetical results by no means guarantee future success, undoubtedly poor backtesting results guarantee failure. First of all, keep in mind that all indicators are lagging. So predicting the future with lagging indicators is not always the best way to develop a strategy.
One of the best ways I know of for backtesting (once you’ve developed a strategy) is to use the “Out-of-Sample Data” feature available on many platforms. Another must is to incorporate realistic slippage and commission rates. This will differ from one market to another, however as a minimum, one needs to deduct at least one tick from each side (entry and exit) for high-volume markets and more for markets with low volume. As an example, for the e-mini S&P 500, I like to use a slippage figure of $16.00 per side. Why so much you ask? Simply to account for those occasions when the market is moving fast, rollover costs and the fact that continuous contracts in futures (although essential and efficient IMHO) add to the cost of trading.
If your platform does not have the “out-of-Sample Data” feature, then you can simply run a test using the same parameters for various periods, sometimes overlapping periods. Finally, when you code a strategy, I think you should forget about it for three to six months. Then, study the results for that “Out-of-Sample” period and compare the results to my backtesting period. If the results are consistent for both periods, you may consider trading live. If not, I’ll go back to the drawing board to analyze the reasons why the methodology did not perform as expected.
Some traders feel they can “Forward-test” their strategy by going live immediately after developing a strategy. I believe that this approach is too risky. Patience not only builds confidence but also costs a lot less. The opportunity cost of not trading for a few months does not justify the risk associated with trading.
Matt