Differences between live and demo

Forex forums are peppered with many comments along the lines that since most EAs lose money when trading live even though they looked good when backtested, then there must be something different about live accounts versus demo accounts. This often moves on to accusations of stop-hunting and other bad or even illegal behaviour by the broker.

The reality is that the behaviour of live and demo accounts is very similar -- it is unusual to see more than a couple of pips of difference between them. Extremely sensitive EAs can be affected by the difference, but the far more likely explanation for the live/demo differences is that the backtest was invalid in some way.

Here are some of the gotchas which will produce an invalid backtest, none of which are caused by differences in demo vs live

• Over-optimisation is the single most common cause. Put simply, over-optimisation means that the backtest is curve-fitting -- cherry-picking good trades and discarding bad ones with insufficient generalisation. Because forex price series are very close to random, the number of variables that can be optimised (the "degrees of freedom", in statistics parlance) must be carefully limited.
• Use of tight stops or take-profits (less than 20 or 30 pips) with the "every tick" option in the strategy tester. The MT4 strategy tester generates ticks from 1 minute data using "fractal interpolation", which is another way of saying that they are completely fictitious. This is why they nominate the quality of the backtest as "90%".
• Great care in the design of an EA is needed when determining trade entries using Bid, Close[0], or indicators with shift = 0
• Use of time-based entries, eg attempting to time entries for the beginning of trading sessions. This is a very difficult type of entry to backtest because of broker timezones and daylight saving changes. The forex price history downloaded by MT4 may not have the same timezone as the broker.

And finally, if I were a scam EA seller who wanted to convince more people to buy my dud EA, here's how I would do it:

1. Design an EA which shows impressive runs of 100% profitability over several weeks, then dies spectacularly with one massive loss. Many EAs will perform like this with a large stoploss of 500 pips or more.
2. Run the EA on a new demo account, wait for it to accumulate a week or two of profit, then market the EA by giving out the demo account number to prospective buyers.  Voila!  A valid forward test, indisputably making money!  This gold-mine is for sale for only $175!
3. Whenever the test EA crashes, goto 2 ...