An early stopping Neural Network strategy

I've written several times in this blog about a Neural Network idea that I've been working on for some time.  I wrote a NN dll for MQL4, but found that the treatment of history in MT4 makes the collection of sufficient data to train the NN very difficult.  However, in the process I did come up with a manual approach to training the NN which showed promise.  With the much faster speed of MQL5, and the very welcome ability to encapsulate code, it is now fully possible to write everything in MQL5, including coding the manual process, meaning that the whole concept can be backtested and optimised.

The broadest descripion of the idea is early stopping, which is not new in the slightest, and in fact most NN literature mentions it.  Early stopping attempts to address the problem with NNs that they are so good at learning that eventually they start to memorise individual data items, which is the familiar problem found in EA optimisation of curve fitting.  Early stopping terminates the learning process at a point where hopefully the NN has only learnt patterns, rather than data.

So this is the hypothesis: a profitable NN EA can be developed ...

1. If a NN can be trained using an early stopping technique to recognise forex patterns
2. If patterns repeat in forex trading, and more importantly, come in clusters

There is no doubt in my mind that item 1 is true, since I have achieved it with manual selection of NNs.  What is somewhat less certain is item 2, although the general technical analysis approach does tend to support it.  Terms such as trending market, or sideways market imply persistence of patterns,  and even a non-profitable EA can often show lengthy periods of profitability.

This last point is another key: rather than discarding all technical analysis in favour of a magical NN black box, why not look first for a basic EA strategy which seems to support item 2 above?  Its equity curve should show extended periods of profitability which could then be selected by a NN filter, in a similar way that the commonly used long term moving average is used to detect a trending market.

The whole idea is already in MQL5 code, with a number of coding errors which I need to find, which will take some time.

Here's the macro view

At last, the VOM article has been published

See the MQL5.com article section.  It was a big effort for me to get it finished, topped off at the end by their somewhat tricky article editing software.  But the editor, Eugene, was very helpful, and it's now up there.  A nice WebMoney payment, and an MQL5.com rating that has jumped to 588.

What now?  It's been hanging over my head for so long.  I get the feeling that it will be rather overwhelming for most readers, although I hope it is taken up by advanced coders and cleaned up in the process.  The fact remains that it is really the only way to manage multiple orders on the one symbol in MT5.  I'm very prepared to spend time responding to questions and fixing bugs.

But what I really want to do now is to finish off my Neural Network EA, and discover whether the approach that I've been thinking about for so long is going to work.  It's coded, but something's wrong with the learning process, and will probably require many hours to find out why.