Regularly emailing the status of an account

Prompted by a query, I thought I'd post a useful little utility that I have used for ages which emails the status of the account every hour.  After lengthy deliberation I decided to call it .... EmailStatus.  With only a small modification it could be used to log the status to a file, and the time interval could be changed also.

Treat it as an indicator, and store it accordingly.  The indicator approach to running scripts is useful because they are reloaded when the terminal is restarted, whereas scripts are not.

Here's a view of what comes in the email

Balance       9999

Equity        9999
Margin         999
Free margin   9999
Open profit     99
Open orders
Open time local Type Lots Symbol  Price   S/L    T/P    Now Profit Comment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
 1 Dec 21:49:04 Sell 0.10 EURUSD 1.3098 1.3401 0.0000 1.3121   -26 xxxxx
Pending orders
Open time local      Type Lots Symbol  Price   S/L    T/P    Now  Comment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Waking up to Neural Network committees

Time to wake up from my blogging slumber and write a few more posts.

A huge amount has happened in my trading life in the well over 12 months since my last post.  I finally completed the early stopping Neural Network library described in my last post of June 2010, wrote an EA or two and played around with it for ages without success.  I also tried the Nearest Neighbour algorithm, (see a subsequent post that I have not yet written but intend to), and became diverted onto many other ideas.

Then … back to the Neural Network idea: how about a committee of Neural Networks?  I had been reluctant to try this purely because of the processing time: as it is, the EA that I have written can take several hours to run a couple of years’ single backtest.

Well, it seems that this may have some promise.  You train a group of NNs with different topologies and parameters, then the simplest method of obtaining a result is to enter a trade when they all agree.  It is a well-established approach in the NN literature.  I’ve tried up to three NNs, although watching grass grow has now become a new fascinating pastime: some of the arrangements I have tried have taken a couple of days to backtest.  Just as with all NN techniques, this is far more art than science: what is a sufficiently different-looking topology to allow the NN to produce a result that is sufficiently independent of its peers?

Attached is a backtest from 1/1/2008 to present for a NN committee of two, trained with a factset of 6/12 EMA cross and both SL and TP set to 100

An early stopping Neural Network strategy

I've written several times in this blog about a Neural Network idea that I've been working on for some time.  I wrote a NN dll for MQL4, but found that the treatment of history in MT4 makes the collection of sufficient data to train the NN very difficult.  However, in the process I did come up with a manual approach to training the NN which showed promise.  With the much faster speed of MQL5, and the very welcome ability to encapsulate code, it is now fully possible to write everything in MQL5, including coding the manual process, meaning that the whole concept can be backtested and optimised.

The broadest descripion of the idea is early stopping, which is not new in the slightest, and in fact most NN literature mentions it.  Early stopping attempts to address the problem with NNs that they are so good at learning that eventually they start to memorise individual data items, which is the familiar problem found in EA optimisation of curve fitting.  Early stopping terminates the learning process at a point where hopefully the NN has only learnt patterns, rather than data.

So this is the hypothesis: a profitable NN EA can be developed ...

1. If a NN can be trained using an early stopping technique to recognise forex patterns
2. If patterns repeat in forex trading, and more importantly, come in clusters

There is no doubt in my mind that item 1 is true, since I have achieved it with manual selection of NNs.  What is somewhat less certain is item 2, although the general technical analysis approach does tend to support it.  Terms such as trending market, or sideways market imply persistence of patterns,  and even a non-profitable EA can often show lengthy periods of profitability.

This last point is another key: rather than discarding all technical analysis in favour of a magical NN black box, why not look first for a basic EA strategy which seems to support item 2 above?  Its equity curve should show extended periods of profitability which could then be selected by a NN filter, in a similar way that the commonly used long term moving average is used to detect a trending market.

The whole idea is already in MQL5 code, with a number of coding errors which I need to find, which will take some time.

Here's the macro view

At last, the VOM article has been published

See the MQL5.com article section.  It was a big effort for me to get it finished, topped off at the end by their somewhat tricky article editing software.  But the editor, Eugene, was very helpful, and it's now up there.  A nice WebMoney payment, and an MQL5.com rating that has jumped to 588.

What now?  It's been hanging over my head for so long.  I get the feeling that it will be rather overwhelming for most readers, although I hope it is taken up by advanced coders and cleaned up in the process.  The fact remains that it is really the only way to manage multiple orders on the one symbol in MT5.  I'm very prepared to spend time responding to questions and fixing bugs.

But what I really want to do now is to finish off my Neural Network EA, and discover whether the approach that I've been thinking about for so long is going to work.  It's coded, but something's wrong with the learning process, and will probably require many hours to find out why. 

Eight months of virtual hosting

A lot has happened since my first blog entry on virtual hosting, which was on 28th August last year.

My original post described what I did to set up a virtual host. In summary

• A simple request to Commercial Network Services that enabled me to have Metatrader 4 up and running in two hours
• Situated in New York to minimise ping times to FXDD.
• $US30 per month

What has happened since then

• I’ve had the most relaxing eight months I can recall in terms of MT4 reliability. The virtual hosting service isolated my MT4 live account from the implications of a 36 hour phone line fault, a 12 hour power outage in my street and two multi-day visits to hospital. Conversely, I experienced one issue with a failed restart command that I requested of the virtual host, which CNS responded to within a couple of hours.
• I started the virtual host with 384MB of memory, but with up to three MT4 clients and two beta MT5 clients running I upgraded to 640MB at the additional cost of $US5.12 per month. The amount of container memory may still seem very low, but it’s the right amount for the purpose now, and can be adjusted immediately any time, in contrast with the purchase of a general purpose PC with multi-GB of memory to run large concurrent applications and to cater for future requirements.
• For some time CNS has been encouraging their customers to move from Virtuosso host software to Microsoft’s Hyper-V, so I did this about a month ago. Before, my container was a Windows 2003 x64 instance, and the new container is Windows XP 32 bit. Everything works fine and this should remove the x64 limitation with running 32 bit dlls, which I have no need to do right now. It is however the one area in which I have less than 100% satisfaction with CNS: their console which shows details of my virtual instance is still pointing to the old Virtuosso host, and their console upgrade to point to the Hyper-V stuff has been “real soon now” ever since I changed over.
• The last change is an interesting one: my FXDD live account is now with FXDD Malta. I believe that this is an offshore strategy to circumvent the new NFA FIFO laws in the US. The funds are still with a secure (?) US bank, but when I get a moment I must check out the ping times from New York, which will be a lot slower if the servers are physically in Malta.

So overall, I thoroughly recommend virtual hosting. At $420 per year it’s a reasonably cost effective alternative to having a dedicated Metatrader PC at home, and it’s certainly a lot more reliable.

I have no association with CNS beyond being one of their customers.

Two VOM shorts on EURUSD

Earlier, I posted an example of two VOM EAs with opposing virtual orders, resulting in no net position at the broker.

Today, there is an example of the two EAs agreeing on price direction. Both the FrAMA Cross EA and the Support Resistance EA are short on EURUSD, and it is instructive to see how this ends up in the VOM and at the server.

FrAMA Cross EA on daily EURUSD chart
Virtual short 0.1 lots at 1.33978. This has a wide protective virtual stoploss 500 (or 5000, in 5 digit terms) pips away.
This was the first order executed, and the server stoploss was set at 550 pips, ie virtual stop plus 50.

Support Resistance EA on hourly EURUSD chart
Virtual short 0.1 lots at 1.34895. This has a virtual stoploss 85 (or 850, in 5 digit terms) pips away.
This is what happened to the server position when this second order was executed

• The server lots moved up to 0.2 short
• The server position open price is now the average of the two orders - this is automatically calculated by the broker.
• The VOM moved the server stoploss down to the tightest virtual stoploss plus 50 (1.36195), which can be seen on the daily chart.

If the virtual stoploss for this EA is hit, then the server stoploss will revert back to the much wider stoploss of the FrAMA Cross EA

All this careful maintenance of the server stoploss at the tightest stop plus 50 (500) pips is there for disaster protection in the event of Internet or PC failure.

Forward tests suspended for a short time

I run all my MetaTrader accounts (MT4 & MT5 demo and MT4 live) on a Commercial Network Services  virtual desktop (see original post here). Although there have been a couple of negative comments on this blog about CNS, it's been completely trouble-free for me.

CNS has been promoting a move to Hyper-V for some time, so I decided to do it. Hyper-V is apparently even more reliable than Virtuozzo, which is what I was running on before. I received an email this morning that the change had occurred, but I didn't have much time before going to work so have only set up my live account at this stage. It only took me about 20 minutes, and it's running fine - I'll write another blog soon on my impressions of Hyper-V and the general experience of remote desktop working. 

Other than being a customer I have no connection whatsoever with CNS.

[ update 18/3 ]
Just waiting for another MT5 build to come out before I can start the forward tests again.  The last two builds broke my VOM code but as usual MetaQuotes have been very responsive in fixing it.