A crude "API" for MetaTrader 4

In response to Ludosm's request to publish this utility, here it is.  Using this API, any other software can drive trade functions in MT4 by generating simple one line CSV files in a similar format to the original MQL4 trade commands.

So where you would write the following in an EA ....
OrderSend(symbol, cmd, volume, price, slippage, stoploss, takeprofit, comment, magic, expiration, arrow_color);

.... the CSV file looks like this
OrderSend, symbol, cmd, volume, price, slippage, stoploss, takeprofit, comment, magic, expiration, arrow_color

A price of Bid or Ask is interpreted correctly.

An important difference is that open orders are located using their Comment, meaning that comments should have a numerical sequence.

So OrderClose(ticket); becomes OrderClose,Comment

There are additional comments at the top of the script.  Hope it's useful.

1500 and counting; what's coming up

Very gratifying to see that I've passed the 1500 mark with hits to this blog - thanks everyone for the interest. So that's a very modest average of 10 hits a day, although it started very slow and has risen to about 50 per day recently. As an aside, it's fascinating to be a microscopic part of the Google machine. The blog has so far had 11 ad click-throughs to generate $18.11 revenue in five months!! Keep on clicking, guys - I'll give up my day job soon :)

In the meantime, I have lots of balls in the air with trading and coding ideas (happy as a pig in sh*t, as they say)

1. First priority - finish off an article which expands on my auto-generated documentation for MQL5 post in this blog. The Metaquotes guys have showed an interest in it and it's likely to be published on the official MQL5 site. The last thing to finish off is a comment processor which trawls through the entire MQL5 folder and subfolders and slightly modifies the comments so that they can be read in by Doxygen and appear in the comiled help documentation.
2. Finish off the MQL5 Virtual Order Manager so people can try it and comment. I've written the VOM itself, but need to polish up the EA I use to test it so it can be debugged more fully. This EA is a series of buttons in a chart window which allow real and virtual orders to be opened and closed. I'll probably release the whole lot then, but also need to follow up quickly with an indicator which reports open virtual orders in a table.
3. The Neural Network EA has been grinding along. I hit a difficult snag with the retrieval of history - other people have also complained that they can't retrieve the full history with Close[xxxxx]. My testing seems to indicate a limit of 1000 bars in the past (or is it 1024?) or only 10 days of 15 minute bars, although some others talk about a gradually increasing value of Bars but they can't work out what triggers it. The EA and associated neural network dll works well functionally, but the neural network can't be trained on a sufficiently long dataset to prove the concept. I can see two ways forward: either write a history reader which reads history directly from the MT4 *.hst files. Or, write the whole damn thing in MQL5, which I have found doesn't have the limitation. The latter will take a lot more time, but I've always had a plan to write a learning neural network in MQL5 in preparation for the trading championships in 2010, so I'm leaning that way. I wish Metaquotes would let us know when the strategy tester will become available.
4. Allow me some (OK, a lot of) misty speculation: if an MQL5 neural network EA proves to be profitable (or any MQL5 EA), then it would be quite easy to write a MetaTrader 5 to MetaTrader 4 messaging utility to enable any MQL5 EA on demo to trade live now using MetaTrader 4. Already sitting in my MT4 code archive somewhere is a crude MT4 API EA which accepts trading commands via *.csv files.

Writing a Virtual Order Manager to enable hedging in MetaTrader 5

Since I wrote about a Virtual Order Manager in August, the MetaTrader 5 public beta commenced, and it has become clear that arguably the biggest change in the transition from MetaTrader 4 to MetaTrader 5 is the management of open trades as positions. At any one time there can be one position only open for each currency pair, and the size of this position adjusts up and down each time orders are processed by the broker. Apart from anything else, this aligns with the FIFO rules recently introduced in the US which effectively outlaw hedging.

To work in this position-centric environment, each Expert Advisor written in MQL5 needs additional programming to record which trades it has open, because it cannot find this out in a simple way from the broker. The most startling example of this would be when two EAs running against the same pair, say a scalper and a trend-follower, issue orders in opposite directions. In MT4, the result would be a long and short order with zero margin used. In MT5, no position would be open at all.

An example of the programming challenge is in the following function. Most people would use something similar in their MT4 EAs to ensure that only one order is open at a time. A similar function would not work as intended in MT5.

int OpenOrders(int magic)
{
   int OrderCount = 0;
   for (int i = OpenOrdersTotal()-1 ; i >= 0 ; i--)
   {
      OrderSelect(i,SELECT_BY_POS, MODE_TRADES);
      if (OrderMagicNumber() == magic
         && (OrderType() == OP_BUY || OrderType() == OP_SELL)
         && OrderSymbol() == Symbol())
      {
         OrderCount++;
      }
   }
   return(OrderCount);
}

So ... here comes a Virtual Order Manager
To work with this new order management environment I am writing a utility which I call a Virtual Order Manager, or VOM, which maintains a local record at the MetaTrader terminal of the orders issued by each EA.

Here are the attributes of the VOM:

Behaviour

• Real orders at the broker reflect total position for that pair, but the VOM also maintains a virtual set of orders at the MT5 terminal. As an example one EA could be virtual long EURUSD 0.2 lots and another EA could be virtual short EURUSD 0.3 lots, with the real position showing in the MT5 open orders list as short 0.1 lots.
• Virtual stoplosses, takeprofits, magic numbers and comments are supported. Virtual stoplosses and takeprofits can be modified
• Real broker stoplosses are maintained at a configurable distance away from the virtual stoplosses as disaster protection against PC or internet connection failures.
• Virtual entry prices, stoplosses and takeprofits can optionally be displayed as horizontal lines on the chart.
• [for eventual development] A virtual_statement.htm can be produced similar to the normal statement.htm which lists open orders and closed order history.

Implementation

• The VOM functionality is added with an #include in each EA.
• I am modelling the API for the VOM on MT4 trading functions. These are familiar, proven functions which will make migration from MT4 EAs easier. So instead of writing OrderSend(….), one writes VirtualOrderManager.OrderSend(…..). I have simplified some of the functions so they are not necessarily a plug-in replacement, but I may also implement the original full versions to make MT4 migration even easier.
• The list of open orders is maintained in a file for all VOM-enabled EAs to access. A table listing all open orders will be viewable using a OpenVirtualOrderViewer.mq5 indicator.

Challenges

Some of the more challenging parts of writing the VOM are listed here

• The VOM must be able to track when a pending order activates. I am using the OnTrade() event in MT5 for this purpose.
• Since multiple EAs read and write to the one file listing virtual open orders, I have to use a global variable as a file locking mechanism.
• The value of the disaster protection stoploss is not easy to determine because it is obtained from a number of virtual orders.

Release

So when will version 1.0 be released? I would hope within two weeks.

Active MQL5 sites

Just thought I'd put in another plug for Metatrader 5, which is the most active Metatrader 5 blog, hosted by miranon. The latest post Indicator Update for MT5 is up to his usual standard, and he has kindly pointed to some of my work on this blog as well.

*** update 20/11/2009 *** http://www.mql5.com/ is now active!

The Russian forum  is bound to be as active as it was in mql4 and is quite readable with Google translate.

That's about it - if you google MQL5 or Metatrader 5 or MT5 you'll find lots of other sites such as Forex-TSD but although there are some interesting discussions, there's not much original MQL5 code.  If anyone reading this has another site they'd like to highlight, feel free to post a comment.

Auto-generated documentation for MQL5 code

I submitted this blog entry to Metaquotes when I wrote it, and I'm pleased to announce that they have now published it on the MQL5.com site.
The article includes a very useful MQL5 source code help file which I intend to update from time to time when new code is released.

Here's the introduction:

Most Java coders will be familiar with the auto-generated documentation that can be created with JavaDocs. The idea is to add comments into the code in a semi-structured way that can then be extracted into an easy to navigate help file.

The C++ world also has a number of documentation auto-generators, with Microsoft's SandCastle and Doxygen being two leaders. I decided to see how well Doxygen could document MQL5, which is in essence a customised subset of C++. To me this is an important step in the maturity of MQL5, because the complexity of the language is easily capable of fostering some quite large class libraries.

The experiment worked very well and I believe the help documentation that Doxygen produces from MQL5 code will add a great deal of value.

NewsTrader

I was looking through my blog posts (yeah, I know, I'm about the only one who does) and I noticed that back a while I had promised a longer description of my NewsTrader.  V2.3 has been forward testing for quite a while now (since 4th September), and has made the princely sum of $82.  This was on a ridiculously small (for a news trader) lot size of 0.1, so I've just bumped it up to a more reasonable 1.0.  With a stoploss of 10 pips this gives a nominal risk of $100 per trade, although I have seen absolutely massive slippage on news in the 30 pip range.  Many traders really get upset by things like that, but with my IT background I've no real problem with it.  Big news events would stress the forex trading network in such a fundamental way that it's not surprising that serious overloads occur.

Anyway, NewsTrader V2_3 reads the news events from the weekly XML list kindly supplied by ForexFactory, and issues a very tight straddle just before each medium or high impact event.  What's a straddle?  It's a BuyStop / SellStop pair that is able to profit from a clean move in either direction.  If you look at the forward test, you'll see a number of indications of how this trader works:-

• Large numbers of deleted stop orders.  This is when the price didn't immediately react to the news at all.
• Stopped out orders at -10 pips.  This is when a spike in one direction preceeds the real direction of the move.
• Closed orders in loss.  I have a timestop which from memory closes the orders after about 2 minutes (yes these trades are short!)
• And, wonder of wonders, a number of profitable orders which hit the TP of 10 pips

NewsTrader V2_4
When I first wrote NewsTrader I was trading from my laptop in Australia.  I've made other comments about the ping times between Australia and the US: suffice to say the theory of News Trading is greatly undermined by network latency across the Pacific.  With the use of my virtual host in New York, I've now reinstated my original code which worked so badly in practice, which was a very tight client-side trailing stoploss.  I'm running 4 pips at the moment.  You wouldn't believe how long it takes for a MetaTrader 4 terminal in Australia to deliver tick updates and do something trivial like close an order.

So that's it: NewsTrader V2_4 is NewsTrader V2_3 with a trailing stoploss of 4 pips and a takeprofit of 30 pips (which will not normally be hit).  It remains to be seen how much better it will work on the faster network.  Forward tested here.

When I get a moment I might add auto screendumps of the news events as they occur.

My first MQL5 Expert Advisor

Update 12/1/2010: this forward test now replaced with a Virtual Order Manager equivalent coexisting with the Support_Resistance EA.  See statement here and live screenshot here.  All links below are still valid.

This EA is a Moving Average Cross system using the Fractal Adaptive Moving Average.  It is almost certainly not a profitable EA - I've just used a simple system like this as an exercise in learning MT5.  Give it to me to code in MT4 and I'd have it done in literally 15 minutes, but this has taken me on an off about a week to code.

I decided to write an EA that is triggered using an indicator to really understand how things work.  The FraMa Cross indicator is here, and includes arrows showing long and short entries.  The default MA periods are 6 and 12.

There are various ways to code an always-in-the-market reversing EA such as this in MT5.  I simply reverse the position by sending an order for double the lots in the opposite direction, which illustrates one of the  largest differences between MT4 and MT5.

Files needed for this EA to compile and run:
FraMa cross EA V1_2.mq5 - the Expert Advisor itself. Store in MQL5/Experts
FraMa cross.mq5 - the indicator.  Store in MQL5/Indicators
EnumToStr.mqh - contains lookup tables for a couple of enums.  Store in MQL5/Include
SimpleLog.mqh - logging utitlity that I used to debug.  Store in MQL5/Include

Here is the statement of a live forward test, which is running on a EURUSD H1 chart with FastMA 6 and SlowMA 12.

[update] and here's a screenshot updated every 10 minutes.  You may need to use refresh on your browser to see updates (the picture to the left is an example only and doesn't update)

[update 2] I have uploaded a new version of the EA. It suddenly stopped working properly because I had made an error by not initialising a double in OpenLots().  A recent MT5 update must have stopped automatically initialising doubles.

[update 3]  The EA hasn't been working for a while - must have been a new build which broke it for some reason.  So I've recompiled after re-writing some of the code, such as OpenLots() (which doesn't need to loop through all positions as I'd orinally written it), and introducing a more versatile function called AdjustPosition().  The code is updated as Frama Cross EA V1_2.mq5