Article Library

3/52
Ranking
Do not worry dear reader, I do not want to scare you by some horrible stories about killing monsters that must be stopped, I'd like to bring your attention to machine learning and also would like to apply and demonstrate some basic neural network sample showing the way how to apply it over spot forex data aiming to predicate future (one day) return.
In recent time machine learning has started to be using as buzzword in many areas of human activities and currency trading does not stand aside. Machine learning has started to be introduced as tool of the future. From trading perspective one can have impression that with regards to machine learning new age is opening before us, because everything seems to be easily driven by some machine learning magic, whatever it means, and so trading also. These ideas are easy to claim though hard to proof them and find an evidence how it could be applied to trading correctly and also if it is possible at all. Another problem is that many people and traders tend to see machine learning like some black box where time series is passed at one side and after some processing by magic machine learning algorithm the super accurate prediction drops on othe…
Read article
Translate to English Show original
Natalia_Kisenko avatar

useful article!

iiivb avatar
iiivb 17 June

though barely understood the programming thing... was super interesting!

mcquak avatar
mcquak 18 June

iiivb thanks alot, appreciate your comment

mcquak avatar
mcquak 18 June

Natalia_Kisenko thank you Natali for stopping by and read the article

mcquak avatar
mcquak 30 June

I'd like to thank all above comments creating great creative atmosphere below my article! And thanks to Dukascopy making this discussion to be possible!

orto leave comments
12/76
Ranking
In this article are described some concepts that can be used to optimize automated strategies.
The topics covered are:
  1. Minimizing cost
  2. Finding an optimal entry/exit within a short time period
  3. Using Big Data to optimize strategies
  4. Statistically evaluating a strategy

Minimizing costs

Costs arise from the spread and commission. Statistically, these costs are the only thing that will cause you to lose money from trades.
The spread is by smallest on the EUR/USD pair but even on this pair, it changes enormously throughout the day. A strategy should therefore follow the spread closely and only enter or exit positions when it’s at its smallest level.
The graph above shows how the average spread and its standard deviation evolves throughout the day. It is important to keep in mind that the spread may change between the moment when you place an order and the moment at which the position is really taken.
Finding an optimal entry/exit within a short time period
Trends are a central point to any strategy. If there were no trends, there would be no means to predict price movements.
It is hard to claim whether trends occur on larger time scales or not but is definitely the case for very s…
Read article
Translate to English Show original
tdbatinkov avatar

Every strategy is winning only... when its settings are in harmony, in resonance with present market conditions...but the market is always changing

Natalia_Kisenko avatar

good advice

zarina avatar
zarina 19 Feb.

Article about the optimization strategies are the most useful!

Olkiss70 avatar
Olkiss70 21 Feb.

very nice job!

Nihad avatar
Nihad 27 Feb.

Good luck buddy

orto leave comments
30/76
Ranking
It won't come as a surprise to many of the readers that modern machine learning techniques can yield significant benefit in crafting automated trading strategies. In fact, any FX trader with solid math background will agree that an automated trading strategy is itself a 'model', whether machine or human learning was used to construct it. This is why it makes sense to borrow and introduce some concepts from this popular branch of modern computer science, and discuss how they might apply to Forex trading.
For our purposes, we can define a model as follows: a mathematical function or formula that takes in a set amount of data (which can be your indicators for example) and produces an output. A trading strategy is one such model. Contrary to machine learning however, we are human learners that use trial and error to come up with a good model for trading currencies. It makes sense that we can learn a few tricks from how machines learn their models, and apply those learnings in Forex for greater return.
Learning a model, for machines, works by running a set of pattern recognition algorithms on a set of data. The data can be anything, ranging from images, music and even financial data. Th…
Read article
Translate to English Show original
orto leave comments
23/40
Ranking
On-line parameter optimization for automated Strategies
Most strategies have parameters to set and depending on the situation, a certain parameter value does best. The result is that most strategies work for a while and then aren't valid anymore. But the entire strategy might not be obsolete, the parameters just need fine-tunning.
Here I’ll show you a way to optimize the parameters while the strategy is running and I compare it to a strategy where the parameters are set once and for all at the beginning of the test period with classical optimization over a period in the past.
When is on-line parameter optimization applicable

Just for the sake of having a real-life example, the method is applied to an automated strategy which I talked about in my previous article this month. However, the method is applicable to any strategy with optimizable parameters. The strategy in question gives a probability that the next movement will be positive or negative:
If you have a strategy that works in the way of the drawing above, you’ll soon notice that having a position open at all times isn’t
the best idea. A better plan might be to only open a position when the probability of a rise is either v…
Read article
Translate to English Show original
Airmike avatar
Airmike 22 Mar.

You can minimize cost of trading by adding an another parameter to machine learning. as limited expected movement. does not make sense to let your algorithms learn from noise. you can increase your profit factor and decrease number of trades. that is a thing I was talking about last time. you used a wrong model for calculation of your costs. cost ratio always depend on volatility (range) and liquidity(spreads)

olga avatar
olga 25 Mar.

Hey, thanks for the comment but I don't realy understand what you mean by the "limited expected movement". I use a minimum movement threshold... however, forex is a noisy time-series, there's no perfect way around it but an LSTM network can deal with noise. As for the calculation of costs, I plain dissagree, high volatility is great but, a priori, you don't know which way the next movement will go so that doesn't impact "cost" it impacts possible profits OR losses. the only things that impact cost is spread and commission, if you didn't have these you could trade "for free" whatever the range.

olga avatar
olga 25 Mar.

Perhaps we only disagree on the interpretation of trading costs.

orto leave comments
10/46
Ranking
Modern techniques like artificial neural networks (ANN) are best used for high frequency trading for several reasons. First, they mimic human intelligence but they mostly don’t reach a human’s level of intelligence, therefore, there is no point in using those techniques on a time scale at which a human could easily be working. Their advantage comes from speed of operation and constant activity. Second, we need a lot of data to train neural networks efficiently and this amount of data will only be found in high frequency trading. Forex has all in all quite few instruments with limited relevant past data on the daily or weekly time-scale. Furthermore, High frequency trading is a type of scalping strategy where we identify noise around the true value of the instrument. This is different from long-term trading that attempts to follow meaningful movements of the instrument according to fundamental analysis.Artificial Neural networks A good time-scale to work on is the minute time-scale. This time-scale is full of noise which will be captured by the algorithm in order to sell at a local high and buy at a local low. This can be proven using a simple neural network trained to predict the f…
Read article
Translate to English Show original
Efegen avatar
Efegen 16 Apr.

Just a thought. If this system works why don't you use it in larger timeframes. So the costs will be more minimal and deviation will be less?

Victor avatar
Victor 18 Apr.

good one.. nicely written

SpecialFX avatar
SpecialFX 30 Apr.

Another very intriguing and interesting article! :) Regarding your comment about Holy Grail, no need to look for it, because it doesn't exist... well, it does, but that will be for another time... ;)

nippur72 avatar
nippur72 21 Nov.

The reason why you are able to predict "high to high" and not "close to close" IMHO is very simple: when you predict "high to high" you are not forecasting the future, but you are forecasting (partially) the past, since the High(-10 to 0) is a point back in time. It's a common pitfall that I've also experienced myself when experimenting with neural networks. As a rule of thumb, when you see hit rates easily going above 65% there is chance you are doing something wrong, e.g. forecasting the preset or the past.

olga avatar
olga 24 Nov.

You are totally right and strangely enough I had realized this before and I made the mistake again. I also found similar pitfalls when training and testing instances aren't totally separated including the period you gather information on and the period you try to predict when you use cross-validation. It’s very easy to get excited about good results and in this article, it totally happened to me. In conclusion, kudos for pointing out the flaw that destroys any interest lying in this article and I believe more and more that high-frequency trading has no solution.

orto leave comments
8/40
Ranking
Building a trading strategy is part of a very special category of problems. First, it is time dependent meaning that the data you analyze is all the same (prices) but shifted in time. Therefore the problem consists in recognizing patterns within charts. Secondly, you don’t want to predict anything about the chart itself, you usually want to predict something about the future, as in weather forecasting or sports bets. In that regards, trading is a lot like gambling.But what exactly do we want to predict? Do we want to predict the high of the following period or perhaps the overall movement of said period? Here are two things that are more logical and easier to predict.The parameters discussed are used to grade single trades, not to mistake with techniques that asses the quality of an entire strategy. These are parameters on which to optimize your strategy.Profit over time The idea behind this parameter is that a trade should make the most profit in as little time as possible, in fact, money that is invested in a trade cannot be invested elsewhere and long-lasting trades will occupy your money and your attention while you could have perhaps made better profit elsewhere. With this ide…
Read article
Translate to English Show original
Efegen avatar
Efegen 28 Mar.

Nice work you are soing but it is sometimes too complex. This on i understood+1

nippur72 avatar
nippur72 16 Nov.

how do you calculate "p", is it just the sum of theorical profit at day 0 + profit at day 1 + .. + 9 ?

e.g. if r[t] are the returns at day t,

for(t=0;t<9;t++) p+= r[t] > 0 ? r[t] : 0;

olga avatar
olga 18 Nov.

The sum of profits depends on the time-scale, if you look at daily bars, one bullish day will add to total profits and a bearish day will add to total drawdowns. You'll get a more accurate value by looking at smaller time-scales. The values for profits and drawdowns will be very different Wether you look at a small or a large time-scale but the proportion of one to the other should stay about the same.

olga avatar
olga 18 Nov.

Actualy what i just said isn't true, a smaller time-scale will push the proportion of profits to drawdowns towards equality but the sign of the ''profits over drawdowns'' indicator will stay the same.

nippur72 avatar
nippur72 21 Nov.

ok, thanks for the clarification. I like your articles on neural networks, I hope you post more articles.

orto leave comments
21/40
Ranking
Introduction Artificial neural networks are a special family of artificial intelligence models and in this article I will explain how they work to predict the forex in a way that anyone can understand but also in a way that people familiar with artificial neural networks may see quite precisely what is being done. When switching from a tick chart to an hour candlestick chart, a huge amount of information has been lost. We would like our automated strategies not to be limited to looking at one or two timeframes at once. We want it to be aware of every movement on every time-scale, identify the important ones, and act accordingly. Today I’ll explain how to identify a single chart feature with the help of a special artificial neural network (ANN). Compressing data/ extracting features This technique requires optimization and I’ll never state this enough; you cannot test your strategy on the same period it’s optimized, that’s just cheating and the strategy will most probably not work on other data!! All the training/optimization/validation here is done using data from January 2009 to June 2011 while the rest of the data up to the present is used for testing and every result reported …
Read article
Translate to English Show original
olga avatar
olga 7 Mar.

Allright yes, I'll explain better how ANNs work in the next article. they are a wonderful family of algorithms that can model any system with sufficient examples don't complain if it bores you to tears though

LSD avatar
LSD 9 Mar.

Good stuff. +1

citikot avatar
citikot 12 Mar.

I sniff the smell of HFT here :)

olga avatar
olga 12 Mar.

You're right, but I'm not sure I can be profitable with high frequency trading because of commissions and spread, I want to achieve low frequency trading but also take into account low time-frame data to optimize entry and exit points. in part two I explain how I intend to do that but I haven't implemented the entire method yet.

drishti avatar
drishti 12 Mar.

@ olga : You are writing in right direction. Using ANN for trading is a very good approach, however there might be only few who can understand the whole part, how it works and how to build it.

orto leave comments
27/51
Ranking
I got the idea for this article in another one from this month. In it, a thorough analysis of EURUSD is given and as interesting as it is, only one question springs to mind; long or short? From a specific chart made up of thousands or even millions of ticks and from fundamentals if you look at them, you want to pull out only this idea, should I buy or should I sell. Similarly, when we look at a picture or listen to a song, our brain processes an enormous amount of data and concatenates it into a single idea that summarizes the entire thing and it does so very efficiently. So let’s look at how the brain functions and see if we can’t build an automated strategy that works a little bit like it. The idea is to simplify charts in a pyramidal manner. Our brains do this for everything. For example, this article is made up of letters that make up words and that in turn form sentences, paragraphs and finally the entire article which is a single idea in our brain. you can do this for everything because that's how our brain functions. So what are weekly charts made up of? They are made up of ticks that together make minutes, hours and days, until the full chart for the past week. That’s logi…
Read article
Translate to English Show original
doctortyby avatar
doctortyby 19 Feb.

I have already tried to replace time frames and range bars charts even with ticks, but there is a maximum of 100 ticks that cannot be correlated with a TF more than 5 minutes. The simple ticks chart is much more reliable for entries and exits. As for the long or short decision, I consider that in any time given the are a few scenarios that have a higher or lower probability of happening. I don't think that trading is as simple as just going Long or Short. That would be just gambling as a retail trader lacks the most important part of trading... Information

olga avatar
olga 19 Feb.

Thanks for the comment. I want to emphasize that allthough your article and the reply by Likerty inspired me to implement a certain tactic, In no way do I mean to critisize your article.

I'd love for you to extend on the beginning of your comment, too little space here...

Now what I actualy say is: given the information, in order to make profit, we need to predict the evolution of the market. then, profit can only go through going long, short or not at all, the only three actions available.

I apologize for the missing picture that summarizes the tactic, I hope it will come on at some point.

kelvindfxguru avatar

nice one, although the tactic picture is not shown.

orto leave comments
26/51
Ranking
IntroductionModern pattern recognition methods have been applied with a lot of success to image recognition. In many ways, the problem of recognizing patterns in a chart is similar to recognizing patterns in a picture and the techniques are adapted with great success to the forex.Defining patterns A very complicated problem that is now mostly solved is how to automatically interpret images. The image is in 2D but the problem is similar for several reasons. The pixels on the image have a definite position and can’t be interchanged; they are time dependent as are forex instruments (actually they are 2d space dependent while forex is 1d time dependent). Secondly, a single image most often has a huge amount of pixels and we need to reduce the amount of data before we can analyze it efficiently, likewise, a pattern on EURUSD is usually made up of thousands of ticks which need to be summarized. In trading, we classically reduce the amount of data with OHLC candlesticks but this is not ideal. The open and close are problematic because they represent a price at a very specific time, the high and low are more lenient and represent a price within a certain time-range. If two patterns are ex…
Read article
Translate to English Show original
orto leave comments
24/51
Ranking
Introduction When designing a strategy, a beginner will pick a few parameters at random, either directly from the price or from indicators, and will attempt to predict future movements based on those parameters. When he’s convinced those parameters don’t hold any information, he’ll choose some slightly different parameters and so forth. This is a sensible way to explore forex instruments however, there are so many parameters to look at and try out that you will most likely die of old age or at least go bankrupt before you find a winning strategy. That’s why I automated the process in an algorithm that attempts to find the optimal parameters for predicting EURUSD movements.A randomizable set of parameters We have to define limits for a space of parameters we want to explore. To do this I first build a list of the lowest price daily, in two days, in three days… until 15 days. The high price could also have been used.Once we have this list, a parameter is defined by three numbers we’re going to call i, j and k. The value for a single parameter for day n is given by taking, in column i, the difference in percent between price n – j and n – k. Thus, our parameters are movements from hi…
Read article
Translate to English Show original
OneGoodTrade avatar

It looks like some big hedge fund's algorithmic trading department is waiting for you :)

Likerty avatar
Likerty 8 Feb.

Pre-programmed trading was always a mystery for me:) Good luck in the contest!

doctortyby avatar
doctortyby 15 Feb.

I agree with One Good Trade... a prop shop is the environement that you need to devellop ;)

orto leave comments
26/43
Ranking
IntroductionMachine learning is a field of artificial intelligence where computer programs learn instead of blindly following a script. With enough training data you can teach those algorithms to drive a car, pilot a helicopter or build the best search engine in the world. Here are the results I obtained with my initial approach at applying machine learning to forex trading.Thechnical ConsiderationsA variety of algorithms are put in place to try and predict the evolution of an instrument with data from only 8 daily bars into the past. For each day, four values are recorded, the first three record information on the movement from the previous day’s close to the day’s high, low and close, in percent while the fourth records the volume for the day. This makes for 32 independent variables total. The data is obtained from three instruments in the dukascopy database, EURUSD, AUDJPY and GBPCHF daily Ask bars from the 1st January 2008 to the 31th December 2011, with weekends blended in the following Monday. For each of the algorithms tested, the first two years were used to train the models while 2012 was used to test them. The open java library for machine learning algorithms used comes f…
Read article
Translate to English Show original
PipSpinner avatar
PipSpinner 13 Dec.

I don't have the reputation to like this yet, but thanks for the great article! Have you tried applying any ML to depth of market and volume data?

olga avatar
olga 16 Dec.

I have applied it to volume data and throughout the day for example it is quite possible to predict the volume based on the time of day. for example it's easy to predict there will be a bottom at midnight and that the overall volume will resemble the previous day but meaningfull differences are hard to predict. I suppose it would be similar for market depth but I have never looked into such data.

Aircooled avatar
Aircooled 22 Jan.

Good article. Unfortunately audience here is not "ready" for such stuff.

campione avatar
campione 13 Sep.

Nice article :)  I assume you used period of 2008~2011 static data as input, I need to do something like this with live data, to analysis the live data and give me live decisions.  Could you implement something like this? I'm happy to help you to do this together if you are up to? ;) best of luck for you in trading

orto leave comments
22/43
Ranking
Introduction I was always very interested in statistics and especially the field of machine learning. These are the “intelligent” algorithms that can learn to do a job rather than blindly following a script. I applied those to the forex with little success. Although I could never really predict the direction of movements based on passed price values, it was always simple to predict the absolute size of the following movement, to some extent, based on previous movements and the time of day. I’ve become convinced that the best strategy should not try to bet on the direction of the market but on its volatility. Take profit and stop losses In my previous article, I posted charts of the distribution of price changes for different time-scales and the results show that the distributions change greatly in shape for different time periods. Partially due to this observation, I want to introduce a different kind of take-profit and stop loss, the Per Period Take-Profit (PPTP) and the Per Period Stop-Loss (PPSL). These limits work like the regular stop-losses and take-profits except we only check if the limits have been passed at the close of the period. They thus have two parameters, the dista…
Read article
Translate to English Show original
PipSpinner avatar
PipSpinner 10 Dec.

Great article, thank you. Always looking to gain more insight into applying ML to my strategies.

Deta avatar
Deta 12 Dec.

Thank you!

Aircooled avatar
Aircooled 22 Jan.

Excellent article. Finally one article that stands out. I am just sorry that I didn't see this on time to vote - it's already January.

ForexSpeaker avatar

great idea olga
keep going

orto leave comments
11/28
Ranking
If you’ve already tried designing your strategy you’ll have noticed a small check-box in the historical tester of the jforex platform named optimization. You can find a little bit about this option on the dukascopy official forums but what is optimization exactly and how does it work? If I optimize my strategy will it necessarily work better in the future? These are some questions I want to tackle in this short article because optimization can improve a strategy or render it worthless while falsely making it look good on back-tests. Over-Learning Over-learning is the enemy of optimization and happens very easily when optimizing a strategy. An over-learned strategy will have a magnificent back-test but will be worthless in the future. Optimizing a strategy means tweaking its parameters to obtain better results. Especially when a strategy has many such parameters, it is very easy to optimize it on any period in time, even very long. Why is that? It’s because the strategy learns the time period by heart. It can tell when an investment is going to be juicy because it has seen it before! This is called over-learning because your strategy adapts to the time-period it’s optimized on too w…
Read article
Translate to English Show original
SpecialFX avatar
SpecialFX 16 Nov.

Interesting article on optimization, and it is good that you use at least 200 trades for statistical purposes. Many traders make assumptions and decisions based on very little data, sometimes less than 20 trades.

fxigor avatar
fxigor 20 Nov.

Hi, where I can read more about two ratios: the ratio of profits over drawdowns and the ratio of mean profit per trade over the standard deviation?

doctortyby avatar
doctortyby 25 Nov.

Good article, I only suggest a 300 trades backtest and statistical data for a better accurancy. of the results

olga avatar
olga 1 Dec.

Sorry for not answering earlier fxigor, I lacked the points to post or send you a message.
This link is an article on the sharpe ratio but you may also read about it on Wikipedia:

http://www.dukascopy.com/fxcomm/fx-article-contest/?Sharpe-Ning-Up-Your-Strategies&action=read&id=29

And don’t forget to read SpecialFX’s article for a bunch more optimization functionshere:

http://www.dukascopy.com/fxcomm/fx-article-contest/?Applying-Simple-Mathematics-To-Evaluate&action=read&id=1082

orto leave comments