Hi Support,
I have an idea for a new Contest.
The contest should be based on backtesting of strategy on previous 2 years, the winning strategy is the strategy
that obtains the best profit on backtesting.

Time of Backtesting: 24 months before the month of contest
Backtesting: Tick All
Update of strategy: partecipant can update his strategy once per week
Minimum trades closed: could be 200 or 400

This idea could improve developing reliable strategies on medium or long period.


Best regards,
Frank

P.S.
Happy Easter

It's a good plan because the current strategies used in the strategy contest aren't really any good... they all pretty much wipe out over a period longer than 1 month.

I do not agree - to make a good strategy just in back test, even for two years is an easy task. I can make hundred's of such strategies. It's the way commercial strategies work - good at back testing, worth nothing at forward testing. To make good back test, one has just to play with optimization.

If a strategy is good at back testing is probably (but is not sure) that it will be good even at forward test.
But If a strategy is not good at back testing is probably (but is not sure) that it will not be good even at forward test.

95% of all strategies used in Strategy Contest will not pass a backtesting of 2 years. If a strategy has lose over last 2 year, will it win for next year ? It's possible but not probably.
These strategies are "one shoot" strategy that means that they win one month and lose all capital the month after.

Best regards,
Frank

Personally, I'd up the prize and make the contest run 2 years back and 1 year forward. There's no real skill in making a strategy that works well for 30 days but then empties your account out a week after that.

A good strategy will use good money management (probably no more than 1-2% risk per trade) but to win the monthly contests you have to purposefully use bad money management (risking 20%+ on a single trade).

I agree with Adask, it does not make any sense to do a contest on backtest history, you just have to play with optimization and that's it.

You have to think that quantitative trading is a game of probability and that market changes will change the probability so ideally you should never use the same strategy for a long period of time unless you are doing artificial intelligence.

The whole idea of that strategy is profitable for the past 2 year it should make money next year is a wrong concept, it all depends on future market conditions not past.

One of my strategy i have posted was profitable for more than one year and did nothing during the contest, one of the biggest difficulties with trading is that high profitable patterns just dont come back that enough to make a good sample!

@pipscity

That is drammatically true from a scientific point of view, but all technical analist make the opposite assumption simply
based on experience and they all are AWARE of this.


Consider the following extract

"14. Although the future is unlikely to exactly mirror the past, assuming that future
market behavior will resemble the past is the best available assumption
on which to base our current decisions. Thus, we can select specific Technical
Market Indicator parameters that would have worked best in the past." (pag.4)

"Back-testing Technical Market Indicators has proved to be useful in actual practice
because the market’s behavior patterns do not change dramatically over time. As
Fed Chairman Alan Greenspan said, “Human psychology molds the value system that
drives a competitive market economy. And that process is inextricably linked to human
nature, which appears essentially immutable and, thus, anchors the future to the
past.” And as philosopher George Santayana wrote, in Life of Reason (1906), “Those
who cannot remember the past are condemned to repeat it.” (pag.7)


from "The Encyclopedia of Technical Market Indicators" by Robert W. Colby


Without this aware assumption all technical analysis should be a nonsense, because for example there isn't a physical law
(like gravitational law) between a cross up of emas (or a cross up of stochastics or a positive momentum ect.)
and an uptrend of price, in fact the gravitational law is true always but the cross up of emas has an up trend of price
only in a percentage of all cases, and the experience says this to us.



That said, let's consider

"How long should you back-test a trading strategy?
The more trades you use in your back-testing, the higher the probability
that your trading strategy will succeed in the future. Look at the following table:

Number of Trades 50 100 200 300 500
<>Margin of Error 14% 10% 7% 6% 4%


More trades mean a smaller margin of error, resulting in a higher predictability of future performance.
Somebody with a Ph.D. in statistics once told me that you need at least
40 trades in order to produce statistically relevant results. "

from "The complete guide to day trading" by Markus Heitkoetter



So let's choose for our backtesting at least 200 trades. Let's consider a common intraday strategy, it will require at lest
one year to produce 200 trades (only scalping strategy have higher frequency trading).
For these reason I suggested at least 200 or 400 trades for backtesting in previous post.


Now few words about "Forward Testing":
it is highly time consuming. If you need 200 trades with a common intraday strategy you need 1 year of testing, after
if the strategy didn't produce profit you have to change the parameters of strategy and start a new 1 year test and so on ...



At the end a question:
if you have to put a robot strategy on a live account what solution should you choose among the following:

--------------------------------------------------
1. BACKTESTED STRATEGY with:
- 24 months of backtesting
- trades: 400
- profit: 2000%
- CAR (Compound Annual Return): 1000%
- MDD (Maximal relative Drawdown): 20%
- CAR/MDD >2
- month with profit: 20
- month with loss: 4 (max loss per month < 15% of equity)
- monthly average profit: 83.3%
- winning trades %: 65%
--------------------------------------------------
2. FORWARDED STRATEGY with:
- 1 month of forward testing
- trades: 22
- profit: 300%
- MDD (Maximal relative Drawdown): 40%
- winning trades %: 65%
--------------------------------------------------
3. FORWARDED STRATEGY with:
a. 12 monts of forward testing
b. profit/Loss: -80% of initial capital

C. if(loss) change parameters and return point a. for a new test

d. after 10 tests (that means after 10 years) the strategy is in profit, destroyed by tests you decide to apply
finally the strategy to a live account without consider the other parameters results of strategy.

--------------------------------------------------



Best regards,
Frank


P.S.
Happy Easter

"How long should you back-test a trading strategy?
The more trades you use in your back-testing, the higher the probability
that your trading strategy will succeed in the future. Look at the following table:

Number of Trades 50 100 200 300 500
<>Margin of Error 14% 10% 7% 6% 4%


More trades mean a smaller margin of error, resulting in a higher predictability of future performance.
Somebody with a Ph.D. in statistics once told me that you need at least
40 trades in order to produce statistically relevant results. "

I do a lot of data analysis and sampling in my job, you can't throw a number like the PHD in statistics said without having any understand and the population you are talking about.

Firstly your sampling is hugely dependent of your distribution, in trading for your sampling size you would also need to take in consideration a lot of parameters such as your ratio SL/TP.

40 trades is a ridiculous number unless your TP/SL are both very high and if they are very high you can be sure that your entry patern are going to be very rare...

I can easily build million buck strategy without difficulty just by looking at past curves... just keep buying AUD for the past years and your done.

Finally as you seem to be reading a lot of stuff you probably already know that Greenspan is not really a reference anymore ,-)

Most traders i have talked too told me they don't believe any "systems" can last forever, you could also talk about efficient market.

Happy easter too.

Hello
as i said, by my side, backtesting is good in case you need to understand if strategy works how you expect so meaning, if i have a strategy buying when macd cross up and selling when macd cross down, i want to TEST if i set correctly all parameters.
also good to check how the strategy behaves in bad conditions, so let's say if/when macd keeps crossing up-down 5-6-7-8-10 times in a row, causing only account blowing. so to set some conditions to avoid some of those situations.
nothing else.
so a strategy causing 100% profit trades till yesterday, could cause 100% loss starting from tomorrow.
also you could test any time range (2 years?) and get 100% profits, put in demo test another 1year and get another 100% profits, then you put it to live just when a change occurs (boj interventations, snb pegs, fed QE, BCE talks, eurozone rumors, cyprus matters... on and on and on) and you see 10-15 losses in a row.

definetly back testing is exactly what it means : BACK - TEST.
byebye


edit: the same applies to manual trading! if you trade looking what happened, you will just blow you account quite quickly. why? try to always sell a price which was going down till last candle
also i agree with adask: one could just optimize the strategy to show great results on past data. much simple, just consider some 2-3 rules, and enlarge the stop loss so much that none of moves could have stopped you out in that particular situation... done: 100% win ratio

Hi Scramble,
I think we shall continue to save our different points of view:
- after developing phase of strategy I shall make numerous backtesting, considering different parameters to validate it,
before applying it to a LIVE account,
- on the other side You are free to develop a strategy, (manual or automatic) and applying it directly to a LIVE account.
Good Luck.


P.S.
"If there were not persistent patterns and trend that we can identify and trade profitably,
then we are all wasting our time." by Howard B. Bandy.

Let me to suggest this book:
"Quantitative Trading System - Pratical Method for Design, Testing and Validation" - 2007 - by Howard B. Bandy

The author uses as developing platform, the program Amibroker.
The program is tremendously reliable, fast and can use data price from many Vendors (including Dukascopy).
Personally I use this approch:
1. I make the dirty work in Amibroker:
- I develop the strategy using Amibroker's object oriented language
- I backtest, validate, optimize the strategy in Amibroker; this simply because this program is fastest (using all
Dukascopy's data tick, since 2007, a backtsting requires less than a minute (the same backtesting in Historical
Tester requires from 1 day to 1 week)
2. If the strategy works,
- I recode it in JForex
- I make the final backtesting in Historical Tester (that usually requires much time).

I don't like polemizing, I hope this can help.

Best regards,
Frank

Like your proposals for a Back-Testing contest - would certainly be a bit more challenging than developing (or copying) a contest strategy that either makes 2-300% in a month - or crashes to zero !!

However, I have been focusing on scalping strategies that aim to simply make ~5% profit per week - which compounding makes good money long term. But with these types of strategies forward-testing is never the same as back-test for the same period, for various reasons.

So I've come to the conclusion that with scalping strategies at least, back-testing is of quite limited use.
If your strategy trades H1 charts or higher, with a small number of set-ups per week - then of course back-testing will be very useful.

Hi fprophet,
when you backtest a scalping strategy you should use always the tick mode (for example "Process all tick" mode), because in tick mode the tester uses as spread the real spread (that is the difference between Ask and Bid), but if you use a candel time the tester uses the fix spread of 2 pips, and this can change results with scalping with little TakeProfit and StopLoss.

Support told this in a post of some years ago, now I didn't find it on forum, so I attached the file that then I saved on my hard disk.

Best regards,
Frank

Understood - I have always used 'Process all ticks' in all back-testing.

Hi Guys,

I'm coming late in this very interesting topic.
I'm coding strategies since 2010 but never dare using one live with closed eyes until now.

For me, the main issues are that a strategy can do well in backtesting over the years on one
instrument. But then, try to test it on an other instrument and it crashes badly... How can
we assume that for instance next year EURUSD will not behave like a GBPUSD ?

So for me, a strategy is good only if it is doing right on more than 5 years on multiple instruments,
otherwise, forget it. IT WILL CRASH one day or an other.

After 2 years of thinking and coding I'm starting to get to that goal. Here are my rules before I even
consider using my strategy with "closed eyes" on my live account :

- Backtest on the past 7 years. (seems representative of market behaviors : bull market -> topping market -> crack -> crisis -> QE -> rebound -> currency war)
- Trade 21 majors and crosses (including USD, JPY, EUR, GBP, AUD, NZD, excluding CHF since the peg to the EUR)
- 200 to 400 trades per year
- Use stops based on current volatility and wide enough so that the gap between backtest and live spreads don't really matter
- Money management 1 or 2% risk per trade
- don't take to strictly the minimum 1:1 ratio of stoploss/take profit. Some markets are just working very fine with 2:1 ratio and a less than 50% win/loss ratio on trending markets, others with 0.5:1 with a 80% win/loss ratio in intraday volatile and not trending markets. Just take the Buck were it is without asking to much questions, what works, works...
- 15-30% return every year is just fine (don't search anything to fancy, any Hedge fund won't do better on the long time)
- don't allow any negative year
- max 10-20% drawdown can happen 1 or 2 times for this period in a really bad market.


I am not far from this. But to be really confident, I am actually working on a solution for
the robot to auto-optimize itself.
The idea is that it runs once a day in the testers optimizer for a period of xx days in order to
make every live optimized settings in xml files based on the best recent backtest results.

In my case, I saw some optimizations lasting for a year then change... fo instance, at one
year market conditions are well for trading 1H candles, in an other 4H...
Even if optimizing settings in the recent past, is still a lagging indicator, it would permit
to catch part of a longer lasting market condition before it changes.

I'll test this on backtests these weeks ahead, but my computer now needs 7 days of calculations
to create my optimized xml history for the past 7 years...

By the way... for the live operation, does anyone know if and how it is possible to launch automatically
the optimizer every day with java ?

Best regards,
Nicolas

If you are using a unix-based system, just create a cron job. For windows, I'm sure there are some equivalents.