Hi,

I've just read the thread "Idea for faster backtests and more effective".

Here are three simple tricks that can hugely speed up your historic backtesting without compromising accuracy.
They don't need a code change on DK side.

trick#1:
Only process ticks that have a price different from last tick's price! (just filter ticks by comparing price to the last processed tick). You don't loose any accuracy in the results with this method.
(You will be amazed how many ticks there are in the historical data with exactly the same price as the previous one...)

trick#2:
Use range bars for backtesting!
This leads the backtesting to only process ticks that have a price move of a set number of pips from the last processed tick.
(The setting would depend on your strategy.)
You already can select range bars for backtestings now! It's already included.

trick#3:
Run backtesting on ticks, and in onTick skip all ticks until price has moved a set number of pips from the last processed tick.
While you wait for this price move to occur register the minimum and maximum tick of that move. When the first of the two reaches the set number of pips process both ticks.

Example: Only process ticks after a 2p price move.
last processed: @... 17:58.121: BID 1.4321
went up to: @... 17:59.340: BID 1.4322.5
now at: @... 18:05.729: BID 1.4219 --> 2p move reached.

==> process tick 17:59.340, followed by 18:01.729
==> wait for the next 2p move.

The precision of this backtesting is higher than pure range bar based backtestings (because DK does only use the close-tick of a range bar) and ways more precise than any backtestings based on period-based interpolations. It's very easy to implement in onTick(...) and it's lightening fast.

You will be amazed, how many ticks get filtered...
We've been able to speed up our backtestings by more than 80x (!!) through enhancements based on the above mentioned tricks.

Enjoy!
Best, RR.

Thanks RoadRunner for your ideas. We will consider those to implement.

Hey Roadrunner,

I think I have a memory leak in my tester, I have 6gb memory and during a test all 6gb is being used with no free memory available. Any idea how to fix this?

Thanks

Hi Saidar,

do you have the "Show Java memory heap" option switched on in the JForex client?
How much use of memory does the heap show?

Also, it would only be a memory leak, if after each run (and clicking on the "clear heap" button in the status bar of the DK client) the heap memory size shown is ever increasing.

Best, RR.

Hi Support,
I even noted that Historical Tester often increases the amount of used memory (heap reaches 400 Mb and the process javaw reaches 500 Mb)
and becomes estremly slowly.
To free memory I have two options:
1. restart Jforex platform
2. or in Java Console press the key "g" to recover memory space

I noted that it happens only when the method onTick contains instructions, so the Java console prints continuosly the following error message:

30.12.2010 21:28:20.594   SEVERE   actions.StrategyTesterAction ] Exception thrown whiler running onTick method: null
java.lang.NullPointerException
   at MyStrategy.onTick(MyStrategy.java:74)
   at com.dukascopy.dds2.greed.agent.strategy.tester.l.run(Unknown Source)
   at com.dukascopy.dds2.greed.agent.strategy.tester.l.vc(Unknown Source)
   at com.dukascopy.dds2.greed.actions.StrategyTesterAction.tm(Unknown Source)
   at com.dukascopy.dds2.greed.actions.StrategyTesterAction.sN(Unknown Source)
   at com.dukascopy.dds2.greed.actions.c.run(Unknown Source)
   at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
   at java.lang.Thread.run(Unknown Source)

When the method onTick is empty there isn't trouble, even if the time-frame in Historical tester is "Tick", at least in my backtestings.

Best regards,
Frank

Hi Frank,

looks like you are referencing an object in line 74 of your strategy that is not initialized or is set to null .

Best, RR.

Hi RoadRunner,
you're right, the error message was for a not initialized object. Thanks.

Best regards,
Frank