Dukascopy
 
 
Wiki JStore Search Login

JFOREX-1491: Backtesting problems
 Post subject: JFOREX-1491: Backtesting problems Post rating: 0   New post Posted: Fri 05 Feb, 2010, 18:07 

User rating: 0
Joined: Fri 25 Sep, 2009, 21:35
Posts: 5
Hi All,

For the last 4-5 months the backtesting of a strategy we have been developing has been going very well. Approx 85% of entries in the backtesting runs had the lots filled. (All the way up to client platform version 2.1.4)

As a side note, one of the development team, has already raised a ticket ID: DKW-979094 on this issue.

On Monday, Jan 12th, using the new client 2.1.5 backtesting the very same unchanged strategy with the same parameters and we found that 80-85% on trades were rejected "Order failed, retrying". The backtesting report once test was complete, stated "canceled because of no liquidity at the price".

We later found out the slippage parameter of 2 (that had been working for 4-5 months previously with client platform prior to v2.1.5) was too low to allow backtest entries to be filled. When we raise up the slippage to 20/30/40 the trades that are rejected are a lot less. (but heavily impacts strategy performance)

We tried recompiling the Java and JFX using a API version 2.5 This did not fix the issue.

It would appear that order filling mechanism for backtesting has changed - can this be something to do with the minimum lot size shift ? (from 0.1 to 0.01)

This is very frustrating after 4-5 months of efforts appearing to be wasted after the platform client software has been upgraded to 2.1.5 (Update: problem still exists with v2.1.13 client)

We had been working on the strategy for a considerable time, but we are now at a loss now and will for the time being, not go ahead with opening a live account. The backtester is a very considerable competitive edge that Duakscopy has compared to all other brokers. MT4 is very limited in its backtesting capabilities, and to my knowledge no one else has developed backtesters with realistic multi-pair and ask-bid price functionality

Thanks.

Code for placing orders is listed below. We are using JFX API 2.5 and jftoolbox 1.6. I am aware the JFToolBox submitOrderSync also waits for an order response, but we wanted wait a little longer if the order was not filled.


private boolean submitOrder(Instrument sym, OrderCommand cmd, double lots, double price, double slippage, int sl, int tp, int magic){
      IOrder tmp;
      int count = 0;
      boolean result = false;
      try {
         while (!result && count < MaximumRetries) {
            box.print("Trying to open order: " + sym + " " + cmd.name() + " " + lots + " " + price + " sl:" + sl + " tp:" + tp );
            
            count++;
            tmp = box.submitOrderSync(sym, cmd, lots, box.roundHalfPip(price), slippage, sl, tp, magic);
            IMessage msg = tmp.waitForUpdate(2 * 1000, TimeUnit.MILLISECONDS);
            if ( tmp.getState() == IOrder.State.FILLED ) {
               result = true;
            } else if ( tmp.getState() == IOrder.State.OPENED && tmp.getState() != IOrder.State.FILLED ) {
               box.print("waiting for order " + tmp.getId() + " to fill");
               msg = tmp.waitForUpdate(5*1000, TimeUnit.MILLISECONDS);

               if ( tmp.getState() == IOrder.State.FILLED || tmp.getState() == IOrder.State.OPENED ) {
                  result = true;
               } else {
                  box.print("Order failed, retrying.  Order State = " + tmp.getState());
                  if ( msg != null ) {
                     box.print(msg.getContent());
                  }
               }
            }
         }
      } catch (JFException e) {
         box.print("Order:  " + e.getMessage());
      }
      return result;
   }




 
 Post subject: Re: JFOREX-1491: Backtesting problems Post rating: 0   New post Posted: Mon 15 Feb, 2010, 10:16 
User avatar

User rating:
Joined: Fri 31 Aug, 2007, 09:17
Posts: 6139
Hello,
here is the example that creates an order every 5 minutes. All orders are filled:
package jforex;

import java.util.*;
import java.util.concurrent.*;

import com.dukascopy.api.*;

public class LimitOrderEvery5Mins implements IStrategy {
   private IEngine engine;
   private IConsole console;
   private IHistory history;
   private IContext context;
   private IIndicators indicators;
   private IUserInterface userInterface;
   
    private long ticks;
    private long time;
    private long realTime;
    private IOrder order;
   
   public void onStart(IContext context) throws JFException {
      this.engine = context.getEngine();
      this.console = context.getConsole();
      this.history = context.getHistory();
      this.context = context;
      this.indicators = context.getIndicators();
      this.userInterface = context.getUserInterface();
   }

   public void onAccount(IAccount account) throws JFException {
   }

   public void onMessage(IMessage message) throws JFException {
   }

   public void onStop() throws JFException {
   }

   public void onTick(Instrument instrument, ITick tick) throws JFException {
        ++ticks;
        if (realTime == 0) {
            realTime = System.currentTimeMillis();
            time = tick.getTime();
        }
        if (time + 24 * 60 * 60 * 1000 < tick.getTime()) {
            long currentTime = System.currentTimeMillis();
            long millis = currentTime - realTime;
            long hours = millis / (60 * 60 * 1000);
            millis -= hours * 60 * 60 * 1000;
            long mins = millis / (60 * 1000);
            millis -= mins * 60 * 1000;
            long secs = millis / 1000;
            millis -= secs * 1000;

            double millisecondsPerTick = ((double) (currentTime - realTime)) / ticks;
           
            console.getOut().println(ticks + " ticks processed in " + hours + "h " + mins + "m " + secs + "s " + millis + "milliseconds. MSPT - " + millisecondsPerTick);
           
            realTime = System.currentTimeMillis();
            time = tick.getTime();
            ticks = 0;
        }
        if (order == null || order.getCreationTime() + 5 * 60 * 1000 < tick.getTime()) {
            if (order != null && (order.getState() == IOrder.State.OPENED || order.getState() == IOrder.State.FILLED)) {
                do {
                    order.close();
                    long safetyTimer = history.getTimeOfLastTick(instrument);
                    while (order.getState() != IOrder.State.CLOSED && order.getState() != IOrder.State.CANCELED && safetyTimer + 10000 > history.getTimeOfLastTick(instrument)) {
                        order.waitForUpdate(1000);
                    }
                } while (order.getState() == IOrder.State.OPENED || order.getState() == IOrder.State.FILLED);
            }
//            order = engine.submitOrder("order", instrument, IEngine.OrderCommand.BUYLIMIT, 0.001, tick.getAsk() - 2 * instrument.getPipValue());
//            order = engine.submitOrder("order", instrument, IEngine.OrderCommand.BUY, 0.001);
            order = submitOrder(instrument, IEngine.OrderCommand.BUY, 0.001, tick.getAsk(), 5, tick.getBid() - 20 * instrument.getPipValue(), tick.getBid() + 20 * instrument.getPipValue(), "order");
        }
        //history.getBars(instrument, Period.TEN_SECS, OfferSide.BID, Filter.ALL_FLATS, 5, history.getTimeForNBarsBack(Period.TEN_SECS, tick.getTime(), 3), 1);
        //indicators.sma(instrument, Period.TEN_SECS, OfferSide.BID, IIndicators.AppliedPrice.CLOSE, 5, Filter.ALL_FLATS, 3, tick.getTime(), 0);
   }
   
    public void onBar(Instrument instrument, Period period, IBar askBar, IBar bidBar) throws JFException {
    }
   
   private IOrder submitOrder(Instrument sym, IEngine.OrderCommand cmd, double lots, double price, double slippage, double sl, double tp, String label){
      IOrder tmp = null;
      int count = 0;
      boolean result = false;
      int MaximumRetries = 5;
      try {
         while (!result && count < MaximumRetries) {
            console.getOut().println("Trying to open order: " + sym + " " + cmd.name() + " " + lots + " " + price + " sl:" + sl + " tp:" + tp );
           
            count++;
            tmp = engine.submitOrder(label, sym, cmd, lots, price, slippage, sl, tp);
            IMessage msg = tmp.waitForUpdate(2 * 1000, TimeUnit.MILLISECONDS);
            if ( tmp.getState() == IOrder.State.FILLED ) {
               result = true;
            } else if ( tmp.getState() == IOrder.State.OPENED && tmp.getState() != IOrder.State.FILLED ) {
               console.getOut().println("waiting for order " + tmp.getId() + " to fill");
               msg = tmp.waitForUpdate(5*1000, TimeUnit.MILLISECONDS);

               if ( tmp.getState() == IOrder.State.FILLED || tmp.getState() == IOrder.State.OPENED ) {
                  result = true;
               } else {
                  console.getOut().println("Order failed, retrying.  Order State = " + tmp.getState());
                  if ( msg != null ) {
                     console.getOut().println(msg.getContent());
                  }
               }
            }
         }
      } catch (JFException e) {
         console.getOut().println("Order:  " + e.getMessage());
      }
      return tmp;
   }
}


I modified your submitOrder code a bit to make it compile without "box" variable.


 
 Post subject: Re: JFOREX-1491: Backtesting problems Post rating: 0   New post Posted: Mon 15 Feb, 2010, 16:09 

User rating: 0
Joined: Fri 25 Sep, 2009, 21:35
Posts: 5
Edit:

Using the modification above does fix the problem we were seeing. When I tried the tests this morning, my client did not see any changes, but after recompiling, I received the results I expected. I don't know what changed between versions of JForex to originally introduce the problem, but its working now. Thanks for the help.

**** Ignore the original text below ****

This code doesn't make the difference.

Prior to the new client, this strategy backtested without any problems. After the new client was implemented, the exact same strategy with the exact same parameters during the exact same time period started having problems during the backtest. 80% of orders now fail. Our code did not change, only the JForex client did.

Something has changed in JForex in the way orders are filled during a backtest. We did some experiments to try and narrow it down the problem and found that if we widen the slippage from a value of 2 to higher values such as 20 to 30, the number failed orders significantly decreases, but the outcome is the strategy is no longer profitable.

To reiterate, prior to the new client, a slippage parameter of 2 ran successfully without any failed orders. After the new client was available. Slippage of 2 causes 80% or more order failures on the same strategy during the same period. Widening the slippage parameter by an order of magnitude (20 to 30 instead of 2), reduces the number of failed orders but changes the outcome of the strategy significantly.

One particular time we are having problems is with the GBPUSD currency pair @ 2007-04-05 10:58:03.

What has changed in the way JForex fills orders during backtesting in the new JForex Client?


 
 Post subject: Re: JFOREX-1491: Backtesting problems Post rating: 0   New post Posted: Mon 15 Feb, 2010, 19:34 
User avatar

User rating: 3
Joined: Wed 18 May, 2011, 16:25
Posts: 331
Location: SwitzerlandSwitzerland
Hi solid54,

I would be *very* cautious with backtesting when the success of your strategy depends on a narrow slippage. I haven't tested with the recent versions, but in previous versions we found that JForex was filling all trades in backtesting with the order amount you were asking for, so no slippage was applied at all. Even when DK now optimizes the entries off the historical data, the slippage you will see is never what you get in the live environment as in historic testing you are not filling the trade from an order book but from recorded historic trades and their volumes.

Best RR.


 
 Post subject: Re: JFOREX-1491: Backtesting problems Post rating: 0   New post Posted: Mon 15 Feb, 2010, 19:56 

User rating: 0
Joined: Fri 25 Sep, 2009, 21:35
Posts: 5
RoadRunner,

I agree, but when I see a significant difference in backtesting between versions of software. I want to be very sure a bug was not introduced that could be causing even more false expectations.

Thanks.


 

Jump to:  

  © 1998-2024 Dukascopy® Bank SA
On-line Currency forex trading with Swiss Forex Broker - ECN Forex Brokerage,
Managed Forex Accounts, introducing forex brokers, Currency Forex Data Feed and News
Currency Forex Trading Platform provided on-line by Dukascopy.com