Dukascopy Support Board • View topic - JFOREX-3411 Order's close time

trottel

Recently I looked into a report created during my historical back tests and recognized that 5844 of 6588 orders have been closed at 37 minute of a hour. It's approximately 89%. And 100% of orders have been closed at 30 second of a minute. What is the reason for that? My strategy doesn't close orders at specific time point. They are closed according to takeprofit/stoploss levels.

API Support

Do you use interpolation for historical data?
If it is used, smaller number of ticks is used when strategy is tested. All these fake ticks happen on same, round time intervals.

trottel

API Support wrote:

Do you use interpolation for historical data?
If it is used, smaller number of ticks is used when strategy is tested. All these fake ticks happen on same, round time intervals.

Yes. I use ITesterClient.InterpolationMethod.FOUR_TICKS. Below is my call of method setDataInterval:

client.setDataInterval(Period.ONE_HOUR, OfferSide.BID, ITesterClient.InterpolationMethod.FOUR_TICKS, dateFrom.getTime(), dateTo.getTime());

Is it possible to ignore this parameter in some way?
That's all information about parameters shown by javadoc:

Parameters:
period defines period that will be used for ticks generation. Default value Period.TICK will be used if this parameter is null
from when to start testing process. Default value will be previous day start
to when to stop testing process. Default value will be previous day end
side 
interpolationMethod

So, no information about "side" and "interpolationMethod" parameters. What is the purpose of "side" parameter? I thought I need both sides for trading/testing?

API Support

If setDataInterval is not set tester will feed all ticks to strategy.

trottel

API Support wrote:

If setDataInterval is not set tester will feed all ticks to strategy.

I have commented the call of this method out. Close times are more realistic now.
But I have only few trades from the last week although with setDataInterval() call my testing starts on 1 Januar 2006 and it were thousands of orders.

API Support

Could you, please, provide a code sample to replicate the problem?

trottel

API Support wrote:

Could you, please, provide a code sample to replicate the problem?

Below is the code which is responsible for starting my strategy in historical back test mode:

      final ITesterClient client = TesterFactory.getDefaultInstance();
      // set the listener that will receive system events
      client.setSystemListener(new ISystemListener() {
         @Override
         public void onStart(long processId) {
            LOGGER.info("Strategy started: " + processId);
         }

         @Override
         public void onStop(long processId) {
            LOGGER.info("Strategy stopped: " + processId);
            File reportFile = new File("resources/report.html");
            try {
               client.createReport(processId, reportFile);
            } catch (Exception e) {
               LOGGER.error(e.getMessage(), e);
            }
            if ((client.getStartedStrategies().size() == 0) && exitOnStop) {
               // uncomment the following line to kill the virtual machine TODO
               System.exit(0);
            }
         }

         @Override
         public void onConnect() {
            LOGGER.info("Connected");
         }

         @Override
         public void onDisconnect() {
            // tester doesn't disconnect
         }
      });
      // connect to the server using jnlp, user name and password
      // connection is needed for data downloading
      client.connect(JNLP_URL, USER_NAME, PASSWORD);

      // wait for it to connect
      int i = 10; // wait max ten seconds
      while (i > 0 && !client.isConnected()) {
         Thread.sleep(1000);
         i--;
      }
      if (!client.isConnected()) {
         LOGGER.error("Failed to connect Dukascopy servers");
         System.exit(1);
      }

      // custom historical data
      String dateFormatString = "dd/MM/yyyy HH:mm:ss";
      //      String dateFromStr = TimeTools.getDateString(TimeTools.getDate(Calendar.DAY_OF_MONTH, -10, new Date()),
      //            dateFormatString);
      String dateFromStr = "01/01/2006 00:00:00";
      //      String dateFromStr = "31/09/2011 00:00:00";
      String dateToStr = TimeUtils.getCurrentDate(dateFormatString);
      //      String dateToStr = "10/08/2011 00:00:00";
      //      String dateToStr = "02/20/2006 00:00:00";
      SimpleDateFormat dateFormat = new SimpleDateFormat(dateFormatString);
      dateFormat.setTimeZone(TimeZone.getTimeZone("GMT"));

      Date dateFrom = dateFormat.parse(dateFromStr);
      Date dateTo = dateFormat.parse(dateToStr);

      //      client.setDataInterval(Period.ONE_HOUR, OfferSide.BID, ITesterClient.InterpolationMethod.FOUR_TICKS,
      //            dateFrom.getTime(), dateTo.getTime());

      // set instruments that will be used in testing
      Set<Instrument> instruments = new HashSet<Instrument>();
      instruments.add(Instrument.EURUSD);
      LOGGER.info("Subscribing instruments...");
      client.setSubscribedInstruments(instruments);
      // setting initial deposit
      double equity = 5000;
      client.setInitialDeposit(Instrument.EURUSD.getSecondaryCurrency(), equity);
      // load data
      LOGGER.info("Downloading data");
      java.util.concurrent.Future<?> future = client.downloadData(null);
      // wait for downloading to complete
      future.get();

      // set parameter values here if necessary
      LOGGER.info("Starting strategy");
      client.startStrategy(strategy);

API Support

In order to get all ticks the following line in code should be changed:

client.setDataInterval(Period.ONE_HOUR, OfferSide.BID, ITesterClient.InterpolationMethod.FOUR_TICKS, dateFrom.getTime(), dateTo.getTime());

trottel

API Support wrote:

In order to get all ticks the following line in code should be changed:

client.setDataInterval(Period.ONE_HOUR, OfferSide.BID, ITesterClient.InterpolationMethod.FOUR_TICKS, dateFrom.getTime(), dateTo.getTime());

How it should be changed?

trottel

trottel wrote:

API Support wrote:

In order to get all ticks the following line in code should be changed:

client.setDataInterval(Period.ONE_HOUR, OfferSide.BID, ITesterClient.InterpolationMethod.FOUR_TICKS, dateFrom.getTime(), dateTo.getTime());

How it should be changed?

Resolved with client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, dateFrom.getTime(), dateTo.getTime());
I didn't know that there is a new API which contains ITesterClient.DataLoadingMethod.ALL_TICKS.

trottel

Still don't understand how it should work.

When I use the following method call

client.setDataInterval(Period.ONE_HOUR, OfferSide.BID, ITesterClient.InterpolationMethod.FOUR_TICKS, dateFrom.getTime(), dateTo.getTime());

and the time period from 1 to 10 January 2006 I can see information about 4 closed orders in my report

But when I use

client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, dateFrom.getTime(), dateTo.getTime());

I cannot see any closed orders in my report

API Support

Sorry for the inconvenience.
Could you please try replacing client.setDataInterval with this:

client.setDataInterval(Period.TICK, OfferSide.BID, ITesterClient.InterpolationMethod.OPEN_TICK, dateFrom.getTime(), dateTo.getTime());

trottel

API Support wrote:

Sorry for the inconvenience.
Could you please try replacing client.setDataInterval with this:

client.setDataInterval(Period.TICK, OfferSide.BID, ITesterClient.InterpolationMethod.OPEN_TICK, dateFrom.getTime(), dateTo.getTime());

Still no trades in the report.

Then I changed the method call to

client.setDataInterval(Period.ONE_HOUR, OfferSide.BID, ITesterClient.InterpolationMethod.OPEN_TICK, dateFrom.getTime(), dateTo.getTime());

ran the historical back tester and it created 3 closed orders in the report (instead of 4). Orders were closed at 19:00:00, 04:00:00 and 22:00:00. There are only 3 orders but I assume that we have the problem with unrealistic close times again.

API Support

Could you please provide a sample strategy that would replicate the problem.

trottel

API Support wrote:

Could you please provide a sample strategy that would replicate the problem.

Sorry, but it would be difficult. My strategy makes use of too many libraries and other projects in my workspace. But however I have debugged a little bit and found something what I hope can say something useful to you.

I use time interval from 1st to 5th January 2006

Before I submit an order I get current market price with:

double marketPrice = history.getLastTick(instrument).getAsk();

And put it later in order's comment:

engine.submitOrder("long_position_" + positionCount++, instrument, IEngine.OrderCommand.BUY, vol, 0,
               slippage, sl.getValue(), tp.getValue(), 0, marketPrice + "|" + context.getAccount().getEquity());

And print the market price with:

System.out.println("marketprice = " + marketPrice);

When I use the following call:

client.setDataInterval(Period.ONE_HOUR, OfferSide.BID, ITesterClient.InterpolationMethod.FOUR_TICKS,
            dateFrom.getTime(), dateTo.getTime());

and run historical back test I see in the report information about 3 closed orders and the output to console contains:

Quote:

marketprice = 1.1865
marketprice = 1.1935
marketprice = 1.20225

But when I use the call:

client.setDataInterval(Period.TICK, OfferSide.BID, ITesterClient.InterpolationMethod.OPEN_TICK,
            dateFrom.getTime(), dateTo.getTime());

and run the test I don't see any information about closed orders and output to console contains:

Quote:

marketprice = 0.0
marketprice = 0.0
marketprice = 0.0
marketprice = 0.0
marketprice = 0.0
marketprice = 0.0
marketprice = 0.0
marketprice = 0.0
marketprice = 0.0

The same result when I use:

client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, dateFrom.getTime(), dateTo.getTime());

Below is the method onBar() of my strategy:

public void onBar(Instrument instrument, Period period, IBar askBar, IBar bidBar) {
   if (!period.equals(Period.ONE_HOUR)) {
      return; // do nothing
   }
   try { // check if an order can be submitted
      analyzer.onBar(instrument, period, askBar, bidBar);
   } catch (JFException e) {
      // TODO Auto-generated catch block
   }
}

API Support

We apologize for the inconvenience. To download all ticks use the following code:

        Calendar start = new GregorianCalendar();
        start.add(Calendar.DATE, -3);
        Calendar finish = new GregorianCalendar();
        
        client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, start.getTimeInMillis(), finish.getTimeInMillis());

trottel

API Support wrote:

We apologize for the inconvenience. To download all ticks use the following code:

        Calendar start = new GregorianCalendar();
        start.add(Calendar.DATE, -3);
        Calendar finish = new GregorianCalendar();
        
        client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, start.getTimeInMillis(), finish.getTimeInMillis());

Should I use Calendar instead of Date?

Calendar start = new GregorianCalendar();
start.set(2006, 0, 1, 0, 0, 0);
Calendar finish = new GregorianCalendar();
finish.set(2006, 0, 5, 0, 0, 0);

client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, start.getTimeInMillis(), finish.getTimeInMillis());

Still no trades for the period from 1st to 5th January 2006...

API Support

There is data problem with the period you work with, for now please select some other time period, e.g. 2008-01-01 to 2008-01-05. This will be fixed with the next JForex API release.