Dukascopy Support Board • View topic - JFOREX-3584 Cannot get previous close price to create flats

cenobit

hi,

got a problem with jforex demo account client. i hope you can help me with this.
i checked out your latest maven JForexClientLibrary project. (jforex version 2.6.64) when executing i get the following error and the client crashes:

Apr 11, 2012 11:32:24 PM TesterMain$1 onConnect
Information: Connected
Apr 11, 2012 11:32:24 PM TesterMain main
Information: Subscribing instruments...
Apr 11, 2012 11:32:24 PM TesterMain main
Information: Downloading data
Apr 11, 2012 11:32:24 PM TesterMain main
Information: Starting strategy
Apr 11, 2012 11:32:24 PM TesterMain$1 onStart
Information: Strategy started: 1
Apr 11, 2012 11:32:24 PM TesterMain$2 dataLoaded
Information: Getting prices before start time
Apr 11, 2012 11:32:24 PM com.dukascopy.charts.data.datacache.LoadNumberOfCandlesAction$TicksLiveFeedListener theEnd
[color=#FF0000]Schwerwiegend: Cannot get previous close price to create flats[/color]
Apr 11, 2012 11:32:24 PM TesterMain$2 dataLoaded
Information: Creating data loading threads
Apr 11, 2012 11:32:24 PM TesterMain$2 dataLoaded
Information: Getting prices before start time
Apr 11, 2012 11:32:24 PM TesterMain$2 dataLoaded
Information: Filling in-progress candles
Apr 11, 2012 11:32:24 PM TesterMain$2 dataLoaded
Information: Executing onStart
Apr 11, 2012 11:32:24 PM TesterMain$2 dataLoaded
Information: Running
Started

Apr 11, 2012 11:32:24 PM TesterMain$2 dataLoaded
Information: Executing onStop
Stopped

Apr 11, 2012 11:32:24 PM TesterMain$1 onStop
Information: Strategy stopped: 1

DoubleDutch

I keep getting the same error while testing any strategy with Eclipse.

2012-04-12 12:09:27.125 INFO  Strategy1 - Connected
2012-04-12 12:09:27.125 INFO  Strategy1 - Subscribing instruments...
2012-04-12 12:09:27.125 INFO  Strategy1 - Downloading data
2012-04-12 12:09:27.156 INFO  Strategy1 - Starting strategy
2012-04-12 12:09:27.218 INFO  Strategy1 - Strategy started: 1
2012-04-12 12:09:27.218 INFO  Strategy1 - Getting prices before start time
2012-04-12 12:09:27.234 ERROR LoadNumberOfCandlesAction - Cannot get previous close price to create flats
2012-04-12 12:09:27.234 INFO  Strategy1 - Creating data loading threads
2012-04-12 12:09:27.296 INFO  Strategy1 - Getting prices before start time
2012-04-12 12:09:27.500 INFO  TesterClientImpl - Authorized
2012-04-12 12:09:28.828 INFO  Strategy1 - Filling in-progress candles
2012-04-12 12:09:28.828 INFO  Strategy1 - Executing onStart
2012-04-12 12:09:28.843 INFO  Strategy1 - Running
2012-04-12 12:09:28.843 INFO  Strategy1 - Executing onStop
2012-04-12 12:09:28.953 INFO  Strategy1 - Strategy stopped: 1

I could get the tester running with the same strategy earlier today, but than it gave me an 'ERROR LocalCacheManager' just before the tester could finish. Then I ran the tester again and now I get the error above. Testing another strategy gives the same error.

I would really appreciate some help.

simonred

Today I used API2.6.64, and I found that need sleep(2000) before call client.startStrategy(), like this:

   SMAStrategy stragegy=new SMAStrategy();
                Thread.sleep(2000);   
      client.startStrategy(stragegy, new LoadingProgressListener() {}....)

otherwise it will be exception,the console will print:

[color=#BF0000]2012-04-12 08:03:05.515 ERROR LoadNumberOfCandlesAction - Cannot get previous close price to create flats
2012-04-12 08:03:05.593 ERROR StrategyRunner - Instrument [EUR/USD] not opened
com.dukascopy.api.JFException: Instrument [EUR/USD] not opened
   at com.dukascopy.api.impl.History.getBar(History.java:166)
   at singlejartest.SMAStrategy.onStart(SMAStrategy.java:88)
   at com.dukascopy.dds2.greed.agent.strategy.tester.StrategyRunner.run(StrategyRunner.java:425)
   at java.lang.Thread.run(Unknown Source)[/color]
Strategy tester: com.dukascopy.api.JFException: Instrument [EUR/USD] not opened @ singlejartest.SMAStrategy.onStart(SMAStrategy.java:88)
Instrument [EUR/USD] not opened: com.dukascopy.api.JFException: Instrument [EUR/USD] not opened
   at com.dukascopy.api.impl.History.getBar(History.java:166)
   at singlejartest.SMAStrategy.onStart(SMAStrategy.java:88)
   at com.dukascopy.dds2.greed.agent.strategy.tester.StrategyRunner.run(StrategyRunner.java:425)
   at java.lang.Thread.run(Unknown Source)
2012-04-12 08:03:05.593 ERROR TesterClientImpl$DefaultStrategyExceptionHandler - Exception thrown while running onStart method: Instrument [EUR/USD] not opened
com.dukascopy.api.JFException: Instrument [EUR/USD] not opened
   at com.dukascopy.api.impl.History.getBar(History.java:166)
   at singlejartest.SMAStrategy.onStart(SMAStrategy.java:88)
   at com.dukascopy.dds2.greed.agent.strategy.tester.StrategyRunner.run(StrategyRunner.java:425)
   at java.lang.Thread.run(Unknown Source)
2012-04-12 08:03:05.593 ERROR TesterClientImpl - Instrument [EUR/USD] not opened
com.dukascopy.api.JFException: Instrument [EUR/USD] not opened
   at com.dukascopy.api.impl.History.getBar(History.java:166)
   at singlejartest.SMAStrategy.onStart(SMAStrategy.java:88)
   at com.dukascopy.dds2.greed.agent.strategy.tester.StrategyRunner.run(StrategyRunner.java:425)
   at java.lang.Thread.run(Unknown Source)
2012-04-12 08:03:07.578 INFO  TesterClientImpl - Authorized

API Support

You need the timeout because subscription to instruments is asynchronous. See:
https://www.dukascopy.com/wiki/#Standalo ... nstruments

yntoxy

Output for the standard TestMain from dukascopy.com:

Quote:

2012-04-12 17:03:24.236 INFO Main - Connecting...
2012-04-12 17:03:26.014 INFO AuthorizationClient - >>
...
2012-04-12 17:03:27.622 INFO Main - Connected
2012-04-12 17:03:27.623 INFO Main - Subscribing instruments...
2012-04-12 17:03:27.623 INFO Main - Downloading data
2012-04-12 17:03:27.636 INFO Main - Starting strategy
2012-04-12 17:03:27.668 INFO Main - Strategy started: 1
2012-04-12 17:03:27.669 INFO Main - Getting prices before start time
2012-04-12 17:03:27.679 ERROR LoadNumberOfCandlesAction - Cannot get previous close price to create flats
2012-04-12 17:03:27.679 INFO Main - Creating data loading threads
2012-04-12 17:03:27.707 INFO Main - Getting prices before start time
2012-04-12 17:03:27.809 INFO TesterClientImpl - Authorized
2012-04-12 17:03:27.816 INFO Main - Filling in-progress candles
2012-04-12 17:03:27.817 INFO Main - Executing onStart
Started

2012-04-12 17:03:27.822 INFO Main - Running
2012-04-12 17:03:27.824 INFO Main - Executing onStop
Stopped

2012-04-12 17:03:27.838 INFO Main - Strategy stopped: 1

API Support

Apparently the instruments don't manage to get subscribed before their data get accessed, try the following approach:
https://www.dukascopy.com/wiki/#Standalone_JForex_API/Subscribing_to_instruments

DoubleDutch

Sorry for being such a dummy but I can't get it to work. Could you please add this method to the script bellow

/
 * Copyright (c) 2009 Dukascopy (Suisse) SA. All Rights Reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 * -Redistribution of source code must retain the above copyright notice, this
 *  list of conditions and the following disclaimer.
 *
 * -Redistribution in binary form must reproduce the above copyright notice,
 *  this list of conditions and the following disclaimer in the documentation
 *  and/or other materials provided with the distribution.
 * 
 * Neither the name of Dukascopy (Suisse) SA or the names of contributors may
 * be used to endorse or promote products derived from this software without
 * specific prior written permission.
 *
 * This software is provided "AS IS," without a warranty of any kind. ALL
 * EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING
 * ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
 * OR NON-INFRINGEMENT, ARE HEREBY EXCLUDED. DUKASCOPY (SUISSE) SA ("DUKASCOPY")
 * AND ITS LICENSORS SHALL NOT BE LIABLE FOR ANY DAMAGES SUFFERED BY LICENSEE
 * AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THIS SOFTWARE OR ITS
 * DERIVATIVES. IN NO EVENT WILL DUKASCOPY OR ITS LICENSORS BE LIABLE FOR ANY LOST
 * REVENUE, PROFIT OR DATA, OR FOR DIRECT, INDIRECT, SPECIAL, CONSEQUENTIAL,
 * INCIDENTAL OR PUNITIVE DAMAGES, HOWEVER CAUSED AND REGARDLESS OF THE THEORY
 * OF LIABILITY, ARISING OUT OF THE USE OF OR INABILITY TO USE THIS SOFTWARE,
 * EVEN IF DUKASCOPY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
 */
package singlejartest;

import com.dukascopy.api.Instrument;
import com.dukascopy.api.LoadingProgressListener;
import com.dukascopy.api.system.ISystemListener;
import com.dukascopy.api.system.ITesterClient;
import com.dukascopy.api.system.TesterFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.File;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashSet;
import java.util.Set;
import java.util.TimeZone;
import java.util.concurrent.Future;

/**
 * This small program demonstrates how to initialize Dukascopy tester and start a strategy
 */
public class TesterMain {
    private static final Logger LOGGER = LoggerFactory.getLogger(Main.class);

    //url of the DEMO jnlp
    private static String jnlpUrl = "https://www.dukascopy.com/client/demo/jclient/jforex.jnlp";
    //user name
    private static String userName = "username";
    //password
    private static String password = "password";

    public static void main(String[] args) throws Exception {
        //get the instance of the IClient interface
        final ITesterClient client = TesterFactory.getDefaultInstance();
        //set the listener that will receive system events
        client.setSystemListener(new ISystemListener() {
            @Override
            public void onStart(long processId) {
                LOGGER.info("Strategy started: " + processId);
            }

            @Override
            public void onStop(long processId) {
                LOGGER.info("Strategy stopped: " + processId);
                File reportFile = new File("C:\\report.html");
                try {
                    client.createReport(processId, reportFile);
                } catch (Exception e) {
                    LOGGER.error(e.getMessage(), e);
                }
                if (client.getStartedStrategies().size() == 0) {
                    System.exit(0);
                }
            }
         // custom historical data
        final SimpleDateFormat dateFormat = new SimpleDateFormat("MM/dd/yyyy HH:mm:ss");
        dateFormat.setTimeZone(TimeZone.getTimeZone("GMT"));

        Date dateFrom = dateFormat.parse("01/26/2010 00:00:00");
        Date dateTo = dateFormat.parse("03/14/2012 00:00:00");

        client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, dateFrom.getTime(), dateTo.getTime());

            @Override
            public void onConnect() {
                LOGGER.info("Connected");
            }

            @Override
            public void onDisconnect() {
                //tester doesn't disconnect
            }
        });
      
        LOGGER.info("Connecting...");
        //connect to the server using jnlp, user name and password
        //connection is needed for data downloading
        client.connect(jnlpUrl, userName, password);

        //wait for it to connect
        int i = 10; //wait max ten seconds
        while (i > 0 && !client.isConnected()) {
            Thread.sleep(1000);
            i--;
        }
        if (!client.isConnected()) {
            LOGGER.error("Failed to connect Dukascopy servers");
            System.exit(1);
        }

        //set instruments that will be used in testing
        Set<Instrument> instruments = new HashSet<Instrument>();
        instruments.add(Instrument.EURUSD);
        LOGGER.info("Subscribing instruments...");
        client.setSubscribedInstruments(instruments);
        //setting initial deposit
        client.setInitialDeposit(Instrument.EURUSD.getSecondaryCurrency(), 50000);
        //load data
        LOGGER.info("Downloading data");
        Future<?> future = client.downloadData(null);
        //wait for downloading to complete
        future.get();
        //start the strategy
        LOGGER.info("Starting strategy");
        client.startStrategy(new MA_Play(), new LoadingProgressListener() {
            @Override
            public void dataLoaded(long startTime, long endTime, long currentTime, String information) {
                LOGGER.info(information);
            }

            @Override
            public void loadingFinished(boolean allDataLoaded, long startTime, long endTime, long currentTime) {
            }

            @Override
            public boolean stopJob() {
                return false;
            }
        });
        //now it's running
    }
}

API Support

DoubleDutch wrote:

Sorry for being such a dummy but I can't get it to work. Could you please add this method to the script bellow

Please replace the

 //set instruments that will be used in testing
        Set<Instrument> instruments = new HashSet<Instrument>();
        instruments.add(Instrument.EURUSD);
        LOGGER.info("Subscribing instruments...");
        client.setSubscribedInstruments(instruments);

With the snippet from wiki:
https://www.dukascopy.com/wiki/#Standalone_JForex_API/Subscribing_to_instruments

API Support

yntoxy wrote:

Subscribing before data get accessed is not work.

I solved it in bad way such that:

        Future<?> future = client.downloadData(null);
        Thread.sleep(5000); //this timeout helped
        future.get();

Please use the approach given in:
https://www.dukascopy.com/wiki/#Standalone_JForex_API/Subscribing_to_instruments

DoubleDutch

This solution doesn't work for me. I still get the same error. Maybe I'm doing something wrong.
Is this right?

public class TesterMain {
private static IConsole console;

//set instruments that will be used in testing
        Set<Instrument> instruments = new HashSet<Instrument>();
        instruments.add(Instrument.EURUSD);
                               
        client.setSubscribedInstruments(instruments);
       
        // wait max 2 seconds for the instruments to get subscribed
        int m = 10;
        while (!client.getSubscribedInstruments().containsAll(instruments)) {
            try {
                console.getOut().println("Instruments not subscribed yet " + m);
                Thread.sleep(200);
            } catch (InterruptedException e) {
                console.getOut().println(e.getMessage());
            }
            m--;
        }

simonred

Thanks your reply

I had added this code

Set<Instrument> instruments = new HashSet<Instrument>();
instruments.add(Instrument.EURSEK);
instruments.add(Instrument.AUDUSD);
                
client.setSubscribedInstruments(instruments);

// wait max 2 seconds for the instruments to get subscribed
int i = 10;
while (!client.getSubscribedInstruments().containsAll(instruments)) {
    try {
        console.getOut().println("Instruments not subscribed yet " + i);
        Thread.sleep(200);
    } catch (InterruptedException e) {
        console.getOut().println(e.getMessage());
    }
    i--;
}

and "client.getSubscribedInstruments().containsAll(instruments)" always return true when first loop,
but exception still occured .
By the way , exception ouccured at statement in fuction onstart()
"singlejartest.SMAStrategy.onStart(SMAStrategy.java:88)":

IBar currBar = history.getBar(this.selectedInstrument,
            Period.FIFTEEN_MINS, OfferSide.ASK, 0);

API Support

tabanyi wrote:

same problem here, the suggested solution in the wiki page does NOT solve this bug

This is the case, as a workaround please consider using the solution already suggested before - add Thread.sleep(5000) after the instrument subscription.

Balazs

I'm having the same problem as cenobit since the new api. I run the TesterMain found in the package (without any modification), but it fails:

2012-04-13 14:36:41.216 INFO  Main - Connecting...
2012-04-13 14:36:45.042 INFO  Main - Connected
2012-04-13 14:36:45.042 INFO  Main - Subscribing instruments...
2012-04-13 14:36:45.042 INFO  Main - Downloading data
2012-04-13 14:36:45.052 INFO  Main - Starting strategy
2012-04-13 14:36:45.077 INFO  Main - Strategy started: 1
2012-04-13 14:36:45.078 INFO  Main - Getting prices before start time
2012-04-13 14:36:45.085 ERROR LoadNumberOfCandlesAction - Cannot get previous close price to create flats
2012-04-13 14:36:45.085 INFO  Main - Creating data loading threads
2012-04-13 14:36:45.096 INFO  Main - Getting prices before start time
2012-04-13 14:36:45.172 INFO  Main - Filling in-progress candles
2012-04-13 14:36:45.172 INFO  Main - Executing onStart
Started

2012-04-13 14:36:45.175 INFO  Main - Running
2012-04-13 14:36:45.176 INFO  Main - Executing onStop
Stopped

2012-04-13 14:36:45.188 INFO  Main - Strategy stopped: 1
2012-04-13 14:36:45.342 INFO  TesterClientImpl - Authorized
BUILD SUCCESSFUL (total time: 5 seconds)

But if I try the TesterMainGUIMode it works perfectly.

With the old api (2.6.60) I did not have any problems, but now I am not able to test my strategies any more.

FXace

Hi Support,

when I run standalone .jar file I have the same problem. Tried deleting EURUSD (the only pair used for the strategy) directory in D:\Users\<my user name>\AppData\Local\JForex\.cache\ cache directory and didn't help.

However when I run exactly the same code from Eclipse (Helios) it doesn't appear and I only get a harmless warning as shown in log below and strategy continues to work for the set time interval...

Please help.

Thanks in advance.

FXace

Hi Support,

the solution from yntoxy worked well for me. I'd still expect

Future<?> future = client.downloadData(null);
//wait for downloading to complete
future.get();

i.e. call to future.get() to ensure that asynchronous data download finish before the thread continues, i.e. strategy starts further in it. At least please update TesterMain.java example to make the workaround more visible...

Cheers,

FXace

cenobit

does not work:
Future<?> future = client.downloadData(null);
future.get(); // seems to sleep not long enough
=> error: Cannot get previous close price to create flats

works:
Future<?> future = client.downloadData(null);
future.get();
Thread.sleep(5000);

cenobit

there still seems to be a problem when working with time intervals like this:

      DataLoadingMethod dataLoadingMethod = DataLoadingMethod.TICKS_WITH_TIME_INTERVAL;
      dataLoadingMethod.setTimeIntervalBetweenTicks(TimeUnit.HOURS.toMillis(1));

      client.setDataInterval(dataLoadingMethod, new Date(2012, 3, 3).getTime(), new Date(2012, 3, 4).getTime());

      Future<?> future = client.downloadData(null);
      // wait for downloading to complete
      future.get();

      Thread.sleep(50000);

i still get: error: Cannot get previous close price to create flats ... even with sleep 50k and the wait-instrument-thing. i hope you are going to release a new version soon...
without intervals it works "fine".

also there seems to be a problem when selecting a custom cache folder via

client.setCacheDirectory(new File(cacheDir));

when executing this code i'll get the following error:

com.dukascopy.charts.data.datacache.DataCacheException: Cannot delete cache files [C:\var\tmp\dc\cache\EURUSD\2012\03\12\ASK_candles_min_1.bi5]
   at com.dukascopy.charts.data.datacache.CacheManager.deleteCacheRecursive(CacheManager.java:595)
   at com.dukascopy.charts.data.datacache.CacheManager.deleteCacheRecursive(CacheManager.java:581)
   at com.dukascopy.charts.data.datacache.CacheManager.deleteCacheRecursive(CacheManager.java:581)
   at com.dukascopy.charts.data.datacache.CacheManager.deleteCacheRecursive(CacheManager.java:581)
   at com.dukascopy.charts.data.datacache.CacheManager.deleteCache(CacheManager.java:563)
   at com.dukascopy.charts.data.datacache.CacheManager.<init>(CacheManager.java:159)
   at com.dukascopy.charts.data.datacache.CacheManager.<init>(CacheManager.java:93)
   at com.dukascopy.charts.data.datacache.LocalCacheManager.<init>(LocalCacheManager.java:99)
   at com.dukascopy.charts.data.datacache.FeedDataProvider.<init>(FeedDataProvider.java:137)
   at com.dukascopy.dds2.greed.agent.strategy.tester.TesterFeedDataProvider.<init>(TesterFeedDataProvider.java:29)
   at com.dukascopy.dds2.greed.agent.strategy.tester.TesterHistory.createTesterFeedDataProvider(TesterHistory.java:153)
   at com.dukascopy.dds2.greed.agent.strategy.tester.TesterHistory.<init>(TesterHistory.java:119)
   at com.dukascopy.dds2.greed.agent.strategy.tester.TesterConfig.<init>(TesterConfig.java:106)
   at com.dukascopy.dds2.greed.agent.strategy.tester.StrategyRunner.run(StrategyRunner.java:394)
   at java.lang.Thread.run(Unknown Source)

disabling the custom cache folder "fixes" the problem

API Support · Fixed in 2.6.66

Fixed in 2.6.66

API Support

Quote:

i still get: error: Cannot get previous close price to create flats ... even with sleep 50k and the wait-instrument-thing. i hope you are going to release a new version soon...
without intervals it works "fine".

This:

new Date(2012, 3, 3)

yields date:

3912-04-02 21:00:00

See the constructor javadocs.
Consider using the following approach:
https://www.dukascopy.com/wiki/#Standalone_JForex_API/Testing_over_a_particular_historical_time_interval

cenobit · loading data now works with jforex 2.6.66 thanks

loading data now works with jforex 2.6.66
thanks