Dukascopy Support Board • View topic - ERROR AbstractHTFlowDataCreator

nicofr0707

Hello,

On SDK maven project, I have issues running ITesterClient using an other period than ticks.

This works fine :

client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, dateFrom.getTime(), dateTo.getTime());

This does an error when the strategy finishes testing :

client.setDataInterval(Period.ONE_MIN, OfferSide.BID, ITesterClient.InterpolationMethod.FOUR_TICKS, dateFrom.getTime(), dateTo.getTime());

Here the error :

Quote:

2014-06-03 10:37:55.592 ERROR AbstractHTFlowDataCreator - to(2009-12-02 00:58:59:999) > last tick time (2009-12-01 23:59:56:295 1259711996295)
java.lang.IllegalArgumentException: to(2009-12-02 00:58:59:999) > last tick time (2009-12-01 23:59:56:295 1259711996295)
at com.dukascopy.charts.data.datacache.customticks.AbstractLoadCustomTicksFromCandlesAction.validateFromTo(AbstractLoadCustomTicksFromCandlesAction.java:65)
at com.dukascopy.charts.data.datacache.customticks.AbstractLoadCustomTicksFromCandlesAction.<init>(AbstractLoadCustomTicksFromCandlesAction.java:59)
at com.dukascopy.charts.data.datacache.customticks.LoadFourTicksAction.<init>(LoadFourTicksAction.java:39)
at com.dukascopy.charts.data.datacache.FeedDataProvider.createCustomTicksLoadingAction(FeedDataProvider.java:2605)
at com.dukascopy.charts.data.datacache.FeedDataProvider.loadCustomTicksSynched(FeedDataProvider.java:2564)
at com.dukascopy.dds2.greed.agent.strategy.tester.dataload.AbstractHTFlowDataCreator.performDataLoad(AbstractHTFlowDataCreator.java:147)
at com.dukascopy.dds2.greed.agent.strategy.tester.dataload.AbstractHTFlowDataCreator.loadNextDataChunk(AbstractHTFlowDataCreator.java:137)
at com.dukascopy.dds2.greed.agent.strategy.tester.dataload.HTCandlesCreator.loadNextDataChunk(HTCandlesCreator.java:171)
at com.dukascopy.dds2.greed.agent.strategy.tester.dataload.HistoricalTesterDataManager$1.run(HistoricalTesterDataManager.java:193)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

Please see my strategy :

package NicoSources;

import singlejartest.*;
import java.awt.Dimension;
import java.awt.FlowLayout;
import java.awt.Toolkit;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.io.File;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.Future;

import javax.swing.BoxLayout;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.SwingUtilities;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.dukascopy.api.IChart;
import com.dukascopy.api.Instrument;
import com.dukascopy.api.LoadingProgressListener;
import com.dukascopy.api.OfferSide;
import com.dukascopy.api.Period;
import com.dukascopy.api.system.Commissions;
import com.dukascopy.api.system.ISystemListener;
import com.dukascopy.api.system.ITesterClient;
import com.dukascopy.api.system.TesterFactory;
import com.dukascopy.api.system.tester.ITesterExecution;
import com.dukascopy.api.system.tester.ITesterExecutionControl;
import com.dukascopy.api.system.tester.ITesterGui;
import com.dukascopy.api.system.tester.ITesterUserInterface;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
import java.util.Locale;
import java.util.SortedMap;
import java.util.TimeZone;
import java.util.Timer;
import java.util.TimerTask;
import java.util.logging.Level;
import javax.swing.JScrollPane;
import javax.swing.JTable;
import javax.swing.table.AbstractTableModel;

/**
 * This small program demonstrates how to initialize Dukascopy tester and start
 * a strategy in GUI mode
 */
@SuppressWarnings("serial")
public class TesterMainGUIMode extends JFrame implements ITesterUserInterface, ITesterExecution {

    private String strDateFrom = "2009/11/30 23:59:00";
    private String strDateTo = "2009/12/02 00:00:00";

    private static final Logger LOGGER = LoggerFactory.getLogger(TesterMainGUIMode.class);
    private boolean DEBUG = false;

    private final int frameWidth = 1000;
    private final int frameHeight = 600;
    private final int controlPanelHeight = 40;

    private JPanel currentChartPanel = null;
    private ITesterExecutionControl executionControl = null;
    private List<ITesterExecutionControl> executionControlList = new ArrayList<ITesterExecutionControl>();

    private JPanel controlPanel = null;
    private JPanel tablePanel = null;
    private JTable orderTable = null;
    private JButton startStrategyButton = null;
    private JButton pauseButton = null;
    private JButton continueButton = null;
    private JButton cancelButton = null;
    private boolean cancelApp = false;

    private int maxStartedStrategies = 1;
    private int nbStartedStrategies;
    private int optimizationListIndex = 0;
    List<String> optimizationList = new ArrayList<String>();
    DateFormat dateFormat;
    DateFormat chronoFormat;
    Calendar cal;
    private long startTimeMillisec = 0;
    private long endTimeMillisec = 0;

    //url of the DEMO jnlp
    private static String jnlpUrl = "https://www.dukascopy.com/client/demo/jclient/jforex.jnlp";
    //user name
    private static String userName = "***";
    //password
    private static String password = "***";

    R_CurrencyIndexRobotFX20140102_mvn strategy1;
    LoadingProgressListener loadingProgressListener;

    public TesterMainGUIMode(List<String> _optimizationList) {

        optimizationList = _optimizationList;

        dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss.SSS", Locale.US);
        chronoFormat = new SimpleDateFormat("HH:mm:ss.SSS", Locale.US);
        cal = Calendar.getInstance();

        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        getContentPane().setLayout(new BoxLayout(getContentPane(), BoxLayout.Y_AXIS));
        showChartFrame();
    }

    @Override
    public void setChartPanels(Map<IChart, ITesterGui> chartPanels) {
        /* if (chartPanels != null && chartPanels.size() > 0) {

         IChart chart = chartPanels.keySet().iterator().next();
         Instrument instrument = chart.getInstrument();
         setTitle(instrument.toString() + " " + chart.getSelectedOfferSide() + " " + chart.getSelectedPeriod());

         JPanel chartPanel = chartPanels.get(chart).getChartPanel();
         addChartPanel(chartPanel);
         } */
    }

    @Override
    public void setExecutionControl(ITesterExecutionControl executionControl) {
        LOGGER.info("setExecutionControl");
        executionControlList.add(executionControl);
    }

    public void startStrategy() throws Exception {
        //get the instance of the IClient interface
        final ITesterClient client = TesterFactory.getDefaultInstance();

        //set the listener that will receive system events
        //
        client.setSystemListener(new ISystemListener() {
            @Override
            public void onStart(long processId) {

                cal = Calendar.getInstance();
                startTimeMillisec = cal.getTimeInMillis();
                LOGGER.info("Strategy started: " + processId + " - Date : " + dateFormat.format(startTimeMillisec));
                updateButtons();
            }

            @Override
            public void onStop(long processId) {

              //  RecursiveDelete.main(null);
                LOGGER.info("Strategy stopped strategy1 before :" + strategy1);
                LOGGER.info("Strategy stopped loadingProgressListener before :" + loadingProgressListener);
                loadingProgressListener = null;
                strategy1 = null;

                LOGGER.info("Strategy stopped strategy1 after :" + strategy1);
                LOGGER.info("Strategy stopped loadingProgressListener after :" + loadingProgressListener);

                cal = Calendar.getInstance();
                endTimeMillisec = cal.getTimeInMillis();
                LOGGER.info("Strategy stopped: " + processId + " - Date : " + dateFormat.format(endTimeMillisec));
                LOGGER.info("ProcessLength: " + chronoFormat.format(endTimeMillisec - startTimeMillisec));

                if (!cancelApp) {
                    nbStartedStrategies = client.getStartedStrategies().size();
                    if (nbStartedStrategies < maxStartedStrategies && optimizationListIndex < optimizationList.size()) {
                        startLoopStrategies(optimizationListIndex, client);
                    }
                } else {
                    resetButtons();
                    System.exit(0);
                    /* File reportFile = new File("C:\\report.html");
                     try {
                     client.createReport(processId, reportFile);
                     } catch (Exception e) {
                     LOGGER.error(e.getMessage(), e);
                     }*/
                    /*   if (client.getStartedStrategies().size() == 0) {
                     //Do nothing
                     }*/
                }
            }

            @Override
            public void onConnect() {
                LOGGER.info("Connected");
            }

            @Override
            public void onDisconnect() {
                //tester doesn't disconnect
            }
        });

        // setting the cacheFolder
        File CacheFolder = new File("W:/JForexCache/.cache");
        client.setCacheDirectory(CacheFolder);

        LOGGER.info("Connecting...");
        //connect to the server using jnlp, user name and password
        //connection is needed for data downloading
        client.connect(jnlpUrl, userName, password);

        //wait for it to connect
        int i = 10; //wait max ten seconds
        while (i > 0 && !client.isConnected()) {
            Thread.sleep(1000);
            i--;
        }
        if (!client.isConnected()) {
            LOGGER.error("Failed to connect Dukascopy servers");
            System.exit(1);
        }

        //set instruments that will be used in testing
        final Set<Instrument> instruments = new HashSet<Instrument>();
        instruments.add(Instrument.EURUSD);
        instruments.add(Instrument.AUDCAD);
        instruments.add(Instrument.AUDJPY);
        instruments.add(Instrument.AUDNZD);
        instruments.add(Instrument.AUDUSD);
        instruments.add(Instrument.CADJPY);
        instruments.add(Instrument.EURAUD);
        instruments.add(Instrument.EURCAD);
        instruments.add(Instrument.EURGBP);
        instruments.add(Instrument.EURJPY);
        instruments.add(Instrument.EURNZD);
        instruments.add(Instrument.GBPAUD);
        instruments.add(Instrument.GBPCAD);
        instruments.add(Instrument.GBPJPY);
        instruments.add(Instrument.GBPNZD);
        instruments.add(Instrument.GBPUSD);
        instruments.add(Instrument.NZDCAD);
        instruments.add(Instrument.NZDJPY);
        instruments.add(Instrument.NZDUSD);
        instruments.add(Instrument.USDCAD);
        instruments.add(Instrument.USDJPY);

        LOGGER.info("Subscribing instruments...");
        client.setSubscribedInstruments(instruments);

        //setting initial deposit
        client.setInitialDeposit(Instrument.EURUSD.getSecondaryCurrency(), 100000);

        Commissions commissions = client.getCommissions();

        SortedMap<Double, Double> depositLimits = commissions.getDepositLimits();
        SortedMap<Double, Double> equityLimits = commissions.getEquityLimits();
        SortedMap<Double, Double> turnoverLimits = commissions.getTurnoverLimits();

        // more elegant way
        for (Map.Entry<Double, Double> entry : depositLimits.entrySet()) {
            System.out.println("Key : " + entry.getKey() + " Value : " + entry.getValue());
            if (entry.getKey() == 5000.0) {
                entry.setValue(48.0);
            } else {
                entry.setValue(38.0);
            }
        }
        for (Map.Entry<Double, Double> entry : equityLimits.entrySet()) {
            System.out.println("Key : " + entry.getKey() + " Value : " + entry.getValue());
            entry.setValue(38.0);
        }

        for (Map.Entry<Double, Double> entry : turnoverLimits.entrySet()) {
            System.out.println("Key : " + entry.getKey() + " Value : " + entry.getValue());
            entry.setValue(38.0);
        }

        // Outputinng commission values
        for (Map.Entry<Double, Double> entry : depositLimits.entrySet()) {
            System.out.println("depositLimits Key : " + entry.getKey() + " Value : " + entry.getValue());
        }
        for (Map.Entry<Double, Double> entry : equityLimits.entrySet()) {
            System.out.println("equityLimits Key : " + entry.getKey() + " Value : " + entry.getValue());
        }
        for (Map.Entry<Double, Double> entry : turnoverLimits.entrySet()) {
            System.out.println("turnoverLimits Key : " + entry.getKey() + " Value : " + entry.getValue());
        }
        //
        final SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
        dateFormat.setTimeZone(TimeZone.getTimeZone("GMT"));

        Date dateFrom = dateFormat.parse(strDateFrom);
        Date dateTo = dateFormat.parse(strDateTo);

        client.setDataInterval(Period.ONE_MIN, OfferSide.BID, ITesterClient.InterpolationMethod.FOUR_TICKS, dateFrom.getTime(), dateTo.getTime());
        //client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, dateFrom.getTime(), dateTo.getTime());

        //load data
        LOGGER.info("Downloading data");
        Future<?> future = client.downloadData(null);
        //wait for downloading to complete
        future.get();
        //start the strategy
        //  LOGGER.info("Starting strategy");

        //workaround for LoadNumberOfCandlesAction for JForex-API versions > 2.6.64
        Thread.sleep(5000);

        for (int j = 0; j < maxStartedStrategies; j++) {
            startLoopStrategies(j, client);
        }
    }

    /**
     * Center a frame on the screen
     */
    private void centerFrame() {
        Toolkit tk = Toolkit.getDefaultToolkit();
        Dimension screenSize = tk.getScreenSize();
        int screenHeight = screenSize.height;
        int screenWidth = screenSize.width;
        setSize(screenWidth / 2, screenHeight / 2);
        setLocation(screenWidth / 4, screenHeight / 4);
    }

    /**
     * Add chart panel to the frame
     *
     * @param panel
     */
    private void addChartPanel(JPanel chartPanel) {
        removecurrentChartPanel();

        this.currentChartPanel = chartPanel;
        chartPanel.setPreferredSize(new Dimension(frameWidth, frameHeight - controlPanelHeight));
        chartPanel.setMinimumSize(new Dimension(frameWidth, 200));
        chartPanel.setMaximumSize(new Dimension(Short.MAX_VALUE, Short.MAX_VALUE));
        getContentPane().add(chartPanel);
        this.validate();
        chartPanel.repaint();
    }

    /**
     * Add buttons to start/pause/continue/cancel actions
     */
    private void addControlPanel() {

        controlPanel = new JPanel();
        FlowLayout flowLayout = new FlowLayout(FlowLayout.LEFT);
        controlPanel.setLayout(flowLayout);
        controlPanel.setPreferredSize(new Dimension(frameWidth, controlPanelHeight));
        controlPanel.setMinimumSize(new Dimension(frameWidth, controlPanelHeight));
        controlPanel.setMaximumSize(new Dimension(Short.MAX_VALUE, controlPanelHeight));

        startStrategyButton = new JButton("Start strategy");
        startStrategyButton.addActionListener(new ActionListener() {
            @Override
            public void actionPerformed(ActionEvent e) {
                startStrategyButton.setEnabled(false);
                Runnable r = new Runnable() {
                    public void run() {
                        try {
                            startStrategy();
                        } catch (Exception e2) {
                            LOGGER.error(e2.getMessage(), e2);
                            e2.printStackTrace();
                            startStrategyButton.setEnabled(true);
                        }
                    }
                };
                Thread t = new Thread(r);
                t.start();
            }
        });

        pauseButton = new JButton("Pause");
        pauseButton.addActionListener(new ActionListener() {
            @Override
            public void actionPerformed(ActionEvent e) {
                if (!executionControlList.isEmpty()) {
                    for (ITesterExecutionControl executionControlObj : executionControlList) {
                        executionControlObj.pauseExecution();
                    }
                    updateButtons();
                }
            }
        });

        continueButton = new JButton("Continue");
        continueButton.addActionListener(new ActionListener() {
            @Override
            public void actionPerformed(ActionEvent e) {
                if (!executionControlList.isEmpty()) {
                    for (ITesterExecutionControl executionControlObj : executionControlList) {
                        executionControlObj.continueExecution();
                    }
                    updateButtons();
                }
            }
        });

        cancelButton = new JButton("Cancel");
        cancelButton.addActionListener(new ActionListener() {
            @Override
            public void actionPerformed(ActionEvent e) {
                if (!executionControlList.isEmpty()) {
                    cancelApp = true;
                    for (ITesterExecutionControl executionControlObj : executionControlList) {
                        executionControlObj.cancelExecution();
                    }
                    updateButtons();
                }
            }
        });

        controlPanel.add(startStrategyButton);
        controlPanel.add(pauseButton);
        controlPanel.add(continueButton);
        controlPanel.add(cancelButton);
        getContentPane().add(controlPanel);

        pauseButton.setEnabled(false);
        continueButton.setEnabled(false);
        cancelButton.setEnabled(false);
    }

    private void updateButtons() {

        if (!executionControlList.isEmpty()) {
            startStrategyButton.setEnabled(executionControlList.get(0).isExecutionCanceled());
            pauseButton.setEnabled(!executionControlList.get(0).isExecutionPaused() && !executionControlList.get(0).isExecutionCanceled());
            cancelButton.setEnabled(!executionControlList.get(0).isExecutionCanceled());
            continueButton.setEnabled(executionControlList.get(0).isExecutionPaused());
        }
    }

    private void resetButtons() {
        startStrategyButton.setEnabled(false);
        pauseButton.setEnabled(false);
        continueButton.setEnabled(false);
        cancelButton.setEnabled(false);
    }

    private void removecurrentChartPanel() {
        if (this.currentChartPanel != null) {
            try {
                SwingUtilities.invokeAndWait(new Runnable() {
                    @Override
                    public void run() {
                        TesterMainGUIMode.this.getContentPane().remove(TesterMainGUIMode.this.currentChartPanel);
                        TesterMainGUIMode.this.getContentPane().repaint();
                    }
                });
            } catch (Exception e) {
                LOGGER.error(e.getMessage(), e);
            }
        }
    }

    public void showChartFrame() {
        setSize(frameWidth, frameHeight);
        centerFrame();
        addControlPanel();
        setVisible(true);
    }

    private void startLoopStrategies(int paramIndex, final ITesterClient _client) {

        String optimisationParameters = optimizationList.get(paramIndex);

        strategy1 = new R_CurrencyIndexRobotFX20140102_mvn();
        optimizationListIndex++;

        if (getExistingXMLFile(optimisationParameters, strategy1.destFolders[0]) == 0) {

            String[] listPlit = optimisationParameters.split("_");
            strategy1.usedPeriodsConfig = Integer.parseInt(listPlit[0]);
            strategy1.stopLossMultiplier = Double.parseDouble(listPlit[1]);
            strategy1.usedSLConfig = Integer.parseInt(listPlit[2]);
            strategy1.atrSLDEVMultiplier = Integer.parseInt(listPlit[3]);

            //start the strategy
            LOGGER.info("Starting strategy");

            loadingProgressListener = new LoadingProgressListener() {
                @Override
                public void dataLoaded(long startTime, long endTime, long currentTime, String information) {
                    // LOGGER.info(information + " - " + nbStartedStrategies);
                }

                @Override
                public void loadingFinished(boolean allDataLoaded, long startTime, long endTime, long currentTime) {
                }

                @Override
                public boolean stopJob() {
                    // LOGGER.info("4 - stopJob()");
                    return false;
                }
            };

            // _client.startStrategy(strategy1);
            _client.startStrategy(strategy1, loadingProgressListener, null, this, this, false);

        } else {
            nbStartedStrategies = _client.getStartedStrategies().size();
            if (nbStartedStrategies < maxStartedStrategies && optimizationListIndex < optimizationList.size()) {
                startLoopStrategies(optimizationListIndex, _client);
            } else {
                resetButtons();
                System.exit(0);
            }
        }

        //now it's running
    }

    //
    // Test if file allready exists
    private int getExistingXMLFile(String _optimisationParameters, String readFolder) {

        int ret = 0;

        // Searching for files in the directory
        File folder = new File(readFolder);
        File[] listOfFiles = folder.listFiles();

        // Searching if there is already a file in the folder named like the parameters
        int cpt = 0;
        boolean found = false;

        while (!found && cpt < listOfFiles.length) {

            String fileName = listOfFiles[cpt].getName();
            String currentParameters = "currencyIndex_" + _optimisationParameters + ".xml";

            LOGGER.info(fileName + " - " + currentParameters);

            if (fileName.equals(currentParameters)) {
                found = true;
                ret = 1;
            } else {
                found = false;
            }
            cpt++;
        }
        return ret;
    }
}

Thanks for the help.

Cosmo

I can confirm this bug with standalone api:

client.setDataInterval(Period.ONE_MIN, OfferSide.BID, InterpolationMethod.CLOSE_TICK, from.getMillis, to.getMillis)

Data range is last year from 6/1/2013 00:00:00 to 5/30/2014 23:59:59

nicofr0707

OK so I'm not the only one having this issue.

When I launch a few instances of the program in order to use all my
CPU cores, this problem is making Netbeans freeze because it
is overwhelming the output.

I hope support can provide us a solution rapidly.

Best regards

nicofr0707

Hi,

Does support have some workaround for this issue ?

best regards.

API Support

The issue has been registered.

nicofr0707

Thanks. Please let us know when this issue is resolved.

Best regards

nicofr0707

Support,

Is there a way to disable the printing of this error temporally while we are waiting
for the bug to be solved ?
Because the output window is making my backtests freeze in Netbeans and I
didn't found any way to disable it in the IDE(even if I close the output tab
it is still printing in the background)

Thanks

API Support

You need to set the logging level OFF for the package of the error-raising class.

nicofr0707

Hi support,

Quote:

You need to set the logging level OFF for the package of the error-raising class.

Please can you provide me with an example for my case ? I tried, but didn't manage to do this.

About this issue, I noticed this today :

Sometimes it shows this error and eventually it disappears after a hundred exceptions when dates are correctly formatted :

Quote:

2014-06-03 10:37:55.592 ERROR AbstractHTFlowDataCreator - to(2009-12-02 00:58:59:999) > last tick time (2009-12-01 23:59:56:295 1259711996295)
java.lang.IllegalArgumentException: to(2009-12-02 00:58:59:999) > last tick time (2009-12-01 23:59:56:295 1259711996295

BUT sometimes the program never manages to "come back to normal" and here we seem to enter an infinite Exception when it looks
like this (look at the value of the year : 292269055-12-02 16:47:04:192 !) :

Quote:

2014-06-19 10:26:59.042 ERROR AbstractHTFlowDataCreator - to(2012-12-15 06:58:59:999) > last tick time (292269055-12-02 16:47:04:192 -9223372036854775808)
java.lang.IllegalArgumentException: to(2012-12-15 06:58:59:999) > last tick time (292269055-12-02 16:47:04:192 -9223372036854775808)

This is a serious issue when backtesting a set of strategies with different parameters in a loop, as it is freezing all the processes
and forces a hard reboot of netbeans.

I'm not sure just disabling logging alone will help, any temporary workaround is welcomed.
Best regards

API Support

nicofr0707 wrote:

Please can you provide me with an example for my case ? I tried, but didn't manage to do this.

Open JForex-SDK\rc\log4j.properties file and add to the log filters the following:

#Filter tester logs
log4j.category.com.dukascopy.dds2.greed.agent.strategy.tester.dataload=OFF

If this does not "mute" this error message, then the property file is probably not in the classpath. To see how to do this, simply download the latest JForex-SDK and see in the pom.xml how resources get defined.
Please mind that "muting" error messages is dangerous, because shall there appear some flawed behaviour, you would not be informed about it.

nicofr0707

Hi,

Quote:

#Filter tester logs
log4j.category.com.dukascopy.dds2.greed.agent.strategy.tester.dataload=OFF

OK it does mute the logging for this error but does not resolve the fact that the tester is
sometimes freezing at the end of the backtest.

But I did find a temporary workaround by stopping the strategy with java one day before
the end of backtest day using LoadingProgressListener() :

loadingProgressListener = new LoadingProgressListener() {
                @Override
                public void dataLoaded(long startTime, long endTime, long currentTime, String information) {
                    //LOGGER.info("dataLoaded startTime : " + dateFormat.format(startTime) + " - endTime : " + dateFormat.format(endTime) + " - currentTime : " + dateFormat.format(currentTime) + " - information : " + information);
                    // Stop manually the strategy 1 day before end time to workaround Dukascopy's BUG
                    if ((currentTime + 86400000) >= endTime) {
                        for (ITesterExecutionControl executionControlObj : executionControlList) {
                            executionControlObj.cancelExecution();
                        }
                    }
                }

                @Override
                public void loadingFinished(boolean allDataLoaded, long startTime, long endTime, long currentTime) {
                   // LOGGER.info("loadingFinished startTime : " + dateFormat.format(startTime) + " - endTime : " + dateFormat.format(endTime) + " - currentTime : " + dateFormat.format(currentTime));
                }

                @Override
                public boolean stopJob() {
                    // LOGGER.info("4 - stopJob()");
                    return false;
                }
            };

This is OK for now, anyway please let us know when this is solved.

Thanks for your help.
Best regards.

nicofr0707

Hello support,

My last post does not resolve entirely the AbstractHTFlowDataCreator error.

It only resolves the error at the end of a backtest for the given strategy.

BUT, this error happens randomly in the middle of a backtest, sometimes
it can work for 8 hours sometimes the test crashes after 2 hours.

It looks like this :

Quote:

2014-06-25 10:25:00.018 ERROR AbstractHTFlowDataCreator - to(2009-11-27 17:58:59:999) > last tick time (292269055-12-02 16:47:04:192 -9223372036854775808)
java.lang.IllegalArgumentException: to(2009-11-27 17:58:59:999) > last tick time (292269055-12-02 16:47:04:192 -9223372036854775808)
at com.dukascopy.charts.data.datacache.customticks.AbstractLoadCustomTicksFromCandlesAction.validateFromTo(AbstractLoadCustomTicksFromCandlesAction.java:65)
at com.dukascopy.charts.data.datacache.customticks.AbstractLoadCustomTicksFromCandlesAction.<init>(AbstractLoadCustomTicksFromCandlesAction.java:59)
at com.dukascopy.charts.data.datacache.customticks.LoadFourTicksAction.<init>(LoadFourTicksAction.java:39)
at com.dukascopy.charts.data.datacache.FeedDataProvider.createCustomTicksLoadingAction(FeedDataProvider.java:2605)
at com.dukascopy.charts.data.datacache.FeedDataProvider.loadCustomTicksSynched(FeedDataProvider.java:2564)
at com.dukascopy.dds2.greed.agent.strategy.tester.dataload.AbstractHTFlowDataCreator.performDataLoad(AbstractHTFlowDataCreator.java:147)
at com.dukascopy.dds2.greed.agent.strategy.tester.dataload.AbstractHTFlowDataCreator.loadNextDataChunk(AbstractHTFlowDataCreator.java:137)
at com.dukascopy.dds2.greed.agent.strategy.tester.dataload.HTCandlesCreator.loadNextDataChunk(HTCandlesCreator.java:171)
at com.dukascopy.dds2.greed.agent.strategy.tester.dataload.HistoricalTesterDataManager$1.run(HistoricalTesterDataManager.java:193)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

I am running 4 instances of the program on 3 servers, with each having its own history data installed.

All 3 are freezing and I have to start again the java programs.
My strategy is near production mode, it is auto-optimizing itself every day, but with these tester issues I
can't even imagine using it live for clients !

Do you know why this error is happening ? Any other workaround ?

Please fix this quickly, it's a real pain for professional use.

Thanks.
Best regards.

API Support

We will take a look at this.

nicofr0707

Hi support,

To help you find what's wrong, since all strategies on all servers are freezing at
the same time, the only common thing they have is the Internet connection.

I have a good 30Mb cable connection, but it seems that sometimes I may have
some micro cuts I don't even notice. This seems to make the backtests freeze
and never manage to continue after reconnection.

I have 100% of the history cache installed on the hard drives. To remember you, I
use startStrategy with writeHTDataToFiles=false and ONE_MIN, FOUR_TICKS tester config.

To reproduce this you just have to launch a backtest then unplug the Ethernet cable and
re-plug it in, you'll see the strategy freezing for ever.

[edit] This morning, the sdk log file had a size of 140Gb ! wow

I hope this may help you.
Best regards

nicofr0707

Hi support.

OK I can confirm that this issue comes from micro cuts in the Internet connection.

I have deployed my strategy on my web server which has a very stable Internet connection
and I didn't had any problem.

However, a webserver with a lot of horsepower is expensive, so I prefer to use local hardware.

Is their a way you would allow some micro cuts with a longer timeout ? I believe most
of your users have DSL and cable connections which in essence aren't 100% reliable.

Best regards

nicofr0707

Hello,

Any news regarding this issue ? At this time I have to pay cloud servers
with very stable Internet connexions in order to avoid ItesterClient to crash.
But it is quite expensive to have the same calculation power of what I have locally.

Best regards.
Nicolas

API Support

The issue has been fixed and will be available with the release of JForex-API 2.9.12.