Dukascopy
 
 
Wiki JStore Search Login

Attention! Read the forum rules carefully before posting a topic.

    Try to find an answer in Wiki before asking a question.
    Submit programming questions in this forum only.
    Off topics are strictly forbidden.

Any topics which do not satisfy these rules will be deleted.

Test over separate periods
 Post subject: Test over separate periods Post rating: 0   New post Posted: Mon 25 Sep, 2017, 12:27 
User avatar

User rating: 2
Joined: Tue 11 Jul, 2017, 09:30
Posts: 24
Location: HungaryHungary
Hello,

I am testing a strategy on every tick, but because of this the historical tester is very slow in JForex platform. I want to test my strategy over some years. But most of the ticks are unnecessary, because I want only the ticks around events.

I would like to know if it is possible to modify the original TesterMain.java to load only ticks of particular time periods.
For example I want the ticks from
2017.01.10 11:00 to 2017.01.10 11:05
2017.02.08 12:10 to 2017.02.08 12:15
2017.03.15 03:00 to 2017.03.15 03:05
.
.
.

Can I somehow download and use only these times without downloading the others?

Thanks


 
 Post subject: Re: Test over separate periods Post rating: 0   New post Posted: Tue 26 Sep, 2017, 08:43 
User avatar

User rating:
Joined: Fri 31 Aug, 2007, 09:17
Posts: 6139
It can be done by using ITesterClient and JForex SDK project:
https://www.dukascopy.com/wiki/en/devel ... e-interval


 
 Post subject: Re: Test over separate periods Post rating: 0   New post Posted: Tue 26 Sep, 2017, 09:58 
User avatar

User rating: 2
Joined: Tue 11 Jul, 2017, 09:30
Posts: 24
Location: HungaryHungary
Thanks for the answer.

I tried to solve this with the ITesterClient methods, and I know that I can give custom time to the tester by client.setDataInterval. But I was able to set only one period at a time. There was a start and an end date. So if I have ~200 periods/events I have to run the test 200 times (if I am not wrong), and I will get 200 distinct reports instead of one.

What I want to achive to run one test, but skip downloading most of the ticks, because they make my tester very slow, and I want only to download some distinct smaller periods.

Image

So for example in the image: I want speed up testing by downloading only the green ticks on a test, because 99% of downloaded ticks are redundant for my test.

So can I somehow filter multiple smaller periods to download for one tester cycle?

I have an idea, and will test it: call client.startStrategy in a loop for all periods I want to include. And I think between loops I have to pass somehow all the parameters to client to begin the next loop with the parameters of the earlier loops (deposit, turnover, commission, closed orders . . .) If I am on the wrong path please tell, I don't want to spend hours on a wrong path :D

Thanks

// edit
as I see ITesterClient doesn't allow me to set too many parameters, at least deposite can be set


// edit
but ITesterReportData data = client.getReportData(id); containes a lot of info, and these can be summarized, unfortunatelly only has getters, not setters


Attachments:
Capture.JPG [42.84 KiB]
Downloaded 244 times
DISCLAIMER: Dukascopy Bank SA's waiver of responsability - Documents, data or information available on this webpage may be posted by third parties without Dukascopy Bank SA being obliged to make any control on their content. Anyone accessing this webpage and downloading or otherwise making use of any document, data or information found on this webpage shall do it on his/her own risks without any recourse against Dukascopy Bank SA in relation thereto or for any consequences arising to him/her or any third party from the use and/or reliance on any document, data or information found on this webpage.
 
 Post subject: Re: Test over separate periods Post rating: 0   New post Posted: Tue 26 Sep, 2017, 10:46 
User avatar

User rating:
Joined: Fri 31 Aug, 2007, 09:17
Posts: 6139
There can be several ways to speed up testing:
1) stop test instance when account equity has fallen below certain limit
2) download instrument data in advance and make sure that each test case does not redownload all of the data, this may involve finding enough free disk space because tick data can be quite large
3) make sure that strategies don't do work with ticks and bars that are not used


 
 Post subject: Re: Test over separate periods Post rating: 0   New post Posted: Tue 26 Sep, 2017, 14:53 
User avatar

User rating: 2
Joined: Tue 11 Jul, 2017, 09:30
Posts: 24
Location: HungaryHungary
Thanks,

I use only one period in onFeed and the onTick, where I have optional returns to avoid unwanted calculation. Maybe I can't decrees the testing time more, but I am investigating my earlier idea. If I have some success I will share here, maybe somebody is interested.


 
 Post subject: Re: Test over separate periods Post rating: 0   New post Posted: Wed 27 Sep, 2017, 12:09 
User avatar

User rating: 2
Joined: Tue 11 Jul, 2017, 09:30
Posts: 24
Location: HungaryHungary
If somebody is interested:

I altered the original TesterMain.java for faster event-trading tests. With this you will not download all the ticks, you will download only the ticks around the time of events. It makes the test really faster if you aim small time periods, but far from each other. But if you aim large periods this will not help.

There are only example dates in it, but you can implement your CSV reader or database reader - this is only an example code.

Here all small timeperiod are distinct runs, but they are launched when the earlier is finished so the parameters can be passed to the next one. At the end it shows the final statistics. Could be more fast if I wouldn't wait the earlier run to finish, because the different dates could run paralel, but this way I couldn't pass the finished deposite.

I will try to implement the optimization feature in it . . . later.

Not the most beautiful code, and if anybody finds somthing logically wrong, please tell.

package singlejartest;

import com.dukascopy.api.IOrder;
import com.dukascopy.api.Instrument;
import com.dukascopy.api.LoadingProgressListener;
import com.dukascopy.api.system.ISystemListener;
import com.dukascopy.api.system.ITesterClient;
import com.dukascopy.api.system.ITesterReportData;
import com.dukascopy.api.system.TesterFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.File;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashSet;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.TimeZone;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
import java.util.logging.Level;

/**
 * This small program demonstrates how to initialize Dukascopy tester and start a strategy
 */
public class MyTesterMain {
    private static final Logger LOGGER = LoggerFactory.getLogger(Main.class);

    //url of the DEMO jnlp
    private static String jnlpUrl = "https://platform.dukascopy.com/demo/jforex.jnlp";
    //user name
    private static String userName = "user";
    //password
    private static String password = "pass";
   

    public static void main(String[] args) throws Exception {
       
        Set<Instrument> instruments;
        final SimpleDateFormat dateFormat = new SimpleDateFormat("MM/dd/yyyy HH:mm:ss");
        dateFormat.setTimeZone(TimeZone.getTimeZone("GMT"));       
        final MyReportData summaReportData = new MyReportData(); // the summarized report
        ITesterReportData singleReportData;
        Map<Date,Date> eventPeriods = new LinkedHashMap(); // report for only one period
       
        //setting example dates - begin/end - can be imported from CSV... in real scenario
        eventPeriods.put(dateFormat.parse("11/14/2014 12:00:00"), dateFormat.parse("11/14/2014 23:15:00"));
        eventPeriods.put(dateFormat.parse("04/23/2015 12:00:00"), dateFormat.parse("04/23/2015 23:15:00"));
        eventPeriods.put(dateFormat.parse("03/03/2016 12:00:00"), dateFormat.parse("03/03/2016 23:15:00"));
        eventPeriods.put(dateFormat.parse("05/05/2017 12:00:00"), dateFormat.parse("05/05/2017 23:15:00"));
        final Iterator<Map.Entry<Date, Date>> dateEntriesIterator = eventPeriods.entrySet().iterator();           
       
        //get the instance of the IClient interface
        final ITesterClient client = TesterFactory.getDefaultInstance();
       
        //set the listener that will receive system events
        client.setSystemListener(new ISystemListener() {
            @Override
            public void onStart(long processId) {
                LOGGER.info("Strategy started: " + processId);
            }

            @Override
            public void onStop(long processId) {
                // saves to file every single strategy run - don't use if, if you don't want all the small details
                makeSingleReportFile(client, processId);
                updateSummaReport(client, processId, summaReportData);
                launchNextDateorExit(client, summaReportData, dateEntriesIterator);
            }

            @Override
            public void onConnect() {
                LOGGER.info("Connected");
            }

            @Override
            public void onDisconnect() {
                //tester doesn't disconnect
            }
        });

        LOGGER.info("Connecting...");
        //connect to the server using jnlp, user name and password
        //connection is needed for data downloading
        client.connect(jnlpUrl, userName, password);

        //wait for it to connect
        int i = 10; //wait max ten seconds
        while (i > 0 && !client.isConnected()) {
            Thread.sleep(1000);
            i--;
        }
        if (!client.isConnected()) {
            LOGGER.error("Failed to connect Dukascopy servers");
            System.exit(1);
        }

        //set instruments that will be used in testing
        instruments = new HashSet<>();
        instruments.add(Instrument.EURUSD);
        LOGGER.info("Subscribing instruments...");
        client.setSubscribedInstruments(instruments);
       
        //setting initial deposit
        client.setInitialDeposit(Instrument.EURUSD.getSecondaryJFCurrency(), 50000);
       
        // the first time we have to call the first strategy explicitly
        if(dateEntriesIterator.hasNext()){
            Map.Entry<Date, Date> dateEntry = dateEntriesIterator.next();
            setUpStrategy(client, dateEntry, summaReportData);
        }
               
    } // end Main

   
    private static void setUpStrategy(ITesterClient client, Map.Entry<Date, Date> dateEntry, MyReportData summaReportData) {
       
            // set initial deposit from the final deposite of the earlier run
            if(summaReportData.getParameterValues() == null) { // if parameters aren't set: this is the first run, set the initial deposit
                client.setInitialDeposit(Instrument.EURUSD.getSecondaryJFCurrency(), 50000); // here comes the initial deposit
            } else {
                client.setInitialDeposit(Instrument.EURUSD.getSecondaryJFCurrency(), summaReportData.getFinishDeposit()); // parameters are set, so this isn't the first run, get the last runs finishedDeposit
            }
           
            // set date
            Date dateFrom = dateEntry.getKey();
            Date dateTo = dateEntry.getValue();           
            client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, dateFrom.getTime(), dateTo.getTime());

            //load data
            LOGGER.info("Downloading data");
            Future<?> future = client.downloadData(null);
            try {
                //wait for downloading to complete
                future.get();
            } catch (InterruptedException ex) {
                java.util.logging.Logger.getLogger(MyTesterMain.class.getName()).log(Level.SEVERE, null, ex);
            } catch (ExecutionException ex) {
                java.util.logging.Logger.getLogger(MyTesterMain.class.getName()).log(Level.SEVERE, null, ex);
            }

            //start the strategy if there isn't other running strategy
                LOGGER.info("Starting strategy");
                client.startStrategy(new MA_Play(), new LoadingProgressListener() {

                    @Override
                    public void dataLoaded(long startTime, long endTime, long currentTime, String information) {
                        LOGGER.info(information);
                    }

                    @Override
                    public void loadingFinished(boolean allDataLoaded, long startTime, long endTime, long currentTime) {
                    }

                    @Override
                    public boolean stopJob() {
                        return false;
                    }
                });       
    } // end setUpStrategy
   
    // saves to file every single strategy run
    private static void makeSingleReportFile(ITesterClient client, long processId){
                LOGGER.info("Strategy stopped: " + processId);
                File reportFile = new File("C:\\report" + processId + ".html");
                try {
                    client.createReport(processId, reportFile);
                } catch (Exception e) {
                    LOGGER.error(e.getMessage(), e);
                }       
    }
   
    // update summaReportData after every strategy run
    private static void updateSummaReport(ITesterClient client, long processId, MyReportData summaReportData){
        if(processId == 1){ // update the strategy parameters and initial deposit only once
            summaReportData.setParameterValues(client.getReportData(processId).getParameterValues());
            summaReportData.setInitialDeposit(client.getReportData(processId).getInitialDeposit());
        }
       
        summaReportData.setFinishDeposit(client.getReportData(processId).getFinishDeposit());
        summaReportData.addToTurnover(client.getReportData(processId).getTurnover());
        summaReportData.addToCommission(client.getReportData(processId).getCommission());
        summaReportData.addToSubRuns();
    }
   
    // checks if new strategy can be launched for the new date or close exit
    private static void launchNextDateorExit(ITesterClient client, MyReportData summaReportData, Iterator<Map.Entry<Date, Date>> dateEntriesIterator){
        if(client.getStartedStrategies().isEmpty() && dateEntriesIterator.hasNext()){
            Map.Entry<Date, Date> dateEntry = dateEntriesIterator.next();
            setUpStrategy(client, dateEntry, summaReportData);
        } else if (client.getStartedStrategies().isEmpty()) {
            LOGGER.info("FINAL STATISTICS: ");
            LOGGER.info("INITIAL DEPOSIT: " + summaReportData.getInitialDeposit());
            LOGGER.info("FINISHED DEPOSIT: " + summaReportData.getFinishDeposit());
            LOGGER.info("TURNOVER: " + summaReportData.getTurnover());
            LOGGER.info("COMMISSION: " + summaReportData.getCommission());
            LOGGER.info("SUBRUNS: " + summaReportData.getSubRuns());
            System.exit(0);
        }       
    }
   
    // Update the Report data on eventloop
    static class MyReportData{
        private List<String[]> parameterValues;
        private double commission;
        private double turnover;
        private double initialDeposit;
        private double finishDeposit;       
        private List<IOrder> openOrders;
        private List<IOrder> closedOrders;
        private int subRuns;

        public MyReportData() {
            this.parameterValues = null;
            this.commission = 0;
            this.turnover = 0;
            this.initialDeposit = 0;
            this.finishDeposit = 0;
            this.openOrders = null;
            this.closedOrders = null;
            this.subRuns = 0;
        }

        public List<String[]> getParameterValues() {
            return parameterValues;
        }

        public void setParameterValues(List<String[]> parameterValues) {
            this.parameterValues = parameterValues;
        }

        public double getCommission() {
            return commission;
        }
        public void setCommission(double commission) {
            this.commission = commission;
        }
       
        public void addToCommission(double commission) {
            this.commission += commission;
        }       

        public double getTurnover() {
            return turnover;
        }
        public void setTurnover(double turnover) {
            this.turnover = turnover;
        }
       
        public void addToTurnover(double turnover) {
            this.turnover += turnover;
        }       

        public double getInitialDeposit() {
            return initialDeposit;
        }

        public void setInitialDeposit(double initialDeposit) {
            this.initialDeposit = initialDeposit;
        }

        public double getFinishDeposit() {
            return finishDeposit;
        }
        public void setFinishDeposit(double finishDeposit) {
            this.finishDeposit = finishDeposit;
        }

        public List<IOrder> getOpenOrders() {
            return openOrders;
        }
        public void addToOpenOrders(IOrder openOrder) {
            this.openOrders.add(openOrder);
        }

        public List<IOrder> getClosedOrders() {
            return closedOrders;
        }
        public void addToClosedOrders(IOrder closedOrder) {
            this.closedOrders.add(closedOrder);
        }

        public int getSubRuns() {
            return subRuns;
        }
       
        public void addToSubRuns() {
            this.subRuns += 1;
        }
       
    } // end MyReportData

} // END CLASS MyTesterMain


 

Jump to:  

cron
  © 1998-2025 Dukascopy® Bank SA
On-line Currency forex trading with Swiss Forex Broker - ECN Forex Brokerage,
Managed Forex Accounts, introducing forex brokers, Currency Forex Data Feed and News
Currency Forex Trading Platform provided on-line by Dukascopy.com