Dukascopy
 
 
Wiki JStore Search Login

Attention! Read the forum rules carefully before posting a topic.

    Try to find an answer in Wiki before asking a question.
    Submit programming questions in this forum only.
    Off topics are strictly forbidden.

Any topics which do not satisfy these rules will be deleted.

ITesterClient.connect() sometimes causes IHistory.getLastTick() to start failing until a new chart is opened
 Post subject: Re: What does it mean if an instrument is subscribed but "not opened"? Post rating: 0   New post Posted: Tue 12 Mar, 2013, 15:49 
User avatar

User rating: 6
Joined: Thu 19 May, 2011, 11:14
Posts: 235
Location: South Africa,
Thank you for unlocking this thread again.
API Support wrote:
We can't proceed without an example program which replicates the case and precise launch conditions.

  1. There is no set of "precise launch conditions" that will reliably induce the bug. Sometimes it happens, sometimes it doesn't, without any apparent changes in launch conditions.
  2. Even if I gave you my program (almost 30k lines of code) you may never be able to reproduce the bug (even unreliably) because your connection with the servers has different characteristics (latency, speed, warp) from mine here in South Africa.

Nevertheless, I will see if I can cut out a minimal skeleton from my program for you that still induces the bug sometimes.
However, given the unpredictable nature of the bug, this is a long-term undertaking; I don't know how long it will take.

In the mean time...

  1. If you put more diagnostic information into that exception thrown by getLastTick (with a view towards exploring why it disagrees with getSubscribedInstruments()), then I can report that diagnostic info to you the next time the bug happens, and this process may lead you to finding the bug.
  2. We already have information about the bug that could be valuable:
    Let state A be the state in which the API does not exhibit the bug, and state B the state in which the API exhibits the bug.
    • A->B happens unpredictably, but is always accompanied by Java console messages of the form
      "WARNING connect.JForexTaskManager ] Got tick for instrument [EUR/USD] that was not processed by FeedDataProvider..."
      These messages actually seem to contain a lot of diagnostic state information (which was truncated here but can be found in earlier posts).
    • B->A happens by user intervention. The transition can be induced by:
      • Opening a new chart of the relevant instrument.
      • Subscribing/unsubscribing to any instrument.


 
 Post subject: Re: What does it mean if an instrument is subscribed but "not opened"? Post rating: 0   New post Posted: Wed 24 Apr, 2013, 13:45 
User avatar

User rating: 6
Joined: Thu 19 May, 2011, 11:14
Posts: 235
Location: South Africa,
New information:
After some trial and error it has become evident that A->B (when it occurs occasionally) is induced by TesterFactory.getDefaultInstance().connect()

Debug.trace("Tester client connecting...");
try{history.getLastTick(Instrument.EURUSD);}catch(JFException ex){Debug.error("Checkpoint C: " + ex);} // diagnostic for bug https://www.dukascopy.com/swiss/english/forex/jforex/forum/viewtopic.php?f=65&t=48927
//connection is needed for data downloading
TesterFactory.getDefaultInstance().connect(
                  "https://www.dukascopy.com/client/demo/jclient/jforex.jnlp",
                  preambleLogin, preamblePassword); // let it throw
try{history.getLastTick(Instrument.EURUSD);}catch(JFException ex){Debug.error("Checkpoint D: " + ex);} // diagnostic for bug https://www.dukascopy.com/swiss/english/forex/jforex/forum/viewtopic.php?f=65&t=48927

...results in...
Strategy Messages wrote:
2013-04-24 12:34:46 TRACE: Tester client connecting...
2013-04-24 12:34:55 ERROR: Checkpoint D: com.dukascopy.api.JFException: Instrument [EUR/USD] not opened

...along with...
Java Console wrote:
24.04.2013 14:34:54.977 WARNING connect.JForexTaskManager ] Got tick for instrument [EUR/USD] that was not processed by FeedDataProvider... Instrument subscription status [false] MarketState [{"tlb":"199.46","apa":"1.30001","a":"1.29986,10.95,0.0,1.29987,4.97,0.0,1.29988,13.5,0.0,1.29989,20.63,0.0,1.29990,24.0,0.0,1.29991,7.13,0.0,1.29992,5.25,0.0,1.29995,3.2,0.0,1.29996,
13.5,0.0,1.29998,52.23,0.0","t":"1366806895940","apb":"1.29967","b":"1.29982,1.13,0.0,1.29981,3.3,0.0,1.29980,7.5,0.0,1.29979,10.33,0.0,1.29978,17.45,0.0,1.29977,12.2,0.0,1.29976,
3.75,0.0,1.29975,19.33,0.0,1.29974,5.63,0.0,1.29973,37.83,0.0","pip":"0.0001","i":"EUR/USD","type":"marketState","tla":"234.38"}]


In other words: The call to TesterFactory.getDefaultInstance().connect() seems to (occasionally and unpredictably) cause history.getLastTick() to start failing (with "JFException: Instrument [EUR/USD] not opened.") even while EUR/USD is subscribed (according to getSubscribedInstruments()).


 
 Post subject: Re: What does it mean if an instrument is subscribed but "not opened"? Post rating: 0   New post Posted: Mon 13 May, 2013, 22:52 
User avatar

User rating: 6
Joined: Thu 19 May, 2011, 11:14
Posts: 235
Location: South Africa,
API Support wrote:
We can't proceed without an example program which replicates the case and precise launch conditions.

Run the following strategy in the JForex client.
package isak.dukamatic.jfs;

import com.dukascopy.api.*;
import com.dukascopy.api.INewsMessage.Action;
import com.dukascopy.api.system.TesterFactory;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.PrintStream;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;

@RequiresFullAccess
public class BugInducingStrategy implements IStrategy {

    @Configurable("Preamble login")
    public String preambleLogin = "DEMO2fDYoE";
    @Configurable("Preamble password")
    public String preamblePassword = "fDYoE";

    private IHistory history;
    private IAccount account;
    private IConsole console;

    private final SimpleDateFormat dateFormat
            = new SimpleDateFormat("yyyy.MM.dd HH:mm:ss.SSS"){{setTimeZone(TimeZone.getTimeZone("GMT"));}};

    public void onTick(Instrument i, ITick t) throws JFException{
        history.getLastTick(Instrument.EURUSD);
    }
    public void onMessage(IMessage im){
       
    }
    public void onAccount(IAccount ia){
       
    }
    public void onStop(){
       
    }
    public void onBar(Instrument i, Period p, IBar b, IBar a){
       
    }

    @Override
    public void onStart(final IContext context) {
            history = context.getHistory();
            account = context.getAccount();
            console = context.getConsole();
           
            console.getOut().println("Checkpoint A: Subscribed to " + context.getSubscribedInstruments() + ".");

            //connection is needed for preamble data downloading
            console.getOut().println("FoundationJfs: Tester client connecting...");
            try {
                try{history.getLastTick(Instrument.EURUSD);}catch(JFException ex){console.getErr().println("Checkpoint B: Subscribed to " + context.getSubscribedInstruments() + ". getLastTick(EURUSD) threw " + ex);}
// diagnostic for bug https://www.dukascopy.com/swiss/english/forex/jforex/forum/viewtopic.php?f=65&t=48927
                TesterFactory.getDefaultInstance().connect(
                        "https://www.dukascopy.com/client/demo/jclient/jforex.jnlp",
                        preambleLogin, preamblePassword);
                try{history.getLastTick(Instrument.EURUSD);}catch(JFException ex){console.getErr().println("Checkpoint C: Subscribed to " + context.getSubscribedInstruments() + ". getLastTick(EURUSD) threw " + ex);}
// diagnostic for bug https://www.dukascopy.com/swiss/english/forex/jforex/forum/viewtopic.php?f=65&t=48927
                console.getOut().println("... connect succeeded");
            } catch (Exception ex) {
                context.getConsole().getWarn().println("... connect failed: " + ex);
                //throw new RuntimeException("Failed to connect TesterFactory: " + ex, ex);
            }
    }
}

The above sometimes generates:
Strategy Console (in reverse chronological order) wrote:
2013-05-13 21:38:06 ... connect succeeded
2013-05-13 21:38:06 FoundationJfs: Tester client connecting...
2013-05-13 21:38:06 Checkpoint A: Subscribed to [EUR/USD].

which is the expected output, and sometimes
Strategy Console (in reverse chronological order) wrote:
2013-05-13 21:28:33 ... connect succeeded
2013-05-13 21:28:33 Checkpoint C: Subscribed to [EUR/USD]. getLastTick(EURUSD) threw com.dukascopy.api.JFException: Instrument [EUR/USD] not opened
2013-05-13 21:28:21 FoundationJfs: Tester client connecting...
2013-05-13 21:28:21 Checkpoint A: Subscribed to [EUR/USD].

which is not expected.

The problem is summarized in the red line: Why does getLastTick() think (sometimes) that the instrument is not opened, when in fact it is subscribed?


 
 Post subject: Re: ITesterClient.connect() sometimes causes IHistory.getLastTick() to start failing until a new chart is opened Post rating: 0   New post Posted: Mon 20 May, 2013, 09:37 
User avatar

User rating:
Joined: Fri 31 Aug, 2007, 09:17
Posts: 6139
On ITesterClient.connect initializes the environment for strategy launching, which includes subscription to instruments.
So basically you reset the feed and instrument subscription from a running strategy by using a functionality which is not designed to be used from the strategy.
Therefore also the indeterministic behaviour of the functionality.


 
 Post subject: Re: ITesterClient.connect() sometimes causes IHistory.getLastTick() to start failing until a new chart is opened Post rating: 0   New post Posted: Mon 20 May, 2013, 14:02 
User avatar

User rating: 6
Joined: Thu 19 May, 2011, 11:14
Posts: 235
Location: South Africa,
API Support wrote:
So basically you reset the feed and instrument subscription...
when all I intended was for ITesterClient to...
Authenticate and connect to dukascopy servers.

So now I have a problem because I don't know what hidden assumptions the API is making about my program.


 
 Post subject: Re: ITesterClient.connect() sometimes causes IHistory.getLastTick() to start failing until a new chart is opened Post rating: 0   New post Posted: Mon 20 May, 2013, 14:27 
User avatar

User rating:
Joined: Fri 31 Aug, 2007, 09:17
Posts: 6139
There are no hidden assumptions - the ITesterClient.connect authenticates, connects to servers and sets up the environment for strategy run in historical testing mode. We reiterate that any ITesterClient method calls from within the strategy are against the JForex-API design and their behaviour from within the strategy id not determined. If you lack some functionality that is meant for strategies please make a feature request.


 
 Post subject: Re: ITesterClient.connect() sometimes causes IHistory.getLastTick() to start failing until a new chart is opened Post rating: 0   New post Posted: Mon 20 May, 2013, 15:23 
User avatar

User rating: 6
Joined: Thu 19 May, 2011, 11:14
Posts: 235
Location: South Africa,
API Support wrote:
There are no hidden assumptions... ...We reiterate that any ITesterClient method calls from within the strategy are against the JForex-API design

But that is a hidden assumption. Some strategies may want to run backtests as part of their decision-making process. Nowhere does ITesterClient's API doc mention that its methods should not be called from a strategy.


 
 Post subject: Re: ITesterClient.connect() sometimes causes IHistory.getLastTick() to start failing until a new chart is opened Post rating: 0   New post Posted: Wed 22 May, 2013, 07:38 
User avatar

User rating: 6
Joined: Thu 19 May, 2011, 11:14
Posts: 235
Location: South Africa,
API Support wrote:
There are no hidden assumptions... ...We reiterate that any ITesterClient method calls from within the strategy are against the JForex-API design and their behaviour from within the strategy id not determined.

So then it is okay to call ITesterClient methods while a strategy is running, as long as they are not called from within a strategy?

But how do the ITesterClient methods even know where they are being called from?


 
 Post subject: Re: ITesterClient.connect() sometimes causes IHistory.getLastTick() to start failing until a new chart is opened Post rating: 0   New post Posted: Fri 24 May, 2013, 16:24 
User avatar

User rating:
Joined: Fri 31 Aug, 2007, 09:17
Posts: 6139
Isak wrote:
Some strategies may want to run backtests as part of their decision-making process.
This would effectively be like setting up Historical Tester from within a strategy.
Isak wrote:
So then it is okay to call ITesterClient methods while a strategy is running, as long as they are not called from within a strategy?
It is ok to call the methods which have their equivalent Historical Tester functionalities in JForex client, that are available during the strategy historical run. Other method calls may lead to indeterministic results and may interfere with strategy run environment.


 
 Post subject: Re: ITesterClient.connect() sometimes causes IHistory.getLastTick() to start failing until a new chart is opened Post rating: 0   New post Posted: Fri 24 May, 2013, 20:24 
User avatar

User rating: 6
Joined: Thu 19 May, 2011, 11:14
Posts: 235
Location: South Africa,
API Support wrote:
This would effectively be like setting up Historical Tester from within a strategy.
Yes. The strategy formulates hypotheses inspired by the current market, then wants to backtest the hypotheses before betting on them. Judging from the API docs this shouldn't be impossible. One would expect ITesterClient (like any reusable software component) to be user-indifferent.

API Support wrote:
There are no hidden assumptions... ...We reiterate that any ITesterClient method calls from within the strategy are against the JForex-API design and their behaviour from within the strategy id not determined.
Isak wrote:
So then it is okay to call ITesterClient methods while a strategy is running, as long as they are not called from within a strategy?
But how do the ITesterClient methods even know where they are being called from?
API Support wrote:
It is ok to call the methods which...
Is it ok to call ITesterClient.connect (while a strategy is running but not from within a strategy)?


 
 Post subject: Re: ITesterClient.connect() sometimes causes IHistory.getLastTick() to start failing until a new chart is opened Post rating: 0   New post Posted: Mon 27 May, 2013, 09:52 
User avatar

User rating:
Joined: Fri 31 Aug, 2007, 09:17
Posts: 6139
Isak wrote:
Is it ok to call ITesterClient.connect (while a strategy is running but not from within a strategy)?
No.


 
 Post subject: Re: ITesterClient.connect() sometimes causes IHistory.getLastTick() to start failing until a new chart is opened Post rating: 0   New post Posted: Wed 29 May, 2013, 19:29 
User avatar

User rating: 6
Joined: Thu 19 May, 2011, 11:14
Posts: 235
Location: South Africa,
Isak wrote:
...So now I have a problem because I don't know what hidden assumptions the API is making about my program.
API Support wrote:
There are no hidden assumptions... ...We reiterate that any ITesterClient method calls from within the strategy are against the JForex-API design and their behaviour from within the strategy id not determined.
Isak wrote:
Is it ok to call ITesterClient.connect (while a strategy is running but not from within a strategy)?
API Support wrote:
No.
So the JFAPI assumes that ITesterClient.connect will not be called (from anywhere) while a strategy is running (in any IClient). Is that correct?

Supposing A and B are both of (polymorphic) type IClient. In general, calling A.connect() has an (undocumented side-)effect on B, and calling B.connect() has an (undocumented side-)effect on A. Is that correct?


 

Jump to:  

  © 1998-2026 Dukascopy® Bank SA
On-line Currency forex trading with Swiss Forex Broker - ECN Forex Brokerage,
Managed Forex Accounts, introducing forex brokers, Currency Forex Data Feed and News
Currency Forex Trading Platform provided on-line by Dukascopy.com