|
JFOREX-3422 onBar doesn't get always called for 30min ? |
FXace
|
Post subject: JFOREX-3422 onBar doesn't get always called for 30min ? |
Post rating: 0
|
Posted: Tue 03 Jan, 2012, 20:54
|
|
User rating: 0
Joined: Tue 05 Jul, 2011, 15:21 Posts: 28 Location: Switzerland,
|
Dear API Support,
could you please help me with the following problem:
1. I'm using standalone API with Eclipse, Windows 7 2. in general I want to run a alert service every half an hour, looking into events only in 30min and 4h timeframes. I'm using an implementation of ITesterClient in order to get only historical data. Batch file calling the exported .jar is being called every 30 minutes by Windows Task Scheduler, starting always 1 minute after 00/30 minutes (i.e 7:01, 7:31 etc). 3. to ensure only last 30min bar is called, I set the time interval in call to client.setDataInterval to end times of the last two 30min bars, see attached code 4. the problem is that onBar method of the IStrategy implementation is NOT being called for 30min Period each and every time Task Scheduler invokes the batch file. For example, today between 7:00 and 20:00 CET it got called only 7 times (7:30, 11:00, 12:31, 13:30 and then from 19:30 each time. For 4 Hours Period it didn't get called at all, despite it should have been at 9:00, 13:00 and 17:00.
If I run the same batch by hand from command line it usually works correctly and onBar gets invoked for 30min Period. This is what I did today around 19:15 - first two times there was the same problem but afterwards it was OK. Interestingly enough after that up until now I got 3 successfull calls.
When running the batch file by hand I do observe on console 2012-01-03 19:27:48.824 WARN CurvesJsonProtocolHandler - Disconnected from data feed server, reason - AUTHORIZATION_TIMEOUT 2012-01-03 19:27:48.884 INFO CurvesJsonProtocolHandler - Connecting to data feed server [d-jd-gva-94-161.dukascopy.com/194.8.15.161:10443] attempt [2] ... but afterwards as mentioned application works correctly
Attached are sample log files (successful calls have entry "Entered onBar AFTER filtering for timeframe") and the code to demonstrate the problem. Please let me know how to get this working reliably.
I've observed such instability in last 3-4 weeks (3-4), before that such setup was working reliably.
Thanks in advance !
P.S. And a fresh example of such problem this morning (04.01). Occured after 5 correct runs
Attachments: |
File comment: 04.01.12 09:34 CET run
sample log - no 30min entry 2.txt.txt [37.76 KiB]
Downloaded 360 times
|
sample log - no 30min entry.txt [35.99 KiB]
Downloaded 348 times
|
sample log - 30min entry.txt [38.62 KiB]
Downloaded 342 times
|
CalledStrategy.java [2.85 KiB]
Downloaded 356 times
|
Strategy30MinRepeat.java [8.15 KiB]
Downloaded 352 times
|
DISCLAIMER: Dukascopy Bank SA's waiver of responsability - Documents, data or information available on
this webpage may be posted by third parties without Dukascopy Bank SA being obliged to make any control
on their content. Anyone accessing this webpage and downloading or otherwise making use of any document,
data or information found on this webpage shall do it on his/her own risks without any recourse against
Dukascopy Bank SA in relation thereto or for any consequences arising to him/her or any third party from
the use and/or reliance on any document, data or information found on this webpage.
|
|
|
|
|
 |
API Support
|
Post subject: Re: onBar doesn't get always called in ITesterClient for 30m |
Post rating: 0
|
Posted: Wed 04 Jan, 2012, 10:28
|
|
User rating: ∞
Joined: Fri 31 Aug, 2007, 09:17 Posts: 6139
|
FXace wrote: 4. the problem is that onBar method of the IStrategy implementation is NOT being called for 30min Period each and every time Task Scheduler invokes the batch file. For example, today between 7:00 and 20:00 CET it got called only 7 times (7:30, 11:00, 12:31, 13:30 and then from 19:30 each time. For 4 Hours Period it didn't get called at all, despite it should have been at 9:00, 13:00 and 17:00. Find in attachments a strategy and a program running it that reproduce the scenario. However we did not manage to reproduce the error. In general, if there are some errors on server (e.g. connectivity problems, data feed server problems etc.) it may result in incomplete data on your local cache. In such case, please consider deleting your local cache directory (in whole or some subfolders) and try launching the strategy again - the cache will be downloaded all over new.
Attachments: |
Strategy30MinRepeat2.java [5.66 KiB]
Downloaded 334 times
|
CalledStrategy3.java [2.36 KiB]
Downloaded 368 times
|
DISCLAIMER: Dukascopy Bank SA's waiver of responsability - Documents, data or information available on
this webpage may be posted by third parties without Dukascopy Bank SA being obliged to make any control
on their content. Anyone accessing this webpage and downloading or otherwise making use of any document,
data or information found on this webpage shall do it on his/her own risks without any recourse against
Dukascopy Bank SA in relation thereto or for any consequences arising to him/her or any third party from
the use and/or reliance on any document, data or information found on this webpage.
|
|
|
|
|
 |
FXace
|
Post subject: Re: onBar doesn't get always called in ITesterClient for 30m |
Post rating: 0
|
Posted: Wed 04 Jan, 2012, 11:52
|
|
User rating: 0
Joined: Tue 05 Jul, 2011, 15:21 Posts: 28 Location: Switzerland,
|
Hello API Support, thanks for the quick answer. Do I understand correctly that onBar doesn't get called for Periods for which data is incomplete on my local cache (due to errors on server) ? Please confirm. In order to reproduce the problem you'd need to set up scheduled task to run and then observe log files after at least a day or so since it doesn't appear immediatelly. Deleting my local cache directory and launching the strategy again by hand is not the preferred option since the whole idea here is to get automatic alerts when not sitting in front of the computer and checking the charts each 30 min. I'd like to be able to ensure that onBar gets called for desired Periods (30 min and 4 hours) by automatically restarting the strategy if necessary. The most promissing way seems to be using onStop method of the attached ISystemListener, something like this: int allowedTries = 0; final ITesterClient client = TesterFactory.getDefaultInstance(); //set the listener that will receive system events client.setSystemListener(new ISystemListener() { @Override public void onStart(long processId) { LOGGER.info("Strategy started: " + processId); }
@Override public void onStop(long processId) { LOGGER.info("Strategy stopped: " + processId); if (!desiredOnBarRun() && allowedTries++ < 3) // checks for example a log file entry that gets created only for desired onBar calls { deleteCache(); IStrategy strategyToRun = new CalledStrategy(); client.startStrategy(strategyToRun, new LoadingProgressListener() { @Override public void dataLoaded(long startTime, long endTime, long currentTime, String information) { LOGGER.info(information); } @Override public void loadingFinished(boolean allDataLoaded, long startTime, long endTime, long currentTime) { } @Override public boolean stopJob() { return false; } }); } if (client.getStartedStrategies().size() == 0) { System.exit(0); } }
@Override public void onConnect() { LOGGER.info("Connected"); }
@Override public void onDisconnect() { //tester doesn't disconnect } });
Any other suggestions how to make such repeated automatic calls of a strategy more robust and independent of temporary data server problems ? One last question: is there any important difference between two setDataInterval signatures: client.setDataInterval(Period.TICK, null, null, startInterval.getMillis(), roundedEnd.getMillis()); client.setDataInterval(ITesterClient.DataLoadingMethod.ALL_TICKS, dateFrom.getTime(), dateTo.getTime());
|
|
|
|
 |
API Support
|
Post subject: Re: onBar doesn't get always called in ITesterClient for 30m |
Post rating: 0
|
Posted: Wed 04 Jan, 2012, 13:10
|
|
User rating: ∞
Joined: Fri 31 Aug, 2007, 09:17 Posts: 6139
|
FXace wrote: Do I understand correctly that onBar doesn't get called for Periods for which data is incomplete on my local cache (due to errors on server) ? Please confirm. This is the case, but we would like to reiterate that we did not manage to reproduce such behavior and still the case is not clear how you achieved such results: - You refer in your description to period 7:00 and 20:00 CET, but the same time your example works only with 30 minute interval.
- The date formatting differ in the two log files, suggesting, that the source code might have differences between the two cases.
- Also the function that you use to calculate the period, makes the case depend on time -> sophisticates the reproduction. You refer to multiple launching instances at different times of the day, meaning that in order to reproduce your case we would also need to launch the program at certain times.
FXace wrote: In order to reproduce the problem you'd need to set up scheduled task to run and then observe log files after at least a day or so since it doesn't appear immediatelly. Does this mean that you are back-testing (working with historical data) or nevertheless working with live data? The program that you provided works with historical data. FXace wrote: One last question: is there any important difference between two setDataInterval signatures: The first option soon will be deprecated.
|
|
|
|
 |
FXace
|
Post subject: Re: onBar doesn't get always called in ITesterClient for 30m |
Post rating: 0
|
Posted: Wed 04 Jan, 2012, 15:28
|
|
User rating: 0
Joined: Tue 05 Jul, 2011, 15:21 Posts: 28 Location: Switzerland,
|
OK let me fully clarify the yesterday's (03.01) case: 1. I have a Windows Task Scheduler task starting every 30 minutes, at 7:01, 7:30 etc. Time interval I refered too, 07:00 - 20:00, is simply test case for which I checked the log files produced by each invocation of my alerter strategy, which is getting called by this task scheduler. Note there is no strategy running all the time, it runs once using last completed 30 min bar and then exits ! In these 13 hours there have been 26 invocations (13 * 2), and only in 7 I managed to get onBar called for Period = 30 Mins. Btw none for Period = 4 Hours. 2. most importantly yes, I indeed work only with historical data, trying to get the very last 30 min bar that just finished 1 minute ago (every 30 minutes obviously). I am not working with live data. 3. key for the above is method calcEndBar() in my test program which ensures that time of the very last finished 30 min bar is calculated correctly, also over weekends. Then I simply decrease this for further 30 mins and use such two values for call to ITesterClient.setDataInterval, see lines: DateTime roundedEnd = calcEndBar(); DateTime startInterval = new DateTime(roundedEnd.minusMinutes(30)); client.setDataInterval(Period.TICK, null, null, startInterval.getMillis(), roundedEnd.getMillis()); 3. yes date formatting slightly changed between log files because I added seconds to be able to distinguish between different invocations for Period = 10 Secs, but otherwise the code is exactly the same 4. so in order to (rather try to) reproduce the problem please take the attached Strategy30MinRepeat3.java (modification of your test implementation to setDataInterval the way described above, compile it into a .jar (together with CalledStrategy4.java) and let such .jar be invoked every 30 mins by Windows Task Scheduler (or any other scheduler if you're on Linux). Note open source library Joda Time ( https://joda-time.sourceforge.net) has been used instead of java.util.Date and java.text.SimpleDateFormat. I modified also CalledStrategy4.java so you get a separate log file for each invocation. Now let such setup run for few hours against demo system and observe the content of all the produced log files - how many of those will have entries for 30 min and 4 Hours bars... Hope the case is bit clearer now. I'm aware it's a bit peculiar but my assumption is that fresh historical bars are available on server immediatelly (i.e. max. 1 minute after the corresponding real time period is finished). Btw it seems to me that after I increased the invocation offset to 4 instead of 1 minute (i.e. starting 7:04, 7:34 etc) that there are less "dropped" onBar calls for Period = 30 min and Period = 4 Hours. Today for example both expected 4h calls at 9:00 CET and 13:00 CET came in correctly. At least but not at least thanks for the tip on setDataInterval signature, I'll switch to the one using ITesterClient.DataLoadingMethod dataLoadingMethod
Attachments: |
File comment: Modified CalledStrategy3.java
CalledStrategy4.java [2.53 KiB]
Downloaded 362 times
|
File comment: Modified Strategy30MinRepeat2.java
Strategy30MinRepeat3.java [7.05 KiB]
Downloaded 362 times
|
DISCLAIMER: Dukascopy Bank SA's waiver of responsability - Documents, data or information available on
this webpage may be posted by third parties without Dukascopy Bank SA being obliged to make any control
on their content. Anyone accessing this webpage and downloading or otherwise making use of any document,
data or information found on this webpage shall do it on his/her own risks without any recourse against
Dukascopy Bank SA in relation thereto or for any consequences arising to him/her or any third party from
the use and/or reliance on any document, data or information found on this webpage.
|
|
|
|
|
 |
API Support
|
Post subject: Re: JFOREX-3422 onBar doesn't get always called for 30min ? |
Post rating: 0
|
Posted: Thu 05 Jan, 2012, 14:38
|
|
User rating: ∞
Joined: Fri 31 Aug, 2007, 09:17 Posts: 6139
|
The issue has been registered.
|
|
|
|
 |
|
Pages: [
1
]
|
|
|
|
|