I created a very simple algorithm to test an issue I was seeing, attached.

I tested the algorithm on 00:00 01.01.2017 - 00:00 01.02.2017 which gave the following outputs:

Time Messages
2017-01-30 00:00:00 bb = 1.0048048186437157, 0.9809684, 0.9571319813562841
2017-01-23 00:00:00 bb = 1.0022123772429579, 0.9801075999999997, 0.9580028227570415
2017-01-16 00:00:00 bb = 1.000154636545928, 0.9797681999999998, 0.9593817634540716
2017-01-09 00:00:00 bb = 1.0063871169748668, 0.9821467999999999, 0.9579064830251328
2017-01-02 00:00:00 bb = 1.0162116594904136, 0.9876237999999996, 0.9590359405095856

I then deleted saved cache files via Settings, Preferences, Advanced, Advanced Settings and waited for the platform to restart.

I then repeated the test on the same period which gave the following outputs:

Time Messages
2017-01-30 00:00:00 bb = 1.0046105893855304, 0.9808675999999997, 0.9571246106144691
2017-01-23 00:00:00 bb = 1.0023088351635872, 0.9800971999999999, 0.9578855648364126
2017-01-16 00:00:00 bb = 1.0002120975717437, 0.9797347999999997, 0.9592575024282558
2017-01-09 00:00:00 bb = 1.0064319470410117, 0.9821093999999998, 0.9577868529589878
2017-01-02 00:00:00 bb = 1.0162116594904136, 0.9876237999999996, 0.9590359405095856

As you can see the results of bbands call on the first bar are the same in each test. But the results of the bbands calls on subsequent bars are different in each test.

Again I deleted saved cache files via Settings, Preferences, Advanced, Advanced Settings and waited for the platform to restart.

I then repeated the test on the same period which gave the following outputs:

Time Messages
2017-01-30 00:00:00 bb = 1.004838725386285, 0.9808731999999997, 0.9569076746137144
2017-01-23 00:00:00 bb = 1.0023078437550896, 0.9800351999999998, 0.9577625562449099
2017-01-16 00:00:00 bb = 1.0002593526463972, 0.9796957999999998, 0.9591322473536024
2017-01-09 00:00:00 bb = 1.0064842811439911, 0.9820695999999998, 0.9576549188560084
2017-01-02 00:00:00 bb = 1.0162116594904136, 0.9876237999999996, 0.9590359405095856

Again the results of the first bbands call match both of the previous tests but the results of subsequent bbands calls are different to either of the previous tests.

Why is this and how do I rectify it?

Hello,

We cannot reproduce this issue on Instrument.EURUSD and specified testing period. What instrument and HT settings are you using? Especially ticks data loading method

Hi,

I'm using AUD/CAD and "Process all ticks"

Any ideas on this one please?

Is the problem still persists?

Yes

Still having this issue. Can somebody help please?

I've taken a look at the source code for that bband function integrated into JForex and could not see any random component in it to explain the number drift.

The fact that the time period being tested is completely in the past (2017) is telling since of course cache data held for this period should be static and consistent on the server.

Question A: What is the Account Currency of the account being used to connect to JForex?
Question B: What is the Account Currency setting in the Historical Tester? (I seem to remember you can set this and some random equity amount)

Few things to try first:

1. Could you modify the onBar() method in your java file to use sma (Simple Moving Average) instead of bband (comment this out) then repeat your original experiment with AUD/CAD?
Standard Deviations and SMAs are the basis of the bollinger bands so you're going to want your SMAs to be consistent in the very first instance.

The SMA indicator to use returns a single double value instead of the array triplet you're picking up from the bband indicator. Use the same value for timePeriod you used for bband which I think was 50 (although for our purposes the value is irrelevant).

If the same issue IS observed with the SMAs then all bets are off and Dukascopy Support will have to dig into what it going on.

If the SMAs are consistent, then

2. Could you conduct the exact experiment using EUR/USD i.e. 3 runs of numbers as you did with AUD/CAD?

If the same issue is NOT observed on EUR/USD then the investigation goes in another direction straight away.

If the same issue IS observed on EUR/USD, then

3. Could you modify the onBar() method in your java file to use OfferSide.BID then repeat your original experiment with AUD/CAD?

If the same issue IS observed, then

4. Could you modify the onBar() method in your java file to isolate completed bars DAILY (instead of weekly) then repeat your original experiment with AUD/CAD?
(remembering to restore the OfferSide.ASK changed above)
This one will of course take a bit longer to run and will produce more output to the log but the idea here is to grab the log and then strip out the same days you have been when filtering by week for comparison purposes.
I'd like to eliminate any internal interpolations being done by the platform to produce daily and/or weekly bar data from static tick or bar data.

First thank you very much for your help. It is greatly appreciated.

On to your questions:

A+B. Both the platform account currency and the historical tester account currency are GBP. I have tried the same test on EURGBP and GBPUSD, i.e. secondary and primary currencies matching the account currency and the same problem as I have with AUDCAD exists.

EURGBP Run 1:

Time Messages
2017-01-30 00:00:00 bb = 0.8781243836389857, 0.8547908000000004, 0.8314572163610151
2017-01-23 00:00:00 bb = 0.8782773750977, 0.8534122000000002, 0.8285470249023004
2017-01-16 00:00:00 bb = 0.873095150567071, 0.8505595999999999, 0.8280240494329287
2017-01-09 00:00:00 bb = 0.8616164783098221, 0.8476044, 0.8335923216901779
2017-01-02 00:00:00 bb = 0.863605669731466, 0.8482232000000001, 0.8328407302685341

EURGBP Run 2:

Time Messages
2017-01-30 00:00:00 bb = 0.8775256874435469, 0.8547226000000003, 0.8319195125564538
2017-01-23 00:00:00 bb = 0.8776240506929911, 0.8532972000000002, 0.8289703493070093
2017-01-16 00:00:00 bb = 0.8721663369850602, 0.8504204000000002, 0.8286744630149402
2017-01-09 00:00:00 bb = 0.8616414513214782, 0.8476121999999999, 0.8335829486785217
2017-01-02 00:00:00 bb = 0.863605669731466, 0.8482232000000001, 0.8328407302685341

GBPUSD Run 1:

Time Messages
2017-01-30 00:00:00 bb = 1.2666886569682503, 1.2361762, 1.2056637430317498
2017-01-23 00:00:00 bb = 1.270527152859354, 1.2371139999999996, 1.2037008471406452
2017-01-16 00:00:00 bb = 1.2762379202273726, 1.2407119999999996, 1.2051860797726266
2017-01-09 00:00:00 bb = 1.2741964494575453, 1.2446782, 1.2151599505424548
2017-01-02 00:00:00 bb = 1.2739289361747512, 1.2467610000000002, 1.2195930638252492

GBPUSD Run 2:

Time Messages
2017-01-30 00:00:00 bb = 1.266737860220125, 1.2363243999999998, 1.2059109397798746
2017-01-23 00:00:00 bb = 1.2704944764860435, 1.2372493999999994, 1.2040043235139553
2017-01-16 00:00:00 bb = 1.2761534436624529, 1.2408497999999994, 1.205546156337546
2017-01-09 00:00:00 bb = 1.2740361582678317, 1.2448002, 1.2155642417321684
2017-01-02 00:00:00 bb = 1.2739289361747512, 1.2467610000000002, 1.2195930638252492


Then to your suggestions:

1. SMA shows the same mismatch as bbands:

Run 1:

Time Messages
2017-01-30 00:00:00 sma = 0.9809144
2017-01-23 00:00:00 sma = 0.9800815999999998
2017-01-16 00:00:00 sma = 0.9797453999999998
2017-01-09 00:00:00 sma = 0.9821549999999999
2017-01-02 00:00:00 sma = 0.9876237999999996

Run 2:

Time Messages
2017-01-30 00:00:00 sma = 0.9809642
2017-01-23 00:00:00 sma = 0.9800735999999998
2017-01-16 00:00:00 sma = 0.9797341999999998
2017-01-09 00:00:00 sma = 0.9821127999999999
2017-01-02 00:00:00 sma = 0.9876237999999996

2. See answers to questions A+B above.

3. The behavior is the same as it is using ASK

Run 1:

Time Messages
2017-01-30 00:00:00 bb = 1.0045043246962264, 0.9803193999999997, 0.9561344753037729
2017-01-23 00:00:00 bb = 1.001884197326623, 0.9794339999999996, 0.9569838026733761
2017-01-16 00:00:00 bb = 0.9998480943177415, 0.9790671999999992, 0.958286305682257
2017-01-09 00:00:00 bb = 1.0058439367646985, 0.9813955999999999, 0.9569472632353012
2017-01-02 00:00:00 bb = 1.015740245379096, 0.9868970000000001, 0.9580537546209043

Run 2:

Time Messages
2017-01-30 00:00:00 bb = 1.0040394528590608, 0.980151, 0.9562625471409392
2017-01-23 00:00:00 bb = 1.0015158522549665, 0.9793019999999995, 0.9570881477450325
2017-01-16 00:00:00 bb = 0.9998106687457076, 0.9789929999999991, 0.9581753312542907
2017-01-09 00:00:00 bb = 1.0058941275050115, 0.9813561999999998, 0.9568182724949881
2017-01-02 00:00:00 bb = 1.015740245379096, 0.9868970000000001, 0.9580537546209043

4. Again the outputs are different from the second bar onwards

Run 1:

Time Messages
2017-02-01 00:00:00 bb = 1.0050217391737029, 0.9811649999999997, 0.9573082608262967
2017-01-31 00:00:00 bb = 1.0048511761911447, 0.9810751999999997, 0.9572992238088547
2017-01-30 00:00:00 bb = 1.0044087543567048, 0.9808083999999997, 0.9572080456432946
2017-01-29 00:00:00 bb = 1.0040046232599933, 0.9806175999999995, 0.9572305767400058
2017-01-28 00:00:00 bb = 1.0035988366848492, 0.9804289999999996, 0.9572591633151499
2017-01-27 00:00:00 bb = 1.0032450212488013, 0.9802789999999998, 0.9573129787511983
2017-01-26 00:00:00 bb = 1.0034169347063615, 0.9803503999999998, 0.9572838652936382
2017-01-25 00:00:00 bb = 1.0035530265512695, 0.9803977999999998, 0.9572425734487301
2017-01-24 00:00:00 bb = 1.00313585929252, 0.9802799999999999, 0.9574241407074798
2017-01-23 00:00:00 bb = 1.0019960906768768, 0.9800114, 0.9580267093231232
2017-01-22 00:00:00 bb = 1.000563545138792, 0.9797062, 0.9588488548612079
2017-01-21 00:00:00 bb = 0.9990475236619303, 0.979401, 0.9597544763380697
2017-01-20 00:00:00 bb = 0.9971474663797587, 0.9790118, 0.9608761336202413
2017-01-19 00:00:00 bb = 0.9954744773019715, 0.9787185999999999, 0.9619627226980283
2017-01-18 00:00:00 bb = 0.996638009380313, 0.9789093999999998, 0.9611807906196866
2017-01-17 00:00:00 bb = 0.998308999159368, 0.9792749999999999, 0.9602410008406318
2017-01-16 00:00:00 bb = 1.0001575499466244, 0.9797101999999999, 0.9592628500533753
2017-01-15 00:00:00 bb = 1.0019620321104727, 0.9801654, 0.9583687678895271
2017-01-14 00:00:00 bb = 1.0036517651151022, 0.9806206000000001, 0.9575894348848979
2017-01-13 00:00:00 bb = 1.0046466297111267, 0.9809518, 0.9572569702888734
2017-01-12 00:00:00 bb = 1.0052230508561288, 0.9811866, 0.9571501491438712
2017-01-11 00:00:00 bb = 1.005831500947365, 0.9814752000000001, 0.9571188990526351
2017-01-10 00:00:00 bb = 1.006144316444042, 0.9817685999999999, 0.9573928835559578
2017-01-09 00:00:00 bb = 1.0064453995982858, 0.9821019999999998, 0.9577586004017138
2017-01-08 00:00:00 bb = 1.0066752260846252, 0.9825973999999998, 0.9585195739153745
2017-01-07 00:00:00 bb = 1.0068608235745924, 0.9830927999999998, 0.9593247764254071
2017-01-06 00:00:00 bb = 1.0076634522402526, 0.9837955999999997, 0.9599277477597468
2017-01-05 00:00:00 bb = 1.0088675507398694, 0.9844985999999998, 0.9601296492601301
2017-01-04 00:00:00 bb = 1.0109815402132158, 0.9854705999999998, 0.9599596597867838
2017-01-03 00:00:00 bb = 1.0139648492349373, 0.9865553999999997, 0.9591459507650621
2017-01-02 00:00:00 bb = 1.0162116594904136, 0.9876237999999996, 0.9590359405095856

Run 2:

Time Messages
2017-02-01 00:00:00 bb = 1.005122034818383, 0.9812839999999997, 0.9574459651816165
2017-01-31 00:00:00 bb = 1.0049532074170493, 0.9811941999999996, 0.95743519258295
2017-01-30 00:00:00 bb = 1.0045160436440157, 0.9809273999999994, 0.9573387563559831
2017-01-29 00:00:00 bb = 1.0041156906958337, 0.9807365999999995, 0.9573575093041653
2017-01-28 00:00:00 bb = 1.0037137053422465, 0.9805479999999994, 0.9573822946577523
2017-01-27 00:00:00 bb = 1.0033629625299663, 0.9803979999999995, 0.9574330374700328
2017-01-26 00:00:00 bb = 1.003555622021956, 0.9804803999999997, 0.9574051779780435
2017-01-25 00:00:00 bb = 1.0036911434827624, 0.9805279999999996, 0.9573648565172368
2017-01-24 00:00:00 bb = 1.0032718849448905, 0.9804089999999996, 0.9575461150551088
2017-01-23 00:00:00 bb = 1.0021386959195004, 0.9801403999999997, 0.958142104080499
2017-01-22 00:00:00 bb = 1.0007144291965444, 0.9798351999999997, 0.958955970803455
2017-01-21 00:00:00 bb = 0.9992077596286039, 0.9795299999999998, 0.9598522403713957
2017-01-20 00:00:00 bb = 0.9973213495362666, 0.9791407999999998, 0.960960250463733
2017-01-19 00:00:00 bb = 0.9956610467900393, 0.9788475999999997, 0.9620341532099601
2017-01-18 00:00:00 bb = 0.9967220666510924, 0.9790193999999999, 0.9613167333489073
2017-01-17 00:00:00 bb = 0.9983961604402237, 0.9793889999999998, 0.960381839559776
2017-01-16 00:00:00 bb = 1.0002061446040929, 0.9798089999999998, 0.9594118553959067
2017-01-15 00:00:00 bb = 1.0019924390989696, 0.9802567999999998, 0.95852116090103
2017-01-14 00:00:00 bb = 1.0036596229106025, 0.9807007999999999, 0.9577419770893973
2017-01-13 00:00:00 bb = 1.004634251705346, 0.9810207999999999, 0.9574073482946539
2017-01-12 00:00:00 bb = 1.0051728103455224, 0.981232, 0.9572911896544776
2017-01-11 00:00:00 bb = 1.005780361057348, 0.9815206000000001, 0.9572608389426521
2017-01-10 00:00:00 bb = 1.006091741902263, 0.9818095999999998, 0.9575274580977367
2017-01-09 00:00:00 bb = 1.00638313982298, 0.9821559999999998, 0.9579288601770195
2017-01-08 00:00:00 bb = 1.0066558398062977, 0.9826193999999998, 0.9585829601937018
2017-01-07 00:00:00 bb = 1.0068390593950256, 0.9831147999999997, 0.9593905406049739
2017-01-06 00:00:00 bb = 1.0076392753600945, 0.9838175999999997, 0.9599959246399051
2017-01-05 00:00:00 bb = 1.0088683587206149, 0.9844973999999997, 0.9601264412793846
2017-01-04 00:00:00 bb = 1.0110263360725096, 0.9854423999999997, 0.9598584639274899
2017-01-03 00:00:00 bb = 1.0139895872430185, 0.9865411999999998, 0.9590928127569811
2017-01-02 00:00:00 bb = 1.0162116594904136, 0.9876237999999996, 0.9590359405095856

NB In all of the above tests saved cache files were deleted between Run 1 and Run 2 as they were in the original test.

So no solution unfortunately but thank you for helping me to eliminate some possibilities.

Hello,

We need some additional info to determine problem. Please, do the following

Run this strategy and save ticksOutput and BarsOutput in files, then clear your cache and run again.
Provide all 4 output files, barsOutput files must be small, so you can post it as attachment to post, ticksOutput you can share at any file hosting service.
Also compare ticks amount in both strategy launches.

All requested files can be found here:

https://we.tl/t-yAgMLo1xDH

2,153,586 ticks in the first run
2,622,217 ticks in the second run

good job running off those experiments, seems we now know for sure that the underlying platform carries a flaw.
hopefully the API/Platform teams will be able to get it sorted out pretty soon.

Hope so. Thanks again for your earlier help.

For some reason in cache data updating process you lose chunks of tick data. Also interesting what each time there is missing different pieces of data. It could happen because of connection issues or some errors with data processing, but anyways it's strange behavior which should be somehow reflected in platform logs. Could you reproduce this issue again and provide your log files?

Log files available here: https://we.tl/t-5dUdf5PfFs

Thanks

Thanks. We determined cause of this bug. Will be fixed in next platform release.

That's great news. Thanks. When do you expect the next release?

Within two weeks.