Hi Support,

Thank you very much indeed for having a stab at fixing bug 3669 (now listed as ‘Indicator for Renko’).

I’m really sorry to have to say, though, that it’s *still* not working right. I don’t know whether this post counts as a new bug report, and I may be covering more than one bug. Please feel free to split the post up or else to attach it to the thread about bug 3669 and move the thread back onto the unfixed bug list if that’s what you’d prefer.

As before, I’m working with the strategy you previously wrote for me which I’ve modified to print out every bar (I’ve attached it again to this posting for convenience). If I run this strategy on a Demo 5-pip chart, it at least now gives the correct opening time and price for each new bar, and the speed problem I mentioned previously (see my post of 5 April) is fixed, all of which is great!

But if I ask it to print out details of the previous bar, as I’ve done at lines 92 & 93, it still prints out the details of the bar before that (i.e. the code asks for the bar with shift 1, but what you get is the bar with shift 2, and as time goes on you sometimes get the bar with shift 3), which is another of the problems I noted in my post of 5 April. Moreover, the opening time of the bar is wrong (e.g. the 14:57:04.402 bar is shown as starting at 14:57:30.000); the opening price given is generally wrong (although not curiously the high, the low or the close); and the indicator values do not match those shown on the chart.

More bizarrely, though, the printed version appears simply to miss the occasional bar. Thus for the bar starting at 15:39:44.940, the printout lists the bar at shift 1 as 15:24:30.000, open 1.33801, high 1.33801, low 1.33751, close 1.33751, which is the bar at shift 3 allowing for the erroneous open price. But at the next bar (15:40:19.690), the printout gives 15:27:30.000, open 1.3173, high 1.31747, low 1.31697, close 1.31697, which is the bar at shift 2, so the bar history never lists the bar at 15:27:17.026 at all. And then at the next bar (15:40:47.026), it prints out the same details for the ‘15:27:30.000’ bar, which is now the bar at shift 3, so that when it prints the bar after that (15:52:52.113) it has to skip another bar to keep up.

Similarly, if I run the strategy on a 5-pip chart in the Historical Tester using EURUSD for 29 – 30 Nov 2011, I find that the printed list of bars mis-states the opening time of every bar (e.g. giving 00:03:30.000, 00:08:30.000 and 00:10:30.000 for the bars which are shown on the chart as 00:03:38.844, 00:08:27.915 and 00:10:51.292 respectively). There are also some bars where all the prices differ from those shown on the chart (e.g. the bar listed at 00:08:30.000 is given as open 1.33008, high 1.33013, low 1.32963, close 1.32963, whereas the bar on the chart appears as open 1.33018, high 1.33018, low 1.32968, close 1.32968), and the opening price of most bars differs from that shown on the chart.

More seriously, the printed version again misses out the occasional bar, apparently at random (e.g. the chart bars at 00:11:25.292 and 01:07:36.293). The printed list is also interrupted by bug 3679 (‘Passed bar in time interval’), which I know you’ve not yet got around to fixing, but as far as I can tell the missing bars are not related to this. Given all of this, it hardly seems surprising that the indicator values the strategy prints out don’t match those shown on the chart.

The progress you’ve made in getting range bars to work properly is splendid; I do hope you can fix these remaining issues quickly so that those of us who use range bars can enjoy all the features your platform promises.

Many many thanks,

Bunyip

This is because the feed works in a different thread. Hence:

• Final values of the completed range bar that arrived in the feed have not been stored in the historical data cache yet.
• The new range bar that has already started forming has not been stored in the historical data cache yet.

This will be fixed, such that history is in sync with the feed.
This is because the feed history for renko and range bars get constructed by the following principle:
viewtopic.php?f=65&t=47024&p=63193#p63193
Whilst the on chart feed gets constructed from ticks. This algorithm, however, is subject to change - the in-progress range bars and renko bars (the ones within the testing period) are going to be constructed from ticks in order to match the on chat feed algorithm.

Hi Support,

Many thanks for your prompt reply.

Let’s see if I’ve understood you correctly. For the strategy used on the Demo chart, you say that the range bar which is currently forming isn’t stored in the historical data cache, which makes sense. But you also say that the bar that’s just finished forming isn’t yet stored their either, which I find harder to understand.

Surely it should be possible to store the completed bar in the cache with the same parameters as are shown on the chart as soon as it’s finished forming. Indeed I’ve written code to do so myself, just to check it’s possible, and I’m no wizard Java programmer (see attached strategy, in case anyone else finds it helpful—the close price can have tiny errors, and you’d have to store the values in array and calculate any indicators on that if you wanted them).

As the built-in data feed for range bars stands, though:
1) the bar that’s finished forming isn’t available in the historical data cache until at least one bar *after* it forms;
2) the delay can sometimes be two bars, apparently at random;
3) some bars on the chart therefore never appear in the historical data cache at all;
4) none of the bars in the historical data cache start at the same time as they do on the chart;
5) very few of the bars in the historical data cache have the same prices that they do on the chart; and
6) as a result, the indicator values calculated by the strategy are different from those shown on the chart.

Basically, with these restrictions it is just not possible to write a strategy that responds properly to events on a range bar chart, and it is misleading your clients to claim otherwise.

I also understand that you’ve chosen to construct large range bars in the Historical Tester from bar data rather assemble them tick by tick, because you believed the latter would be too slow.

But in practice, the Historical Tester is quite fast enough for my purposes in tick by tick mode, and I have no reason to use any of the faster data feeds. This, I take it, is why you left the choice of data feed to the user in the first place.

The consequence, though, is that I cannot access an accurate start time or price for any given range bar in the Tester, and so cannot even work around the problems in the historical data cache as I’ve done for the Demo version.

In fact, it looks to me as though the bars that are stored in the historical data cache are constructed from time bars even in the Demo version (given that, for instance, they always start at a fixed number of seconds in the minute), which would explain the very strange behaviour I’ve listed above. If I’m right, it makes it all the more important that you fix this problem as you’ve indicated you intend to do.

I do hope that you make this a top priority, as I’m clearly not your only client who would like to use range bars, and it will not be possible to do so until these issues are resolved.

Best regards,

Bunyip

Please note that there are two issues registered for your post. Both parts of our previous reply also point to that. The two will get fixed as soon as possible.

Hi Support,

I really appreciate your undertaking to fix both these problems, but it would be great to have some idea when the fixes will be available. It’s been two weeks since your last post, and it’s clear that ‘as soon as possible’ is a lot longer than those of us who use range bars would like.

Looking forward to hearing from you,

Bunyip

I’m really sorry, but what makes you think these two bugs are fixed?

I’m using version 2.6.66 and I’ve downloaded the latest version of the code just to be sure. And it’s true that if I run the strategy I attached above (MacdEmaOnFeed4, which is just a minor modification of your own MacdEmaOnFeed) on a new 5-pip Demo chart, it initially gives bar prices and indicator values that are the same as those shown on the chart.

Sadly, that’s just because what the chart shows for the bars before it was loaded are synthetic bars built from higher time frames, which are the same thing the strategy gets. If I continue running the strategy, however, it still gives all the problems I’ve mentioned in previous posts. For what it’s worth, it still looks to me as though they stem from the fact that the strategy only has access to bars built from higher time frames, whereas both Demo and Historical tester charts build bars from individual ticks once they’re started.

For your clients to use range bars effectively, it’s imperative that their strategies should have access to the same bars that are shown on the charts, and surely it should not be hard to feed the same data the chart gets to a strategy. Please either advise what changes need to be made to the code or to the setup so that it works properly, or shift this thread back into the unsolved bugs section and make a thorough attempt to sort the matter out as soon as possible.

I look forward to hearing from you,

Bunyip

In case anyone’s wondering, my previous post on the thread regarding this bug was in response to a post from Support that claimed to have fixed the bug. Sadly this was not the case, and Support have since deleted their post, although the thread remains locked for some reason (I'd be very grateful if this could be fixed!).

Just to follow on a bit, here’s a new version of the previous strategy that prints out 1) the parameters of each bar and 2) the value of a simple indicator (SMA3) both from the feed (as Support originally proposed) and from the actual tick feed data collected in an array.

You’ll see that when the strategy starts, neither data source gives accurate values for the indicator (as they both have to rely on back-data that’s loaded when the strategy starts—data for both outputs is arranged with the most recent values at the left). The array data does, however, give a value for the indicator on the last complete bar, which the data from the feed just omits.

But once the strategy has been running for a few bars, you’ll see that the values given by the array differ from those given by the feed and correspond exactly to those shown on screen. Strangely, this means that the code for the indicator must have access to the true data for each bar in a way that the code for the strategy does not.

I don’t know why this is, but it makes me think that it shouldn’t be *so* hard to arrange that strategies have access to the same data. Please, Support, this is a problem that urgently needs to be fixed. It’s just about possible to use the work-around I’ve posted here on real-time data, but it fails completely in the Historical Tester, and how else are we to trouble-shoot our code?

Looking forward to hearing from you,

Bunyip

API 2.6.67

Hi Support,

What’s going on here? Yesterday I posted the message below regarding bug 3778 ‘Further feed problems’, and today I find it’s been deleted without any further acknowledgement.

Does this mean that the bug will be fixed in the next release? Or was my post just deleted by accident? Certainly the bug’s still there this morning in version 2.6.69.

Looking forward to hearing from you,

Bunyip

Original message follows:

Many many thanks for tackling this. I’m afraid it’s still not working properly, though, so please take this thread out of ‘Fixed Bugs’.

When I run the same strategy as before (‘MacdEmaOnFeed5’) on a Demo 5-pip chart, the bars from the datafeed at least now have start times that are specific to the last millisecond, which is progress. Unfortunately, they’re not the same start times as those shown on screen, and the open, high, low and close prices are all the same. Unsurprisingly, the indicator values calculated from the feed are therefore quite different from those shown on screen.

The good news is that when I run the strategy in the Historical Tester, the values for the bars from the datafeed are more or less accurate, which is great! The only trouble is that trying to calculate any indicator from the datafeed gives the ‘Passed time has to be in interval’ bug (no. 3679), which doesn’t seem to have been fixed at all (and may be related to the ‘Historical Tester does not work’ bug, no 3819).

I’d really appreciate it if you could sort out the remaining issues as soon as possible. It’s very important to get accurate indicator values and be able to use the Historical Tester, and the response to bug 3819 shows I’m not the only person who needs a solution.

Best regards,

Bunyip

We cannot reproduce the issue. Please provide us with detailed steps and a sample code.

Please find attached MacdEmaOnFeed10, which (like the strategies I’ve attached above) is code you wrote yourselves modified to print out the time and prices for each successive bar.

In my hands, it now works perfectly in the Historical Tester, which is great! Sadly, though, it’s not so simple with a Demo chart.

The prices from the datafeed are printed out as ‘Last bar on feed’ and come from history.getRangeBar(instrument, side, priceRange, 0). Unfortunately, the open, high, low and close displayed for each bar are all the same, and correspond to the open price of the bar that’s forming.

In addition, if you modify the code to history.getRangeBar(instrument, side, priceRange, 1), it gives a start time that’s rounded to 15 seconds (e.g. Last bar on screen: 2012-06-18 17:00:45.000; 1.25627; 1.25688; 1.25627; 1.25638) with prices that don’t correspond to anything on the chart.

Interestingly, the SMA indicator values calculated from the feed are accurate for the first few bars, although they don’t include the value for the last complete bar. After that, though, they become completely different, and there are also individual bars where either no new SMA value is shown or two new SMA values are shown.

As before, I’ve also included a work-around which gets the price data for each bar working from the starting time. The values this gives are shown as ‘Last bar on chart’, and do correspond to those on the screen. Moreover, calculating an SMA on them as an array gives the values shown as ‘SMA on array’, which also correspond to those on the screen after the first few bars.

I hope that this clarifies the problem, and that you can now sort it out quickly.

Many thanks,

Bunyip

What do you mean by "Demo chart"? The in-progress data? Do you claim that in Historical Tester everything works fine, but the problems you describe only occur with in-progress data?

I know it seems odd, but that's what I find. The problems occur with in-progress data, but not in-progress data from which you can trade real money.

We managed to replicated the issue

Hi Support,

I don’t know why this thread is now in Fixed Bugs, but I’m really sorry to say that the bug is *still* not fixed.

If you run MacdEmaOnFeed10 (the same program I attached above) on in-progress data for some time, the data feed now gives the same start time for each bar as that shown on screen, but it still only gives a single price (the open price) for the high, low and close of the bar.

Moreover, the SMA values given by the data feed still don’t correspond to those shown on the screen.

I do hope that you can now sort this problem out quickly, as it’s frustrating that we’ve had to wait almost three months for a solution.

Many thanks,

Bunyip

A bug fix will be provided in a new platform version.

I believe that I am seeing the same issues with Renko bars in API 2.7.
Support could you confirm which API this fix is targeted for?

Please specify exactly which issues and please provide an example strategy which reproduces them.