Dukascopy
 
 
Wiki JStore Search Login

Historical Data Errors
 Post subject: Historical Data Errors Post rating: 0   New post Posted: Sat 12 Apr, 2014, 20:18 

User rating: 0
Joined: Sat 12 Apr, 2014, 11:41
Posts: 1
Location: Russian Federation,
I needed data for offline testing and debugging expert. I downloaded the historical data and found errors.

E1. Non-identical periods when selecting export and received data.
Date From: 2001.01.01; Date To: 2013.12.31; Format: CSV; Date Format: YYYY.MM.DD HH:MM:SS; Delimiter: ,; Data Type: Ticks; All Data; EURUSD
Export file: EURUSD_UTC_Ticks_2003.05.04_2013.12.30.csv
but:
Date From: 2003.05.04; Date To: 2013.12.31; Format: CSV; Date Format: YYYY.MM.DD HH:MM:SS; Delimiter: ,; Data Type: Ticks; All Data; EURUSD
Export file: EURUSD_UTC_Ticks_2003.05.04_2013.12.31.csv

Q1. getTicks и getBars have the same error?

E2. I got 11 files:
EURUSD_UTC_Ticks_2003.05.04_2003.12.31.csv
EURUSD_UTC_Ticks_2004.01.01_2004.12.31.csv
EURUSD_UTC_Ticks_2005.01.01_2005.12.31.csv
EURUSD_UTC_Ticks_2006.01.01_2006.12.31.csv
EURUSD_UTC_Ticks_2007.01.01_2007.12.31.csv
EURUSD_UTC_Ticks_2008.01.01_2008.12.31.csv
EURUSD_UTC_Ticks_2009.01.01_2009.12.31.csv
EURUSD_UTC_Ticks_2010.01.01_2010.12.31.csv
EURUSD_UTC_Ticks_2011.01.01_2011.12.30.csv
EURUSD_UTC_Ticks_2012.01.01_2012.12.31.csv
EURUSD_UTC_Ticks_2013.01.01_2013.12.31.csv
and combined them in the file 2.csv:
cat EURUSD_UTC_Ticks_2003.05.04_2003.12.31.csv | grep 'Time'  >'path/2.csv'
cat *.csv | grep -v 'Time'  >>'path/2.csv'


I got 1 file for all period:
EURUSD_UTC_Ticks_2003.05.04_2013.12.31.csv
and renamed it in 1.csv

Files 1.csv and 2.csv are different size:
1.csv 8879835040
2.csv 8879834763

I compared the number of lines in the files:
wc -l <1.csv
180625606
wc -l <2.csv
180625606

The files have the same number of lines. Good. It is hoped that when I get historical data again, the number of ticks will be unchanged.

Compare files line by line:
bdiff 1.csv 2.csv >bdiff.log

Log comparison attached to this message.

The log analysis revealed the different historical data, depending on the period of exports.

Q.2. Why different volumes depending on different periods? Volumes in your base are more than 5 digits after the decimal point and rounded different methods? getTicks and getBars receive the same different data? What could be the problem?
Examples:
46490616c46490616
< 2007.04.03 17:37:01.537,1.33495,1.33485,11.2,30.70605
---
> 2007.04.03 17:37:01.537,1.33495,1.33485,11.2,30.70606

112547040c112547040
< 2010.12.24 16:49:12.008,1.31176,1.31162,1.25199,2.625
---
> 2010.12.24 16:49:12.008,1.31176,1.31162,1.252,2.625

P.S. Sorry, log not attached.


 

Jump to:  

  © 1998-2025 Dukascopy® Bank SA
On-line Currency forex trading with Swiss Forex Broker - ECN Forex Brokerage,
Managed Forex Accounts, introducing forex brokers, Currency Forex Data Feed and News
Currency Forex Trading Platform provided on-line by Dukascopy.com