|
| 74% performance degradation backtesting No-op IStrategy between 2.7.9 and 2.8.0 |
|
GPEngine
|
| Post subject: 74% performance degradation backtesting No-op IStrategy between 2.7.9 and 2.8.0 |
Post rating: 1
|
Posted: Tue 27 Aug, 2013, 06:29
|
|
User rating: 2
Joined: Tue 11 Jun, 2013, 05:24
Posts: 7
|
|
I observed a performance degradation when backtesting strategies when I upgraded from (JForex-API 2.7.9 + JForex-SDK 2.4) to (JForex-API 2.8.0 + JForex-SDK 2.25.6)
The attached trivial IStrategy illustrates the performance hit. On my machine, using 2.7.9 library, it finishes in 31 seconds. With 2.8.0 it finishes in 54 seconds, a 74% performance hit.
My machine is 6-core 64-bit, 20GB ram, Linux 3.2.0-51-generic x86_64 GNU/Linux
with default /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java
What happened?
Even the old performance of 30 seconds to backtest merely 2 weeks of data, without any activity on the account, is long compared to other backtesting platforms I have used. Am I doing something wrong?
$ grep -C3 '2\.' pom.xml
<modelVersion>4.0.0</modelVersion>
<groupId>com.dukascopy.dds2</groupId>
<artifactId>JForex-SDK</artifactId>
<version>2.24</version>
<organization>
<name>Dukascopy Bank SA</name>
<url>https://www.dukascopy.com</url>
--
<dependency>
<groupId>com.dukascopy.dds2</groupId>
<artifactId>DDS2-jClient-JForex</artifactId>
<version>2.24</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
--
<dependency>
<groupId>com.dukascopy.api</groupId>
<artifactId>JForex-API</artifactId>
<version>2.7.9</version>
<classifier>sources</classifier>
</dependency>
<dependency>
$ /usr/local/netbeans-7.3.1/java/maven/bin/mvn clean install
...
$ time /usr/local/netbeans-7.3.1/java/maven/bin/mvn -Dexec.classpathScope=runtime "-Dexec.args=-classpath %classpath soup.Metric" -Dexec.executable=/usr/lib/jvm/default-java/bin/java process-classes org.codehaus.mojo:exec-maven-plugin:1.2:exec
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Project 2.24
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for org.rosuda:JRI:jar:0.5-5 is missing, no dependency information available
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ JForex-SDK ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ JForex-SDK ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 35 source files to /home/gpengine/project/target/classes
[INFO]
[INFO] --- reflections-maven:0.9.9-RC1:reflections (default) @ JForex-SDK ---
[INFO] Reflections took 72 ms to scan 1 urls, producing 19 keys and 49 values
[INFO] Reflections successfully saved in /home/gpengine/project/target/classes/META-INF/reflections/JForex-SDK-reflections.xml using XmlSerializer
[INFO]
[INFO] --- exec-maven-plugin:1.2:exec (default-cli) @ JForex-SDK ---
2013-08-25 22:33:14.500 WARN WLabelData - White Label Image is not received
2013-08-25 22:33:16.920 WARN WLabelData - White Label Image is not received
2013-08-25 22:33:29.314 WARN Metric - Finish deposit = 5000.0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.969s
[INFO] Finished at: Sun Aug 25 22:33:29 PDT 2013
[INFO] Final Memory: 7M/293M
[INFO] ------------------------------------------------------------------------
real 0m31.042s
user 0m54.767s
sys 0m3.144s
versus
$ grep -C3 '2\.' pom.xml
<modelVersion>4.0.0</modelVersion>
<groupId>com.dukascopy.dds2</groupId>
<artifactId>JForex-SDK</artifactId>
<version>2.25.6</version>
<organization>
<name>Dukascopy Bank SA</name>
<url>https://www.dukascopy.com</url>
--
<dependency>
<groupId>com.dukascopy.dds2</groupId>
<artifactId>DDS2-jClient-JForex</artifactId>
<version>2.25.6</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
--
<dependency>
<groupId>com.dukascopy.api</groupId>
<artifactId>JForex-API</artifactId>
<version>2.8.0</version>
<classifier>sources</classifier>
</dependency>
<dependency>
$ /usr/local/netbeans-7.3.1/java/maven/bin/mvn clean install
...
$ time /usr/local/netbeans-7.3.1/java/maven/bin/mvn -Dexec.classpathScope=runtime "-Dexec.args=-classpath %classpath soup.Metric" -Dexec.executable=/usr/lib/jvm/default-java/bin/java process-classes org.codehaus.mojo:exec-maven-plugin:1.2:exec
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Project 2.25.6
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for org.rosuda:JRI:jar:0.5-5 is missing, no dependency information available
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ JForex-SDK ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ JForex-SDK ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 35 source files to /home/gpengine/project/target/classes
[INFO]
[INFO] --- reflections-maven:0.9.9-RC1:reflections (default) @ JForex-SDK ---
[INFO] Reflections took 70 ms to scan 1 urls, producing 19 keys and 49 values
[INFO] Reflections successfully saved in /home/gpengine/project/target/classes/META-INF/reflections/JForex-SDK-reflections.xml using XmlSerializer
[INFO]
[INFO] --- exec-maven-plugin:1.2:exec (default-cli) @ JForex-SDK ---
2013-08-25 22:27:11.779 WARN WLabelData - White Label Image is not received
2013-08-25 22:27:14.193 WARN WLabelData - White Label Image is not received
2013-08-25 22:27:51.810 WARN Metric - Finish deposit = 5000.0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 53.026s
[INFO] Finished at: Sun Aug 25 22:27:52 PDT 2013
[INFO] Final Memory: 7M/300M
[INFO] ------------------------------------------------------------------------
real 0m54.068s
user 1m59.567s
sys 0m10.221s
| Attachments: |
Metric.java [3.57 KiB]
Downloaded 485 times
|
|
DISCLAIMER: Dukascopy Bank SA's waiver of responsability - Documents, data or information available on
this webpage may be posted by third parties without Dukascopy Bank SA being obliged to make any control
on their content. Anyone accessing this webpage and downloading or otherwise making use of any document,
data or information found on this webpage shall do it on his/her own risks without any recourse against
Dukascopy Bank SA in relation thereto or for any consequences arising to him/her or any third party from
the use and/or reliance on any document, data or information found on this webpage.
|
|
|
|
|
|
 |
|
API Support
|
| Post subject: Re: 74% performance degradation backtesting No-op IStrategy between 2.7.9 and 2.8.0 |
Post rating: 0
|
Posted: Tue 27 Aug, 2013, 15:12
|
|
User rating: ∞
Joined: Fri 31 Aug, 2007, 09:17
Posts: 6139
|
|
We did not observe any performance deterioration both in Windows and Linux, could you please post the results of the attached program for both JForex-SDK versions that you cite?
You can avoid receiving other logs by setting their levels to ERROR in the log4j.properties.
| Attachments: |
Metric.java [3.35 KiB]
Downloaded 481 times
|
|
DISCLAIMER: Dukascopy Bank SA's waiver of responsability - Documents, data or information available on
this webpage may be posted by third parties without Dukascopy Bank SA being obliged to make any control
on their content. Anyone accessing this webpage and downloading or otherwise making use of any document,
data or information found on this webpage shall do it on his/her own risks without any recourse against
Dukascopy Bank SA in relation thereto or for any consequences arising to him/her or any third party from
the use and/or reliance on any document, data or information found on this webpage.
|
|
|
|
|
|
|
|
GPEngine
|
| Post subject: Re: 74% performance degradation backtesting No-op IStrategy between 2.7.9 and 2.8.0 |
Post rating: 0
|
Posted: Tue 27 Aug, 2013, 17:05
|
|
User rating: 2
Joined: Tue 11 Jun, 2013, 05:24
Posts: 7
|
|
The supplied program does not terminate after 15 minutes. Using 2.7.9
|
|
|
|
|
|
|
GPEngine
|
| Post subject: Re: 74% performance degradation backtesting No-op IStrategy between 2.7.9 and 2.8.0 |
Post rating: 0
|
Posted: Wed 28 Aug, 2013, 06:37
|
|
User rating: 2
Joined: Tue 11 Jun, 2013, 05:24
Posts: 7
|
|
Got it. I needed to change my logging level to INFO to see your intended messages.
The program does not terminate after completing the 10th iteration.
2.7.9:
2013-08-27 22:34:17.376 INFO MetricAlt - Started 1
2013-08-27 22:34:29.765 INFO MetricAlt -
Execution time in secs: 12.3880
2013-08-27 22:34:31.402 INFO MetricAlt - Started 2
2013-08-27 22:34:42.781 INFO MetricAlt -
Execution time in secs: 11.3790
2013-08-27 22:34:44.426 INFO MetricAlt - Started 3
2013-08-27 22:34:55.356 INFO MetricAlt -
Execution time in secs: 10.9300
2013-08-27 22:34:56.445 INFO MetricAlt - Started 4
2013-08-27 22:35:07.295 INFO MetricAlt -
Execution time in secs: 10.8500
2013-08-27 22:35:08.469 INFO MetricAlt - Started 5
2013-08-27 22:35:19.582 INFO MetricAlt -
Execution time in secs: 11.1120
2013-08-27 22:35:21.489 INFO MetricAlt - Started 6
2013-08-27 22:35:32.512 INFO MetricAlt -
Execution time in secs: 11.0230
2013-08-27 22:35:34.509 INFO MetricAlt - Started 7
2013-08-27 22:35:45.477 INFO MetricAlt -
Execution time in secs: 10.9440
2013-08-27 22:35:46.552 INFO MetricAlt - Started 8
2013-08-27 22:35:57.306 INFO MetricAlt -
Execution time in secs: 10.7530
2013-08-27 22:35:58.572 INFO MetricAlt - Started 9
2013-08-27 22:36:09.391 INFO MetricAlt -
Execution time in secs: 10.8180
2013-08-27 22:36:10.594 INFO MetricAlt - Started 10
2013-08-27 22:36:22.013 INFO MetricAlt -
Execution time in secs: 11.4190
^C
2.8.0:
2013-08-27 22:28:17.333 INFO MetricAlt - Started 1
2013-08-27 22:28:46.141 INFO MetricAlt -
Execution time in secs: 28.8070
2013-08-27 22:28:47.357 INFO MetricAlt - Started 2
2013-08-27 22:29:17.862 INFO MetricAlt -
Execution time in secs: 30.4850
2013-08-27 22:29:19.399 INFO MetricAlt - Started 3
2013-08-27 22:29:47.732 INFO MetricAlt -
Execution time in secs: 28.3330
2013-08-27 22:29:49.419 INFO MetricAlt - Started 4
2013-08-27 22:30:18.385 INFO MetricAlt -
Execution time in secs: 28.9650
2013-08-27 22:30:19.440 INFO MetricAlt - Started 5
2013-08-27 22:30:47.952 INFO MetricAlt -
Execution time in secs: 28.5120
2013-08-27 22:30:49.461 INFO MetricAlt - Started 6
2013-08-27 22:31:18.988 INFO MetricAlt -
Execution time in secs: 29.5270
2013-08-27 22:31:20.483 INFO MetricAlt - Started 7
2013-08-27 22:31:48.998 INFO MetricAlt -
Execution time in secs: 28.5150
2013-08-27 22:31:50.504 INFO MetricAlt - Started 8
2013-08-27 22:32:18.048 INFO MetricAlt -
Execution time in secs: 27.5430
2013-08-27 22:32:19.519 INFO MetricAlt - Started 9
2013-08-27 22:32:48.194 INFO MetricAlt -
Execution time in secs: 28.6750
2013-08-27 22:32:49.554 INFO MetricAlt - Started 10
2013-08-27 22:33:18.277 INFO MetricAlt -
Execution time in secs: 28.7210
^C
|
|
|
|
|
|
|
API Support
|
| Post subject: Re: 74% performance degradation backtesting No-op IStrategy between 2.7.9 and 2.8.0 |
Post rating: 0
|
Posted: Thu 29 Aug, 2013, 14:05
|
|
User rating: ∞
Joined: Fri 31 Aug, 2007, 09:17
Posts: 6139
|
|
We managed to replicate the issue on Linux. On windows with both API versions the test performance does not change. The issue is under further investigation.
|
|
|
|
|
|
|
CnF
|
| Post subject: Re: 74% performance degradation backtesting No-op IStrategy between 2.7.9 and 2.8.0 |
Post rating: 0
|
Posted: Sat 07 Sep, 2013, 19:42
|
|
User rating: 1
Joined: Fri 02 Aug, 2013, 12:45
Posts: 6
Location: ItalyItaly
|
|
I have the same issue on Linux
|
|
|
|
|
|
|
CnF
|
| Post subject: Re: 74% performance degradation backtesting No-op IStrategy between 2.7.9 and 2.8.0 |
Post rating: 0
|
Posted: Tue 10 Sep, 2013, 11:04
|
|
User rating: 1
Joined: Fri 02 Aug, 2013, 12:45
Posts: 6
Location: ItalyItaly
|
|
Any news about this issue? Have you been able to reproduce it? It is almost blocking on more complex strategies...
Can we work around it in some way, for example using an older jforex.jnlp with previous packets versions? If yes, can Support post such jnlp here?
Thanks a lot.
|
|
|
|
|
|
 
|
Pages: [
1
]
|
|
|
|
|
