I am trying to perform a portfolio optimisation over a 4 year period with multiple currency pairs, only the cache manager is attempting to open too any files to access my cache. At 4 currency pairs (and actually running - any more pairs and it starts genrating errors) it has in excess of 600 files and over 500 threads open. Most of the files are candles/bars on multiple timeframes (in form cache/PAIR/YR/MTH/BID/ASK_candles_Period.bif), but many are also file of the format /tmp/jforexNNNNNNNNN.tmp Once the test gets well under way, these tmp files are closed, but the candles are not.

The actual error is see is "java.lang.OutOfMemoryError: unable to create new native thread" I have tried lowering the thread stack size but it was no help (and 500 threads is not too much for this) I suspect its some form of open file descriptor limit. I can attempt to increase these limits but my question is why does it attempt to open so many cache files at the same time

I see that the cache keeps the candles open for the duration of the test

Is this broken, necessary, a configurable item, too many pairs/backtest data?

The environment in Fedora 14, 64-bit. JavaVm only option is -server, testing from within Eclipse (not JForex gui) - no other jvm options in use - Java 1.6.0_23 - AMD Phenom 6-core, 16GB

If the thread stack size decreasing does not help, try to decrease java heap size also.
Java thread stacks are not stored in the Java heap, they are stored in the general memory of the virtual machine.
This effectively limits the number of threads that an application can create.

If the general memory is not enough it can be reason of a problem.
Thus the java heap size decreasing will increase the general memory allowing the creation of a larger number of threads.