Hi

I am wishing to set up a trading system which will -

1) collect hourly bar data in real time and save this data (to a database?)

2) pre-process this data and prepare it as input to a neural network

3) run the neural network and use output to setup a trade order

4) repeat - every x hours - closing any open trade and commencing new trade

I have the neural network aspect sorted but am seeking advice on how to bring this all together in one java program

Any suggestions or advise greatly appreciated

Thank you

Bob M

Hi us_copiosus:

In my opinion, you can avoid some steps...

Just integrate the "neural network" inside your strategy, and this "neural network" can read and process the data in real time... you need the information of last bars, you can get them with history bars, if you don't "stored" them before.

I suppose you have the "neural" part in java.

trade well

JL

Hi JLongo

Yes - I agree with your thoughts but I am struggling to understand how to setup a database or whatever off to one side where the basic hourly bar data is stored.

The input into the neural network is dynamic - new instances are always added and the oldest ones dropped off

Yes - the neural network is coded in java

Are you suggesting that one doesn't need to have a database off to one side which stores basic data and is updated regularly?
Are you saying that this should all be done within the strategy?

Bob M

Hi us_copiosus:

Exactly that, if you don't need a great number of data that exceeds memory limits...

If it is as you say, some data in, some data out (FIFO method)... you can store all needed data in memory for fast process.

Each bar object stores 5 double values (OHLC values and volume), if you use bid and ask bar will be 10 doubles (80 bytes per bar in simple way)... Of corse, you need to take in consideration the number of bars you need and the number of instruments to use.
Reading from database or other stored format takes time and makes no sense if you don't need all data over time, but some from last elapsed time.

Without more information is hard to advice the use or not of some store method, database, file, whatever...

Trade well

JL

EDIT: There exist some examples on wiki how to connect to a database (in this case mysql) on "Pratices" section. But you can use Java DB as it is included on java JDK, or other simple databases as HSQLDB or DB4O, as they don't need to work only on server mode (you can use local db files without the need to deploy a db server).

1) data and memory limits
I would be storing 5 double values OHLC and volume - bid only
Memory limits - I have no idea

2) Number of bars and number of instruments
I am using a 6 hour turn around, so every 6 hours I need to collect the last 6 hour bar's data
Pre-process them and input to NN.
Pre-process includes calculating 11 attributes as input to NN.
Number of instruments - I am starting with one but would like to move on to using six say

3) database
I was intending to try and use the java DB (Derby) database

All my pre-processing is currently done in excel where I am more familiar

Talking about deploying a db server - this is where you loose me completely
But I think you are saying that if I do use a simply database it runs on my laptop and I don't need to deploy a db server

I also need to store all past trades in a database in addition to the historical bar data and my on-going attribute calculations

Overall, I feel that using a dbase would be the way to go, rather than storing more and more within the java programme

Bob M

Well, if you need the persistence of some data... the history of bars and the resulting process and orders data, yes, the use of database is the way to go depending on amount of data or the way we want to consult the data.

You can use Java DB (the same as Derby) in server mode, or in embedded mode (server mode you need to start the server part outside your application, embedded mode, it runs inside your application but only your application can access the db - this all in a simple way because some variations are allowed)...

Deploying a server, i mean we need to configure and start a db server... to be capable to access it.

Now... about the processing part... i think by what you describe, you don't need to store the bars to the database at beginning (before the process part). You get them from history if you don't have them at start, and you will have the next ones on real time. After processing, and if you want to keep a record of the bars processed and the result of it, at this moment you store it to the db...

Again, just my point of view with the limited information you provided.


Trade well

JL

Many thanks for your thoughts and also the series of articles
I have a copy of all of them and have learnt so much from them

Kind regards

Bob M
Dunedin
New Zealand

Hello again

Strategy running on JForex platform - demo mode

I have successfully coded the part of the strategy which collects new data periodically
I have also pre-processed this new data ready to add a new record to my DB (Derby)

I believe that I wish to run Derby in embedded mode....

Can somebody well-versed in this area provide clear step-by-step instructions on what I need to get the connection working

I do understand the area of adding/deleting records in the DB database.

Many thanks

Bob M

I guess my main problem at the moment is :-

For mysql - one has 3 jar files required in files folder
Commons-pool, commons-dbcp and mysql java connector

For DB (Derby) - one requires ?????

Bob M

Hi us_copiosus:

Java DB is part of the JDK from version 1.6 if i'm not in mistake... it's located on java base directory/db/lib.
If you use the JRE only, you need to add the libraries and a classpath to this library (and ofcourse use @Library inside the strategy).

Trade well

JL

So are you saying that if I am running Java JDK 7, I don't have to add anything extra?

Bob M

Hi us_copiosus:

Simple answer: yes... As you may have some more questions, a good start is -> https://www.oracle.com/technetwork/java/ ... 56714.html.

Trade well

JL

Many thanks

Bob M