technical advise: the Java implementation from Oracle/Sun on Apple OSX is very slow compared to Windows, on Linux I have no experience. Last year or so I did a performance benchmark, and the JVM running on Windows was nearly twice as dast as running on OSX on the same machine.
On Wednesday, January 16, 2013 12:01:01 AM UTC+1, mwsailing wrote: > I wanted to see if anyone would post additional advice on optimization > workflows that work, and perhaps help me explore ways to improve my own. > Here are some things that work for me so far, but I'm not yet satisfied > with the efficiency. > > What works for me: > - I come up with model theory in my head, about what might improve my > trades. I do this by observation of prior trades (backtest utility), and I > try to see obvious things the Strategy didn't get right. For this, I love > the "chart" feature of the backtest data. > - I then code some changes, and watch backtest again, knowing that it > isn't optimized, but I look for some improvement with any algo changes. > - Then I run divide-conquer optimization, across a broad range of > min/max. But, I never can tell at what scale I want to operate on. How > fast is a fast ema period, etc.... I'm always a bit unsure if even my > broad range is broad enough, but I eventually settle on something. > - If the divide and conquer method started to look promising, I note (and > hope to see) hot spots in the heat map. Unfortunately, it is usually too > spares to really see and understand what is going on. So, I take a guess at > some ranges for each param. > - Finally, if all went well, I setup an overnight run (usually around > 3,000,000+ "combinations", which takes forever on my blazing fast new > macbook retina with 16GB ram, etc, etc.. Usually 20 hours with the fan on > high. Maybe I need Applecare after-all. > - Now maybe I have something, then I take the best island area in this > process, back-test it, and start to study trades again. Mostly, this puts > me back into a loop of trying to make another Strategy change to improve it > further. > > So, this is the process I came up with so far. It works, I get > improvements, but it is very slow going. > > What might need improvement in either my process or the jbooktrader: > - I wish to see good/bad trades fast! The chart graphs everything, and > hence takes forever. I am thinking of a "trade view" chart, which shows > only the area around the trade, in great detail, then maybe advance to the > next trade fast. This would speed up my review of trades. > - Optimizer, figuring out why it takes so long. I know, 3,000,000 > combinations is a lot of work. But, I feel like maybe studying this in a > profiler, and trying to make some improvements. Nevermind the wild ideas I > have about farming out work to a dynamic cluster of EC2 servers using > hadoop. > - Indicator graphing. I have placed indicators I don't even use into > Strategies, just to see them in the graph, and get a feel for if they make > sense. Maybe people use some external program for this? If so, how to > tell if your indicator logic is correct. So, I dirty up my strategies with > unused indicators, just to see them on the graphs. > - I considered trying to wire up scalalab (and learn it) to my java > strategies, so i can play with more advanced analysis without rolling my > own code for math routines. For example, write scalalab adapter to run my > strategies, and use it like matlab to make improvements. I could then use > the same java code for both math software package, and trading. A cool > idea, but I only got as far as installing scalalab, nevermind learning it. > - Move this onto a giant linux box in my home, and offload this > optimization for now. > > As you can see, I'm all over the place, any advice? Did anyone else have > any of these ideas? Maybe you did, and already executed on it? Since I > have so many thoughts, each which require a lot of work, I'm seeking some > feedback from people who have been doing this for years. I'm a newbie. > Maybe I just need to learn how to use what we have better? > > Meanwhile, running a giant CL optimization, maybe I'll have something to > trade soon. > > Marcus > > > > > > > -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To view this discussion on the web visit https://groups.google.com/d/msg/jbooktrader/-/XxtGLaGhyhAJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.
