Steve, You're a lucky man. My last CMAE optimization took 19 days to iterate 37,000 times on a 3.11 Ghz octo core CPU with 4Gb of memory (32 bit Win XP). Though, only 1 core really ever gets stressed at a time.
I suspect that a lot depends on - The nature of your result space - Your code - The number of symbols being used - The time span covered - Usage of QuickAFL In my case, the fitness function was UPI multiplied by a number of penalty weights for such things as number of trades, trade duration, etc. The optimization was over 2 years of all NYSE, AMEX, NASDAQ, included calls to Foreign, AddToComposite and employed custom backtesting for determining position size and calculation of the fitness function. I did not use QuickAFL. I've been too busy to look into it yet. But, I'm assuming that something like MCO would not apply to my scenario since I use composites. If anyone knows differently, please feel free to speak up. Mike --- In [email protected], "Steve Dugas" <sjdu...@...> wrote: > > Hi Ton - It is the norm for me, when I start an optimization, for AB to tell me that it will take several months or more to run, I think I remember a few where it said 50 or 100 *years*, something like that. But with the IO engines you can go ahead and run it anyway, for me they always finish probably within an hour, sometimes much quicker. The times can vary a bit, I think maybe it depends on where in the opt space they start, what paths they take from there and what it leads them to... Sometimes they will finish and report results in 5 or 10 minutes, other times can be an hour or maybe a little more, most will be somewhere in the middle. At least that is how it always works for me. I definitely agree with you, I am not looking for peaks like the one you posted either but sometimes there are smaller and more profitable plateaus that are tradable for months and I like to at least find them and know they are there... > > Steve > ----- Original Message ----- > From: Ton Sieverding > To: [email protected] > Sent: Wednesday, February 04, 2009 11:27 AM > Subject: Re: [amibroker] SPSO vs Trib vs CMAE, was: random optimization? > > > Your are talking about ".... up into the trillions and more.". How are you handling the time problem ? These should be optimizations of several months ... Even with CMAE etc. it will be still a question of weeks ( about 1/4 of the time ). Let's say I want to optimize 100 different systems on 10 different Symbols or 1.000 combinations times your trillions. It's a life time. I just do not have a solution for this. Do you ? > > And again optimization on points is not what I would like to do. Because of the underneath picture ... I would like to optimize on areas in stead of points. I hoped to get this with CMAE. The result was negative ... > > Regards, Ton. > > ----- Original Message ----- > From: Steve Dugas > To: [email protected] > Sent: Wednesday, February 04, 2009 12:21 AM > Subject: Re: [amibroker] SPSO vs Trib vs CMAE, was: random optimization? > > > > Hi Ton - The 2 MA crossover system was just a simple example for illustration purposes. In real life I would do an exhaustive opt on that one since it would only have maybe 100 x 100 = 10,000 combinations, and perhaps the small optimization space is why CMAE was able to find the peak. The systems I test with the IO engines generally have at least millions of possible combos and some up into the trillions and more. FWIW, I have done lots of these tests and I will have to stand by my earlier remarks because that is my honest experience, but perhaps others may see different results... > > Steve > ----- Original Message ----- > From: Ton Sieverding > To: [email protected] > Sent: Tuesday, February 03, 2009 3:40 PM > Subject: Re: [amibroker] SPSO vs Trib vs CMAE, was: random optimization? > > > Interesting ... > > ..... results, then ran lots of IO tests and compared them to the exhaustive > results to see what the IO's found and also what they missed. You could say > that CMAE seems to take the "safe" approach, IMHO it finds the broad > plateaus pretty well but as you might guess they are usually far from the > most profitable. In my experience, the other two IO engines will generally > find those too but they also find a lot of the smaller and more profitable > ones, which you can then run a mini exhaustive opt on to get a more complete > picture .... > > Is that true ? Does CMAE really take the 'safe' approach ? Look to following > picture and see what CMAE gave me as an optimum ... > > > > I got the left peak and hoped to get the plateau in the middle ... > > Regards, Ton. > > > > > ----- Original Message ----- > From: Steve Dugas > To: [email protected] > Sent: Tuesday, February 03, 2009 7:54 PM > Subject: Re: [amibroker] SPSO vs Trib vs CMAE, was: random optimization? > > > Hi Steve - Once you have done an IO and found some results that look > promising, then you can run a mini exhaustive opt if you want. For a simple > example, you run an IO on a MA crossover system, testing both MA's with > periods from 1 to 100. You won't see all possible combos reported but maybe > the results show that MA1=10 and MA2=20 might be good. So to see all the > other values in that neighborhood you could then run a little exhaustive > opt, say MA1 = 5 thru 15 and MA2 = 15 thru 25, something like that, which > will run in a reasonable time. > > To test the built-in IO engines, I ran a few exhausive opts and saved the > results, then ran lots of IO tests and compared them to the exhaustive > results to see what the IO's found and also what they missed. You could say > that CMAE seems to take the "safe" approach, IMHO it finds the broad > plateaus pretty well but as you might guess they are usually far from the > most profitable. In my experience, the other two IO engines will generally > find those too but they also find a lot of the smaller and more profitable > ones, which you can then run a mini exhaustive opt on to get a more complete > picture. > > Regarding the trade-off you mentioned, I would think it is a matter of > personal taste. How greedy are you? 8 - ) How risk-averse? I am > inclined to try the smaller and higher plateaus first, as long as they have > a little play on each side and are doing well right now, and knowing that > they will fail eventually and I need to keep a close eye on them... Good > luck! > > Steve > > ----- Original Message ----- > From: "Steve Davis" <_sda...@...> > To: <[email protected]> > Sent: Monday, February 02, 2009 5:01 PM > Subject: [amibroker] SPSO vs Trib vs CMAE, was: random optimization? > > > Steve, > > > > I would like to hear more about your system optimization process. How > > were you able to determine the size of the plateaus discovered by the > > built-in optimizers, and how did you decide which solutions had the > > best trade-off between plateau size and profitability? > > > > Thanks, > > another Steve > > > > --- In [email protected], "Steve Dugas" <sjdugas@> wrote: > >> > >> Hi - I have spent lots of time playing with the built-in intelligent > >> optimizers, in my experience SPSO will return the same results every > > time if > >> the settings are the same. Trib and CMAE will probably return different > >> results each time. FWIW, I find CMAE to be the worst of the three and I > >> don't use it anymore, it will find plateaus but nearly always misses > > the > >> much more profitable but smaller plateaus. Using a quad- core I can > > run 4 > >> simultaneous instances and I find that by running 1 SPSO and 3 > > Trib's and > >> then comparing the 4 results together, it will generally point me to > > some > >> pretty good param values. Good luck! > >> > >> Steve > >> > >> ----- Original Message ----- > >> From: "gabriel_id@" <finance@> > >> To: <[email protected]> > >> Sent: Monday, February 02, 2009 7:25 AM > >> Subject: [amibroker] Re: random optimization? > >> > >> > >> > OK.. > >> > > >> > Can u give me what type of engine and with what kind of settings will > >> > get the same results when i optimize this lines: > >> > > >> > N = Optimize("N-minutes", 33, 1, 60, 1); > >> > TimeFrameSet( N * in1Minute ); > >> > MA1 = MA( Close, 10); > >> > MA2 = MA( Close, 20); > >> > BuySignal = Cross( MA1, MA2); > >> > sellSignal = Cross( MA2, MA1); > >> > TimeFrameRestore(); > >> > > >> > Buy = TimeFrameExpand(BuySignal , N*in1Minute); > >> > Sell = TimeFrameExpand(sellSignal , N*in1Minute); > >> > > >> > I tried cmae, 5 , 1000, have variable results.. on walkforward > >> > i tried spso, 5, 1000, same variables results.. > >> > and also trib, 5, 1000.. > >> > > >> > > >> > --- In [email protected], "Mike" <sfclimbers@> wrote: > >> >> > >> >> Tribes is a non exhaustive optimizer, meaning that it does not > >> >> evaluate every possible combination. > >> >> > >> >> As such, it is possible that it will find different "optimal" > >> >> solutions every time, depending on the nature of the surface being > >> >> optimized. For example; If the surface has many similar peaks, it may > >> >> land on a different one each time (local optima) instead of the one > >> >> true optimal solution (global optima). > >> >> > >> >> Try increasing the number of Runs and/or MaxEval. If you have more > >> >> than 2 or 3 optimization variables, 1000 MaxEval is not enough. > >> >> > >> >> http://amibroker.com/guide/h_optimization.html > >> >> > >> >> Mike > >> >> > >> >> --- In [email protected], "gabriel_id@" <finance@> wrote: > >> >> > > >> >> > hi there, > >> >> > > >> >> > i am a bit confused, i run the same optimization process.. on same > >> >> > data range.. and i got different results each time :) > >> >> > > >> >> > and the engine was trib, 5, 1000... > >> >> > > >> >> > thx, > >> >> > GV > >> >> > > >> >> > >> > > >> > > >> > > >> > ------------------------------------ > >> > > >> > **** IMPORTANT **** > >> > This group is for the discussion between users only. > >> > This is *NOT* technical support channel. > >> > > >> > ********************* > >> > TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail > > directly to > >> > SUPPORT {at} amibroker.com > >> > ********************* > >> > > >> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > >> > http://www.amibroker.com/devlog/ > >> > > >> > For other support material please check also: > >> > http://www.amibroker.com/support.html > >> > > >> > ********************************* > >> > Yahoo! Groups Links > >> > > >> > > >> > > >> > > >> > > > > > > > > ------------------------------------ > > > > **** IMPORTANT **** > > This group is for the discussion between users only. > > This is *NOT* technical support channel. > > > > ********************* > > TO GET TECHNICAL SUPPORT from AmiBroker please send an e- mail directly to > > SUPPORT {at} amibroker.com > > ********************* > > > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > > http://www.amibroker.com/devlog/ > > > > For other support material please check also: > > http://www.amibroker.com/support.html > > > > ********************************* > > Yahoo! Groups Links > > > > > > > > >
