You know TJ, now that you put it that way, I think you are right. ( Not sure if I should call them "plateaus" as I am testing with 5 dimensions/Opt() statements, but I am referring to groups of similar param values that return consistant results ).
Actually, it does seem to find *some* plateau or another almost always, but often it is not the higher ( i.e., most profitable ) ones that I know are there and want it to find. So I think the shorter/less profitable plateaus are probably broader and more numerous and easier to find, and if it finds a short one first then perhaps it concentrates largely on the local area for a while and doesn't get to find the good ones before time runs out. Well for now, using lots of runs seems to do the trick for the most part. Thank you for reminding me about the source code, it looks like you put a lot of work into commenting and making it easy for us ( thank you! ) so I think I can understand enough of it to make some tweaks if I want to try that later... Steve ----- Original Message ----- From: Tomasz Janeczko To: [email protected] Sent: Sunday, August 10, 2008 3:48 AM Subject: Re: [amibroker] CMAE question CMAE via its iternal convergence logic tends to find plateaus, however if the size of the problem is large enough and number of tests is several orders of magnitude smaller it may find local ones instead of global ones. With 31 runs you are getting 2^(31-10) more tests thats why you are getting more consistency too. Using source codes provided it is possible to add extra sensitivity tests into but that does not add as much good as one may think in case of CMAE. Best regards, Tomasz Janeczko amibroker.com ----- Original Message ----- From: Steve Dugas To: [email protected] Sent: Sunday, August 10, 2008 3:10 AM Subject: Re: [amibroker] CMAE question Thanks Fred, that is what I thought and it makes sense but I was not sure how it works internally. I guess it just does fewer tests but there is a logic that determines which points it tests next, based on results from recently tested points. So for the heck of it, let me reverse the question. Assuming it finds something good, and that it then tests tests some other points in the area, I wonder if we might assume, since it only returned the one line at 200%, that most likely the surrounding values were not so good? BTW, I have at times found some pretty nice plateaus with CMAE, sometimes they jump right out at you the same as with an exhaustive opt. The problem was that it would often miss them too. Even though the math may not make sense, I find that results seem much more consistant with 31 runs. Go figure? Steve ----- Original Message ----- From: Fred Tonetti To: [email protected] Sent: Saturday, August 09, 2008 7:39 PM Subject: RE: [amibroker] CMAE question The only way to purposely find plateaus is to test for them during in sample optimization . Not doing so may result in some sort of plateau any way but you won't really know where your parameter values are relative to the geography of the plateau or if other larger plateaus exist -------------------------------------------------------------------------- From: [email protected] [mailto:[EMAIL PROTECTED] On Behalf Of Steve Dugas Sent: Saturday, August 09, 2008 3:44 PM To: [email protected] Subject: Re: [amibroker] CMAE question Thank you TJ - Would you mind answering one more question about the results? Suppose I am optimizing on CAR. And when the CMAE opt is done, the first line shows 200%, followed by a bunch of lines at 100% with closely related params. In exhaustive opt, I would ignore the 200% in favor of all the 100%'s. In CMAE opt, does the fact that it returned one 200% line mean that it has found some sort of a plateau around those param values - i.e., would you consider trying those param values going forward? Thanks very much! Steve ----- Original Message ----- From: Tomasz Janeczko To: [email protected] Sent: Saturday, August 09, 2008 3:24 PM Subject: Re: [amibroker] CMAE question Hello, Zero as default is good. In that case internally it is set to 900 * square of ( number of params + 3 ). As for Runs you should never go beyond 10 because of exponential growth of search space. Runs 31 means 2^31 (=2147483648 ) times larger search. Makes no sense. Best regards, Tomasz Janeczko amibroker.com ----- Original Message ----- From: Steve Dugas To: [email protected] Sent: Saturday, August 09, 2008 8:52 PM Subject: Re: [amibroker] CMAE question Hi TJ - I read in the ReadMe that the default for MaxEval is 0, and it is advised not to change it. If I define it myself as 0, is that different from not defining it at all and just accepting the default? Yes, I read that it is advised not to go over 10 runs, but in experimenting, Runs = 31 and MaxEval = 0 seems to return quite good and consistant results whereas 5 or 10 runs seems hit-or-miss and results can vary quite a bit. But maybe I am screwing things up by defining Runs as 0? Steve ----- Original Message ----- From: Tomasz Janeczko To: [email protected] Sent: Saturday, August 09, 2008 2:29 PM Subject: Re: [amibroker] CMAE question Hello, This is entirely wrong combination of params. MaxEval should be AT LEAST 2000. Runs should NEVER be set to anything greater than 10, with 5 being resonable max. Best regards, Tomasz Janeczko amibroker.com ----- Original Message ----- From: Steve Dugas To: Yahoo - AmiBroker Sent: Saturday, August 09, 2008 8:03 PM Subject: [amibroker] CMAE question Hi - I am playing around with the CMAE opt engine. MaxEval is set to 0 by me (instead of just accepting default). For Runs, if I enter any number up to 31, it will warn me about taking lots of time, and at 31 runs it says I have about 1.5B opt steps. If I enter any any number > 31, it may or may not warn me, and when it doesn't warn me, behavior of the progress box appears different. Am I overflowing the max allowed opt steps? Is there some limit to this algorithm? Any other thoughts why things are different sometimes? Thanks very much! Steve -------------------------------------------------------------------------- I am using the free version of SPAMfighter for private users. It has removed 522 spam emails to date. Paying users do not have this message in their emails. Try SPAMfighter for free now!
