It seems 5.01 breaks the Optimize/Backtest for individual stocks of a watchlist set in the Filter with RangeFromDate and RangeToDate set. Since the GUI only provides portofolio Optimize, you'll see the problem only running it in script code.
Please give it a try with the following parameters set: AA.ApplyTo = 2; AA.RangeMode = 3; AA.RangeFromDate = "01/02/2001"; AA.RangeToDate = "12/30/2001"; AA.Optimize(1); It works fine for AA.Optimize(0) or AA.Optimize(2); It pops "Internal Error" for AA.Optimize(1). Please verify. Thanks, Rolly --- In [email protected], "nhall" <[EMAIL PROTECTED]> wrote: > > I think you misunderstood what I was trying to say. When I'm talking > about signals, I'm not talking about the signals that you actually see > in your Optimization/Backtester results pane, I am talking about the > RAW signals that AB generates during the first pass of backtesting. > These are just temporary signals that can be saved until the end of > processing. Let me try to be a little more clear. This is how I > understand the optimizer currently works for optimizing over a portfolio: > > - Loop through the parameter combinations > - For each parameter combination, loop through every symbol > - Calculate the signals from the AFL for this one symbol > - Once the raw signals have been generated for all the > symbols for this particular combination, generate the > final buy/sell signals and results of the portfolio > backtest for the given parameter combination. > > When I do a portfolio optimization I can see that this order of events > is being done because as soon as one combination of parameters has > been backtested, the results are immediately outputted in the results > pane of the Optimization/Backtest window. > > This method switches between symbols (# Symbols * # Optimization > combinations) times. > > > After reorganizing the order of events: > - Loop through every symbol > - For each symbol, loop through each parameter combination > - Calculate the signals from the AFL for this one symbol > - After the AFL has been run for this symbol for every > optimization combination, save these raw signals in > memory and move on to the next symbol > - Once the raw signals have been generated for all the > symbols, for all combinations, then the final buy/sell > signals and results are generated for all parameter > combinations. > > The downside of this is that the optimization results are not obtained > for any parameter combination until all the combinations have been > backtested (not a big deal in my opinion). > > The upside is that it switches between the symbols only (# Symbols) > times. Say you have a 5000 symbol database and are doing a 100 > combination optimization, that means 500,000 symbol switches with the > current method. By reorganizing the sequence of events that means only > 5000 symbol switches. You would have to save more raw signals to > memory, but this would probably be better than doing all those symbol > switches. The goal is to be processing over one section of memory for > a longer period of time, which will help the cache hit ratio and make > it less problematic to expand to a multi-core processor. > > Nick > > > --- In [email protected], "vlanschot" <vlanschot@> wrote: > > > > Nick, you make one wrong assumption: that the default optimization > > process is based on seperate optimizations per individual symbol. > > Instead, default optimization is at the portfolio level, which is why > > ALL symbols of the backtest-universe need to be included. Optimized > > parameters for one symbol don't have any value if I do not know what > > their impact is at the portfolio level, i.e. on all the other > > symbols. Hope this makes sense. > > > > PS > > > > --- In [email protected], "nhall" <c-yahoo@> wrote: > > > > > > Hello Tomasz, > > > > > > Thanks for all you've done with AmiBroker. It is a great program. In > > > your response to your dual-core optimization comments, there's > > > something I've been wondering about. You said that the core cache > > > becomes a limitation for AFL because backtesting is so memory > > > intensive and the memory interface speed is fixed no matter the > > number > > > of cores on a processor. > > > > > > Currently, if I do an Optimization over a large watchlist, AB will > > run > > > the AFL with the given parameters for the entire watchlist all the > > way > > > until it has the overall system result for the entire watchlist, for > > > the given parameters. Then it alters the parameters according to the > > > Optimization and reruns the AFL for the entire watchlist again. This > > > continues for all the Optimization iterations. > > > > > > As I understand you, the problem with this is that the data for the > > > watchlist takes up a lot more memory than will fit in a core's > > cache, > > > so if you have multiple cores doing this processing simultaneously, > > > they will be fighting each other for memory bandwidth. > > > > > > Would it be possible to alter the order of events? If I'm running an > > > Optimization with 100 combinations I don't need to see the results > > > from each combination until the entire set has been processed. What > > if > > > the Optimization sequence of events was changed to run the AFL for > > > just one symbol from the watchlist, then alter the parameters and > > run > > > the AFL again for the *same symbol*, just with different parameters, > > > and continue this for all the combinations of the Optimization. > > After > > > signals have been generated for this particular symbol for all > > > parameter combinations, the signals can be stored in memory and then > > > it can move on to the next symbol. After all the symbols have been > > > processed, AB can do the backtesting for all the signals. > > > > > > The advantage of this is if I am Optimizing 100 combinations of > > > parameters on a watchlist of 5000 symbols, hopefully 1 symbol can > > fit > > > in the processor's cache and it can do 100 runs through the AFL, > > > generating signals, before it has to fetch more data from the > > memory. > > > This could provide some concurrency as another core could do the > > same > > > thing for a different symbol. The Optimization would be more > > efficient > > > with more combinations. > > > > > > Does this make sense? I know that I glossed over details of how the > > > cache really works and also that I do not know the internals of > > > AmiBroker and could be missing some critical information and > > therefore > > > this idea may not work at all. But maybe it could help. Thanks for > > > listening! > > > > > > Nick > > > > > > > > > --- In [email protected], "Tomasz Janeczko" <groups@> wrote: > > > > > > > > Hello, > > > > > > > > It is perfectly valid question. > > > > > > > > First it does not really matter if the process goes through both > > ends or > > > > sequentially but one core goes though odd and another through even > > > steps, > > > > and at first look it seems like this would give significant speed > > up. > > > > > > > > BUT... in real world things are more ugly that in theory. > > > > I did lots of testing and profiling (measuring time of execution > > of > > > code on function-level), > > > > and dual thread execution on dual core processor is faster if and > > > only if > > > > each core can execute accessing data only from its own on- chip > > data > > > cache. > > > > This is unfortunatelly NOT the case for backtesting/optimization. > > > > On-chip caches are usually limited to well below 1MB. Almost every > > > backtest > > > > requires way more than 1MB. > > > > Now what happens if you run code that uses more memory - > > > > BOTH cores need to access on-board (regular) RAM. Both cores do > > this > > > > through single memory interface that is SHARED between cores and > > > > access one memory that runs at fixed speed (no matter if 1 or 8 > > > cores access > > > > the memory - it can not respond quicker than factory limit and one > > > core is > > > > fast enough to actually need to WAIT for memory). > > > > > > > > Now if you run on 2 or more cores, they have to wait for the same, > > > single shared memory > > > > that runs at constant pace, slow enough for one core, not to > > mention > > > more. > > > > > > > > Net result is that if you actually try to run something that needs > > > more than 1MB > > > > of data and does not fit into individual data cache, the > > performance > > > drops down > > > > to actually single-core. What's more it can run slower because of > > > additional overhead with > > > > thread management. > > > > > > > > And it is not imagination or theory. I did actual code profiling > > and > > > I was surprised to when I tested multi-threaded > > > > code. It works upto 2x faster, on dual core BUT ONLY IF you don't > > > access more than > > > > the size of on-chip per-core data cache. Or your code needs way > > more > > > calculation than memory access. > > > > If your code does a LOT of memory access (more than 1MB) and does > > it > > > QUICKLY > > > > (backtesting is extremely memory intensive and AFL scans through > > mem > > > like crazy) > > > > all advantages of running in multiple cores are gone. > > > > > > > > BTW: what I did in this upgrade to speed up the > > > backtest/optimization was to reduce the COUNT > > > > of memory accesses to absolute minimum required. As it turns out > > > even single CPU core was waiting for memory. > > > > > > > > Best regards, > > > > Tomasz Janeczko > > > > amibroker.com > > > > ----- Original Message ----- > > > > From: "tipequity" <tagroups@> > > > > To: <[email protected]> > > > > Sent: Friday, October 05, 2007 4:20 AM > > > > Subject: [amibroker] Re: Optimization speed increase in 5.01 > > > > > > > > > > > > > Tomasz, at the risk of sounding stupid, I am gonna run this > > idea by > > > > > you. Since AB during backtest and optimizations work on a list > > of > > > > > stocks why not have one cpu (dual core CPUs) work on symbols > > from top > > > > > of the list and another cpu to work on symbols from the bottom > > of the > > > > > list. Like buring candles from both end. > > > > > > > > > > Regards > > > > > > > > > > Kam > > > > > > > > > > > > > > > --- In [email protected], "Tomasz Janeczko" <groups@> > > > > > wrote: > > > > >> > > > > >> Hello, > > > > >> > > > > >> If you are running optimizations using new version I would > > love to > > > > > hear about the timings you get > > > > >> compared with old one. > > > > >> Note that optimization with new version may run even 2 times > > faster > > > > > (or more), > > > > >> but actual speed increase depends how complex the formula is > > and > > > > > how often system > > > > >> trades and how large baskets. Speed increases are larger with > > > > > simpler formulas, > > > > >> because AFL execution speed did NOT change. The only things > > that > > > > > has changed > > > > >> is collection of signals (1st backtest phase) and entire 2nd > > phase > > > > > of backtest. > > > > >> As it turns out, when backtesting very simple formulas the AFL > > code > > > > > execution is only less > > > > >> than 20% of total time, the rest is collecting signals and > > sorting > > > > > them > > > > >> according to score and 2nd phase of the backtest (actual > > trading > > > > > simulation). > > > > >> These latter areas were the subject of performance tweaking. > > > > >> > > > > >> Best regards, > > > > >> Tomasz Janeczko > > > > >> amibroker.com > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > Please note that this group is for discussion between users > > only. > > > > > > > > > > To get support from AmiBroker please send an e-mail directly to > > > > > SUPPORT {at} amibroker.com > > > > > > > > > > For NEW RELEASE ANNOUNCEMENTS and other news always check > > DEVLOG: > > > > > http://www.amibroker.com/devlog/ > > > > > > > > > > For other support material please check also: > > > > > http://www.amibroker.com/support.html > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
