Hello, Turn on QuickAFL in AA.
Other than that, there is very simple reason why more data leads to performance degradation: every bar of your data is 40 bytes. Multiply number of bars x 40. You will get memory requirement for data alone per symbol. Now compare this to CPU cache size. If your CPU has say 2MB on-chip L2 cache it can keep on-chip only 2MB/40 = 50K bars. Anything more than 50000 bars does not fit onto on-chip cache. Now since system RAM is 10 times slower than CPU cache, as soon as you reach the size of on-chip CPU cache you will hit performance drop. Next boundary is when your system runs out of physical free RAM and starts swaping. This is easily 3 orders of magnitude slower than RAM. For this simple reason - keep your databases *small* if you want speed. Best regards, Tomasz Janeczko amibroker.com ----- Original Message ----- From: "davemabe2000" <[email protected]> To: <[email protected]> Sent: Thursday, July 30, 2009 2:07 PM Subject: [amibroker] Maximizing Backtesting Performance on Large Data Sets > When I create a database and insert 1.5 years of 5 minute bar data, the > optimization I run takes around 2.5 hours. > > When I add more data to the database to end up with 8 years of 5 minute bar > data, the same optimization takes 4 days+ > (exponentially worse). > > Even when I restrict the date range to a 1.5 year period in the larger > database, a single backtest is noticeably longer than the > same backtest with just the 1.5 years in the database. > > Is there anything that I can do to improve the performance of the > optimization over the larger period? > > My plan now is to create a database for each 2 year period and run the > optimizations on each individually. > > Any other ideas? Any tricks for getting more performance on a large data set? > > > > ------------------------------------ > > **** IMPORTANT PLEASE READ **** > This group is for the discussion between users only. > This is *NOT* technical support channel. > > TO GET TECHNICAL SUPPORT send an e-mail directly to > SUPPORT {at} amibroker.com > > TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at > http://www.amibroker.com/feedback/ > (submissions sent via other channels won't be considered) > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > http://www.amibroker.com/devlog/ > > Yahoo! Groups Links > > >
