Actually Bruce Robinson pointed this out to me long before I employed the technique in IO . I guess for some reason it didn't initially sink in .
_____ From: [email protected] [mailto:[EMAIL PROTECTED] On Behalf Of brian_z111 Sent: Friday, May 02, 2008 8:47 AM To: [email protected] Subject: [amibroker] Re: Copying/Limiting Database history Gotcha. That's a "nice work" (for being father of the idea) from me too. brian_z --- In [EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com, Fred Tonetti <[EMAIL PROTECTED]> wrote: > > As I stated and you reiterated, IO Clones symbols in order to increase speed > of optimization. See the comparative run time chart below. > > > > It does this in essence by manufacturing and running an AFL to export > symbols with limited data history and then importing them under a different > easily identifiable symbol. This is relatively fast and from the > perspective of having to match dates totally painless as AB will match the > dates when the cloned data is imported. > > > > IO does it this way because one might be using IO Servers which can not be > expected to have the same completeness of data for the symbol in the same > data directory nor even the same symbol in the same data directory nor even > the same data directory. > > > > > > > > _____ > > From: [EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com [mailto:[EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com] On Behalf > Of brian_z111 > Sent: Friday, May 02, 2008 6:08 AM > To: [EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com > Subject: [amibroker] Re: Copying/Limiting Database history > > > > Tomasz, > > I empathise with your position. > > When you produce 'customiseable' and complex software the questions > are almost infinite (10000^N where N == newusers). > > In a world where transparent exchange of information is the benchmark > isn't getting any easier. > > When I said "discussion isn't permanent and info is best provided in > docs or a KB" I put my money where my mouth is. > > My new AmiBrokerTradersProject (ATP) is intended to go to the next > level above and beyond the UKB (of course it is up to AB and the > community if they want that or not -if either of those parties don't > want it I will just go to the beach instead). > > > It does so for indicators, commentaries, tooltip display and > >interpretations. > > It does NOT do so in AA. > > It would have been enough for me if you had only said the above > + "the exception is when using Pad & Align" (from memory I don't > think that particular special 'bars loaded behaviour' of P&A is > spelled out anywhere in docs - from my side of the screen I can't > guess if all bars are loaded or not, depending on the circumstances). > > I am sure a lot of newcomers will be happy to receive your > explanation though. > > > I have posted artificial ticker code earlier. If I implement it > >in "easier" way, I am still > > going to "hide" it from "casual user" because of points rised above. > > Creating an artificial 'market day' ticker isn't a difficult problem > (rather tedious matching the calendar/market holidays for each market > though but that isn't an AB problem). > > I should thing 'advanced' users would be happy to use a function to > over-ride default AA 'load all bars behaviour' and then have to deal > with the consequences (as we do now in Foreign(fixup == 0)). > > I don't know about other people but I know how many bars I need at > all times and in all places. > > BTW isn't there already some 'problems' to deal with, if using P&A > for optimization truncation, because data with holes will be > automatically fixed, whether the user wants it or not???? > > Wouldn't we need something like SetBarsRequired(100,0, enabledinAA = > true, fixup = true)???? > > Anyway, thanks for your answer. > > As always you make me think about things. > > brian_z > > --- In [EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com, > "Tomasz Janeczko" <groups@> > wrote: > > > > Hello, > > > > > I thought AB default was to load a 'few' more bars than required > by > > > AFL > > > > It does so for indicators, commentaries, tooltip display and > interpretations. > > > > It does NOT do so in AA. > > > > Why? It was discussed in the past already. > > I am wondering if any other vendor on this planet gives you inside > out information > > about literally everything. I should probably start saying "because > of technical reasons. period". > > But OK, lets go over it once again. > > There are two main reasons for that: > > > > a) AddToComposite use. > > When you create composites you want them to include all data > consistently, > > not only say one last bar if it happens that you run scan on one > last bar only > > > > b) Trading system consistency and accurracy. > > Your trading system may rely on indicators that have recursive > formulation > > (functions like Cum, EMA, AMA) and others having "unknown" lookback > period > > (ValueWhen). Because of such formulation they use unlimited or > nearly unlimited > > lookback period. Your trading system results may be severely > affected if > > you truncate time series. What does not matter for charts (due to > display limitations > > you can not really see difference of less than 0.5..1% on charts), > may be important > > if you use trading system. The same also applies for exploration. > You probably > > want full accurracy. > > > > If AmiBroker truncated data series in AA by default, we would > simply get > > dozens of e-mails saying "results are inaccurrate/incorrect" simply > because > > most people are unaware of things like recursion. I have seen > people comparing > > absolute values of indicators such as Chaikin Oscillator, without > understading > > its concept of running sum over all loaded bars. Now if AA did not > load all data, > > they would run the same exploration code with different "start" > date and they would > > get different result at the same "end" date. > > > > At support we already spend significant amount of time educating on > such basic things. > > > > For advanced users, who know what they are doing, there is a pad > and align feature > > that allows to use artificial ticker to control on how many bars > your code is run in AA. > > I have posted artificial ticker code earlier. If I implement it > in "easier" way, I am still > > going to "hide" it from "casual user" because of points rised above. > > > > Best regards, > > Tomasz Janeczko > > amibroker.com > > ----- Original Message ----- > > From: "brian_z111" <brian_z111@> > > To: <[EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com> > > Sent: Friday, May 02, 2008 4:29 AM > > Subject: [amibroker] Re: Copying/Limiting Database history > > > > > > >> Fred and I and probably > > >> others clone symbols to a specified date minus 252 bars to allow > > > long > > >> lookback functions to stabilize. This data truncation has a > > >>PROFOUND > > >> impact on execution times. > > > > > > Can anybody please explain why this speeds up execution. > > > > > > I thought AB default was to load a 'few' more bars than required > by > > > AFL, so if the formula filters by datenum OR range OR barindex > etc > > > why is that slower than using 'Pad and Align' with a 'shortened' > > > symbol. > > > > > > Doesn't AFL only load and process from the filtered date on? > > > If not, what does it do? > > > > > > What does a 'Pad and Align',with a 100 bar reference, do > differently > > > to a barindex filter of 100 days, as far as loading and > processing > > > time goes? > > > > > > I would appreciate it if anyone can clear this up for me. > > > > > > brian_z > > > > > > > > > --- In [EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com, > "bruce1r" <brucer@> wrote: > > >> > > >> Herman - > > >> > > >> First, I agree, the new status functions look wonderful and will > > > allow > > >> some interesting app's. > > >> > > >> In reference to your note about cloning data, I've tried > lobbying > > > TJ > > >> at conferences for 2 years to add this to the Settings. He > > >> acknowledges the gain, so maybe someday soon. Fred and I and > > > probably > > >> others clone symbols to a specified date minus 252 bars to allow > > > long > > >> lookback functions to stabilize. This data truncation has a > > > PROFOUND > > >> impact on execution times. Depending on the amount of AFL code, > and > > >> if you only need a couple years of history, speedups of 3x+ are > not > > >> uncommon. Very important to optimizations. > > >> > > >> There is another way, though, and that is why I thought I'd > write a > > >> quick note. If this method was available through COM > automation, I > > >> would have used it instead of cloning because the overhead is > > > minimal > > >> and the benefits are almost as great. > > >> > > >> Go into settings and set "Pad and align...", and set the > reference > > >> symbol to a ticker that covers the desired date range minus a > number > > >> of bars for functions to stabilize. For example, if you were > using > > >> the QP3 database, you might set it to IEZ, which started in > 5/5/2006 > > >> to get a 1 year history minus 1 year of bars for stabilization. > You > > >> will get a significant speedup. > > >> > > >> Plus it is simpler than an export/import. Just remember to "un- > set" > > >> it when it isn't needed. > > >> > > >> Bruce R. > > >> > > > > > > > > > > > > ------------------------------------ > > > > > > Please note that this group is for discussion between users only. > > > > > > To get support from AmiBroker please send an e-mail directly to > > > SUPPORT {at} amibroker.com > > > > > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > > > http://www.amibroke <http://www.amibroke <http://www.amibroker.com/devlog/> r.com/devlog/> r.com/devlog/ > > > > > > For other support material please check also: > > > http://www.amibroke <http://www.amibroke <http://www.amibroker.com/support.html> r.com/support.html> > r.com/support.html > > > Yahoo! Groups Links > > > > > > > > > > > >
