Tomasz, I think that when "Import Files" is sued, it sorts my selection (I select an entire years worth of data files at once). My file names are like this: "nasdaq_20010201.txt" (that is with yyyymmdd in the name). Is AmiBroker sorting those file names? or does it import in the order that was selected?
--- In [email protected], "Tomasz Janeczko" <[EMAIL PROTECTED]> wrote: > > Hello, > > 5 years of data on my end imports in *minutes* not hours. > > a) you have small size of "in-memory" cache so it requires a lot of disk mangling > b) the data you import are not sorted or sorted improperly (in reverse order) and AB needs to re-sort. > > For fast import It is advised to use data that is sorted in OLDEST QUOTES FIRST - NEWEST LAST > and to have one file per symbol. > > The worst possible case (it's probably yours) is having sorted in opposite direction > (newest first) and different symbol in every line. > > Best regards, > Tomasz Janeczko > amibroker.com > ----- Original Message ----- > From: "jacklweinberg" <[EMAIL PROTECTED]> > To: <[email protected]> > Sent: Tuesday, May 15, 2007 2:59 PM > Subject: [amibroker] Re: Massive data loading? > > > > Tomasz, > > Perhaps my original message wasn't clear. > > I have the data in txt files, and am using AmiBroker "Import Files" > > to load them into the database. That is the process that is taking > > over 32 hours! > > > > --- In [email protected], "Tomasz Janeczko" <groups@> > > wrote: > >> > >> Hello, > >> > >> It is MUCH quicker if you change the settings. > >> Default settings are fail-safe. You can however > >> change "number of simultaneous downloads" to 5 or more. > >> BY setting it say to 10 you can get 10x speed up, > >> however Yahoo may block you after some excessive downloading > >> and you will need to revert to defaults and wait a couple of > > minutes > >> for Yahoo to unblock you. > >> > >> Best regards, > >> Tomasz Janeczko > >> amibroker.com > >> ----- Original Message ----- > >> From: "jacklweinberg" <jacklweinberg@> > >> To: <[email protected]> > >> Sent: Monday, May 14, 2007 7:33 PM > >> Subject: [amibroker] Massive data loading? > >> > >> > >> > Does anyone have experience at loading up an entirely new > > database > >> > with AmiBroker? > >> > For example, loading AB with all of Nasdaq with 5 years of data? > >> > I used the file import command, and selected a series of .txt > > files, > >> > and it takes around 32 hours of elapsed time per year!!! (On a > > fast > >> > machine, no less!) > >> > Does anyone know of a quicker method? > >> > > >> > > >> > > >> > > >> > Please note that this group is for discussion between users only. > >> > > >> > To get support from AmiBroker please send an e-mail directly to > >> > SUPPORT {at} amibroker.com > >> > > >> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > >> > http://www.amibroker.com/devlog/ > >> > > >> > For other support material please check also: > >> > http://www.amibroker.com/support.html > >> > > >> > Yahoo! Groups Links > >> > > >> > > >> > > >> > > >> > > >> > > > > > > > > > > Please note that this group is for discussion between users only. > > > > To get support from AmiBroker please send an e-mail directly to > > SUPPORT {at} amibroker.com > > > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > > http://www.amibroker.com/devlog/ > > > > For other support material please check also: > > http://www.amibroker.com/support.html > > > > Yahoo! Groups Links > > > > > > > > > > >
