Hello, 1. Data are stored internally and externally in the base interval you specify in the Database Settings. If you have chosen 1-minute - it will be 1-minute, if tick - tick. So whatever you choose - it is used.
2. Yes it uses FIFO. Best regards, Tomasz Janeczko amibroker.com ----- Original Message ----- From: "murthysuresh" <[EMAIL PROTECTED]> To: <[email protected]> Sent: Thursday, September 13, 2007 3:30 PM Subject: [amibroker] Re: Is 500,000 the maximum number of bars allowable in a database? > Tomasz > Dollars and cents aside, i would like to understand the criteria that > is used in the data that would be purged. > Here is the case that i am trying to understand. > > My AB has # o fbars 50000 in base interval 1 minute > The data feed comign in is a tick level data feed. > > Questions are > 1. Does AB store it as a Tick data internally. The reason is that if > i want to change to see tick data later on, could i see it for the > timeframe without it going to the plugin to get the tick data > 2. AB tells me 50k bars = 104 days with 1562 kb per symbol. So does > it use FIFO to purge the bar that are >104 days. This means that i > can backup my database files every 104 days to get some older data if > i needed. > > Seede > --- In [email protected], "Tomasz Janeczko" <[EMAIL PROTECTED]> > wrote: >> >> That's special price for AmiBroker users :-) >> >> Best regards, >> Tomasz Janeczko >> amibroker.com >> ----- Original Message ----- >> From: Don Lindberg >> To: [email protected] >> Sent: Thursday, September 13, 2007 6:37 AM >> Subject: RE: [AmiBroker] Re: Is 500,000 the maximum number of > bars allowable in a database? >> >> >> Tomasz, >> >> Your to cheap. Programming consultants are in the $80 - $100 hr > range! >> >> >> >> Donald F Lindberg >> >> >> -------------------------------------------------------------------- > ---------- >> >> From: [email protected] > [mailto:[EMAIL PROTECTED] On Behalf Of eric tao >> Sent: Wednesday, September 12, 2007 8:55 PM >> To: [email protected] >> Subject: [amibroker] Re: Is 500,000 the maximum number of bars > allowable in a database? >> >> >> >> $60/hours, Good to know :) >> >> --- In [email protected], "Tomasz Janeczko" <groups@> > wrote: >> > >> > > This isn't even a major change. It just removes the direct >> > > relationship of per symbol data on disk to the amount of > symbol data >> > > Amibroker processes. >> > >> > I will tell you something: you have NO idea, absolutely NO > IDEA, if >> it is major change or not. >> > As far as AmiBroker internals are considered your guesses are as >> accurate as forecasting the value of SP500 in December 2017. >> > >> > The one you think isn't major is in fact big (requires in fact > LOTS >> of changes) and no-one except 0.5% of AB user base would benefit >> > from it. >> > >> > Are you willing to pay for custom development? $60/hour is the > rate. >> > It will take about 10 days of development and testing. Are you >> willing to pay $4800 to develop it for you? >> > If so, no problem, I will do that for $4800. >> > >> > Best regards, >> > Tomasz Janeczko >> > amibroker.com >> > ----- Original Message ----- >> > From: "scourt2000" <stevehite@> >> > To: <[email protected]> >> > Sent: Wednesday, September 12, 2007 4:43 PM >> > Subject: [amibroker] Re: Is 500,000 the maximum number of bars >> allowable in a database? >> > >> > >> > > >> > > Tomasz, >> > > >> > > Just because I store large amounts of data in one file (aka, a >> > > symbol), it doesn't mean that Amibroker has to process all of > that >> > > data. Take the example of sqlite, the free database stores > all of its >> > > information in one file but users don't process all of that > data just >> > > because it's in one physical file on a disk. The stock data in >> > > Amibroker is a simple database. >> > > >> > > Please address this point: >> > > >> > > 1. Amibroker should never delete old stock/futures data just > because >> > > new data is coming in. All that is needed is for Amibroker to > limit >> > > the amount of data it is willing to pull in historically from > some >> > > specified start point in the past. Whether that is a certain > number >> > > of bars or a specific date range, that's fine. Like I alluded > to, if >> > > I have 5 years worth of something like ES tick data as a > continuous >> > > contract in one file and I want to backtest (or view) a date > range in >> > > that data, then I should have some way in Amibroker of > specifying a >> > > start date and end date for the data segment to be pulled in > for >> > > processing. >> > > >> > > Here's what Amibroker currently 'says': "I'm going to use all > of the >> > > data you have for a symbol. If you have too much, then I am > going to >> > > blow out the PC's memory. Game over." >> > > >> > > Here's what Amibroker should do: If there's too much data for > a >> > > symbol, let the user specify a date range for testing/viewing > or so >> > > many bars leading up to the present bar for testing/viewing. > The >> > > actual physical size of that data on disk remains intact, > just not all >> > > available for processing due to memory limitations. >> > > >> > > This isn't even a major change. It just removes the direct >> > > relationship of per symbol data on disk to the amount of > symbol data >> > > Amibroker processes. >> > > >> > > And like I said previously, there is a workaround for this. I > have to >> > > chop up my data into separate, artificial symbol names. Where > does >> > > this put the burden of the workload? On me, of course. What is >> > > software supposed to do? It's supposed to take the burden of > the >> > > workload off of the user as much as possible. >> > > >> > > Steve >> > > >> > > >> > > --- In [email protected], "Tomasz Janeczko" <groups@> > wrote: >> > >> >> > >> Hello, >> > >> >> > >> Why don't you move to Metastock that has 65000 bar limit ? >> > >> >> > >> The truth is that AmiBroker is the most capable in this area > because >> > >> it stores way more data than other softwares. Also, by > changing the >> > > registry >> > >> limit you can set it to any value you need. The limit is not > the >> > > software but the hardware >> > >> >> > >> It is funny when people make statements like yours without > taking >> > > calculator and >> > >> actually checking how much memory you need to store that. >> > >> >> > >> 20 million bars - OK, but did you actually count how much > memory you >> > > need for that? >> > >> Each quote is 64 bytes, now 20 million of bars is 1.2GB of > memory >> > > per symbol. >> > >> >> > >> You may say "but you don't need to load it at once". But >> > > unfortunatelly you do need that. >> > >> How you can backtest without loading the data ? >> > >> And each AFL array in such case would need 4 * 20 million = > 80 MB of >> > > RAM. >> > >> Usually user formulas use hundreds of such arrays. >> > >> >> > >> Wise old words are: "You can't eat your cake and have it > too". >> > >> >> > >> If you want more data stored - the price for that is > increased >> > > memory requirements >> > >> and more CPU power needed to process it. >> > >> >> > >> Now you can say: you can process one bar at a time - yes you > can - >> > > but you need >> > >> to store intermediate results anyway for next iteration and > that >> > > means that memory requirements >> > >> would not be even one byte less than they are. Plus changing >> > > execution model to 1-bar at a time will >> > >> mean (number-of-bars)-times slow down. >> > >> >> > >> So you would end up with system that is slow, slow and even > more >> slow. >> > >> >> > >> Best regards, >> > >> Tomasz Janeczko >> > >> amibroker.com >> > >> ----- Original Message ----- >> > >> From: "scourt2000" <stevehite@> >> > >> To: <[email protected]> >> > >> Sent: Wednesday, September 12, 2007 2:00 PM >> > >> Subject: [amibroker] Re: Is 500,000 the maximum number of > bars >> > > allowable in a database? >> > >> >> > >> >> > >> > >> > >> > And this is my biggest pet peeve about Amibroker: >> > >> > >> > >> > If I have the disk space available, Amibroker should never, >> ever, EVER >> > >> > just silently blow away my historical data because of some >> arbitrary >> > >> > system limit like 500,000 bars or 1,000,000. If I want 20 > million >> > >> > bars historical and I have the disk space to store it, then >> fine. If >> > >> > Amibroker needs far less to function well, then that's > fine too. >> > >> > >> > >> > The solution is NOT to delete old data. The solution is to >> have a way >> > >> > for Amibroker to limit the number of past bars (for > calculation >> > >> > purposes) or even have me set a beginning and end date in > an >> > >> > historical range WITHOUT destroying that historical data > (i.e., the >> > >> > present day should not have to be the last bar that I view > or >> use for >> > >> > backtesting). >> > >> > >> > >> > Currently, the only solution is to set some registry value > to >> go over >> > >> > 500,000, but that in itself is not solution. Because of >> Amibroker's >> > >> > implementation, you'll get your memory stressed out big > time and, >> > >> > again, if your new limit gets reached, Amibroker will > remove your >> > >> > oldest data to make room for new data. >> > >> > >> > >> > No one ever said that you had to look at stock/futures data >> from the >> > >> > complete past to today. If I had 20 years of one symbol's >> data and >> > >> > that data was far too much to make available at one time > because of >> > >> > memory problems, then it should be segmentable without me > having to >> > >> > split of the data in artificial symbols. >> > >> > >> > >> > Of course there's a workaround (there usually is). You > have to >> split >> > >> > your data up (e.g., tick data) into segments with different >> artificial >> > >> > symbol names (e.g., I use eSignal continuous futures > contracts). >> > >> > >> > >> > >> > >> > --- In [email protected], Graham <kavemanperth@> > wrote: >> > >> >> >> > >> >> I believe the database limit is set to 500,000 but can be >> changed in >> > >> >> the windows registry. TJ has mentioned this in a previous > post >> here, >> > >> >> somewhere sometime. >> > >> >> >> > >> >> >> > >> >> -- >> > >> >> Cheers >> > >> >> Graham Kav >> > >> >> AFL Writing Service >> > >> >> http://www.aflwriting.com >> > >> >> >> > >> >> >> > >> >> On 12/09/2007, Ara Kaloustian <ara1@> wrote: >> > >> >> > >> > >> >> > >> > >> >> > You can have as much as you want, but performance > degrades. >> > >> >> > >> > >> >> >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > Please note that this group is for discussion between > users only. >> > >> > >> > >> > To get support from AmiBroker please send an e-mail > directly to >> > >> > SUPPORT {at} amibroker.com >> > >> > >> > >> > For NEW RELEASE ANNOUNCEMENTS and other news always check > DEVLOG: >> > >> > http://www.amibroker.com/devlog/ >> > >> > >> > >> > For other support material please check also: >> > >> > http://www.amibroker.com/support.html >> > >> > >> > >> > Yahoo! Groups Links >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> >> > > >> > > >> > > >> > > >> > > Please note that this group is for discussion between users > only. >> > > >> > > To get support from AmiBroker please send an e-mail directly > to >> > > SUPPORT {at} amibroker.com >> > > >> > > For NEW RELEASE ANNOUNCEMENTS and other news always check > DEVLOG: >> > > http://www.amibroker.com/devlog/ >> > > >> > > For other support material please check also: >> > > http://www.amibroker.com/support.html >> > > >> > > Yahoo! Groups Links >> > > >> > > >> > > >> > > >> > > >> > >> > > > > > Please note that this group is for discussion between users only. > > To get support from AmiBroker please send an e-mail directly to > SUPPORT {at} amibroker.com > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > http://www.amibroker.com/devlog/ > > For other support material please check also: > http://www.amibroker.com/support.html > > Yahoo! Groups Links > > > > >
