That's special price for AmiBroker users :-) Best regards, Tomasz Janeczko amibroker.com ----- Original Message ----- From: Don Lindberg To: [email protected] Sent: Thursday, September 13, 2007 6:37 AM Subject: RE: [AmiBroker] Re: Is 500,000 the maximum number of bars allowable in a database?
Tomasz, Your to cheap. Programming consultants are in the $80 - $100 hr range! Donald F Lindberg ------------------------------------------------------------------------------ From: [email protected] [mailto:[EMAIL PROTECTED] On Behalf Of eric tao Sent: Wednesday, September 12, 2007 8:55 PM To: [email protected] Subject: [amibroker] Re: Is 500,000 the maximum number of bars allowable in a database? $60/hours, Good to know :) --- In [email protected], "Tomasz Janeczko" <[EMAIL PROTECTED]> wrote: > > > This isn't even a major change. It just removes the direct > > relationship of per symbol data on disk to the amount of symbol data > > Amibroker processes. > > I will tell you something: you have NO idea, absolutely NO IDEA, if it is major change or not. > As far as AmiBroker internals are considered your guesses are as accurate as forecasting the value of SP500 in December 2017. > > The one you think isn't major is in fact big (requires in fact LOTS of changes) and no-one except 0.5% of AB user base would benefit > from it. > > Are you willing to pay for custom development? $60/hour is the rate. > It will take about 10 days of development and testing. Are you willing to pay $4800 to develop it for you? > If so, no problem, I will do that for $4800. > > Best regards, > Tomasz Janeczko > amibroker.com > ----- Original Message ----- > From: "scourt2000" <[EMAIL PROTECTED]> > To: <[email protected]> > Sent: Wednesday, September 12, 2007 4:43 PM > Subject: [amibroker] Re: Is 500,000 the maximum number of bars allowable in a database? > > > > > > Tomasz, > > > > Just because I store large amounts of data in one file (aka, a > > symbol), it doesn't mean that Amibroker has to process all of that > > data. Take the example of sqlite, the free database stores all of its > > information in one file but users don't process all of that data just > > because it's in one physical file on a disk. The stock data in > > Amibroker is a simple database. > > > > Please address this point: > > > > 1. Amibroker should never delete old stock/futures data just because > > new data is coming in. All that is needed is for Amibroker to limit > > the amount of data it is willing to pull in historically from some > > specified start point in the past. Whether that is a certain number > > of bars or a specific date range, that's fine. Like I alluded to, if > > I have 5 years worth of something like ES tick data as a continuous > > contract in one file and I want to backtest (or view) a date range in > > that data, then I should have some way in Amibroker of specifying a > > start date and end date for the data segment to be pulled in for > > processing. > > > > Here's what Amibroker currently 'says': "I'm going to use all of the > > data you have for a symbol. If you have too much, then I am going to > > blow out the PC's memory. Game over." > > > > Here's what Amibroker should do: If there's too much data for a > > symbol, let the user specify a date range for testing/viewing or so > > many bars leading up to the present bar for testing/viewing. The > > actual physical size of that data on disk remains intact, just not all > > available for processing due to memory limitations. > > > > This isn't even a major change. It just removes the direct > > relationship of per symbol data on disk to the amount of symbol data > > Amibroker processes. > > > > And like I said previously, there is a workaround for this. I have to > > chop up my data into separate, artificial symbol names. Where does > > this put the burden of the workload? On me, of course. What is > > software supposed to do? It's supposed to take the burden of the > > workload off of the user as much as possible. > > > > Steve > > > > > > --- In [email protected], "Tomasz Janeczko" <groups@> wrote: > >> > >> Hello, > >> > >> Why don't you move to Metastock that has 65000 bar limit ? > >> > >> The truth is that AmiBroker is the most capable in this area because > >> it stores way more data than other softwares. Also, by changing the > > registry > >> limit you can set it to any value you need. The limit is not the > > software but the hardware > >> > >> It is funny when people make statements like yours without taking > > calculator and > >> actually checking how much memory you need to store that. > >> > >> 20 million bars - OK, but did you actually count how much memory you > > need for that? > >> Each quote is 64 bytes, now 20 million of bars is 1.2GB of memory > > per symbol. > >> > >> You may say "but you don't need to load it at once". But > > unfortunatelly you do need that. > >> How you can backtest without loading the data ? > >> And each AFL array in such case would need 4 * 20 million = 80 MB of > > RAM. > >> Usually user formulas use hundreds of such arrays. > >> > >> Wise old words are: "You can't eat your cake and have it too". > >> > >> If you want more data stored - the price for that is increased > > memory requirements > >> and more CPU power needed to process it. > >> > >> Now you can say: you can process one bar at a time - yes you can - > > but you need > >> to store intermediate results anyway for next iteration and that > > means that memory requirements > >> would not be even one byte less than they are. Plus changing > > execution model to 1-bar at a time will > >> mean (number-of-bars)-times slow down. > >> > >> So you would end up with system that is slow, slow and even more slow. > >> > >> Best regards, > >> Tomasz Janeczko > >> amibroker.com > >> ----- Original Message ----- > >> From: "scourt2000" <stevehite@> > >> To: <[email protected]> > >> Sent: Wednesday, September 12, 2007 2:00 PM > >> Subject: [amibroker] Re: Is 500,000 the maximum number of bars > > allowable in a database? > >> > >> > >> > > >> > And this is my biggest pet peeve about Amibroker: > >> > > >> > If I have the disk space available, Amibroker should never, ever, EVER > >> > just silently blow away my historical data because of some arbitrary > >> > system limit like 500,000 bars or 1,000,000. If I want 20 million > >> > bars historical and I have the disk space to store it, then fine. If > >> > Amibroker needs far less to function well, then that's fine too. > >> > > >> > The solution is NOT to delete old data. The solution is to have a way > >> > for Amibroker to limit the number of past bars (for calculation > >> > purposes) or even have me set a beginning and end date in an > >> > historical range WITHOUT destroying that historical data (i.e., the > >> > present day should not have to be the last bar that I view or use for > >> > backtesting). > >> > > >> > Currently, the only solution is to set some registry value to go over > >> > 500,000, but that in itself is not solution. Because of Amibroker's > >> > implementation, you'll get your memory stressed out big time and, > >> > again, if your new limit gets reached, Amibroker will remove your > >> > oldest data to make room for new data. > >> > > >> > No one ever said that you had to look at stock/futures data from the > >> > complete past to today. If I had 20 years of one symbol's data and > >> > that data was far too much to make available at one time because of > >> > memory problems, then it should be segmentable without me having to > >> > split of the data in artificial symbols. > >> > > >> > Of course there's a workaround (there usually is). You have to split > >> > your data up (e.g., tick data) into segments with different artificial > >> > symbol names (e.g., I use eSignal continuous futures contracts). > >> > > >> > > >> > --- In [email protected], Graham <kavemanperth@> wrote: > >> >> > >> >> I believe the database limit is set to 500,000 but can be changed in > >> >> the windows registry. TJ has mentioned this in a previous post here, > >> >> somewhere sometime. > >> >> > >> >> > >> >> -- > >> >> Cheers > >> >> Graham Kav > >> >> AFL Writing Service > >> >> http://www.aflwriting.com > >> >> > >> >> > >> >> On 12/09/2007, Ara Kaloustian <ara1@> wrote: > >> >> > > >> >> > > >> >> > You can have as much as you want, but performance degrades. > >> >> > > >> >> > >> > > >> > > >> > > >> > > >> > Please note that this group is for discussion between users only. > >> > > >> > To get support from AmiBroker please send an e-mail directly to > >> > SUPPORT {at} amibroker.com > >> > > >> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > >> > http://www.amibroker.com/devlog/ > >> > > >> > For other support material please check also: > >> > http://www.amibroker.com/support.html > >> > > >> > Yahoo! Groups Links > >> > > >> > > >> > > >> > > >> > > >> > > > > > > > > > > Please note that this group is for discussion between users only. > > > > To get support from AmiBroker please send an e-mail directly to > > SUPPORT {at} amibroker.com > > > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > > http://www.amibroker.com/devlog/ > > > > For other support material please check also: > > http://www.amibroker.com/support.html > > > > Yahoo! Groups Links > > > > > > > > > > >
