Hello,
There is no such thing as "no limit". Your HARDWARE IS LIMITED, 3rd
party data vendor HAS LIMITS on amounts of data sent.
The limits are there for a reason. To allow smooth REAL TIME OPERATION.
If you override limits you are shooting yourself in a foot and will face
problems with real-time operation.
*DO NOT override limits*
Real-time database must be able to keep up with data stream counting
even thousands of ticks per second PER SYMBOL.
Everyone who thinks that his/her hardware has no limits is simply very,
very, very wrong.
The person who thinks so is deeply wrong because he/she does not know
how the amount of memory affects the performance.
You guys should run memtest86 program to see what is the difference
between processing speed of ON-CHIP CPU CACHE
and normal memory. The difference is TEN FOLD (or larger). There are in
fact differences as large as 2x or 3x in speed between level-1
and level-2 caches. If you allow that your data are larger than your CPU
cache is, you will automatically
get 10x performance hit because of limited memory thoughput. CPU will
WAIT for memory. Memory speeds are way behind processor speeds.
If you need want "endless" database use LOCAL DATABASE only without
external source. Such database working OFF-LINE
is not subject to real-time constraints.
Best regards,
Tomasz Janeczko
amibroker.com
On 2010-03-11 11:19, Magesh wrote:
Some how my mails aren't getting through into the group. Hoping to get
luckier this time.
Thanks to [email protected], for I got clarity on this mail that #
of Bar limitations is meant only for plugin-driven databases.
Though that solved my worries, as I use a local database for caching
RT data, I still am curious to understand how such scenarios are
handled in real-time while using Plugin driven databases
In other words, how to preserve my data from getting truncated once
the size limit exceeds in a Plugin-Driven database.
Any thoughts or suggestions are much appreciated.
thanks and regards
Magesh
On 10/03/10 11:01 AM, Magesh wrote:
Does this mean a limitation of AB for saving data to infinite without
truncating?
I am using a real-time stream, Base Time Interval is "Tick" and No.
of Bars=500,000.
Total no of symbols = 10 (all with real-time Tick).
I have a fixed layout, and no frequent change in symbols.
And with my rough math of a tick for every 2 seconds,
# of bars per trading day = 11700 bars
Max # of days of history (@ 500,000 bars)= 43 days
Max # of days of history (@ 1,500,000 bars) = 128 days
My questions are:
1) Is my understandings on the bar count, size per symbol and limit
is correct?
2) Is there any ways of overcoming this limit, my reason being, I
would like to hold the historical tick size data for more than a year
(at-least to start with)?
3) I think its scalable by altering "Base Time Interval" from "Tick"
to "5/15 seconds" and will allow me to store up to 18 & 60 months
approx. of data respectively. Is this right?
4) What's the max integer limit (size of variable) for MaximumNoOfBars?
5) Can alternate dBs (MySQL)be used in synch with AB's database, so
that periodic push/flushing of AB's data into an external dBase? (as
I won't be looking or using historical intraday's in realtime or
atleast not so frequently)
Btw, this is a total scaled down version of my requirement, as my
database is just about a month old and currently I have about 30
symbols and just begun learning AFL and back-testing/explorations.
And with the current bar count setting, I think my database will
start getting truncated anytime sooner (for few of the symbols
updated via RT).
So I would like to keep my options very open for any kind/type of
suggestions as my current attempt is to have AB retain data and stop
draining LIFO.
Thanks in Advance,
Magesh
P.S.:I couldn't find any updates beyond this chain on this email
thread subject from gmane, so any pointers or suggestions in this
direction will be of great help for me.
On 02/08/08 3:59 PM, Herman wrote:
Hi Tomasz (or anyone who knows),
Could you define "advanced users and powerful computers"?
Who would use this feature and what what exactly does it do?
Thanks,
herman
Friday, August 1, 2008, 7:21:11 AM, you wrote:
> No, it shouldn't. This key is a "hidden extra feature" for advanced
> users having computers powerful enough to handle extra load.
> By default the key does not exist and then the default max is 500000.
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: "progster01"
<progster-sfbTBCYm2/[email protected]
<mailto:progster-sfbTBCYm2/[email protected]>>
> To: <[email protected]
<mailto:[email protected]>>
> Sent: Friday, August 01, 2008 12:55 PM
> Subject: [amibroker] Re: Database settings and the amount of data saved
>>> Make sure you check the path exactly! The key name is
>>> HKEY_CURRENT_USER\Software\TJP\Broker\Settings\MaximumNumber
>>> OfBars
>>> (watch out for line truncations)
>>> The key is a DWORD.
>> This key does not exist on the machine I'm looking at (which has AB
>> 5.15 installed).
>> Obviously I could add it myself, but "should" it already be there?
>> IOW, what is the design scenario for the creation of this key?
>> ------------------------------------
>> Please note that this group is for discussion between users only.
>> To get support from AmiBroker please send an e-mail directly to
>> SUPPORT {at} amibroker.com
>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
>> http://www.amibroker.com/devlog/
>> For other support material please check also:
>> http://www.amibroker.com/support.html
>> Yahoo! Groups Links
> ------------------------------------
> Please note that this group is for discussion between users only.
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
> For other support material please check also:
> http://www.amibroker.com/support.html
> Yahoo! Groups Links
> (Yahoo! ID required)
> mailto:[email protected]
<mailto:[email protected]>
