There are various ones on the market. Google RAMDisk and you will find free versions, evaluation versions and expensive versions :)
I have not yet done any research on the pros and cons of each, but I suspect it's all got to do with functionalty around the product rather than the speed, which essentially would be the speed of your RAM. There are many different versions. I suggest you have a look and download a demo siutable for your current system. E.g. www.dataram.com do a freeware version for up to 4GB.. and many others out there. cheers, eToke --- In [email protected], "sidhartha70" <sidharth...@...> wrote: > > Thanks. > > But what did you use to create the RAM drive HDD on your machine currently...? > > TJ, anything to be aware of using this approach...? > > TIA > > --- In [email protected], "etoketrader" <etoke@> wrote: > > > > I am not sure about the technical details, but I can say that until now on > > today's current market volume, what was taking 6-7 minutes yesterday is > > taking around 1m 20sec... that's an incredible increase in performance. > > > > Everything else seems stable and responsive. > > > > I am not sure about the 64bit RAM drive, but I am running 64bit windows Xp > > and 32b ones I downloaded didn't work. I am not sure whether a 32bit system > > is capable of creating a RAM drive beyond the 4GB limit... cannot test that > > as I haven't got a 32bit OS here. > > > > cheers, > > eToke > > > > --- In [email protected], "sidhartha70" <sidhartha70@> wrote: > > > > > > Very interesting. So what's taking the time here...?? Streaming the data > > > onto your HD...? Hence you remove that bottleneck by moving your HD into > > > RAM...? > > > > > > Why would you want to a 64 bit compatible RAM Drive utility...? You are > > > running 32 bit AB right...? How did you create your current 4GB RAM > > > HDD...? > > > > > > Thanks > > > > > > --- In [email protected], "etoketrader" <etoke@> wrote: > > > > > > > > All, > > > > > > > > I was able to drastically improve the performance of the exploration > > > > during market hours from 6 minutes to just over a minute!! That is an > > > > incredible 6X improvement! > > > > > > > > Outside of market hours I was able to get over 2X improvement. > > > > > > > > I have 8GB of RAM on my 64bit machine yet I have to run 32bit AB due to > > > > the fact that IQFeed do not support a 64bit plugin - hence half my RAM > > > > is practically unusable. > > > > > > > > My AB database is currently approx 3.75GB. This is what I did: I > > > > created a 4GB RAMDRIVE (this will take 4GB of RAM and create a virtual > > > > HDD), then copied the entire AB folder into it, and ran the program > > > > from there. The result was a fascinating improvement: everything runs > > > > just so much faster. > > > > > > > > Since I am running a large 1 minute DB I keep my cache symbols low to > > > > 11. This I suspect causes a lot of HDD interaction under normal > > > > circumstances, but since the HDD is now in RAM, this isn't an issue any > > > > more. > > > > > > > > Also, the physical HDD is now silent and hardly active any longer. Over > > > > all I think this saves some life out of it as a tick-by-tick update > > > > every single market day cannot do it much good. > > > > > > > > There are various free/cheap RAM Drive utilities out there. Most > > > > importantly one would need to pick up a 64bit compatible one. I have > > > > not done any research whatsoever on these programs... I just picked up > > > > the first one I found. > > > > > > > > There are downsides of course: most notably is that you would lose all > > > > the data if the power goes down, however since I am planning to use it > > > > this way throughout market hours, it is not an issue for me. I can take > > > > the latest backup and update it with a backfill run. > > > > > > > > I am curious to see whether anybody has experimented with this before > > > > and can compare results. > > > > > > > > Also, anybody aware of other potential downsides when using this method? > > > > > > > > cheers, > > > > eToke > > > > > > > > --- In [email protected], "Paul Ho" <paul.tsho@> wrote: > > > > > > > > > > The following test would help you determine which "elements" are > > > > > critical to improving exporation speed > > > > > 1. > > > > > Temporary change your IQFeed database from IQFeed to local, make sure > > > > > local storage is enabled. run your exploration, That is the speed > > > > > your hardware can do with your afl and with no limitation on > > > > > connection bandwidth. > > > > > 2. I suspect that IQFeed connection at application level, not at adsl > > > > > level, is your bottle neck. To improve it you have to consider: > > > > > 3. Backfilling is far more time consuming than using RTGetData > > > > > 4. The more array data your afl uses, the slower it is. > > > > > 5. There is only certain amount improvement you can make with new > > > > > computer hardware. most of the improvements can be gained by > > > > > reorganising your afl so things that doesnt change change intraday > > > > > are scanned overnight, and use RTGETDATA as a replacement to array > > > > > data as much as possible. > > > > > > > > > > --- In [email protected], "etoketrader" <etoke@> wrote: > > > > > > > > > > > > Hi Tomasz, > > > > > > > > > > > > Agreed on load imposed by the rapid update of 500 symbols. No > > > > > > question about it. > > > > > > > > > > > > What I would like to understand is what elements (hardware, or > > > > > > system optimisation) would help reduce the duration of a scan. > > > > > > > > > > > > Unfortunately running the 64bit version is out of the question > > > > > > because of the realtime plugin issue. This rules out the > > > > > > possibility of adding a large amount of RAM although I don't know > > > > > > whether that alone would help. > > > > > > > > > > > > Are there any other ways I might be able to increase the speed? > > > > > > Would copying the entire db on several GB of a RAM disk before a > > > > > > market session be worth trying? > > > > > > > > > > > > Best regards, > > > > > > eToke > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --- In [email protected], "Tomasz Janeczko" <groups@> wrote: > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > You need to understand that there can be even 800 ticks PER > > > > > > > SECOND on one symbol > > > > > > > If you are for example following 500 symbols, it presents the > > > > > > > stream of 400000 updates per second. > > > > > > > Even when processing took as little as 2 microseconds per update > > > > > > > (1 millionth part of the second) > > > > > > > this would mean 80% CPU utilisation. So yes, if there are lots of > > > > > > > quotes streaming > > > > > > > on lots of symbols you will see higher CPU usage and possible > > > > > > > also slower exploration. > > > > > > > > > > > > > > Best regards, > > > > > > > Tomasz Janeczko > > > > > > > amibroker.com > > > > > > > ----- Original Message ----- > > > > > > > From: "etoketrader" <etoke@> > > > > > > > To: <[email protected]> > > > > > > > Sent: Tuesday, August 04, 2009 6:09 PM > > > > > > > Subject: [amibroker] Re: Speeding exporation intraday > > > > > > > > > > > > > > > > > > > > > > hi Tomasz, > > > > > > > > > > > > > > > > that makes sense. > > > > > > > > > > > > > > > > However in my case all my symbols are already active both on > > > > > > > > and off market hours. > > > > > > > > > > > > > > > > When quotes are being updated constantly during market hours, a > > > > > > > > test exploration code which displays the closing price takes > > > > > > > > approx. 6 - 10 times the duration of when the quotes are also > > > > > > > > streaming but outside of market hours (i.e. hardly any updates). > > > > > > > > > > > > > > > > My question is if there is anything I can do to speed this up. > > > > > > > > > > > > > > > > BR > > > > > > > > > > > > > > > > eToke > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --- In [email protected], "Tomasz Janeczko" <groups@> > > > > > > > > wrote: > > > > > > > >> > > > > > > > >> Hello, > > > > > > > >> > > > > > > > >> That depends if given symbol is in the "active" (i.e. > > > > > > > >> streaming) list or not. > > > > > > > >> If it is - then turning on/off "wait for backfill" does not > > > > > > > >> make difference. > > > > > > > >> > > > > > > > >> If it is not, then with "wait for backfill" turned on it will > > > > > > > >> of course wait for "missing" bars > > > > > > > >> before proceeding to next symbol. > > > > > > > >> > > > > > > > >> Charts on the other hand, always refresh immediatelly with > > > > > > > >> data that are present right now. > > > > > > > >> If symbol is not present in the active list it is > > > > > > > >> automatically added and backfill request > > > > > > > >> may be sent but this does not block chart from refreshing - it > > > > > > > >> will refresh immediatelly, > > > > > > > >> and once backfill data eventually arrives it will refresh once > > > > > > > >> more. > > > > > > > >> In other words - charts are asynchronous with regards to > > > > > > > >> backfill. AA can be synchronous > > > > > > > >> ("wait for backfill" turned ON) or asynchronous ("wait for > > > > > > > >> backfill" turned OFF). > > > > > > > >> > > > > > > > >> That's the only difference. AFL execution IS IDENTICAL > > > > > > > >> regardless where it occurs. > > > > > > > >> Of course with different zoom levels / different AA range you > > > > > > > >> may be using different number of bars, > > > > > > > >> but that should be obvious. > > > > > > > >> > > > > > > > >> Best regards, > > > > > > > >> Tomasz Janeczko > > > > > > > >> amibroker.com > > > > > > > >> ----- Original Message ----- > > > > > > > >> From: "sidhartha70" <sidhartha70@> > > > > > > > >> To: <[email protected]> > > > > > > > >> Sent: Tuesday, August 04, 2009 5:14 PM > > > > > > > >> Subject: [amibroker] Re: Speeding exporation intraday > > > > > > > >> > > > > > > > >> > > > > > > > >> > I'd be interested to hear TJ's answer on this eToke... > > > > > > > >> > > > > > > > > >> > Because I was asking from very similar questions of support > > > > > > > >> > & TJ recently... namely that equivalent AFL on charts seems > > > > > > > >> > to > > > > > > > >> > execute > > > > > > > >> > much quicker than a scan or exploration. Which obviously > > > > > > > >> > leads one to think that other operations that occur in a > > > > > > > >> > scan or > > > > > > > >> > exploration, outside of the AFL execution, are what take the > > > > > > > >> > time... > > > > > > > >> > > > > > > > > >> > Why this time is greater with a streaming feed I don't > > > > > > > >> > know... > > > > > > > >> > > > > > > > > >> > --- In [email protected], "etoketrader" <etoke@> > > > > > > > >> > wrote: > > > > > > > >> >> > > > > > > > >> >> Hi, for testing I am currently running: > > > > > > > >> >> > > > > > > > >> >> n Last quotations = 1 > > > > > > > >> >> A very basic AFL code which lists closing price. > > > > > > > >> >> QuickAFL turned on > > > > > > > >> >> > > > > > > > >> >> I am concerned about the comparison between how it runs > > > > > > > >> >> outside of market hours vs how it runs during. > > > > > > > >> >> > > > > > > > >> >> BR, > > > > > > > >> >> eToke > > > > > > > >> >> > > > > > > > >> >> > > > > > > > >> >> > > > > > > > >> >> --- In [email protected], "sidhartha70" > > > > > > > >> >> <sidhartha70@> wrote: > > > > > > > >> >> > > > > > > > > >> >> > How many quotations are you running the explorations > > > > > > > >> >> > over...? > > > > > > > >> >> > > > > > > > > >> >> > --- In [email protected], "etoketrader" <etoke@> > > > > > > > >> >> > wrote: > > > > > > > >> >> > > > > > > > > > >> >> > > Tried that. I always run the first backfill before > > > > > > > >> >> > > market hours. After that, backfill setting does not > > > > > > > >> >> > > seem to make any > > > > > > > >> >> > > difference as long as everything has already been > > > > > > > >> >> > > backfilled and all symbols are streaming. > > > > > > > >> >> > > > > > > > > > >> >> > > BR > > > > > > > >> >> > > eToke > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > --- In [email protected], "murthysuresh" > > > > > > > >> >> > > <murthysuresh@> wrote: > > > > > > > >> >> > > > > > > > > > > >> >> > > > backfill before the start of the market hours. turn > > > > > > > >> >> > > > off backfill as you only need to do it at the > > > > > > > >> >> > > > beginning of the > > > > > > > >> >> > > > session. > > > > > > > >> >> > > > > > > > > > > >> >> > > > --- In [email protected], "etoketrader" > > > > > > > >> >> > > > <etoke@> wrote: > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > Hi, > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > I currently run IQFeed on 500 symbols. Exploration > > > > > > > >> >> > > > > outside of market hours run under 1 minute but slow > > > > > > > >> >> > > > > down to 6 > > > > > > > >> >> > > > > minutes > > > > > > > >> >> > > > > during market hours. > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > Can anybody recommend some ideal settings to speed > > > > > > > >> >> > > > > up intraday explorations while streaming? > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > thanks, > > > > > > > >> >> > > > > eToke > > > > > > > >> >> > > > > > > > > > > > >> >> > > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > >> >> > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > ------------------------------------ > > > > > > > >> > > > > > > > > >> > **** IMPORTANT PLEASE READ **** > > > > > > > >> > This group is for the discussion between users only. > > > > > > > >> > This is *NOT* technical support channel. > > > > > > > >> > > > > > > > > >> > TO GET TECHNICAL SUPPORT send an e-mail directly to > > > > > > > >> > SUPPORT {at} amibroker.com > > > > > > > >> > > > > > > > > >> > TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at > > > > > > > >> > http://www.amibroker.com/feedback/ > > > > > > > >> > (submissions sent via other channels won't be considered) > > > > > > > >> > > > > > > > > >> > For NEW RELEASE ANNOUNCEMENTS and other news always check > > > > > > > >> > DEVLOG: > > > > > > > >> > http://www.amibroker.com/devlog/ > > > > > > > >> > > > > > > > > >> > Yahoo! Groups Links > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------ > > > > > > > > > > > > > > > > **** IMPORTANT PLEASE READ **** > > > > > > > > This group is for the discussion between users only. > > > > > > > > This is *NOT* technical support channel. > > > > > > > > > > > > > > > > TO GET TECHNICAL SUPPORT send an e-mail directly to > > > > > > > > SUPPORT {at} amibroker.com > > > > > > > > > > > > > > > > TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at > > > > > > > > http://www.amibroker.com/feedback/ > > > > > > > > (submissions sent via other channels won't be considered) > > > > > > > > > > > > > > > > For NEW RELEASE ANNOUNCEMENTS and other news always check > > > > > > > > DEVLOG: > > > > > > > > http://www.amibroker.com/devlog/ > > > > > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
