> I'm not sure what you mean by whether or not it might be a smart >thing to do
I just meant that I am not absolutely sure about my bright ideas since I don't have enough computing knowledge to be certain that I am not making ridiculous statements. Thanks for your answers - appreciated. brian_z --- In [email protected], Fred Tonetti <[EMAIL PROTECTED]> wrote: > > Ohhh it certainly wasn't my idea to use sessions . It was Paul's offhand > comment about already doing it that brought light to the subject for me . > > > > If I had been aware enough to realize this could have been done I would have > done it years ago . > > > > In regards to having multithreading within individual programs . yes of > course . CAD/CAM programs have been doing this for years . > > > > In regards to doing the same thing in IO . Yes of course . MCO was the proof > of concept if you will . The capability within IO already exists now in test > mode . I'm not sure what you mean by whether or not it might be a smart > thing to do but imho it certainly is useful as is MCO to speed up exhaustive > search. > > > > While you may not have the same chip . you will have the same underlying > instruction set . > > > > As far as backward compatibility is concerned . I don't think this is an > issue . It is relatively simple to detect how many processors are available > and make use of more than one if it exists. > > > > I can't say that I see anything all that wonderful out of CMA-ES either but > that is of course just one person's opinion based on limited testing. The > problem with intelligent algorithms in general is that by and large most are > not all that wonderful in their originally published version for the kinds > of tasks that they'd be put to in AB . Some have very limited scope and > others have characteristics and limitations that make them of limited > usefulness as published . Thus many need to be played with to see what makes > them better and that playing takes oodles of time just to see if some > particular one is worthwhile pursuing . > > _____ > > From: [email protected] [mailto:[EMAIL PROTECTED] On Behalf > Of brian_z111 > Sent: Sunday, June 29, 2008 10:44 AM > To: [email protected] > Subject: [amibroker] Re: Another side to the multi-core support argument > > > > You're not sorry - you enjoyed it ;-) > > Exactly as I imagined it - a perfect example of a single AB task that > lends itself perfectly to, what I would call, a distributed approach. > > I think that is about as simple as AB MCP could get since such > symmetry (repetition) of code would be rare, outside of opt? > > Carefully avoiding any further reading of docs, I was also referring > to the fact that you must have utilized the OS to 'force' the ABnn > sessions (since we are in the accreditation mood I had better add I > think someone else kicked in the idea to use 'sessions' but what the > hell?). > > You said that MCO was a pretty basic approach (or words to that > effect) but it seems to me that utilizing the capabilities of the OS > to distribute tasks to CPU resources, in various ways, can also be > done from within a program without the need for sessions. > > As things stand at the moment couldn't you go ahead (if you wanted > to) and write an I/O plugin that utilizes all available cores WIWO > Tomasz changing anything in AB? (not to say it is a smart idea but > possible, yes?) > > As a general comment - not particularly for you - what I am kicking > around is the idea that it would be a heck of a task to isolate the > bits of AB code that lend themselves to it, chop them up for > processing etc and then keep that up over the years (especially since > we won't all have the same OS/chip and/or number of cores) - I am > wondering of a less optimum but more achieveable path might be to use > the task scheduling capabilities of the OS to produce a more generic, > maintainable way to use all available cores. > > The other point I am throwing on the table is - why would any > developer maintain backward compatibility for uniprocessors when as > of now no one is buyig one and realistically computers only have a > life of a few years. > > brian_z > > brian_z > > FTR > > I am not convinced that CMA-ES is the ants pants of optimization but > I thought I would sit on the sidelines and let you two guys slug it > out (of itself it might be nice and it could be a very good move as > part of an overall, yet to be seen AB strategy but I would say it has > some limitations) . > > brian_z > > --- In [EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com, Fred > Tonetti <ftonetti@> wrote: > > > > Nope . Didn't do anything like that . > > > > > > > > What I did was simply as you state . where a sequence of code lends > itself > > to being chopped up, shoved through the processor and put back > together > > again on the other side . > > > > > > > > Even this description is not entirely accurate . What really > happens is that > > virtually the same AFL with different ranges of values for the > optimization > > statements is sent to multiple AB instances to be worked on by > different > > cores and then the results combined . > > > > > > > > From the doc ( Sorry to semi force you to read this ) > > > > > > > > o MCO can deal with most forms of optimization statements. > MCO will > > check each of the Optimize statements in the AFL looking for the > one that > > has a numeric Min, Max and Increment arguments and needs the most > steps to > > get from the Min to Max value. Individual AFL's will be cloned for > each > > ABnn session with modifications to this Optimize statement so that > each core > > performs its share of the process. If for example the AFL to be > run had the > > following Optimize statements . > > > > > > > > Len1 = Optimize("Len1", 10, 1, 20, 1); > > > > Len2 = Optimize("Len2", 50, 1, 100, 1); > > > > > > > > . then it is the second Optimize statement that would be altered > for each of > > the AFL's to be used by the ABnn sessions. If one were using three > cores > > without any /load command line arguments then the resulting Optimize > > statement for each of the three AFL's would look like what is below > > effectively splitting the workload as closely as possible across > the user > > specified cores. > > > > > > > > Len2 = Optimize("Len2", 50, 1, 33, 1); > > > > Len2 = Optimize("Len2", 50, 34, 66, 1); > > > > Len2 = Optimize("Len2", 50, 67, 100, 1); > > > > > > > > _____ > > > > From: [EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com > [mailto:[EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com] > On Behalf > > Of brian_z111 > > Sent: Sunday, June 29, 2008 4:39 AM > > To: [EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> ps.com > > Subject: [amibroker] Re: Another side to the multi-core support > argument > > > > > > > > Sorry, I used the term 'symmetric processing' incorrectly (I now > see > > that it has a formal definition whereas I used it informally). > > > > What I meant was that I don't think software applications for MCP > are > > limited only to single tasks where a sequence of code lends itself > to > > being chopped up, shoved through the processor and put back > together > > again on the other side. > > > > Couldn't software modularise its tasks/features and present them to > > the OS as if they are separate programs e.g. downloading RT data, > in > > AB, and saving to a local database, could be allocated one core, by > > the OS, and rendering that data into a chart/processing indicators > > allocated to another - possibly AB could use some smart > > prioritisation of its own tasks and work in tandem with the OS to > > prioritise access to core CPU's? > > > > It seems to me we are limiting ourselves with regard to the > > possibilities. > > > > (I haven't read, or tried, Freds MCO stuff - maybe that is what he > > did there already, in a basic way). > > > > If and when AB goes ahead with multicore application I have to > wonder > > why anyone wouldn't want to upgrade - could that be the time to end > > the unlimited backward compatibility rule (the benefits of > > maintaining backward compatibility for a few would no longer be > worth > > the effort and even detrimental to the many)? > > > > brian_z > > > > --- In [EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> > ps.com, > > "brian_z111" <brian_z111@> wrote: > > > > > > Re: what can traders do with multicore processing (MCP)?. > > > > > > Gee, where are all the Codesters when you need one? > > > > > > Are you guys holding out on us? > > > > > > MCP applications aren't limited to symmetric processing are they? > > > > > > http://en.wikipedia > <http://en.wikipedia <http://en.wikipedia.org/wiki/Computer_multitasking> > .org/wiki/Computer_multitasking> > > .org/wiki/Computer_multitasking > > > > > > http://en.wikipedia <http://en.wikipedia > <http://en.wikipedia.org/wiki/Multi-core> .org/wiki/Multi-core> > > .org/wiki/Multi-core > > > > > > "The amount of performance gained by the use of a multicore > > processor > > > depends on the problem being solved and the algorithms used, as > > well > > > as their implementation in software (Amdahl's law). For so- > > > called "embarrassingly parallel" problems, a dual-core processor > > with > > > two cores at 2GHz may perform very nearly as fast as a single > core > > of > > > 4GHz.[1] Other problems though may not yield so much speedup. > This > > > all assumes however that the software has been designed to take > > > advantage of available parallelism. If it hasn't, there will not > be > > > any speedup at all. However, the processor will multitask better > > > since it can run two programs at once, one on each core." > > > > > > > > > To what extent does it depend on the hardware/OS e.g. are the > > > possibilties for AB different on a MSFT system with an AMD chip > > > compared to another OS with a different chip - if so how would AB > > > handle that across a wide range of users - it seems to me the AB > > > solution will have to be generic rather than specialist? > > > > > > > > > As a layperson, it appears that multi-tasking is a more exciting > > > prospect, for the majority of traders, than optimising at warp > > speeds. > > > > > > Can't muliple processors handle multi-tasking at speeds unheard > of > > > before and hence bring some things into the realm of > possibilities? > > > > > > > > > How about: > > > > > > a) different databases running in one instance of AB - maybe even > > > with the databases in different timeframes/formats/coming from > > > different providers and being cross referenced in indicators. > > > > > > Can MCP allow users,who now run more than one installation of AB > on > > a > > > single computer (for whatever reason) to get the same results > from > > > one installation. > > > > > > > > > b) in the past Tomasz would enable AB's native database for extra > > > data (historical fields other than OHLCVOI) - as I understand it > > > because it would weigh AB down with things not used by the > majority. > > > > > > Perhaps MCP will make the weight of an open structure AB database > a > > > featherweight (PremiumData has already expressed an interest in > > > synching their database with AB so it is not as if there is no > > demand > > > for this feature). > > > > > > c) a task scheduler built into AB - set the tasks at the start of > > the > > > day - if the keyboard/processor is idle AB starts to run the > tasks > > > (backtesting/optimising). > > > > > > d) if we want to run our trading platform and AB on real time > with > > > two monitors on one computer do we have enough grunt to do that > now > > > OR would MCP allow us to run processing intensive tasks like that > > > concurrently OR maybe with a minimised AB chart (on tick data) > > > floating on top of our trading platform window. > > > > > > I don't know - I am merely speculating! > > > > > > Surely someone out there has some better ideas? > > > > > > brian_z > > > > > > > > > > > > > > > > > > > > > > > > --- In [EMAIL PROTECTED] <mailto:amibroker%40yahoogroups.com> > ps.com, > > "sidhartha70" <sidhartha70@> > > > wrote: > > > > > > > > Hi Tomasz, > > > > > > > > I've been reading the arguments for and against multi-core > support > > > > with interest, and I just wanted to add my own voice to it and > > > mention > > > > an angle which hasn't currently been focused on. > > > > > > > > Since I bought AmiBroker a couple of months back I've been > blown > > > away > > > > by not only what a great piece of software it is, but also by > it's > > > > value and the fantastic customer support. Your own > responsiveness > > to > > > > customer needs and ability to efficiently add new features > leaves > > me > > > > with no hesitations in recommending AmiBroker to my fellow > > traders. > > > > > > > > However, I use AB in two ways. First for swing trading... for > > which > > > it > > > > is not only awesome, but also works perfectly happily on one > core > > > > because of the lower overhead in terms of data through put and > > > > therefore calculation overhead on that data. > > > > > > > > I also use AB for higher frequency day trading... and it's here > > for > > > me > > > > that multi-core support could really add something. For obvious > > > > reasons the multi-core argument has so far been almost > exclusively > > > > focused on optimization. However, for higher frequency traders > > like > > > > myself, working off just one core can be a significant > bottleneck. > > > > > > > > I have an IQ Feed 'tick' database... and some quite complex > chart > > > > setup's & indictaors. Currently, with chart refresh interval > set > > to > > > 2 > > > > seconds (I'd like it quikcer) I can see one core of my computer > > > > working away at about 50%. However, if I increase the update > > > interval > > > > to zero to bring in every tick, I see useage on one core go up > to > > > > 80-90%. At this point, you can imagine, AB becomes noticeably > > clunky > > > > and difficult to use. Things get even worse if I start to think > > > about > > > > symbol linking. (for reference I am only looking at a couple of > e- > > > mini > > > > symbols) > > > > The frustrating aspect is that I have another 7 x 3Ghz cores > > sitting > > > > there twiddling their fingers while AB is grinding up on one > core. > > > > > > > > So, I do think, for those of us who use AB for higher frequncy > > > > trading, there is a very good argument to consider adding multi- > > core > > > > support outside of the optimization argument. > > > > > > > > Love to hear your (or anyone's!) thoughts on this. > > > > > > > > Many Thanks > > > > > > > > > > > > > > > > > _____ > > > > I am using the free version of SPAMfighter for private users. > > It has removed 492 spam emails to date. > > Paying users do not have this message in their emails. > > Try SPAMfighter <http://www.spamfigh <http://www.spamfighter.com/len> > ter.com/len> for free now! > > > > > > > _____ > > I am using the free version of SPAMfighter for private users. > It has removed 492 spam emails to date. > Paying users do not have this message in their emails. > Try SPAMfighter <http://www.spamfighter.com/len> for free now! >
