Hi, just a comment / idea. This is thread is mostly about how to make larger processing power to bear. (Grid-computing is just one way.)
I was only wondering. The kind of computing (parallel number- crunching) that JBT does seems to me to be the same kind of computing some frameworks that rely on the processing power of graphics cards could bring to bear. Has anyone made experiences with this approach.. - how much faster is it actually? And how well does this work with Java / existing code? (Just an idea) Klaus On 25 Jul., 14:34, Martin Koistinen <[email protected]> wrote: > OK, I'm getting more serious about this. > > Here's the architecture I propose. Please let me know if there's anything > flawed here. > > - Someone to implement GridGain into JBT (Eugene, are you up for this?) > - Someone to implement allowing reference to a web-based locations (S3) > for back-test data into JBT. (possibly me, if I am able). > - I will locate or create a suitable public AMI (Amazon Machine Image) > that is based on something like Ubuntu with Java and with GridGain already > installed -- possibly starting-up at boot-time. > - Interested parties would then create their own AWS S3 storage > facilities. This would be used for their own strategies and backtest data. > This S3 storage would be accessible from JBT and would allow their local > machine AND any number of EC2 instances to access all the same data at the > same URI. > - Interested parties would then simply launch JBT locally, possibly > configure locations of remote instances and the location of their S3-based > back-test data. Next, they simply run the optimization as usual, albeit > harnessing the power of the number of cloud machines they choose. > > Interested parties would be able to choose the size and number of the > machines they wish to use. Each user of this set-up would be responsible > for their own EC3 and S3 costs. Since the AMI would be all ready to go, it > should be straight-forward to launch the cluster and put it to work almost > immediately. > > Linux-based AMI's used to reduce running costs to 2/3rd vs. Windows. S3 is > efficient because EC2 access to S3 is free and because, in theory all the > machines can access it through the same, universally-accessible URL. > > I'm interested in hearing from anyone who knows whether this is the right > model and/or can suggest improvements. I'm also looking for people who want > to help out in this effort. > > > > On Tue, Jul 21, 2009 at 10:28 PM, MKoistinen <[email protected]> wrote: > > > GridGain looks *perfect* for JBT, sadly, I don't have so many machines > > at my disposal. Just my 2.53 GHz MacBookPro and my PC (also running > > on another 2.16 GHz MacBookPro). I suspect I could get it to work > > across the two, but I don't think it'd be worth it. After all, it'd > > be easier to split the range of one of the parameters in half and run > > each half on different machines. > > > I was playing around with machines at GoGrid today, their fastest > > machine was disappointing and with its 6 cores @ 3GHz each and 8 GBs > > ram, was only a little bit faster than my 2x 2.5 GHz and 4GB > > MacBookPro. And, considering I've already paid for my Mac, it seemed > > a bit silly to pay $1.50 an hour for the GoGrid machine. Having said > > that, it was wicked easy to get going with them. > > > I think the real gains will come from spawning about 16 cloud machines > > and putting them into a single grid with GridGain (or similar). > > > On 20 July, 19:42, nonlinear5 <[email protected]> wrote: > > > > I think this has been brought up before here, but has anyone actually > > > > used Amazon EC2 for JBT yet? I'm just curious about how effective > > > > this would be for back-testing -- especially where 64 bits, many cores > > > > and mucho RAM would be required. > > > > I have a pretty powerful 8-core processor with 6Gb of RAM, so I didn't > > > have the need for EC2 yet, even when I ran optimization with the large > > > data sets. However, I did experiment with GridGain, which is a > > > framework for distributed optimization, and I found it very intuitive. > > > With just a few lines of code, JBT optimizer can actually run > > > distributed: > >http://groups.google.com/group/jbooktrader/browse_thread/thread/2d79d... > > > > There was at least one person who was planning to use EC2 with JBT, > > > but I am not sure how it went: > >http://groups.google.com/groups/profile?enc_user=gg85sxIAAAChfaE5hFby... --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en -~----------~----~----~----~------~----~------~--~---
