Looks like GridGain are trying to make it even easier for us by providing a HOWTO wiki and making GridGain-enabled AMI's available for public use. Sweet.
On 25 July, 13:34, Martin Koistinen <[email protected]> wrote: > OK, I'm getting more serious about this. > > Here's the architecture I propose. Please let me know if there's anything > flawed here. > > - Someone to implement GridGain into JBT (Eugene, are you up for this?) > - Someone to implement allowing reference to a web-based locations (S3) > for back-test data into JBT. (possibly me, if I am able). > - I will locate or create a suitable public AMI (Amazon Machine Image) > that is based on something like Ubuntu with Java and with GridGain already > installed -- possibly starting-up at boot-time. > - Interested parties would then create their own AWS S3 storage > facilities. This would be used for their own strategies and backtest data. > This S3 storage would be accessible from JBT and would allow their local > machine AND any number of EC2 instances to access all the same data at the > same URI. > - Interested parties would then simply launch JBT locally, possibly > configure locations of remote instances and the location of their S3-based > back-test data. Next, they simply run the optimization as usual, albeit > harnessing the power of the number of cloud machines they choose. > > Interested parties would be able to choose the size and number of the > machines they wish to use. Each user of this set-up would be responsible > for their own EC3 and S3 costs. Since the AMI would be all ready to go, it > should be straight-forward to launch the cluster and put it to work almost > immediately. > > Linux-based AMI's used to reduce running costs to 2/3rd vs. Windows. S3 is > efficient because EC2 access to S3 is free and because, in theory all the > machines can access it through the same, universally-accessible URL. > > I'm interested in hearing from anyone who knows whether this is the right > model and/or can suggest improvements. I'm also looking for people who want > to help out in this effort. > > On Tue, Jul 21, 2009 at 10:28 PM, MKoistinen <[email protected]> wrote: > > > GridGain looks *perfect* for JBT, sadly, I don't have so many machines > > at my disposal. Just my 2.53 GHz MacBookPro and my PC (also running > > on another 2.16 GHz MacBookPro). I suspect I could get it to work > > across the two, but I don't think it'd be worth it. After all, it'd > > be easier to split the range of one of the parameters in half and run > > each half on different machines. > > > I was playing around with machines at GoGrid today, their fastest > > machine was disappointing and with its 6 cores @ 3GHz each and 8 GBs > > ram, was only a little bit faster than my 2x 2.5 GHz and 4GB > > MacBookPro. And, considering I've already paid for my Mac, it seemed > > a bit silly to pay $1.50 an hour for the GoGrid machine. Having said > > that, it was wicked easy to get going with them. > > > I think the real gains will come from spawning about 16 cloud machines > > and putting them into a single grid with GridGain (or similar). > > > On 20 July, 19:42, nonlinear5 <[email protected]> wrote: > > > > I think this has been brought up before here, but has anyone actually > > > > used Amazon EC2 for JBT yet? I'm just curious about how effective > > > > this would be for back-testing -- especially where 64 bits, many cores > > > > and mucho RAM would be required. > > > > I have a pretty powerful 8-core processor with 6Gb of RAM, so I didn't > > > have the need for EC2 yet, even when I ran optimization with the large > > > data sets. However, I did experiment with GridGain, which is a > > > framework for distributed optimization, and I found it very intuitive. > > > With just a few lines of code, JBT optimizer can actually run > > > distributed: > >http://groups.google.com/group/jbooktrader/browse_thread/thread/2d79d... > > > > There was at least one person who was planning to use EC2 with JBT, > > > but I am not sure how it went: > >http://groups.google.com/groups/profile?enc_user=gg85sxIAAAChfaE5hFby... --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en -~----------~----~----~----~------~----~------~--~---
