Re: Starting up hadoop cluster programatically

2011-01-19 Thread Tom White
The Jsch source doesn't have any references to proxyHost, so I'm guessing these properties don't have any effect. However, the Jsch home page does mention connection through HTTP proxy. Perhaps you could ask on the Jsch list how to achieve this? Cheers, Tom On Wed, Jan 19, 2011 at 6:40 AM,

Re: Running Mapred jobs after launching cluster

2011-01-27 Thread Tom White
You don't need to add anything to the classpath, but you need to use the configuration in the org.apache.whirr.service.Cluster object to populate your Hadoop Configuration object so that your code knows which cluster to connect to. See the getConfiguration() method in HadoopServiceController for

Re: Error while running cassandra using whirr

2011-01-28 Thread Tom White
On Fri, Jan 28, 2011 at 12:53 AM, Ashish paliwalash...@gmail.com wrote: Folks, I followed the instructions from http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2 Using whirr-0.2.0-incubating stable release I suggest trying with 0.3.0 (out soon, or available from svn

Re: Running Mapred jobs after launching cluster

2011-01-28 Thread Tom White
jars, as well as their dependencies. If you look at the POMs in the source code they will tell you the dependencies. Cheers Tom Thanks Praveen -Original Message- From: ext Tom White [mailto:tom.e.wh...@gmail.com] Sent: Friday, January 28, 2011 2:12 PM To: whirr-user

[ANNOUNCE] Apache Whirr 0.3.0-incubating released

2011-01-30 Thread Tom White
The Apache Whirr team is pleased to announce the release of Whirr 0.3.0-incubating from the Apache Incubator. Apache Whirr is a set of libraries for running cloud services such as Apache Hadoop, HBase, ZooKeeper, and Cassandra. The release is available here:

Re: Whirr and HBase

2011-02-01 Thread Tom White
Try removing the CDH lines. I don't think that this combination works yet. Tom On Feb 1, 2011 7:53 AM, Paolo Castagna castagna.li...@googlemail.com wrote: Andrei Savu wrote: Could you share the recipe? I want to try to replicate the issue on my computer.

Re: Whirr supports redhat?

2011-02-18 Thread Tom White
the information about what the whirr.image-id property should be? Praveen On Feb 18, 2011, at 7:12 PM, ext Tom White tom.e.wh...@gmail.com wrote: Hi Praveen, I haven't tried Hadoop on Cloudservers with Redhat, but the scripts do support RPM-based systems (like Amazon's Linux AMI). Please let

Re: image-id to specify a m1.large hardware in EC2

2011-04-13 Thread Tom White
Hi Patricio, In the past I've used whirr.hardware-id=m1.large whirr.image-id=us-east-1/ami-da0cf8b3 whirr.location-id=us-east-1 Hope that helps. Tom 2011/4/13 Patricio Echagüe patric...@gmail.com: Hi all, I need to create m1.large EC2 nodes in EC2 and was wondering if someone knows what

Re: Custom install and config functions when calling Whirr from code

2011-05-26 Thread Tom White
Hi John, The functions directory itself needs to be on the classpath. You can achieve this by including it in your application JAR (like the Whirr service JARs do), or by adding it to the application classpath (like the bin/whirr script does). Hope that helps. Cheers, Tom On Thu, May 26, 2011

Re: How to use OtherAction?

2011-06-02 Thread Tom White
On Thu, Jun 2, 2011 at 3:06 PM, Andrei Savu savu.and...@gmail.com wrote: I understand. Tom should be able to tell us more about the intended usage scenario for OtherAction. The other action call was just to cover the case if new events were added and not explicitly exposed in

[ANNOUNCE] Apache Whirr 0.5.0-incubating released

2011-06-05 Thread Tom White
The Apache Whirr team is pleased to announce the release of Whirr 0.5.0-incubating from the Apache Incubator. This is the fifth incubating release of Apache Whirr, a set of libraries for running cloud services such as Apache Hadoop, HBase, ZooKeeper, and Cassandra. The release is available here:

Re: set up two cassandra clusters?

2011-06-07 Thread Tom White
Do the clusters have different names? Can you supply the stacktrace you're getting from whirr.log. Cheers, Tom On Tue, Jun 7, 2011 at 4:45 PM, Khanh Nguyen nguyen.h.kh...@gmail.com wrote: Hi, I want to launch another cassandra clusters on EC2 but I keep getting an exception like this

Re: hadoop security and ssh proxy

2011-06-14 Thread Tom White
The proxy is not used for security (which would be better provided by a firewall), but to make the datanode addresses resolve correctly for the client. Without the proxy the datanodes return their internal addresses which are not routable by the client (which runs in an external network

Re: Is Service.launchCluster thread safe?

2011-06-14 Thread Tom White
On Tue, Jun 14, 2011 at 3:46 PM, John Conwell j...@iamjohn.me wrote: So as an FYI, I just tested using the whirr API to start multiple clusters at the same time using Futures, and it (seems to) works great.  really cuts down on the time to ramp up a set of clusters (like 4 or more).  Yay

Re: hadoop security and ssh proxy

2011-06-15 Thread Tom White
. Agreed, but it is possible - see the Hadoop integration tests which are an example of spinning up a Hadoop cluster from Java in a completely automated fashion. Tom On Tue, Jun 14, 2011 at 5:13 PM, Tom White tom.e.wh...@gmail.com wrote: The proxy is not used for security (which would be better

Re: Is Service.launchCluster thread safe?

2011-06-15 Thread Tom White
at 5:20 PM, Tom White tom.e.wh...@gmail.com wrote: On Tue, Jun 14, 2011 at 3:46 PM, John Conwell j...@iamjohn.me wrote: So as an FYI, I just tested using the whirr API to start multiple clusters at the same time using Futures, and it (seems to) works great.  really cuts down on the time

Re: Execute a cmd via jclouds SshClient that requires sudo privs on VM started by Whirr

2011-06-16 Thread Tom White
You could write your own predicate that does a cast. See ClusterController.runningInGroup() for something similar. Cheers, Tom On Thu, Jun 16, 2011 at 9:47 AM, John Conwell j...@iamjohn.me wrote: Pulled the code from RunScriptCommand as an example, and I think I'm good in that respect. I'm

Re: Run commands via whirr

2011-08-24 Thread Tom White
Looks great - I've often wanted something like this. I think adding whirr run-cmd would be the way to add this since then it's integrated with the whirr command. Thanks, Tom On Wed, Aug 24, 2011 at 2:06 AM, Karel Vervaeke ka...@outerthought.org wrote: I'd be happy to do the jira dance. On