[
https://issues.apache.org/jira/browse/WHIRR-330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Savu updated WHIRR-330:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I've just committed this. Thanks John for the extended bug description.
> BYON doesn't work with HadoopConfigurationBuilder
> -------------------------------------------------
>
> Key: WHIRR-330
> URL: https://issues.apache.org/jira/browse/WHIRR-330
> Project: Whirr
> Issue Type: Bug
> Components: core, service/hadoop
> Affects Versions: 0.5.0
> Environment: CentOS 5.5
> Reporter: John Amos
> Assignee: Andrei Savu
> Fix For: 0.6.0
>
> Attachments: WHIRR-330.patch
>
>
> I created a hadoop-byon.properties file in recipes similar to
> recipes/zookeeper-byon.properties, and I customized recipes/nodes-byon.yaml
> for my machines, then I ran:
> {noformat}
> bin/whirr launch-cluster --config recipes/hadoop-byon.properties
> {noformat}
> I see the following error:
> {noformat}
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduceConfiguration(HadoopConfigurationBuilder.java:106)
> at
> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduce(HadoopConfigurationBuilder.java:71)
> at
> org.apache.whirr.service.hadoop.HadoopNameNodeClusterActionHandler.beforeConfigure(HadoopNameNodeClusterActionHandler.java:80)
> at
> org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:48)
> at
> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:75)
> at
> org.apache.whirr.ByonClusterController.launchCluster(ByonClusterController.java:54)
> at
> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:61)
> at org.apache.whirr.cli.Main.run(Main.java:68)
> at org.apache.whirr.cli.Main.main(Main.java:106)
> {noformat}
> The code looks like this:
> {code:java}
> Hardware hardware = Iterables.getFirst(taskTrackers, null)
> .getNodeMetadata().getHardware();
> int coresPerNode = 0;
> for (Processor processor : hardware.getProcessors()) {
> coresPerNode += processor.getCores();
> }
> {code}
> The problem is that ByonClusterAction.java passes a null NodeMetadata when it
> creates a new Instance:
> {code:java}
> public Instance apply(NodeMetadata node) {
> String publicIp = Iterables.get(node.getPublicAddresses(), 0);
> return new Instance(credentials, roles, publicIp, publicIp, node.getId(),
> null);
> }
> {code}
> I changed null to node in the above code and it only helps a little because
> then a NullPointerException is thrown at HadoopConfigurationBuilder.java:109
> because the NodeMetadata has a null Hardware attribute. It seems like one of
> two things could happen:
> (1) recipes/nodes-byon.yaml could be enhanced to allow specification of
> hardware, including processors and cores per processor (and the first
> NullPointerException at line 106 would still need to be fixed)
> (2) The machines could be queried at runtime using /proc/cpuinfo, but that
> would probably need to happen in jclouds.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
