[
https://issues.apache.org/jira/browse/WHIRR-342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121142#comment-13121142
]
Tom White commented on WHIRR-342:
---------------------------------
This looks good to me. A few comments:
* In configure_cdh_hadoop.sh CONFIGURE_HADOOP_DONE is never read. I think it
would be more robust to check for the existence of a directory to test for
previous installation or configuration (e.g. /usr/local/hadoop,
/etc/hadoop/conf).
* HadoopClusterActionHandler#afterConfigure() should be moved to the subclasses
since it should not be called by HadoopDataNodeClusterActionHandler and
HadoopTaskTrackerClusterActionHandler, and for
HadoopNameNodeClusterActionHandler and HadoopJobTrackerClusterActionHandler
they should do different things. The local files (config, proxy) should only be
written in one place - e.g. by HadoopNameNodeClusterActionHandler.
* The imports are still shuffled. Can you change your default import order so
that "import com..." comes before "import java..."?
* I think that a Hadoop integration test for the separate nodes case would be
useful.
* It would be nice to support HDFS-only clusters too (i.e. make MapReduce
optional), but this can be done separately.
> hadoop/hbase configuration & active roles on a node
> ---------------------------------------------------
>
> Key: WHIRR-342
> URL: https://issues.apache.org/jira/browse/WHIRR-342
> Project: Whirr
> Issue Type: Improvement
> Components: service/hadoop, service/hbase
> Reporter: Bruno Dumon
> Fix For: 0.7.0
>
> Attachments: WHIRR-342.patch, WHIRR-342.patch, WHIRR-342.patch
>
>
> The following limitations exist with the generation of
> hadoop-(core|hdfs|mapred).xml and hbase-site.xml (assuming WHIRR-339 applied):
> * they are not generated by all roles (e.g. tasktracker, thrift server,
> ...), by consequence running these roles by themselves on a node
> unaccompagnied of a role that does generate them will not work.
> * running two roles on the same node that generate the same files does not
> work as it should, as the generated contents gets appended twice to the same
> file, causing non-well-formed XML. This is because of the usage of jcloud's
> Statements.appendFile.
> The cheapest solution would be to replace Statements.appendFile with
> something similar but without the 'append' behavior, thus rather a
> 'Statements.overwriteFile' (not available in jclouds afaics).
> This of course assumes that when different roles are writing the same files,
> that they put the same contents in them, so that the overwriting does not
> matter.
> Alternatively, things could be made smarter so that the same configuration is
> only generated once for all roles.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
