[
https://issues.apache.org/jira/browse/S4-25?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501281#comment-13501281
]
Matthieu Morel edited comment on S4-25 at 11/20/12 4:32 PM:
------------------------------------------------------------
I just uploaded some updates to the previous patch, in commit a021481
It includes improvements to parameters parsing, and in particular, the handling
of memory and other JVM parameters. The memory parameter passed to YARN is used
to set the maximum heap size, and thus is actually less than the maximum amount
of memory that can be required by S4 nodes, since that would also include
thread stack size and other things, and I'm not sure how to exactly determine
the total maximum memory of the VM before launching it.
Other pending issues are still : stopping the application (rather than just the
application master) and test dependencies.
was (Author: mmorel):
I just uploaded some updates to the previous patch, in commit b590862
It includes improvements to parameters parsing, and in particular, the handling
of memory and other JVM parameters. The memory parameter passed to YARN is used
to set the maximum heap size, and thus is actually less than the maximum amount
of memory that can be required by S4 nodes, since that would also include
thread stack size and other things, and I'm not sure how to exactly determine
the total maximum memory of the VM before launching it.
Other pending issues are still : stopping the application (rather than just the
application master) and test dependencies.
> Write S4 Application Master to deploy S4 in Yarn
> ------------------------------------------------
>
> Key: S4-25
> URL: https://issues.apache.org/jira/browse/S4-25
> Project: Apache S4
> Issue Type: New Feature
> Reporter: J Mohamed Zahoor
> Fix For: 0.6
>
> Attachments: S4-ApplicationMaster.diff, S4-Client.diff,
> S4-Constants.diff, S4-YARN-1.patch
>
>
> On the lines of s4PigWrapper, write a s4 application master to host s4 piper
> inside Hadoop Yarn. This could be useful not only for reading data stored in
> hadoop ( to build or train a model)... But we could make use of the resource
> manager to deploy s4 instances in remote machine and monitor them. In short,
> we could make use of most of the resource management , scheduling and other
> good stuff in Yarn.
> - Yarn is useful to deploy and launch s4 instances.
> - It still requires deploying node managers on each box which means it will
> be useful if one is running more than one s4 process on a node.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
