Hi Konstantinos,
Thank you for your answer,
I tried to set this property to 'yarn' just because I found that it
was a valid parameter for another property, namely
'mapreduce.framework.name'. However, I've just verified that the value
does not matter for giraph, as long as it is not set to 'local', in
which case the condition of the if clause I pointed you at in a
previous e-mail is satisfied. So, 'yarn' is certainly not an
appropriate value, but I am still not convinced that you should leave
it to 'local'.
Both these properties ('mapreduce.framework.name' and
'mapreduce.jobtracker.address') have a default value of 'local'
because the default design is for single-node executions. I believe
that both have to be changed for execution in a cluster of nodes.
The first property is set to 'yarn' in the client node of the juju
bundle but I believe that the second property should be also changed
to point to one of the slaves. However, I do not have a counter
example (yet :) ).
--Panagiotis
2016-04-14 5:00 GMT+03:00 Konstantinos Tsakalozos
<[email protected]>:
> Hi Panagiotis,
>
> Apologies for the late response. The issues you are reporting needed some
> work and research from our side. We appreciate your feedback.
>
> The first (and easiest) ask is to provide a way for the user to restart
> services. We are currently working on having the services setup as standard
> system services. We also merged the fix to restart yarn using the resource
> manager's actions. Note, that restarting yarn from the resource manager
> should trigger service restarts on the slaves (fixed in
> https://jujucharms.com/apache-hadoop-resourcemanager/trusty/2).
>
> Regarding the config variables, we are considering a way to enable users to
> update hadoop properties. We are still in the planning phase so I cannot say
> when this will become available. For the specific property you are
> suggesting, namely "mapreduce.jobtracker.address" we are are using the
> default value and we are a bit skeptical if we should deviate from that. How
> did you reach to the "yarn" variable for the "mapreduce.jobtracker.address"?
> In the code you showed us, checkLocalJobRunnerConfiguration checks that the
> job tracker address is not "local" (through a deprecated property), which is
> actually the default for yarn. Is it possible that Giraph is not supposed to
> work in yarn?
>
> For getting a configuration of Giraph and Hadoop that seem to work together
> you could look at Apache Bigtop that packages and deploys the respective
> projects:
> https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/giraph/templates/giraph-site.xml
> https://github.com/apache/bigtop/blob/master/bigtop-deploy/puppet/modules/hadoop/templates/mapred-site.xml
> Note however that since the last release of Giraph (Nov 2014) a lot has
> changed on the Hadoop side.
>
> Again, we would like to thank you for your feedback,
> Konstantinos
>
--
Juju mailing list
[email protected]
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
