How do we know we have a valid hadoop conf on the client? Should we augment client error messages/info messages to indicate where properties are being found?
On Wed, Sep 17, 2014 at 10:28 AM, Billie Rinaldi <[email protected]> wrote: > That would be great if we could drop the hadoop conf properties from the > slider-client.xml when we have a good hadoop conf available. > > On Wed, Sep 17, 2014 at 2:19 AM, Steve Loughran <[email protected]> > wrote: > > > I think there's some JIRAs on that. What we could do is add the hadoop > conf > > dir env variables to the classpath (put them after slider-conf dir to > pick > > up slider's log4j first). > > > > I'd argue for only doing that in the .py script and then we change the > > build to copy bin/slider.py to bin/slider (somehow) so there is then only > > one script to keep up to date. And as the script gets more complex, > > switching to python gets more and more compelling > > > > > > On 17 September 2014 02:01, Sumit Mohanty <[email protected]> > wrote: > > > > > Ran some tests. Looks like the easiest option is to drop yarn-site.xml > in > > > Slider conf dir. > > > > > > However, if hadoop is installed on the host and /etc/hadoop/conf > exists, > > > should we just use the site.xml files at /etc/hadoop/conf? > > > > > > -Sumit > > > > > > On Tue, Sep 16, 2014 at 5:45 PM, Sumit Mohanty < > [email protected] > > > > > > wrote: > > > > > > > Looks like we need the following properties in the slider-client when > > RM > > > HA > > > > is enabled. > > > > > > > > * yarn.resourcemanager.ha.enabled > > > > * yarn.resourcemanager.ha.rm-ids > > > > * yarn.resourcemanager.hostname.[one entry per ids specified above] > > > > * yarn.resourcemanager.zk-address > > > > * yarn.resourcemanager.cluster-id > > > > > > > > Does the list look complete? > > > > > > > > *Steve*, related question: > > > > Should we go the route where we drop yarn-site.xml in the Slider > config > > > > directory and it gets read by the client? *Or, does it already work?* > > > > > > > > *-*Sumit > > > > > > > > -- > > > > CONFIDENTIALITY NOTICE > > > > NOTICE: This message is intended for the use of the individual or > > entity > > > to > > > > which it is addressed and may contain information that is > confidential, > > > > privileged and exempt from disclosure under applicable law. If the > > reader > > > > of this message is not the intended recipient, you are hereby > notified > > > that > > > > any printing, copying, dissemination, distribution, disclosure or > > > > forwarding of this communication is strictly prohibited. If you have > > > > received this communication in error, please contact the sender > > > immediately > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > > > > -- > > > thanks > > > Sumit > > > > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
