Re: nodemanager listen on 0.0.0.0

2015-12-08 Thread Harsh J
Thank you Varun, https://issues.apache.org/jira/browse/MAPREDUCE-5938 is the one. On Tue, Dec 8, 2015 at 10:02 PM Varun Saxena wrote: > There is already a JIRA for it - MAPREDUCE-5938(and other related JIRAs' > MAPREDUCE-6402 and YARN-4119) > > Regards, > Varun Saxena > > On Tue, Dec 8, 2015 at

Re: nodemanager listen on 0.0.0.0

2015-12-08 Thread Li Li
the hadoop cluster may be used as crawler so it should visit public network. On Tue, Dec 8, 2015 at 7:18 PM, Namikaze Minato wrote: > Listening on 0.0.0.0 means listening on all interfaces. Do you really > NEED to have it listen only on internal addresses? If you're not > connected to a public ne

Experience with Sahara on OpenStack

2015-12-08 Thread Andreas Fritzler
Hi, Does somebody have any experience with the Sahara components [1] on OpenStack? The tutorial videos on youtube look very appealing and easy. However they are mostly tailored towards ETL based workloads. But how about more complex scenarios like Streaming, Data Science centric, etc.? Also how a

Re: nodemanager listen on 0.0.0.0

2015-12-08 Thread Varun Saxena
There is already a JIRA for it - MAPREDUCE-5938(and other related JIRAs' MAPREDUCE-6402 and YARN-4119) Regards, Varun Saxena On Tue, Dec 8, 2015 at 9:56 PM, Harsh J wrote: > Hello, > > Could you file a JIRA for this please? Currently the ShuffleHandler will > always bind to wildcard address due

Re: nodemanager listen on 0.0.0.0

2015-12-08 Thread Harsh J
Hello, Could you file a JIRA for this please? Currently the ShuffleHandler will always bind to wildcard address due to the code being that way (in both branch-2 and trunk presently: https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduc

Re: nodemanager listen on 0.0.0.0

2015-12-08 Thread Varun Vasudev
Hi Li, What have you set yarn.nodemanager.bind-host to? From https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-common/yarn-default.xml - "The actual address the server will bind to. If this optional address is set, the RPC and webapp servers will bind to this address and the p

unsubscribe

2015-12-08 Thread ml_143
unsubscribe

Re: nodemanager listen on 0.0.0.0

2015-12-08 Thread Namikaze Minato
Listening on 0.0.0.0 means listening on all interfaces. Do you really NEED to have it listen only on internal addresses? If you're not connected to a public network it should not matter, right? Anyway, your cluster should work as-is. Regards, LLoyd PS: FYI, your e-mail ended up in spams. On 8 D

nodemanager listen on 0.0.0.0

2015-12-08 Thread Li Li
I want to setup a yarn cluster. It's not allowed to visit from public network so I changed many configuration files to let all service bind to the internal network address. All thing is fine exception the nodemanager which bind 0.0.0.0:13562. I searched the default-mapred-default.xml and foun

How to junit test customized Hadoop InputFormat/OutputFormat

2015-12-08 Thread Todd
Hi, I have defined an InputFormat class and an OutputFormat class. It looks to me that I have to create a MR to test whether they work. I would ask whether there is way to junit customized Hadoop InputFormat/OutputFormat without kicking off an MR application. Thanks!