Re: Compile Hadoop 2.4.1 (with Tests and Without Tests)

2014-08-05 Thread Tsuyoshi OZAWA
Hi, Unfortunately, sometimes we face unexpected test failures. Please check whether the problem has been registered or resolved on Hadoop's JIRAs. * https://issues.apache.org/jira/browse/HADOOP * https://issues.apache.org/jira/browse/HDFS * https://issues.apache.org/jira/browse/MAPREDUCE * https:

Re: hadoop 2.4.1 build failure on CentOS 6

2014-08-05 Thread Romu
I'm afraid the log is too big to be pasted here :( Thanks Romu 2014-08-05 18:13 GMT+08:00 Binglin Chang : > Could you attach the entire log? > > > On Tue, Aug 5, 2014 at 3:43 PM, Romu wrote: > >> Yes, cmake is installed. >> >> # rpm -qa|grep cmake >> cmake-2.6.4-5.el6.x86_64 >> >> >> Regards,

Using specific nodes for hadoop 2 jobs

2014-08-05 Thread Robert Chu
Hi everyone, I was wondering if hadoop 2 contains any mechanisms for restricting the nodes that task JVMs are launched on. YARN seems to expose the ability to procure containers on specific hosts through its API, but does the hadoop 2 jobtracker allow you to choose hosts to run on? Robert Chu

Re: hadoop 2.4.1 build failure on CentOS 6

2014-08-05 Thread Romu
They are all installed: # rpm -qa|grep cmake cmake-2.6.4-5.el6.x86_64 # rpm -qa|grep zlib zlib-1.2.3-29.el6.x86_64 zlib-devel-1.2.3-29.el6.x86_64 # rpm -qa|grep ssl openssl-1.0.1e-16.el6_5.14.x86_64 openssl-devel-1.0.1e-16.el6_5.14.x86_64 Thanks Romu 2014-08-06 5:39 GMT+08:00 Tsuyoshi OZAWA :

How to do a date range with globs?

2014-08-05 Thread Mark Kerzner
Hi, experts, I have partitioned my output, so that I have directories like 2014/08/03 2014/08/04 2014/08/05 and I am trying to process a range of dates. Regular globs like [1-9]/[1-9] don't seem to work, because each position changes independently. So instead I wrote a parser that creates a reg

Capacity Scheduler Vcore Allocation in Yarn 2.2

2014-08-05 Thread Josh Clum
Hi, Does the Yarn 2.2 capacity scheduler only allocate 1 core per container or are all the cores on the machine on which a container is running available to that container? Thanks, Josh

Re: Hadoop 2.4.1 Verifying Automatic Failover Failed: ResourceManager and JobHistoryServer do not auto-failover to Standby Node

2014-08-05 Thread Akira AJISAKA
You need additional settings to make ResourceManager auto-failover. http://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html JobHistoryServer does not have automatic failover feature. Regards, Akira (2014/08/05 20:15), arthur.hk.c...@gmail.com wrote: Hi I have

Re: hadoop 2.4.1 build failure on CentOS 6

2014-08-05 Thread Tsuyoshi OZAWA
Hi, Could you check that the following packages which are mentioned in BUILDING.txt are installed? * CMake 2.6 or newer (if compiling native code) * Zlib devel (if compiling native code) * openssl devel ( if compiling native hadoop-pipes ) Thanks, - Tsuyoshi On Tue, Aug 5, 2014 at 7:13 PM, Bing

Re: Datanode not allowed to connect to the Namenode in Hadoop 2.3.0 cluster.

2014-08-05 Thread S.L
There is no entry there. Sent from my HTC - Reply message - From: "hadoop hive" To: Subject: Datanode not allowed to connect to the Namenode in Hadoop 2.3.0 cluster. Date: Tue, Aug 5, 2014 6:36 AM Remove the entry from dfs.exclude if there is any On Aug 4, 2014 3:28 AM, "S.L" wrote:

Access remote Linux cluster from Windows client

2014-08-05 Thread Daniel
Hello everyone, I have a remote Linux cluster which runs YARN and HDFS (Hadoop 2.4.1). Then I create a MapReduce job that should be executed on the remote cluster. If I execute the it on a local Linux client, it works fine. But if I try to execute the same job on a local Windows client, I recei

Hadoop 2.4.1 Verifying Automatic Failover Failed: ResourceManager and JobHistoryServer do not auto-failover to Standby Node

2014-08-05 Thread arthur.hk.c...@gmail.com
Hi I have set up the Hadoop 2.4.1 with HDFS High Availability using the Quorum Journal Manager. I am verifying Automatic Failover: I manually used “kill -9” command to disable all running Hadoop services in active node (NN-1), I can find that the Standby node (NN-2) now becomes ACTIVE now whi

Re: Datanode not allowed to connect to the Namenode in Hadoop 2.3.0 cluster.

2014-08-05 Thread hadoop hive
Remove the entry from dfs.exclude if there is any On Aug 4, 2014 3:28 AM, "S.L" wrote: > Hi All, > > I am trying to set up a Apache Hadoop 2.3.0 cluster , I have a master and > three slave nodes , the slave nodes are listed in the > $HADOOP_HOME/etc/hadoop/slaves file and I can telnet from the sl

Re: Datanode not allowed to connect to the Namenode in Hadoop 2.3.0 cluster.

2014-08-05 Thread Wellington Chevreuil
You should have /etc/hosts properly configured on all your cluster nodes. On 5 Aug 2014, at 07:28, S.L wrote: > when you say /etc/hosts/ file , you mean only on the master of on both the > master and slaves? > > > > > On Tue, Aug 5, 2014 at 1:20 AM, Satyam Singh > wrote: > You have not gi

Re: hadoop 2.4.1 build failure on CentOS 6

2014-08-05 Thread Binglin Chang
Could you attach the entire log? On Tue, Aug 5, 2014 at 3:43 PM, Romu wrote: > Yes, cmake is installed. > > # rpm -qa|grep cmake > cmake-2.6.4-5.el6.x86_64 > > > Regards, > Romu > > > 2014-08-05 15:36 GMT+08:00 Binglin Chang : > > do you install cmake? if not, do not use -Pnative to build hadoo

Re: hadoop 2.4.1 build failure on CentOS 6

2014-08-05 Thread Romu
Yes, cmake is installed. # rpm -qa|grep cmake cmake-2.6.4-5.el6.x86_64 Regards, Romu 2014-08-05 15:36 GMT+08:00 Binglin Chang : > do you install cmake? if not, do not use -Pnative to build hadoop:) > > > On Tue, Aug 5, 2014 at 2:57 PM, Romu wrote: > >> Hi, >> >> I tried to build hadoop 2.4.1

Re: hadoop 2.4.1 build failure on CentOS 6

2014-08-05 Thread Binglin Chang
do you install cmake? if not, do not use -Pnative to build hadoop:) On Tue, Aug 5, 2014 at 2:57 PM, Romu wrote: > Hi, > > I tried to build hadoop 2.4.1 in a CentOS 6 x86_64 vm but failed. Maven > 3.2.2 (installed from official bin tar ball) > > >- Maven: 3.2.2 >- Java: Oracle jdk 1.8.0