Hi, The pom seems alright. When you run it with maven, do you have the cluster config (e.g. /etc/hadoop/conf) in the classpath as well?
Terence > On Dec 10, 2014, at 3:25 PM, Srinivas Reddy Kancharla <[email protected]> > wrote: > > Hi Terence, > > I am using mvn to compile and run my sample application. If I use *mvn*, I > am seeing that all my containers are getting launched on the same node from > where I am executing my program. > > *but* when I am not using mvn i.e. compiling the same file and executing > with proper classpath (having all required .jars), I am seeing that all > nodes are properly getting used. i.e. container 0 on first node and > container 1 and 2 on other nodes. > > I know this sounds strange but working on it to check if any config I am > missing. Below is my POM.xml setting: > > <dependency> > > <groupId>org.apache.twill</groupId> > > <artifactId>twill-api</artifactId> > > <version>0.3.0-incubating</version> > > </dependency> > > > <dependency> > > <groupId>org.apache.twill</groupId> > > <artifactId>twill-yarn</artifactId> > > <version>0.3.0-incubating</version> > > </dependency> > > > <dependency> > > <groupId>org.apache.hadoop</groupId> > > <artifactId>hadoop-hdfs</artifactId> > > <version>2.4.0</version> > > </dependency> > > > <dependency> > > <groupId>org.apache.hadoop</groupId> > > <artifactId>hadoop-common</artifactId> > > <version>2.4.0</version> > > </dependency> > > > <dependency> > > <groupId>org.apache.hadoop</groupId> > > <artifactId>hadoop-client</artifactId> > > <version>2.4.0</version> > > </dependency> > > > <dependency> > > <groupId>org.apache.hadoop</groupId> > > <artifactId>hadoop-yarn-api</artifactId> > > <version>2.4.0</version> > > </dependency> > > > > > On Thu, Dec 4, 2014 at 10:49 PM, Terence Yim <[email protected]> wrote: > >> Sounds like you don't have capacity in the cluster. Have you check the rm >> status page (at port 8080) to see how many resources are available in the >> cluster? >> >> Terence >> >> Sent from my iPhone >> >>> On Dec 4, 2014, at 10:43 PM, Srinivas Reddy Kancharla < >> [email protected]> wrote: >>> >>> HI, >>> >>> When I am trying to execute my progrom which is trying to execute one >>> AbstractTwillRunnable class, its not getting executed in any of my slave >>> node (have a setup of 3 nodes ..1 master and 2 slaves) . Its in a loop as >>> shown below (Showing status as"*ACCEPTED*"). This log is from my master >>> node where "ResourceManager" is running. On the same node, I am trying to >>> launch my program. Could you please let me know what is that i am missing >>> here. >>> >>> >>> 22:37:28.884 [ STARTING] DEBUG o.a.twill.yarn.YarnTwillPreparer - Submit >> AM >>> container spec: application_1417760674314_0004 >>> 22:37:28.907 [IPC Parameter Sending Thread #0] DEBUG >>> org.apache.hadoop.ipc.Client - IPC Client (1561117309) connection to / >>> 0.0.0.0:8032 from srini sending #1 >>> 22:37:28.911 [IPC Client (1561117309) connection to /0.0.0.0:8032 from >>> srini] DEBUG org.apache.hadoop.ipc.Client - IPC Client (1561117309) >>> connection to /0.0.0.0:8032 from srini got value #1 >>> 22:37:28.912 [ STARTING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call: >>> submitApplication took 6ms >>> 22:37:28.915 [IPC Parameter Sending Thread #0] DEBUG >>> org.apache.hadoop.ipc.Client - IPC Client (1561117309) connection to / >>> 0.0.0.0:8032 from srini sending #2 >>> 22:37:28.916 [IPC Client (1561117309) connection to /0.0.0.0:8032 from >>> srini] DEBUG org.apache.hadoop.ipc.Client - IPC Client (1561117309) >>> connection to /0.0.0.0:8032 from srini got value #2 >>> 22:37:28.916 [ STARTING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call: >>> getApplicationReport took 1ms >>> 22:37:28.924 [ STARTING] INFO o.a.h.y.c.api.impl.YarnClientImpl - >>> Submitted application application_1417760674314_0004 >>> 22:37:28.926 [IPC Parameter Sending Thread #0] DEBUG >>> org.apache.hadoop.ipc.Client - IPC Client (1561117309) connection to / >>> 0.0.0.0:8032 from srini sending #3 >>> 22:37:28.927 [IPC Client (1561117309) connection to /0.0.0.0:8032 from >>> srini] DEBUG org.apache.hadoop.ipc.Client - IPC Client (1561117309) >>> connection to /0.0.0.0:8032 from srini got value #3 >>> 22:37:28.927 [ STARTING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call: >>> getApplicationReport took 2ms >>> 22:37:28.927 [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - >>> Application HelloWorldRunnable with id application_1417760674314_0004 >>> submitted >>> 22:37:28.928 [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - >>> Checking yarn application status for HelloWorldRunnable >>> application_1417760674314_0004 >>> 22:37:28.928 [IPC Parameter Sending Thread #0] DEBUG >>> org.apache.hadoop.ipc.Client - IPC Client (1561117309) connection to / >>> 0.0.0.0:8032 from srini sending #4 >>> 22:37:28.929 [IPC Client (1561117309) connection to /0.0.0.0:8032 from >>> srini] DEBUG org.apache.hadoop.ipc.Client - IPC Client (1561117309) >>> connection to /0.0.0.0:8032 from srini got value #4 >>> 22:37:28.929 [ STARTING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call: >>> getApplicationReport took 1ms >>> 22:37:28.929 [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn >>> application status for HelloWorldRunnable application_1417760674314_0004: >>> *ACCEPTED* >>> 22:37:29.930 [IPC Parameter Sending Thread #0] DEBUG >>> org.apache.hadoop.ipc.Client - IPC Client (1561117309) connection to / >>> 0.0.0.0:8032 from srini sending #5 >>> 22:37:29.932 [IPC Client (1561117309) connection to /0.0.0.0:8032 from >>> srini] DEBUG org.apache.hadoop.ipc.Client - IPC Client (1561117309) >>> connection to /0.0.0.0:8032 from srini got value #5 >>> 22:37:29.932 [ STARTING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call: >>> getApplicationReport took 2ms >>> 22:37:29.933 [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn >>> application status for HelloWorldRunnable application_1417760674314_0004: >>> *ACCEPTED* >>> 22:37:30.643 [Kafka-Consumer-log-0] DEBUG >>> o.a.t.i.k.client.SimpleKafkaConsumer - No leader for topic partition >>> TopicPartition{topic=log, partition=0}. >>> 22:37:30.933 [IPC Parameter Sending Thread #0] DEBUG >>> org.apache.hadoop.ipc.Client - IPC Client (1561117309) connection to / >>> 0.0.0.0:8032 from srini sending #6 >>> 22:37:30.935 [IPC Client (1561117309) connection to /0.0.0.0:8032 from >>> srini] DEBUG org.apache.hadoop.ipc.Client - IPC Client (1561117309) >>> connection to /0.0.0.0:8032 from srini got value #6 >>> 22:37:30.936 [ STARTING] DEBUG o.a.hadoop.ipc.ProtobufRpcEngine - Call: >>> getApplicationReport took 3ms >>> 22:37:30.936 [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn >>> application status for HelloWorldRunnable application_1417760674314_0004: >>> *ACCEPTED* >>> >>> Thanks and regards, >>> Srini >>
