Redirecting to Apache pig user list On Thu, Nov 29, 2012 at 1:01 AM, Johnny Kowalski <[email protected]>wrote:
> Another hint? Is is some permission issue? > > MY EXAMPLE: > in = LOAD '/user/myUser/aTest' USING PigStorage(); > DUMP in; > > When I try to run above pig in grunt shell with "pig -x local" it > successfully prints output. > * > And when I do it in mapreduce pig grundshell it hangs... on this line:" > > 2012-11-28 15:24:27,709 [main] INFO org.apache.pig.backend.hadoop. > * > ***executionengine.****mapReduceLayer.****MapReduceLauncher - 0% complete" > * > > > W dniu środa, 28 listopada 2012 15:44:22 UTC+1 użytkownik Johnny Kowalski > napisał: > >> Hi, after configuring whole stack I've got another issue. My pig jobs >> hangs. >> I've created completly new user added a dir for him >> sudo -u hdfs hadoop fs -mkdir /user/newUser >> sudo -u hdfs hadoop fs -chown newUser:newUser /user/newUser >> >> and wanted to run pig that runs fine on another yarn configuration >> cluster. And got something like this: >> >> >> PIG CONSOLE OUTPUT >> >> 2012-11-28 15:24:26,759 [Thread-4] INFO >> org.apache.hadoop.mapreduce.**lib.input.FileInputFormat >> - Total input paths to process : 1 >> 2012-11-28 15:24:26,760 [Thread-4] INFO org.apache.pig.backend.hadoop.** >> executionengine.util.**MapRedUtil - Total input paths to process : 1 >> 2012-11-28 15:24:26,783 [Thread-4] INFO org.apache.pig.backend.hadoop.** >> executionengine.util.**MapRedUtil - Total input paths (combined) to >> process : 1 >> 2012-11-28 15:24:26,872 [Thread-4] INFO >> org.apache.hadoop.mapreduce.**JobSubmitter >> - number of splits:1 >> 2012-11-28 15:24:26,889 [Thread-4] WARN >> org.apache.hadoop.conf.**Configuration >> - fs.default.name is deprecated. Instead, use fs.defaultFS >> 2012-11-28 15:24:26,890 [Thread-4] WARN >> org.apache.hadoop.conf.**Configuration >> - mapreduce.job.counters.limit is deprecated. Instead, use >> mapreduce.job.counters.max >> 2012-11-28 15:24:26,891 [Thread-4] WARN >> org.apache.hadoop.conf.**Configuration >> - mapred.job.tracker is deprecated. Instead, use >> mapreduce.jobtracker.address >> 2012-11-28 15:24:26,892 [Thread-4] WARN >> org.apache.hadoop.conf.**Configuration >> - dfs.https.address is deprecated. Instead, use dfs.namenode.https-address >> 2012-11-28 15:24:26,893 [Thread-4] WARN >> org.apache.hadoop.conf.**Configuration >> - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum >> 2012-11-28 15:24:26,894 [Thread-4] WARN >> org.apache.hadoop.conf.**Configuration >> - mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir >> 2012-11-28 15:24:27,563 [Thread-4] INFO >> org.apache.hadoop.mapred.**ResourceMgrDelegate >> - Submitted application application_1354100898821_0011 to ResourceManager >> at userver/192.168.56.101:8032 >> 2012-11-28 15:24:27,629 [Thread-4] INFO org.apache.hadoop.mapreduce.**Job >> - The url to track the job: http://userver:8088/proxy/** >> application_1354100898821_**0011/<http://userver:8088/proxy/application_1354100898821_0011/> >> 2012-11-28 15:24:27,709 [main] INFO org.apache.pig.backend.hadoop.** >> executionengine.**mapReduceLayer.**MapReduceLauncher - 0% complete >> >> Here I've noticed logs keep growing while mine job is in status RUNNING >> >> /var/log/hadoop-yarn# less hadoop-cmf-yarn1-NODEMANAGER-**userver.log.out >> >> 2012-11-28 15:39:16,227 INFO org.apache.hadoop.yarn.server.**nodemanager. >> **NodeStatusUpdaterImpl: Sending out status for container: container_id >> {, app_attempt_id {, application_id {, id: 11, cluster_timestamp: >> 1354100898821, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: >> "", exit_status: -1000, >> 2012-11-28 15:39:17,230 INFO org.apache.hadoop.yarn.server.**nodemanager. >> **NodeStatusUpdaterImpl: Sending out status for container: container_id >> {, app_attempt_id {, application_id {, id: 11, cluster_timestamp: >> 1354100898821, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: >> "", exit_status: -1000, >> 2012-11-28 15:39:18,233 INFO org.apache.hadoop.yarn.server.**nodemanager. >> **NodeStatusUpdaterImpl: Sending out status for container: container_id >> {, app_attempt_id {, application_id {, id: 11, cluster_timestamp: >> 1354100898821, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: >> "", exit_status: -1000, >> 2012-11-28 15:39:18,356 INFO org.apache.hadoop.yarn.server.** >> nodemanager.containermanager.**monitor.ContainersMonitorImpl: Memory >> usage of ProcessTree 25203 for container-id container_1354100898821_0011_ >> **01_000001: 124.5mb of 1.5gb physical memory used; 1.9gb of 3.1gb >> virtual memory used >> 2012-11-28 15:39:19,238 INFO org.apache.hadoop.yarn.server.**nodemanager. >> **NodeStatusUpdaterImpl: Sending out status for container: container_id >> {, app_attempt_id {, application_id {, id: 11, cluster_timestamp: >> 1354100898821, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: >> "", exit_status: -1000, >> 2012-11-28 15:39:20,241 INFO org.apache.hadoop.yarn.server.**nodemanager. >> **NodeStatusUpdaterImpl: Sending out status for container: container_id >> {, app_attempt_id {, application_id {, id: 11, cluster_timestamp: >> 1354100898821, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: >> "", exit_status: -1000, >> 2012-11-28 15:39:21,245 INFO org.apache.hadoop.yarn.server.**nodemanager. >> **NodeStatusUpdaterImpl: Sending out status for container: container_id >> {, app_attempt_id {, application_id {, id: 11, cluster_timestamp: >> 1354100898821, }, attemptId: 1, }, id: 1, }, state: C_RUNNING, diagnostics: >> "", exit_status: -1000, >> (END) >> >> >> please help >> >
