I believe Shravan resolved this issue. It was due to misconfiguration of his environment.
Olga > -----Original Message----- > From: pi song [mailto:[EMAIL PROTECTED] > Sent: Wednesday, April 30, 2008 7:57 PM > To: [email protected] > Subject: Re: Use of MiniCluster > > Shravan, > > Have you got the solution on this yet? I'm also expecting the > answer from someone else but seems like nobody knows. I guess > what you have to do is to access MiniCluster via PigServer > API instead (=just a tiny : )pointer). > > Please tell me how if you've already got this working. > > Pi > > > > On 4/30/08, Shravan M Narayanamurthy <[EMAIL PROTECTED]> wrote: > > > > Hi, > > I am working on the types branch and am trying to use the > > MiniCluster class for testing Job Submission and Monitoring > in the redesigned pipeline. > > I have hit into an issue. > > > > The MiniCluster's Map Reduce part does not seem to work. I tried to > > submit a wordCount MR job that exists in the hadoop > examples jar. It does not work. > > Here are the steps I followed: > > > > (i) Started the MiniCluster using this: > > > > package org.apache.pig.test; > > > > public class TestMiniCluster { > > public static void main(String[] args) throws > > InterruptedException { > > MiniCluster cluster = MiniCluster.buildCluster(); > > while(true){ > > Thread.sleep(10000); > > } > > } > > } > > > > (ii) hadoop --config ~/pigtest/conf dfs -put /etc/passwd passwd > > > > (iii) hadoop --config ~/pigtest/conf dfs -ls > > Found 1 items > > /user/shravanm/passwd <r 3> 2768 2008-04-30 04:37 > > > > Verified that the file had been created > > > > (iv) hadoop --config ~/pigtest/conf jar hadoop-0.15.3-examples.jar > > wordcount passwd passwdCnt > > Got the following exceptions: > > 08/04/30 04:37:41 INFO mapred.FileInputFormat: Total > input paths to > > process : 1 > > 08/04/30 04:37:41 INFO mapred.JobClient: Running job: > > job_200804300435_0001 > > 08/04/30 04:37:42 INFO mapred.JobClient: map 0% reduce 0% > > 08/04/30 04:37:54 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_m_000000_0, Status : FAILED > > 08/04/30 04:37:55 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50061/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000000_0&filter=stdout > > 08/04/30 04:37:55 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50061/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000000_0&filter=stderr > > 08/04/30 04:37:55 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_m_000001_0, Status : FAILED > > 08/04/30 04:37:55 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50063/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000001_0&filter=stdout > > 08/04/30 04:37:55 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50063/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000001_0&filter=stderr > > 08/04/30 04:37:55 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_r_000000_0, Status : FAILED > > 08/04/30 04:37:55 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50062/tasklog?plaintext=true &taskid=task_200804300435_0001_r_000000_0&filter=stdout > > 08/04/30 04:37:55 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50062/tasklog?plaintext=true &taskid=task_200804300435_0001_r_000000_0&filter=stderr > > 08/04/30 04:37:58 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_m_000001_1, Status : FAILED > > 08/04/30 04:37:58 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50062/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000001_1&filter=stdout > > 08/04/30 04:37:58 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50062/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000001_1&filter=stderr > > 08/04/30 04:37:58 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_m_000000_1, Status : FAILED > > 08/04/30 04:37:58 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50063/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000000_1&filter=stdout > > 08/04/30 04:37:58 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50063/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000000_1&filter=stderr > > 08/04/30 04:38:01 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_r_000000_1, Status : FAILED > > 08/04/30 04:38:01 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50063/tasklog?plaintext=true &taskid=task_200804300435_0001_r_000000_1&filter=stdout > > 08/04/30 04:38:01 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50063/tasklog?plaintext=true &taskid=task_200804300435_0001_r_000000_1&filter=stderr > > 08/04/30 04:38:05 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_m_000000_2, Status : FAILED > > 08/04/30 04:38:05 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50060/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000000_2&filter=stdout > > 08/04/30 04:38:05 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50060/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000000_2&filter=stderr > > 08/04/30 04:38:05 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_r_000000_2, Status : FAILED > > 08/04/30 04:38:05 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50060/tasklog?plaintext=true &taskid=task_200804300435_0001_r_000000_2&filter=stdout > > 08/04/30 04:38:05 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50060/tasklog?plaintext=true &taskid=task_200804300435_0001_r_000000_2&filter=stderr > > 08/04/30 04:38:05 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_m_000001_2, Status : FAILED > > 08/04/30 04:38:05 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50061/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000001_2&filter=stdout > > 08/04/30 04:38:05 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50061/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000001_2&filter=stderr > > 08/04/30 04:38:08 INFO mapred.JobClient: map 100% reduce 100% > > 08/04/30 04:38:08 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_r_000000_3, Status : TIPFAILED > > 08/04/30 04:38:08 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50061/tasklog?plaintext=true &taskid=task_200804300435_0001_r_000000_3&filter=stdout > > 08/04/30 04:38:08 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50061/tasklog?plaintext=true &taskid=task_200804300435_0001_r_000000_3&filter=stderr > > 08/04/30 04:38:08 INFO mapred.JobClient: Task Id : > > task_200804300435_0001_m_000000_3, Status : TIPFAILED > > 08/04/30 04:38:08 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50062/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000000_3&filter=stdout > > 08/04/30 04:38:08 WARN mapred.JobClient: Error reading task > > outputhttp:// > > > richbreak-lr.eglbp.corp.yahoo.com:50062/tasklog?plaintext=true &taskid=task_200804300435_0001_m_000000_3&filter=stderr > > java.io.IOException: Job failed! > > at > org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:831) > > at > org.apache.hadoop.examples.WordCount.run(WordCount.java:147) > > at > org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > > at > org.apache.hadoop.examples.WordCount.main(WordCount.java:153) > > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess > orImpl.java:39) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth > odAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > > > > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke (ProgramDriver.java:68) > > at > > > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) > > at > > > org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:49) > > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess > orImpl.java:39) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth > odAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at org.apache.hadoop.util.RunJar.main(RunJar.java:155) > > > > The same sequence when tried with an actual cluster setup > on my local > > machine works perfectly. > > > > Am I doing something wrong? Should I use the MiniCluster in a > > different way? > > I am using revision 631443, a bit old but this is the one > that exists > > in the type branch.. > > > > Thanks, > > --Shravan > > > > > > >
