Embed Pig in Java does not display exception message when error occurs
----------------------------------------------------------------------
Key: PIG-2401
URL: https://issues.apache.org/jira/browse/PIG-2401
Project: Pig
Issue Type: Bug
Affects Versions: 0.9.1
Reporter: Prashant Kommireddi
Embedding Pig in Java does not display exception messages on console (logs are
not produced either, which is known)
I tried the mapreduce mode example
http://pig.apache.org/docs/r0.9.1/cont.html#Usage+Examples-N101CF
{code}
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.pig.PigServer;
public class idmapreduce {
public static void main(String[] args) {
try {
PigServer pigServer = new PigServer("mapreduce");
runIdQuery(pigServer, "passwd");
} catch (Exception e) {}
}
public static void runIdQuery(PigServer pigServer, String inputFile) throws
IOException {
pigServer.registerQuery("A = load '" + inputFile + "' using
PigStorage(':');");
pigServer.registerQuery("B = foreach A generate $0 as id;");
pigServer.store("B", "idout");
}
}
{code}
The issue with above code is that "idout" already exists on HDFS. Output when
the above is run:
{code}
pkommireddi@pkommireddi-wsl:~/misc/pig$ java -cp
/home/pkommireddi/gridforce_ws/hadoop-test/lib/pig-0.9.1.jar:.:$HADOOP_CONF_DIR
idmapreduce
11/12/05 19:36:46 INFO executionengine.HExecutionEngine: Connecting to hadoop
file system at: hdfs://xyz.net:54310
11/12/05 19:36:46 INFO executionengine.HExecutionEngine: Connecting to
map-reduce job tracker at: xyz.net:54311
11/12/05 19:36:47 INFO pigstats.ScriptState: Pig features used in the script:
UNKNOWN
{code}
There is no exception message spit out to the console.
It runs fine once I delete the output directory
{code}
pkommireddi@pkommireddi-wsl:~/misc/pig$ hadoop fs -rmr idout
Moved to trash: hdfs://xyz.net:54310/user/pkommireddi/idout
pkommireddi@pkommireddi-wsl:~/misc/pig$ java -cp
/home/pkommireddi/gridforce_ws/hadoop-test/lib/pig-0.9.1.jar:.:$HADOOP_CONF_DIR
idmapreduce
11/12/05 19:37:14 INFO executionengine.HExecutionEngine: Connecting to hadoop
file system at: hdfs://xyz.net:54310
11/12/05 19:37:14 INFO executionengine.HExecutionEngine: Connecting to
map-reduce job tracker at: xyz.net:54311
11/12/05 19:37:14 INFO pigstats.ScriptState: Pig features used in the script:
UNKNOWN
11/12/05 19:37:14 INFO mapReduceLayer.MRCompiler: File concatenation threshold:
100 optimistic? false
11/12/05 19:37:14 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size before
optimization: 1
11/12/05 19:37:14 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size after
optimization: 1
11/12/05 19:37:14 INFO pigstats.ScriptState: Pig script settings are added to
the job
11/12/05 19:37:14 INFO mapReduceLayer.JobControlCompiler:
mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
11/12/05 19:37:17 INFO mapReduceLayer.JobControlCompiler: Setting up single
store job
11/12/05 19:37:17 INFO mapReduceLayer.MapReduceLauncher: 1 map-reduce job(s)
waiting for submission.
11/12/05 19:37:17 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the arguments. Applications should implement Tool for the same.
11/12/05 19:37:17 INFO input.FileInputFormat: Total input paths to process : 1
11/12/05 19:37:17 INFO util.MapRedUtil: Total input paths to process : 1
11/12/05 19:37:17 INFO util.MapRedUtil: Total input paths (combined) to process
: 1
11/12/05 19:37:17 INFO mapReduceLayer.MapReduceLauncher: 0% complete
11/12/05 19:37:18 INFO mapReduceLayer.MapReduceLauncher: HadoopJobId:
job_201111102203_3624
11/12/05 19:37:18 INFO mapReduceLayer.MapReduceLauncher: More information at:
http://xyz.net:50030/jobdetails.jsp?jobid=job_201111102203_3624
11/12/05 19:37:30 INFO mapReduceLayer.MapReduceLauncher: 50% complete
11/12/05 19:37:38 INFO mapReduceLayer.MapReduceLauncher: 100% complete
11/12/05 19:37:38 INFO pigstats.SimplePigStats: Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
0.20.2 0.9.1 pkommireddi 2011-12-05 19:37:14 2011-12-05 19:37:38
UNKNOWN
Success!
Job Stats (time in seconds):
JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime
MaxReduceTime MinReduceTime AvgReduceTime Alias Feature Outputs
job_201111102203_3624 1 0 6 6 6 0 0
0 A,B MAP_ONLY idout,
Input(s):
Successfully read 49 records (2289 bytes) from:
"hdfs://xyz.net:54310/user/pkommireddi/passwd"
Output(s):
Successfully stored 49 records (2078 bytes) in: "idout"
Counters:
Total records written : 49
Total bytes written : 2078
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_201111102203_3624
11/12/05 19:37:38 INFO mapReduceLayer.MapReduceLauncher: Success!
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
