Hi All,
Could you please help to take a look at this issue ? thanks a lot...
I have encoutner a testcase failed for apache pig 0.15.0 with apache hive
1.2.0, but there are no any error with pig0.15+hive0.14.
Detail testcase:
org.apache.pig.builtin.TestOrcStoragePushdown.java ->
testPredicatePushdownTimestamp()
{ testPredicatePushdown(INPUT, "f10 >= ToDate('20100101', 'yyyyMMdd',
'UTC')", 3000, 400000)}
This case failed with PredicatePushdownOptimizer enabled, if
PredicatePushdownOptimizer is disabled, then no any error.
There are no any error for other pushdown predicate like: BigDecimal,
FloatDouble, IntLongString and so on..
Error msg:
ob_1433820152892_0004_conf.xml perm=root:supergroup:rwxrwx---
proto=rpc
2015-06-08 19:24:10,799 [main] WARN
org.apache.pig.backend.hadoop.executionengine.Launcher - There is no log
file to write to.
2015-06-08 19:24:10,799 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.Launcher - Backend error
message
Error: java.lang.ClassNotFoundException:
org.apache.commons.lang3.exception.ExceptionUtils
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateRange
(RecordReaderImpl.java:382)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateProto
(RecordReaderImpl.java:338)
at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl
$SargApplier.pickRowGroups(RecordReaderImpl.java:711)
at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups
(RecordReaderImpl.java:752)
at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe
(RecordReaderImpl.java:778)
at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe
(RecordReaderImpl.java:987)
at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow
(RecordReaderImpl.java:1020)
at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>
(RecordReaderImpl.java:205)
at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions
(ReaderImpl.java:539)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.createReaderFromFile
(OrcInputFormat.java:230)
at org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat
$OrcRecordReader.<init>(OrcNewInputFormat.java:69)
at
org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat.createRecordReader
(OrcNewInputFormat.java:51)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initNextRecordReader
(PigRecordReader.java:250)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.<init>
(PigRecordReader.java:121)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader
(PigInputFormat.java:114)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>
(MapTask.java:515)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:758)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs
(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
2015-06-08 19:24:10,801 [main] ERROR org.apache.pig.tools.pigstats.PigStats
- ERROR 0: java.lang.ClassNotFoundException:
org.apache.commons.lang3.exception.ExceptionUtils
2015-06-08 19:24:10,802 [main] ERROR
org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job
(s) failed!
2015-06-08 19:24:10,802 [main] INFO
org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script
Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
2.7.0-IBM-4 0.15.0_IBM_1-SNAPSHOT root 2015-06-08 19:23:47
2015-06-08 19:24:10 FILTER
Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
job_1433820152892_0004 D,E MAP_ONLY Message: Job failed!
build/test/TestOrcStorage_out/TestOrcStorage_4,
Input(s):
Failed to read data from
"hdfs://localhost:42758/user/root/build/test/TestOrcStorage_in/TestOrcStorage_1"
Output(s):
Failed to produce result in
"build/test/TestOrcStorage_out/TestOrcStorage_4"
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Thanks and best regard,
Shiju
------------------------------------------------------
Shi Ju, Xie
InfoSphere BigInsights, IBM China Software Development Laboratory
Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road,
Haidian District Beijing P.R.China 100193
Phone: +86 10 82450799
e-mail: [email protected]
------------------------------------------------------
