commons-lang3 is a new dependency in Hive 1.2.0. Will need to add the
following line to ivy.xml and recompile:
<dependency org="org.apache.commons" name="commons-lang3"
rev="${commons-lang3.version}"
conf="compile->master" />
Daniel
On 6/17/15, 1:44 AM, "Shi Ju SJ Xie" <[email protected]> wrote:
>
>
>Hi All,
>
>Could you please help to take a look at this issue ? thanks a lot...
>
>I have encoutner a testcase failed for apache pig 0.15.0 with apache hive
>1.2.0, but there are no any error with pig0.15+hive0.14.
>
>Detail testcase:
>org.apache.pig.builtin.TestOrcStoragePushdown.java ->
>testPredicatePushdownTimestamp()
> { testPredicatePushdown(INPUT, "f10 >= ToDate('20100101', 'yyyyMMdd',
>'UTC')", 3000, 400000)}
>This case failed with PredicatePushdownOptimizer enabled, if
>PredicatePushdownOptimizer is disabled, then no any error.
>
>There are no any error for other pushdown predicate like: BigDecimal,
>FloatDouble, IntLongString and so on..
>
>
>Error msg:
>
>ob_1433820152892_0004_conf.xml perm=root:supergroup:rwxrwx---
>proto=rpc
>2015-06-08 19:24:10,799 [main] WARN
>org.apache.pig.backend.hadoop.executionengine.Launcher - There is no log
>file to write to.
>2015-06-08 19:24:10,799 [main] ERROR
>org.apache.pig.backend.hadoop.executionengine.Launcher - Backend error
>message
>Error: java.lang.ClassNotFoundException:
>org.apache.commons.lang3.exception.ExceptionUtils
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
>org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateRange
>(RecordReaderImpl.java:382)
> at
>org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateProto
>(RecordReaderImpl.java:338)
> at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl
>$SargApplier.pickRowGroups(RecordReaderImpl.java:711)
> at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups
>(RecordReaderImpl.java:752)
> at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe
>(RecordReaderImpl.java:778)
> at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe
>(RecordReaderImpl.java:987)
> at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow
>(RecordReaderImpl.java:1020)
> at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>
>(RecordReaderImpl.java:205)
> at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions
>(ReaderImpl.java:539)
> at
>org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.createReaderFromFile
>(OrcInputFormat.java:230)
> at org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat
>$OrcRecordReader.<init>(OrcNewInputFormat.java:69)
> at
>org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat.createRecordReader
>(OrcNewInputFormat.java:51)
> at
>org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordRead
>er.initNextRecordReader
>(PigRecordReader.java:250)
> at
>org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordRead
>er.<init>
>(PigRecordReader.java:121)
> at
>org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputForma
>t.createRecordReader
>(PigInputFormat.java:114)
> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>
>(MapTask.java:515)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:758)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs
>(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
>
>2015-06-08 19:24:10,801 [main] ERROR
>org.apache.pig.tools.pigstats.PigStats
>- ERROR 0: java.lang.ClassNotFoundException:
>org.apache.commons.lang3.exception.ExceptionUtils
>2015-06-08 19:24:10,802 [main] ERROR
>org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job
>(s) failed!
>2015-06-08 19:24:10,802 [main] INFO
>org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script
>Statistics:
>
>HadoopVersion PigVersion UserId StartedAt FinishedAt Features
>2.7.0-IBM-4 0.15.0_IBM_1-SNAPSHOT root 2015-06-08 19:23:47
>2015-06-08 19:24:10 FILTER
>
>Failed!
>
>Failed Jobs:
>JobId Alias Feature Message Outputs
>job_1433820152892_0004 D,E MAP_ONLY Message: Job failed!
>build/test/TestOrcStorage_out/TestOrcStorage_4,
>
>Input(s):
>Failed to read data from
>"hdfs://localhost:42758/user/root/build/test/TestOrcStorage_in/TestOrcStor
>age_1"
>
>Output(s):
>Failed to produce result in
>"build/test/TestOrcStorage_out/TestOrcStorage_4"
>
>Counters:
>Total records written : 0
>Total bytes written : 0
>Spillable Memory Manager spill count : 0
>Total bags proactively spilled: 0
>Total records proactively spilled: 0
>
>
>
>
>Thanks and best regard,
>Shiju
>------------------------------------------------------
>Shi Ju, Xie
>InfoSphere BigInsights, IBM China Software Development Laboratory
>Address: 28,ZhongGuanCun Software Park,No.8 Dong Bei Wang West Road,
>Haidian District Beijing P.R.China 100193
>
>Phone: +86 10 82450799
>e-mail: [email protected]
>------------------------------------------------------
