[
https://issues.apache.org/jira/browse/PIG-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-5075:
------------------------------------
Fix Version/s: (was: 0.17.0)
0.18.0
> Pig ORCStorage with Snappy Compression will fail with NoClassDefFoundError
> org/iq80/snappy/Snappy
> -------------------------------------------------------------------------------------------------
>
> Key: PIG-5075
> URL: https://issues.apache.org/jira/browse/PIG-5075
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.16.0
> Reporter: Prabhu Joseph
> Fix For: 0.18.0
>
>
> Pig Script to store a text file into ORC with Snappy compression enabled
> fails with java.lang.NoClassDefFoundError: org/iq80/snappy/Snappy
> The hive-exec jar which comes with Pig does not have snappy jars whereas the
> hive-exec jar comes with hive has that.
> {code}
> [root@prabhuSpark3 lib]# jar tvf
> /usr/hdp/2.4.2.0-258/pig/lib/hive-exec-1.2.1000.2.4.2.0-258-core.jar | grep
> iq80
> [root@prabhuSpark3 lib]#
> [root@prabhuSpark3 lib]# jar tvf
> /usr/hdp/2.4.2.0-258/hive/lib/hive-exec-1.2.1000.2.4.2.0-258.jar | grep iq80
> 0 Mon Apr 25 06:49:28 UTC 2016 org/iq80/
> 0 Mon Apr 25 06:49:28 UTC 2016 org/iq80/snappy/
> 1577 Mon Apr 25 06:49:28 UTC 2016 org/iq80/snappy/Snappy.class
> {code}
> Repro:
> {code}
> [root@prabhuSpark3 lib]# hadoop fs -cat /tmp/data
> hadoop,5
> hive,4
> pig,3
> tez,2
> hawq,1
> MYFILE = LOAD '/tmp/data' using PigStorage(',') As (name:chararray,age:int);
> Store MYFILE into '/tmp/orcsnappydata' using OrcStorage('-c SNAPPY');
> 2016-09-22 03:29:06,830 [main] ERROR
> org.apache.pig.backend.hadoop.executionengine.Launcher - Backend error message
> Error: org/iq80/snappy/Snappy
> 2016-09-22 03:29:06,831 [main] ERROR org.apache.pig.tools.pigstats.PigStats -
> ERROR 0: org.apache.pig.backend.executionengine.ExecException: ERROR 2997:
> Unable to recreate exception from backed error: Error: org/iq80/snappy/Snappy
> 2016-09-22 03:29:06,831 [main] ERROR
> org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job(s)
> failed!
> {code}
> Workaround:
> Register /usr/hdp/2.4.2.0-258/hive/lib/hive-exec-1.2.1000.2.4.2.0-258.jar;
> As part of this Bug, we want to include the Snappy jars which comes into Pig
> hive-exec jar.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
