Hello,
I cloned the github source of parquet-mr. Then
mvn clean package -Plocal
which returned
[INFO] ------------------------------------------------------------
------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Parquet MR (Incubating) .................... SUCCESS [1.536s]
[INFO] Apache Parquet Common (Incubating) ................ SUCCESS [1.327s]
[INFO] Apache Parquet Generator (Incubating) ............. SUCCESS [0.449s]
[INFO] Apache Parquet Encodings (Incubating) ............. SUCCESS [4.469s]
[INFO] Apache Parquet Column (Incubating) ................ SUCCESS [9.453s]
[INFO] Apache Parquet Jackson (Incubating) ............... SUCCESS [0.649s]
[INFO] Apache Parquet Hadoop (Incubating) ................ FAILURE [8.899s]
(
mvn --version
Apache Maven 3.0.5 (Red Hat 3.0.5-16)
Maven home: /usr/share/maven
Java version: 1.8.0_101, vendor: Oracle Corporation
Java home: /opt/jdk1.8.0_101/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-327.28.2.el7.x86_64", arch: "amd64",
family: "unix"
on centos 7 )
Im surely missing something simple, but what could it be? A bit of
searching didnt turn up much
(that failure was preceded by: )
Jul 31, 2017 8:30:01 PM INFO: parquet.hadoop.codec.CodecConfig:
Compression: GZIP
Jul 31, 2017 8:30:01 PM INFO: parquet.hadoop.codec.CodecConfig:
Compression: SNAPPY
Jul 31, 2017 8:30:01 PM INFO: parquet.hadoop.codec.CodecConfig:
Compression: SNAPPY
Jul 31, 2017 8:30:01 PM INFO: parquet.hadoop.codec.CodecConfig: Compression
set through hadoop codec: parquet.hadoop.codec.SnappyCodec
Jul 31, 2017 8:30:01 PM INFO: parquet.hadoop.codec.CodecConfig:
Compression: SNAPPY
Jul 31, 2017 8:30:01 PM INFO: parquet.hadoop.codec.CodecConfig: Compression
set through hadoop codec: parquet.hadoop.codec.SnappyCodec
Jul 31, 2017 8:30:01 PM INFO: parquet.hadoop.codec.CodecConfig:
Compression: SNAPPY
Jul 31, 2017 8:30:01 PM WARNING: parquet.hadoop.codec.CodecConfig: codec
class not found: Compression codec unexistedCodec was not found.
java.lang.IllegalArgumentException: Compression codec unexistedCodec was
not found.
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.
getOutputCompressorClass(FileOutputFormat.java:112)
at parquet.hadoop.codec.CodecConfig$MapreduceCodecConfig.
getHadoopOutputCompressorClass(CodecConfig.java:135)
at parquet.hadoop.codec.CodecConfig.getHadoopCompressionCodec(
CodecConfig.java:104)
...
ul 31, 2017 8:30:01 PM INFO: parquet.hadoop.codec.CodecConfig: Compression:
UNCOMPRESSED
Jul 31, 2017 8:30:01 PM WARNING: parquet.hadoop.codec.CodecConfig: codec
class not found: Compression codec unexistedCodec was not found.
java.lang.IllegalArgumentException: Compression codec unexistedCodec was
not found.
at org.apache.hadoop.mapred.FileOutputFormat.getOutputCompressorClass(
FileOutputFormat.java:89)
at parquet.hadoop.codec.CodecConfig$MapredCodecConfig.
getHadoopOutputCompressorClass(CodecConfig.java:161)
at parquet.hadoop.codec.CodecConfig.getHadoopCompressionCodec(
CodecConfig.java:104)
at parquet.hadoop.codec.CodecConfig.getCodec(CodecConfig.java:90)
...
Jul 31, 2017 8:30:01 PM INFO: parquet.hadoop.codec.CodecConfig:
Compression: UNCOMPRESSED
Jul 31, 2017 8:30:01 PM INFO: parquet.hadoop.codec.CodecConfig: Compression
set through hadoop codec: org.apache.hadoop.io.compress.DefaultCodec
Jul 31, 2017 8:30:01 PM WARNING: parquet.hadoop.codec.CodecConfig: codec
defined in hadoop config is not supported by parquet
[org.apache.hadoop.io.compress.DefaultCodec]
and will use UNCOMPRESSED
parquet.hadoop.codec.CompressionCodecNotSupportedException: codec not
supported: org.apache.hadoop.io.compress.DefaultCodec
at parquet.hadoop.metadata.CompressionCodecName.fromCompressionCodec(
CompressionCodecName.java:47)
at parquet.hadoop.codec.CodecConfig.getHadoopCompressionCodec(
CodecConfig.java:106)
at parquet.hadoop.codec.CodecConfig.getCodec(CodecConfig.java:90)
at parquet.hadoop.codec.CodecConfigTest.shouldUseHadoopFlagToSetCodec(
CodecConfigTest.java:67)
at parquet.hadoop.codec.CodecConfigTest.testReadingCodecs(
CodecConfigTest.java:42)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
