This is an automated email from the ASF dual-hosted git repository.
gabor pushed a change to branch encryption
in repository https://gitbox.apache.org/repos/asf/parquet-mr.git.
omit 7b7099f PARQUET-1373: Encryption key tools (#615)
omit cc5e268 PARQUET-1807: Encryption: Interop and Function test suite for
Java version (#782)
omit 06b5372 PARQUET-1229: Parquet MR encryption (#776)
omit 42827b1 PARQUET-1817: Crypto Properties Factory (#769)
omit add2497 PARQUET-1818: Fix bloom/encryption collision in
format-structures (#771)
omit 01460a0 PARQUET-1286: Crypto package (#614)
omit c68e688 PARQUET-1228: Format Structures encryption (#613)
add 3db93ea PARQUET-1826: Document Hadoop configuration options (#781)
add 60e1cda PARQUET-1842: Update jackson-databind version (#785)
add 783c3b7 PARQUET-1763: Add SLF4J to TestCircularReferences (#746)
add 1456c34 PARQUET-1756: Remove Dependency on Maven Plugin
semantic-versioning (#741)
add b835df9 PARQUET-1853: Minimize shaded fastutil in parquet-avro (#790)
add 94f89aa PARQUET-1750: Reduce Memory Usage of RowRanges Class (#735)
add a2459b6 PARQUET-1728: Simplify NullPointerException Handling in
AvroWriteSupport (#716)
add c6c553b PARQUET-1775: Deprecate AvroParquetWriter Builder Hadoop Path
(#750)
add b2d366a PARQUET-1863: Configure protoc-jar-maven-plugin to add
generated sources to test sources path (#792)
add 0acade0 PARQUET-1850: Fix dictionaryPageOffset flag setting in
toParquetMetadata method
add 1c63c93 PARQUET-1868: Fix bloom filter toggle in reader options
builder (#795)
add 958ed6f PARQUET-1684: Do not store default protobuf values as null
for proto3 (#702)
add dc40b59 PARQUET-1866: Replace Hadoop ZSTD with JNI-ZSTD (#793)
add 84c954d PARQUET-1827: UUID type currently not supported by parquet-mr
(#778)
add e4988f3 Parquet-1872: Add TransCompression command to parquet-tools
(#796)
add 00b42d5 Parquet-1872: Add TransCompression command to parquet-tools -
Add the command to registry to complete (#799)
add 2589cc8 PARQUET-1879: MapKeyValue is not a valid Logical Type (#798)
add fadbe6e Parquet-1860: Add missing Builder Class to ProtoParquetWriter
Class (#791)
add 9a1fbc4 PARQUET-1778: Do Not Consider Class for Avro Generic Record
Reader (#751)
add 4516e97 PARQUET-1228: Format Structures encryption (#613)
add f85a962 PARQUET-1286: Crypto package (#614)
add 42dfc34 PARQUET-1818: Fix bloom/encryption collision in
format-structures (#771)
add 71ad811 PARQUET-1817: Crypto Properties Factory (#769)
add b60f4a4 PARQUET-1229: Parquet MR encryption (#776)
add dcf7d7e PARQUET-1807: Encryption: Interop and Function test suite for
Java version (#782)
add 22a5ae1 PARQUET-1373: Encryption key tools (#615)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (7b7099f)
\
N -- N -- N refs/heads/encryption (22a5ae1)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
parquet-avro/README.md | 44 +++
parquet-avro/pom.xml | 1 +
.../org/apache/parquet/avro/AvroConverters.java | 16 +
.../org/apache/parquet/avro/AvroParquetReader.java | 32 ++
.../org/apache/parquet/avro/AvroParquetWriter.java | 8 +
.../apache/parquet/avro/AvroRecordConverter.java | 15 +-
.../apache/parquet/avro/AvroSchemaConverter.java | 34 ++-
.../org/apache/parquet/avro/AvroWriteSupport.java | 73 +++--
.../java/org/apache/parquet/avro/AvroTestUtil.java | 21 +-
.../parquet/avro/TestAvroSchemaConverter.java | 42 ++-
.../parquet/avro/TestCircularReferences.java | 6 +-
.../parquet/avro/TestGenericLogicalTypes.java | 49 +++-
.../org/apache/parquet/avro/TestReadWrite.java | 50 +++-
.../parquet/avro/TestReflectLogicalTypes.java | 289 +++++++++++++++++-
.../src/main/java/org/apache/parquet/cli/Main.java | 2 +
.../cli/commands/TransCompressionCommand.java | 96 ++++++
parquet-column/pom.xml | 7 +
.../internal/filter2/columnindex/RowRanges.java | 44 ++-
.../apache/parquet/schema/ConversionPatterns.java | 5 +-
.../parquet/schema/LogicalTypeAnnotation.java | 44 +++
.../parquet/schema/PrimitiveStringifier.java | 27 ++
.../org/apache/parquet/schema/PrimitiveType.java | 7 +
.../main/java/org/apache/parquet/schema/Types.java | 53 ++--
.../parquet/schema/TestPrimitiveStringifier.java | 39 ++-
.../apache/parquet/schema/TestTypeBuilders.java | 56 ++--
.../schema/TestTypeBuildersWithLogicalTypes.java | 14 +
.../hadoop/metadata/CompressionCodecName.java | 2 +-
parquet-hadoop/README.md | 325 ++++++++++++++++++++-
parquet-hadoop/pom.xml | 5 +
.../org/apache/parquet/ParquetReadOptions.java | 2 +-
.../format/converter/ParquetMetadataConverter.java | 22 +-
.../org/apache/parquet/hadoop/CodecFactory.java | 4 +-
.../org/apache/parquet/hadoop/MemoryManager.java | 4 +-
.../apache/parquet/hadoop/ParquetFileReader.java | 3 +-
.../apache/parquet/hadoop/ParquetOutputFormat.java | 4 +-
.../{SnappyCodec.java => ZstandardCodec.java} | 69 +++--
...ressorStream.java => ZstdCompressorStream.java} | 56 ++--
.../hadoop/codec/ZstdDecompressorStream.java | 32 +-
.../parquet/hadoop/util/CompressionConverter.java | 271 +++++++++++++++++
.../parquet/hadoop/util/H1SeekableInputStream.java | 2 +-
.../converter/TestParquetMetadataConverter.java | 198 +++++++++++++
.../apache/parquet/hadoop/TestZstandardCodec.java | 167 +++++++++++
.../hadoop/example/TestInputOutputFormat.java | 1 +
.../hadoop/util/CompressionConveterTest.java | 319 ++++++++++++++++++++
.../parquet/statistics/TestColumnIndexes.java | 4 +
.../ql/io/parquet/TestHiveSchemaConverter.java | 10 +-
.../org/apache/parquet/pig/PigSchemaConverter.java | 3 +-
.../apache/parquet/pig/TestPigSchemaConverter.java | 20 +-
.../parquet/pig/TestTupleRecordConsumer.java | 10 +-
parquet-protobuf/pom.xml | 29 +-
.../apache/parquet/proto/ProtoParquetWriter.java | 39 +++
.../apache/parquet/proto/ProtoWriteSupport.java | 74 +++--
.../parquet/proto/ProtoInputOutputFormatTest.java | 294 +++++++++++++++++++
.../parquet/proto/ProtoWriteSupportTest.java | 50 +++-
.../java/org/apache/parquet/proto/TestUtils.java | 30 +-
.../thrift/TestThriftToParquetFileWriter.java | 22 +-
.../parquet/thrift/TestParquetWriteProtocol.java | 30 +-
.../parquet/thrift/TestThriftSchemaConverter.java | 8 +-
.../TestThriftSchemaConverterProjectUnion.java | 12 +-
.../org/apache/parquet/tools/command/Registry.java | 1 +
.../tools/command/TransCompressionCommand.java | 92 ++++++
pom.xml | 9 +-
62 files changed, 2966 insertions(+), 331 deletions(-)
create mode 100644 parquet-avro/README.md
create mode 100644
parquet-cli/src/main/java/org/apache/parquet/cli/commands/TransCompressionCommand.java
copy
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/{SnappyCodec.java
=> ZstandardCodec.java} (53%)
copy
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/{NonBlockedCompressorStream.java
=> ZstdCompressorStream.java} (52%)
copy
parquet-common/src/main/java/org/apache/parquet/ParquetRuntimeException.java =>
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/codec/ZstdDecompressorStream.java
(55%)
create mode 100644
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/CompressionConverter.java
create mode 100644
parquet-hadoop/src/test/java/org/apache/parquet/hadoop/TestZstandardCodec.java
create mode 100644
parquet-hadoop/src/test/java/org/apache/parquet/hadoop/util/CompressionConveterTest.java
create mode 100644
parquet-tools/src/main/java/org/apache/parquet/tools/command/TransCompressionCommand.java
