This is an automated email from the ASF dual-hosted git repository.
xiangfu0 pushed a change to branch
xiangfu0/codex/codec-stack/07-integration-docs
in repository https://gitbox.apache.org/repos/asf/pinot.git
discard c8810abb269 Add codec pipeline integration tests and design doc
discard aedae05dd94 Support codecSpec on segment reload and enable codecSpec
end to end
discard 669848ce4a9 Add V7 raw forward index format for codec pipelines
discard bdc7c330979 Add T64 and GORILLA packing transform codecs
discard 432f34d4110 Add DELTA and DELTADELTA transform codecs
discard bb596e66c96 Harden Snappy decode bounds, reserve CODEC in the
registry, reject ZSTD(0)
discard f41789af4f1 Add bounded codec runtime and compression handlers
discard ba1a944cd4c Document why the codec DSL is not parsed by the SQL
expression parser
discard 2d9d6c1ecb4 Pin codec DSL equality and limit symmetry with tests
discard 240b090de3c Reject leading-zero codec arguments and tighten the spec
length cap
discard 5cbbe1b53ef Add codec spec DSL and configuration plumbing
add c7903e96037 Add codec spec DSL and configuration plumbing
add 822620de9e1 Reject leading-zero codec arguments and tighten the spec
length cap
add 7fc32733919 Pin codec DSL equality and limit symmetry with tests
add e3e58f452a2 Document why the codec DSL is not parsed by the SQL
expression parser
add c7d3f5ed80f Address codec configuration review feedback
add 29eddbd4680 Centralize codec DSL syntax validation
add c9a7afc7197 Add bounded codec runtime and compression handlers
add e8f70f4a100 Harden Snappy decode bounds, reserve CODEC in the
registry, reject ZSTD(0)
add e99c0e8400b Address codec runtime review feedback
add 7204585a04b Add DELTA and DELTADELTA transform codecs
add 968323c3252 Add stable delta codec decode fixtures
add e4af8ac902b Make delta wire decoding byte-order independent
add 4dcc8d16a94 Add T64 and GORILLA packing transform codecs
add 1d1b651b54f Harden packing codec frames and allocations
add d251cca93c5 Pin packing codec wire byte order
add d024edc32a7 Add V7 raw forward index format for codec pipelines
add 3a5cace1a6d Harden V7 codec pipeline I/O
add e784709d4a8 Validate V7 reader metadata invariants
add a318111a625 Support codecSpec on segment reload and enable codecSpec
end to end
add affc903373b Harden codec reload and compression stats
add 948d51c7b17 Add codec pipeline integration tests and design doc
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (c8810abb269)
\
N -- N -- N
refs/heads/xiangfu0/codex/codec-stack/07-integration-docs (948d51c7b17)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
docs/design/codec-pipeline-v7.md | 112 +++++++++-------
.../ColumnCompressionStatsContribution.java | 3 +-
.../resources/ColumnCompressionStatsInfo.java | 3 +-
.../tests/custom/CodecPipelineIntegrationTest.java | 129 ++++++++++++++++---
.../local/io/codec/BaseDeltaCodecDefinition.java | 5 +
.../local/io/codec/CodecPipelineExecutor.java | 141 ++++++++++++++++-----
.../segment/local/io/codec/CodecRegistry.java | 12 +-
.../local/io/codec/GorillaCodecDefinition.java | 49 ++++++-
.../local/io/codec/GzipCodecDefinition.java | 19 +--
.../segment/local/io/codec/T64CodecDefinition.java | 66 +++++++---
.../impl/FixedByteChunkForwardIndexWriterV7.java | 36 +++++-
...ompressionStatsTrackingForwardIndexCreator.java | 3 +-
.../fwd/SingleValueFixedByteRawIndexCreator.java | 8 ++
.../impl/openstruct/OpenStructColumnSplitter.java | 2 +-
.../index/forward/CompressionStatsMetadata.java | 6 +-
.../index/forward/ForwardIndexCreatorFactory.java | 4 +-
.../index/forward/ForwardIndexReaderFactory.java | 10 +-
.../segment/index/forward/ForwardIndexType.java | 5 +-
.../segment/index/loader/ForwardIndexHandler.java | 37 +++---
.../index/readers/forward/ChunkReaderContext.java | 16 +++
.../FixedByteChunkSVForwardIndexReaderV7.java | 47 +++++--
.../local/io/codec/CodecPipelineExecutorTest.java | 93 +++++++++++++-
.../io/codec/CompressionCodecCorruptInputTest.java | 35 +++++
.../local/io/codec/DeltaCodecRoundTripTest.java | 56 ++++++++
.../local/io/codec/GorillaCodecDefinitionTest.java | 111 +++++++++++++++-
.../local/io/codec/T64CodecDefinitionTest.java | 102 ++++++++++++++-
.../forward/CodecPipelineForwardIndexTest.java | 97 ++++++++++++++
.../forward/CompressionStatsMetadataTest.java | 6 +
...eChunkSVForwardIndexReaderV7CorruptionTest.java | 82 +++++++++++-
...dByteChunkSVForwardIndexReaderV7GoldenTest.java | 95 ++++++++++++++
.../forward/ForwardIndexReaderFactoryTest.java | 17 ++-
.../index/loader/ForwardIndexHandlerTest.java | 117 ++++++++++++++++-
.../readers/forward/ChunkReaderContextTest.java | 24 ++--
.../segment/local/utils/TableConfigUtilsTest.java | 18 +++
.../pinot/segment/spi/codec/CodecDslSyntax.java | 87 +++++++++++++
.../pinot/segment/spi/codec/CodecInvocation.java | 61 ++-------
.../pinot/segment/spi/codec/CodecPipeline.java | 6 +-
.../pinot/segment/spi/codec/CodecSpecParser.java | 59 +++------
.../segment/spi/index/ForwardIndexConfig.java | 19 +--
.../segment/spi/codec/CodecInvocationTest.java | 1 +
.../segment/spi/index/ForwardIndexConfigTest.java | 20 ++-
.../resources/SegmentCompressionStatsReader.java | 2 +-
.../SegmentCompressionStatsReaderTest.java | 77 +++++++++++
43 files changed, 1581 insertions(+), 317 deletions(-)
create mode 100644
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/forward/FixedByteChunkSVForwardIndexReaderV7GoldenTest.java
copy
pinot-common/src/test/java/org/apache/pinot/common/utils/StringUtilTest.java =>
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/readers/forward/ChunkReaderContextTest.java
(57%)
create mode 100644
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/codec/CodecDslSyntax.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
