xiangfu0 opened a new pull request, #18188:
URL: https://github.com/apache/pinot/pull/18188
## Summary
- add `CompressionCodecSpec` parsing and validation so forward-index
compression stays externally string-based while supporting optional integer
parameters
- thread parameterized codec specs through forward-index creation, metadata
persistence, load/rewrite paths, and compressor factory resolution
- keep existing plain codec configs and deprecated compatibility paths
working, and add focused tests for serde, config resolution, factory wiring,
and compression round-trips
## User manual
Raw forward indexes can now set `fieldConfigList[].compressionCodec` with
either a plain codec name or a codec plus integer level.
Examples:
```json
"compressionCodec": "ZSTD"
"compressionCodec": "ZSTD(3)"
"compressionCodec": "GZIP(6)"
```
Existing values such as `SNAPPY`, `LZ4`, `PASS_THROUGH`, and legacy
`chunkCompressionType` / `dictIdCompressionType` configs continue to work.
## Sample table config
```json
{
"fieldConfigList": [
{
"name": "column1",
"encodingType": "RAW",
"compressionCodec": "ZSTD(3)"
}
]
}
```
## Testing
- `./mvnw spotless:apply -q`
- `./mvnw license:format -q`
- `./mvnw -pl pinot-segment-local,pinot-segment-spi,pinot-spi,pinot-tools
-am -DskipTests compile -q`
- `./mvnw -pl pinot-segment-local,pinot-segment-spi,pinot-spi,pinot-common
-am
-Dtest=ForwardIndexConfigTest,CompressionCodecSpecTest,CompressionCodecSpecValidatorTest,TableConfigSerDeUtilsTest,ForwardIndexCreatorFactoryTest,BaseSegmentCreatorTest,ForwardIndexTypeTest,IndexLoadingConfigTest,ForwardIndexHandlerTest,TestCompression
-Dsurefire.failIfNoSpecifiedTests=false test -q`
- `./mvnw checkstyle:check -q`
- `./mvnw license:check -q`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
