KKcorps opened a new pull request, #20113: URL: https://github.com/apache/pinot/pull/20113
The segment data CRC covers only `.fwd` and `.dict`. A null is stored as the default value in the forward index, so replicas that differ only in nulls get the same data CRC, and `hasSameCRC` lets a replica keep a divergent local copy at commit. This replaces #19706, which changed the CRC for everyone. This adds data CRC version 2, which also covers `.bitmap.nullvalue`. A new server config `pinot.server.instance.segment.data.crc.version` picks the version for newly built segments. It defaults to 1, so segment bytes and CRCs are unchanged unless a deployment opts in. Version 2 segments write the version as an int after the data CRC in `creation.meta`, and older readers just ignore it. Version 2 equals version 1 for segments with no null vectors, so during a rollout only segments with nulls that straddle versions miss on the data CRC and fall back to the full CRC. Tested with a new `CrcUtilsTest` case: in version 1 a null and a stored default give the same data CRC, in version 2 they differ, and segments without nulls match across versions. `CrcUtilsTest` and `SegmentMetadataImplTest` pass, and spotless, checkstyle and license checks pass on the touched modules. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
