tibrewalpratik17 opened a new pull request, #14668: URL: https://github.com/apache/pinot/pull/14668
Related to issue: https://github.com/apache/pinot/issues/13491#issuecomment-2225868886 A fix has already been implemented to address potential segment CRC mismatches between Zookeeper (ZK) and Deepstore: https://github.com/apache/pinot/pull/14506 Despite the fix, many tables in Uber's Pinot production environment still exhibit CRC mismatches that were introduced earlier (before the fix). As a result, the compaction task consistently fails for these segments, effectively halting compaction for affected tables. Task failure is almost consistent for these tables: <img width="1712" alt="Screenshot 2024-12-16 at 11 52 23 PM" src="https://github.com/user-attachments/assets/1cd34ab9-484b-46be-ad63-9e426e9c143c"; /> Default Setting: The option is disabled (false) by default and should only be manually enabled when necessary. To unblock these tables compaction, **we are introducing an `ignoreCrcMismatch` task config for upsert compaction**. When enabled, the task will proceed with compaction even if a CRC mismatch is detected. Default value = false. As another follow-up, adding a Controller API will allow manual correction of segment CRC mismatches. However, given the scale of the current issue (with hundreds of affected segments), triggering fixes manually via the API might not be practical. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
