Jackie-Jiang opened a new pull request, #18473: URL: https://github.com/apache/pinot/pull/18473
## Summary `testCommitTimeCompactionPreservesDeletedRecords` in `CommitTimeCompactionIntegrationTest` configured `deleteRecordColumn` via `updateTableConfig` *after* `addTableConfig` had already created each of the three tables. The pre-commit assertion at line 1126 (`validatePreCommitState`) expects 2 logical records but observes 4, because soft-deleted rows are never filtered out. ## Root cause The upsert metadata manager captures `_deleteRecordColumn` once at construction time — it is a `final` field on `BasePartitionUpsertMetadataManager` (line 89), populated from `UpsertContext` in `BaseTableUpsertMetadataManager#init` (`BaseTableUpsertMetadataManager.java:125-146`). A subsequent `updateTableConfig` writes the new value to ZooKeeper but does **not** rebuild the in-memory manager, so the consumer never recognizes the `deleted` column and the valid-doc bitmap is not narrowed. This regression dates back to commit `93de8f0dc7` (Add support for commit time compaction with columnMajorBuild, #16769), which refactored the test from setting the delete column on `UpsertConfig` *before* `setUpTable(...)` (the pattern still used in `UpsertTableIntegrationTest`) to the `createUpsertTable(...)` + post-create `updateTableConfig(...)` pattern that doesn't take effect. ## Fix Add an overload of `createUpsertTable` that accepts an optional `deleteRecordColumn` and threads it onto the `UpsertConfig` before `addTableConfig`. Route the three table-setup blocks in `testCommitTimeCompactionPreservesDeletedRecords` through the new overload and drop the now-unnecessary post-create `setDeleteRecordColumn` + `updateTableConfig` calls. The other tests in the file are unaffected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
