Vamsi-klu opened a new pull request, #19532: URL: https://github.com/apache/pinot/pull/19532
Ledger-only foundation for first-class column deletion ([#18808](https://github.com/apache/pinot/issues/18808)). This is **not** a public delete API and does **not** replace [#18831](https://github.com/apache/pinot/pull/18831). ## What this adds - `SchemaDiff` in `pinot-spi`: dedicated helper for deleted / added / retained-incompatible columns. Not an overload of `Schema.isBackwardCompatibleWith`. Primary-key equality stays exact-name. `ignoreCase` uses `Locale.ROOT` for column-name matching only. - PropertyStore path `/COLUMN_DELETION_METADATA/<tableNameWithType>`. - Format-versioned ledger (`PREPARED` → `PENDING` / `RECLAIMING` → `COMPLETE` / `FAILED`). Newer stored formats are refused. `write(..., -1)` is create-only; updates CAS the version from the last read. - Dirty-set helper: `RefreshSegmentTask.time` is not a success marker. Equal `ctime` is dirty. An empty live set is not `COMPLETE`. Callers may pass a physical-presence override. This PR does not read segment files. ## What this does not add - No REST or client `allowColumnDeletion` - No `reclaimDeletedColumnsOnReload` - No `BaseDefaultColumnHandler` REMOVE expansion - No schema-update path that actually deletes a column ## Tests Unit tests in `pinot-spi` (`SchemaDiffTest`, `SchemaTest`) and `pinot-common` (`ColumnDeletionMetadataTest`, `ColumnDeletionMetadataAccessHelperTest`, `ColumnDeletionDirtySegmentPredicateTest`). ## Compatibility Additive only. New ZK prefix is unused until a later writer exists. Mixed-version controllers that cannot parse a newer ledger format refuse to overwrite it. Labels: `feature`, `release-notes` (new public `SchemaDiff` type in `pinot-spi`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
