andygrove opened a new pull request, #3468: URL: https://github.com/apache/datafusion-comet/pull/3468
## Summary - Marks `BatchReader` as `@Deprecated` (since 0.14.0) — kept for Iceberg compatibility via `@IcebergApi` - Removes all prefetch internals from `BatchReader` (fields, methods, `PrefetchTask` inner class) - Removes `COMET_SCAN_PREFETCH_ENABLED` and `COMET_SCAN_PREFETCH_THREAD_NUM` configs - Removes `CometPrefetchThreadPool` - Deletes `CometParquetPartitionReaderFactory` and `CometParquetScan` (V2 Parquet scan dead code) - Simplifies `CometScanExec.prepareRDD` to always use `newFileScanRDD` - Cleans up `EliminateRedundantTransitions` V2 dead code path - Removes prefetch tests and `BatchReader` benchmark case - Cleans up `CometParquetScan` references in tests ## Context `BatchReader` is annotated `@IcebergApi` and must be kept for Iceberg compatibility, but Comet's own production code no longer uses it. The prefetch feature was entirely built on `BatchReader` and is dead code now that the `native_iceberg_compat` path uses `NativeBatchReader`. Comet also no longer accelerates V2 Parquet scans, so `CometParquetScan` and `CometParquetPartitionReaderFactory` are dead code. ## Test plan - [x] `mvn compile -pl common -am` passes - [x] `mvn compile -pl spark -am` passes - [x] `mvn test-compile -pl spark -am` passes - [x] `mvn test -pl common` passes - [x] Grep confirms no dangling references to `CometPrefetchThreadPool`, `COMET_SCAN_PREFETCH`, `CometParquetPartitionReaderFactory`, or `CometParquetScan` 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
