MisterRaindrop opened a new issue, #2019: URL: https://github.com/apache/cloudberry/issues/2019
Part of #2008. Letters (A, B0–B7, C, D, E) are the PRs listed there; this is **D**. ### Scope - Fragments from the engine's plan (files, row-group ranges) assigned to segments; one large file may be split by row group. - A CustomScan (planner hook, design decision D10) that opens a reader per fragment and decodes batches into slots. - Projection by field id from the table's Iceberg schema (`ProjectionSet`), so files written before a column was added or renamed read correctly; only needed columns are read. - Row-group pruning from the query's quals against Parquet statistics; `EXPLAIN ANALYZE` reports row groups skipped. - With it: #1989 name mapping for files without field ids. ### Out of scope Delete files (E), time travel (#1683 §2.3), ANALYZE beyond the current zero-sample no-op. ### Depends on A, B2, B4. Skeleton, projection and pruning are testable against the stub and local files first. ### Acceptance - Same rows as Spark on the same table, including one Spark evolved (column added, renamed, dropped, int promoted to long). - A file split into three fragments reads the same as whole; every segment reads only its fragments. - `EXPLAIN ANALYZE` shows pruning for a selective predicate and none for `WHERE true`. - Unsupported or dictionary-encoded columns are refused naming the column; cancel mid-scan releases the reader. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
