wangzhigang1999 opened a new pull request, #11169:
URL: https://github.com/apache/paimon/pull/11169

   ### Purpose
   
   Closes #11168.
   
   For a column-list projection such as `['value', 'rid']`, the reader can 
already return the requested columns in order. The extra output projection then 
recomputes the adapter and schema and wraps the same arrays in another 
`RecordBatch` for each batch.
   
   Skip this step in `TableRead.__init__` only when:
   
   - The projection is unnamed and its output names and source paths exactly 
match the top-level fields in `read_type`, in the same order.
   - The reader adapter returns those same fields and paths, without flattening 
nested fields or changing their types.
   
   Reuse the computed adapter when projection is still needed. This also covers 
directly constructed `TableRead` instances. Keep the builder's projection so 
`explain()` continues to show the user's request.
   
   ### Tests
   
   - Targeted tests passed with Python and Native execution, covering direct 
construction, nested fields, aliases, duplicate columns, empty projections, 
filters on columns omitted from the output, authorization and `explain()`.
   - Ruff and ty reported no new diagnostics in the changed files compared with 
the baseline.
   
   Compared with `6d50c1ab` on a telemetry table with 1,151,842 rows and TPC-H 
SF10 tables. Each query ran three times per version after a warmup, alternating 
between versions. The table shows median elapsed time; negative change means 
faster.
   
   | Query | Before (s) | After (s) | Change |
   | --- | ---: | ---: | ---: |
   | Telemetry: all columns | 5.577 | 5.604 | +0.5% |
   | Telemetry: one column | 0.345 | 0.311 | -9.9% |
   | Telemetry: reordered columns | 0.477 | 0.446 | -6.4% |
   | Telemetry: full MAP column | 5.654 | 5.553 | -1.8% |
   | Telemetry: filter with no matches | 0.252 | 0.251 | -0.4% |
   | Telemetry: filter on an omitted column | 0.248 | 0.247 | -0.1% |
   | Telemetry: batch iteration | 0.478 | 0.447 | -6.6% |
   | SF10: orders narrow projection | 0.636 | 0.521 | -18.2% |
   | SF10: Q6 scan and filter | 7.425 | 6.771 | -8.8% |
   | SF10: customer segment filter | 0.348 | 0.330 | -5.2% |
   | SF10: part 10% range filter | 0.231 | 0.228 | -1.7% |
   
   For every run, returned rows and schema matched the expected results 
computed from the source data.
   
   These measurements used the Python reader with Python 3.11 and Arrow 19, 
local append-only Parquet tables, separate processes and warm OS page cache. 
Timing includes reading and collecting results, plus Arrow IPC writes for 
TPC-H. Q6 measures its scan and filter, not the full SQL query. Native, Ray and 
OSS performance were not measured.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to