2162174202-sudo commented on issue #14995:
URL: https://github.com/apache/iceberg/issues/14995#issuecomment-7055935287

   Confirming `@joyhaldar` and `@alexandrefimov`'s findings with a 3-engine 
cross-check on the **same frozen Apache Iceberg 1.11.0 table snapshot** (REST 
Catalog, format-version 2, generated via our multi-engine Lakehouse 
differential fuzzer):
   
   Query pattern (unescaped `_`, snake_case prefix):
   
   ```sql
   SELECT id, city
   FROM iceberg.default.t_main
   WHERE city LIKE 'prefix_16_bytes_alpha%';
   ```
   
   ### Physical Plan Comparison (Same Frozen Iceberg Snapshot)
   
   1. **Apache Spark 3.5.5 + Iceberg 1.11.0** — no prefix filter pushed to 
`BatchScan` (only `IS NOT NULL`), falling back to a full scan:
   
   ```text
   (1) BatchScan iceberg.default.t_main
   iceberg.default.t_main (branch=null) [filters=city IS NOT NULL, groupedBy=]
   ...
   Condition : (isnotnull(city) AND city LIKE prefix_16_bytes_alpha%)
   ```
   
   2. **Trino 470** — extracts the literal prefix `'prefix'` before the first 
`_` into a conservative range filter pushed to the Iceberg scan, while keeping 
`$like(...)` as a residual check:
   
   ```text
   ScanFilter[
     table = iceberg:default.t_main$data@...,
     filterPredicate = ((city >= varchar 'prefix') AND (city < varchar 
'prefiy') AND $like(...))
   ]
   ```
   
   3. **DuckDB 2.0.0 (`iceberg` extension `4c0c0874c4`)** — pushes `Filters:` 
directly into `ICEBERG_SCAN`.
   
   This confirms the gap is specifically in Spark Catalyst's 
`LikeSimplification` / DSv2 pushdown path: even for **unescaped `_`** (not just 
`\_` / `ESCAPE`), Spark does not derive a conservative auxiliary 
`StartsWith(col, <literal_prefix_before_first_wildcard>)` for DSv2 
`pushPredicates(...)` while keeping the original `LIKE` as a residual filter. 
As `@alexandrefimov` suggested, filing a focused Spark Jira proposal 
highlighting the DSv2 / Iceberg file-pruning impact (rather than replacing 
`Like` in-place) seems like the cleanest path forward.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to