2162174202-sudo opened a new issue, #19051:
URL: https://github.com/apache/iceberg/issues/19051

   ### Apache Iceberg version
   
   1.11.0
   
   ### Query engine
   
   Flink
   
   ### Please describe the bug 🐞
   
   In Flink batch mode, planning (`EXPLAIN PLAN FOR`) a query that embeds a 
non-correlated scalar subquery containing a `LEFT JOIN` (with a derived table) 
and a `RIGHT JOIN` inside an outer Iceberg table scan consistently times out 
(>60s) on a 4-row Iceberg table.
   
   This reproduces on both **Flink 2.1.0 + Iceberg 1.11.0** (newest supported 
pairing) and **Flink 1.19.1 + Iceberg 1.8.1**. By contrast, the exact same SQL 
shape and 4-row dataset over Flink's built-in `filesystem` CSV connector plans 
in under 1 second on Flink 2.3.0, indicating a planning interaction specific to 
the Iceberg table source rather than a generic Flink SQL-shape issue.
   
   ### Environment
   
   - **Primary reproduction**:
     - Image: `flink:2.1.0-scala_2.12-java17`
     - Iceberg runtime: `iceberg-flink-runtime-2.1-1.11.0.jar`
     - FileIO: `org.apache.iceberg.aws.s3.S3FileIO` 
(`iceberg-aws-bundle-1.11.0.jar`)
     - Catalog: Iceberg REST catalog backed by MinIO/S3
     - Execution mode: `SET 'execution.runtime-mode' = 'batch';`
   - **Earlier reproduction**:
     - Flink `1.19.1` + Iceberg `1.8.1`
   
   ### Steps to reproduce
   
   1. Configure Flink SQL Gateway in batch mode with an Iceberg REST catalog 
and a 4-row Iceberg table (`order_id INT, region STRING, v INT`):
   
   ```sql
   SET 'execution.runtime-mode' = 'batch';
   SET 'table.dynamic-table-options.enabled' = 'true';
   
   CREATE CATALOG iceberg WITH (
     'type' = 'iceberg',
     'catalog-type' = 'rest',
     'uri' = 'http://iceberg-rest:8181',
     'warehouse' = 's3://warehouse/wh/',
     'io-impl' = 'org.apache.iceberg.aws.s3.S3FileIO',
     's3.endpoint' = 'http://minio:9000',
     's3.path-style-access' = 'true',
     's3.access-key-id' = 'admin',
     's3.secret-access-key' = 'password'
   );
   USE CATALOG iceberg;
   
   -- Table schema and 4-row fixture used in our tests:
   -- CREATE TABLE ld_pd_target_20261006a (order_id INT, region STRING, v INT);
   -- INSERT INTO ld_pd_target_20261006a VALUES (1, 'west', 11), (2, 'west', 
12), (5, 'east', 15), (6, 'east', 16);
   ```
   
   2. Run `EXPLAIN PLAN FOR` (in our reproduction, the 4-row snapshot was 
pinned with `/*+ OPTIONS('snapshot-id'='2218517376986569101') */`; when 
reproducing on a freshly created table, you can omit the `snapshot-id` hint):
   
   ```sql
   EXPLAIN PLAN FOR
   SELECT
     ((SELECT t4.order_id
       FROM ld_pd_target_20261006a /*+ 
OPTIONS('snapshot-id'='2218517376986569101') */ AS t2
       LEFT JOIN (
         SELECT x.region AS q2_col_1, x.v AS q2_col_2, x.v AS q2_col_3
         FROM ld_pd_target_20261006a /*+ 
OPTIONS('snapshot-id'='2218517376986569101') */ AS x
       ) AS sq3 ON t2.region <> sq3.q2_col_1
       RIGHT JOIN ld_pd_target_20261006a /*+ 
OPTIONS('snapshot-id'='2218517376986569101') */ AS t4
         ON sq3.q2_col_1 NOT LIKE '%sample_19%'
       ORDER BY t4.order_id DESC
       LIMIT 1) < t1.v AND t1.order_id BETWEEN 29 AND 65),
     t1.order_id,
     t1.v
   FROM ld_pd_target_20261006a /*+ OPTIONS('snapshot-id'='2218517376986569101') 
*/ AS t1;
   ```
   
   ### Observed behavior
   
   **Flink 2.1.0 + Iceberg 1.11.0 (60s operation timeout, `EXPLAIN PLAN FOR` 
only):**
   
   | Case | Status | Elapsed |
   |---|---|---:|
   | `EXPLAIN PLAN FOR SELECT 1` | ok | 11,254 ms |
   | Control Iceberg scan (`SELECT order_id, region, v FROM 
ld_pd_target_20261006a LIMIT 1`) | ok | 6,430 ms |
   | Reproducer attempt 1 | `TimeoutError` (>60s) | 65,099 ms |
   | Reproducer attempt 2 | `TimeoutError` (>60s) | 66,683 ms |
   | Reproducer attempt 3 | `TimeoutError` (>60s) | 67,511 ms |
   
   **Flink 1.19.1 + Iceberg 1.8.1 (20s operation timeout, 3 independent healthy 
sessions):**
   - Reproducer timed out 3/3 times (`25,184 ms`, `25,127 ms`, `25,073 ms` 
including timeout cleanup), while health checks before each attempt succeeded 
in `2,105 ms`, `1,913 ms`, and `1,469 ms`.
   
   ### Isolation & controls
   
   - **Flink 2.3.0 filesystem CSV control (same 4 rows & identical SQL shape, 
`EXPLAIN PLAN FOR`)**: succeeded 3/3 in `939 ms`, `759 ms`, and `489 ms` 
(identical 5,126-character plan).
   - **Standalone scalar subquery** (without outer Iceberg scan): planned in 
`1,653 ms` and executed in `2,575 ms` on Flink 1.19.1.
   - **Equivalent pure `VALUES` query**: executed in `1,855 ms`.
   - **Disabling source reuse or sub-plan reuse**: did not remove the timeout.
   
   ### Expected behavior
   
   `EXPLAIN PLAN FOR` on a 4-row Iceberg table should complete planning 
promptly (comparable to the control scan and filesystem connector) rather than 
hanging until timeout.
   
   
   ### Willingness to contribute
   
   - [ ] I can contribute a fix for this bug independently
   - [x] I would be willing to contribute a fix for this bug with guidance from 
the Iceberg community
   - [ ] I cannot contribute a fix for this bug at this time


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

  • [I] Flink: EXPLAIN / quer... 2162174202-sudo (via GitHub) via issues

Reply via email to