morningman opened a new issue, #67367: URL: https://github.com/apache/doris/issues/67367
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version Apache Doris 4.1.3-rc02, commit `31263df4dc1d4d3a27517d264802cd4d6b92c874` Client: Python + ADBC Flight SQL driver (`adbc_driver_flightsql`), FE `arrow_flight_sql_port` = 41070. The MySQL/JDBC protocol is used as the control path for comparison. ### What's Wrong? If an ADBC Flight SQL connection is opened *before* a `VARIANT` table is created and written, the first read of a persisted empty JSON object on that connection can return an empty string instead of `{}`. Running exactly the same query a second time on the same connection returns `{}`, and a freshly opened ADBC connection also returns `{}` on its first try. The raw Arrow schema is `int32, string, string`: - pre-existing connection, first read: `["1", "", ""]` - same connection, second read: `["1", "{}", "{}"]` - new connection after the write: `["1", "{}", "{}"]` ### What You Expected? The first read and all subsequent reads should be identical. An empty JSON object should come back as `{}` for both nullable and `NOT NULL` `VARIANT` columns. ### How to Reproduce? 1. Open the ADBC Flight SQL connection first, without querying the target table yet. 2. Over a separate MySQL/JDBC connection, run the DDL and `INSERT` below. 3. On the pre-existing ADBC connection, run the same `SELECT` twice in a row. 4. Compare the two Arrow results, then open a new ADBC connection and run it once more as a control. ```sql DROP TABLE IF EXISTS adbc_variant_first_read; CREATE TABLE adbc_variant_first_read ( id INT, v_nullable VARIANT NULL, v_not_null VARIANT NOT NULL ) DUPLICATE KEY(id) DISTRIBUTED BY HASH(id) BUCKETS 1 PROPERTIES("replication_num"="1"); INSERT INTO adbc_variant_first_read VALUES (1, PARSE_JSON('{}'), PARSE_JSON('{}')); SELECT id, CAST(v_nullable AS STRING), CAST(v_not_null AS STRING) FROM adbc_variant_first_read ORDER BY id; ``` Client side: ```python import adbc_driver_flightsql.dbapi as flight_sql # Open this connection BEFORE the JDBC session runs CREATE TABLE and INSERT. conn = flight_sql.connect(uri="grpc://127.0.0.1:41070", db_kwargs={"username": "root", "password": ""}) cur = conn.cursor() sql = ("SELECT id, CAST(v_nullable AS STRING), CAST(v_not_null AS STRING) " "FROM adbc_variant_first_read ORDER BY id") cur.execute(sql) print(cur.fetch_arrow_table().to_pylist()) # first read may contain empty strings cur.execute(sql) print(cur.fetch_arrow_table().to_pylist()) # returns {} ``` ### Anything Else? This looks like stale per-connection schema/metadata state on the Flight SQL session: the connection was established before the table existed, and the first read resolves the variant subcolumns against that stale state. **Workaround:** retry the identical read, or re-establish the ADBC connection after the write completes. A silent retry was deliberately not added to the test path, because it would hide the consistency problem. Also reproducible through the `variant_p0/variant_hirachinal` regression case, in addition to the isolated table above. Tracking issue: #65615 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
