morningman opened a new issue, #67372: URL: https://github.com/apache/doris/issues/67372
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version Apache Doris 4.1.3-rc02, commit `31263df4dc1d4d3a27517d264802cd4d6b92c874` Client: Python + ADBC Flight SQL driver (`adbc_driver_flightsql`), FE `arrow_flight_sql_port` = 41070. The MySQL/JDBC protocol is used as the control path for comparison. ### What's Wrong? Doris `MAP` semantics allow a `NULL` key, but the Arrow `MAP` specification requires the key field to be non-nullable. When a result contains a `NULL` map key, the Flight SQL `DoGet` fails and the whole result becomes unreadable: ``` Can not write null value of map key to arrow ``` Projecting the same value as `map_entries(m)`, i.e. `ARRAY<STRUCT<key, value>>`, is returned over ADBC correctly and preserves the `NULL` key. ### What You Expected? Flight SQL should be able to return `MAP` data that Doris itself accepts, either through a compatible Arrow representation that preserves the `NULL` key, or with a clear, documented limitation; it should not fail the entire result at `DoGet` time. ### How to Reproduce? 1. Create a Doris table with a `MAP<STRING,INT>` column. 2. Insert `map(NULL,100)`. 3. Query the raw `MAP` column over Python ADBC; `DoGet` fails. 4. Query `map_entries(m)`; the same data is returned through the compatible structure. ```sql DROP TABLE IF EXISTS adbc_null_map; CREATE TABLE adbc_null_map ( k INT, m MAP<STRING,INT> ) DUPLICATE KEY(k) DISTRIBUTED BY HASH(k) BUCKETS 1 PROPERTIES("replication_num"="1"); INSERT INTO adbc_null_map VALUES (1, map(NULL,100)); -- Fails through Arrow Flight SQL. SELECT m FROM adbc_null_map; -- Succeeds and preserves the NULL key. SELECT map_entries(m) FROM adbc_null_map; ``` Client side: ```python import adbc_driver_flightsql.dbapi as flight_sql conn = flight_sql.connect(uri="grpc://127.0.0.1:41070", db_kwargs={"username": "root", "password": ""}) cur = conn.cursor() cur.execute("SELECT m FROM adbc_null_map") cur.fetch_arrow_table() ``` ### Anything Else? This is a genuine model mismatch between Doris `MAP` and Arrow `MAP`, so it needs an explicit decision rather than only a test skip: either serialize such maps as `list<struct<key, value>>` (which round-trips the `NULL` key, as `map_entries` already shows), or reject them with a clear, documented error at plan/schema time instead of failing mid-stream in `DoGet`. Either way the limitation belongs in the Arrow Flight SQL documentation. Related: #65182 skipped the Arrow-incompatible Map null-key regression cases; this issue tracks the underlying behavior. **Workaround:** use `map_entries(m)` in SQL to project the `MAP` as `ARRAY<STRUCT<key, value>>`, or filter/replace `NULL` keys when the business logic allows it. Tracking issue: #65615 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
