dbnl-renaud opened a new issue, #2684:
URL: https://github.com/apache/iceberg-python/issues/2684
### Question
I am not sure if this is a bug or things working as expected, but when
writing to a table with an optional map field, if the input data is missing
that field entirely, the write will fail. This is because even though the map
is optional, the map key is required. This does not happen with other types.
Here's a script to reproduce.
```
from pyiceberg.catalog import load_catalog
import pyarrow as pa
warehouse_path = "/tmp/warehouse"
catalog = load_catalog(
"default",
**{
'type': 'sql',
"uri": f"sqlite:///{warehouse_path}/pyiceberg_catalog.db",
"warehouse": f"file://{warehouse_path}",
},
)
catalog.create_namespace_if_not_exists("test")
schema = pa.schema({
"id": pa.int64(),
"text": pa.string(),
"map": pa.map_(pa.string(), pa.string())
})
table = catalog.create_table_if_not_exists("test.table", schema)
table.append(pa.Table.from_pylist([{"id": 1}]))
```
The solution I found is to cast the input data to the table schema when
writing, but it's not always practical.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
