pkhetrapal commented on issue #6979:
URL: https://github.com/apache/iceberg/issues/6979#issuecomment-1461101722
Hi @Fokko, thanks for the response.
My use case is to handle streaming data with variable fields in the struct
column.
I was able to find the missing fields from the struct column on the fly and
was able to update the table using the `ADD COLUMN`, but now, even if the
fields are nullable, I can't add the data with missing fields.
For eg:
I have the below struct col in the table
```
my_col: struct (nullable = true)
| |-- key1: string (nullable = true)
| |-- key2: string (nullable = true)
| |-- key3: string (nullable = true)
```
but when I try to add a row where `key3` is missing in the `my_col` column,
I get:
```
pyspark.sql.utils.AnalysisException: Cannot write incompatible data:
- Struct 'my_col' missing fields: 'key3'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
