Joris Van den Bossche created ARROW-16651: ---------------------------------------------
Summary: [Python] Casting Table to new schema ignores nullability of fields Key: ARROW-16651 URL: https://issues.apache.org/jira/browse/ARROW-16651 Project: Apache Arrow Issue Type: Bug Components: Python Reporter: Joris Van den Bossche Similar to ARROW-15478, but not for nested fields but just for casting a full Table (in theory that could be the same code, but currently the Table.cast logic is implemented in cython). So currently when casting a Table to a new schema, the nullability of the fields in the schema is ignored (and as a result you get an "invalid" schema indicating a field is non-nullable that actually can have nulls): {code} >>> table = pa.table({'a': [None, 1]}) >>> table pyarrow.Table a: int64 ---- a: [[null,1]] >>> new_schema = pa.schema([pa.field("a", "int64", nullable=False)]) >>> table.cast(new_schema) pyarrow.Table a: int64 not null ---- a: [[null,1]] {code} -- This message was sent by Atlassian Jira (v8.20.7#820007)