[
https://issues.apache.org/jira/browse/ARROW-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Li updated ARROW-16651:
-----------------------------
Labels: good-first-issue good-second-issue kernel (was: )
> [Python] Casting Table to new schema ignores nullability of fields
> ------------------------------------------------------------------
>
> Key: ARROW-16651
> URL: https://issues.apache.org/jira/browse/ARROW-16651
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Joris Van den Bossche
> Priority: Major
> Labels: good-first-issue, good-second-issue, kernel
>
> Similar to ARROW-15478, but not for nested fields but just for casting a full
> Table (in theory that could be the same code, but currently the Table.cast
> logic is implemented in cython).
> So currently when casting a Table to a new schema, the nullability of the
> fields in the schema is ignored (and as a result you get an "invalid" schema
> indicating a field is non-nullable that actually can have nulls):
> {code}
> >>> table = pa.table({'a': [None, 1]})
> >>> table
> pyarrow.Table
> a: int64
> ----
> a: [[null,1]]
> >>> new_schema = pa.schema([pa.field("a", "int64", nullable=False)])
> >>> table.cast(new_schema)
> pyarrow.Table
> a: int64 not null
> ----
> a: [[null,1]]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
