[
https://issues.apache.org/jira/browse/SPARK-61178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-61178:
-----------------------------------
Labels: pull-request-available (was: )
> Make StructType.fields the list of the struct, and the name, dataType and
> nullable of StructField assignable, so that their edits show in the struct,
> in the Python client as PySpark does
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61178
> URL: https://issues.apache.org/jira/browse/SPARK-61178
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> PySpark's StructType holds its fields in a list, fields, which it gives back
> each time: edits of it, such as st.fields.append(f), st.fields[0] = f,
> st.fields.pop() and st.fields.sort(...), change the struct, and a list kept
> before st.add(...) sees the new field. The name, dataType and nullable of a
> StructField are plain attributes, so f.nullable = False changes the field and
> the struct it is in (a common way to change the nullability of a schema).
> The Python client's StructType.fields gives a new list on each access, so
> these edits are lost, and StructField's name, dataType and nullable cannot be
> set:
> AttributeError: attribute 'nullable' of 'pyspark.sql.types.StructField'
> objects is not writable
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
