[
https://issues.apache.org/jira/browse/FLINK-41228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu reassigned FLINK-41228:
-------------------------------
Assignee: Sangkyoon Nam
> Thread-mode PyFlink DataStream converters mishandle rows in arrays, lists and
> maps
> ----------------------------------------------------------------------------------
>
> Key: FLINK-41228
> URL: https://issues.apache.org/jira/browse/FLINK-41228
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Affects Versions: 2.3.0
> Reporter: Sangkyoon Nam
> Assignee: Sangkyoon Nam
> Priority: Major
>
> In PyFlink DataStream thread mode, the container cases below give wrong
> values or fail; process mode handles them correctly. A {{map}} that returns
> {{Row([Row(1), Row(2), Row(3)])}} with output type
> {{Types.ROW([Types.OBJECT_ARRAY(Types.ROW([Types.INT()]))])}} emits
> {{+I[[+I[3], +I[3], +I[3]]]}} in thread mode and {{+I[[+I[1], +I[2],
> +I[3]]]}} in process mode. Map values of rows and arrays of tuples behave the
> same way. A {{LIST}} of rows fails with {{ClassCastException: class
> pemja.core.object.PyObject cannot be cast to class [Ljava.lang.Object;}}, and
> a null array field fails with {{TypeError: 'NoneType' object is not
> iterable}}.
> There are three causes:
> * Java {{RowDataConverter}} and {{TupleDataConverter}} reuse one result
> object across calls, so a container holds several references to the same
> object, which later elements overwrite.
> * Python {{from_type_info_proto}} compares {{type_info}} instead of
> {{type_name}} with {{LIST}}, so lists fall back to {{IdentityDataConverter}}.
> * Python {{ArrayDataConverter}} calls {{tuple()}} on {{None}}.
> I reproduced this on a 2.4-SNAPSHOT build. {{PythonTypeUtils.java}} and
> {{converters.py}} are the same in 2.3.0 and on master.
> I have a fix with tests. Could someone assign this to me?
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
