[
https://issues.apache.org/jira/browse/FLINK-41245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu resolved FLINK-41245.
-----------------------------
Fix Version/s: 2.3.1
Resolution: Fixed
Merged to release-2.3 via d6142775c7217d829444648bbcbf0702e45426c9
> PyFlink mypy check fails on release-2.3: create_arrow_schema has no
> allow_nested parameter
> ------------------------------------------------------------------------------------------
>
> Key: FLINK-41245
> URL: https://issues.apache.org/jira/browse/FLINK-41245
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Affects Versions: 2.3.1
> Reporter: Martijn Visser
> Assignee: Dian Fu
> Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 2.3.1
>
>
> Since the FLINK-41032 backport 79907771be7, the python leg on release-2.3
> fails mypy. It passes allow_nested to create_arrow_schema, which only exists
> from FLINK-40472 (2.4.0) on:
> https://github.com/apache/flink/actions/runs/40486299256/job/122134071075
> https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=80592&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602
> {code}
> pyflink/table/table.py:952: error: Unexpected keyword argument "allow_nested"
> for "create_arrow_schema" [call-arg]
> Found 1 error in 1 file (checked 73 source files)
> {code}
> Before #29634 the FLINK-34625 compile break hid it.
> h3. Timestamp precision in empty pandas results
> After restoring the missing {{allow_nested}} support, the same backport also
> exposes a timestamp precision issue on release-2.3.
> {{_from_java_data_type()}} hardcodes {{TIMESTAMP}} precision to 3 and uses
> the default precision of 6 for {{TIMESTAMP_LTZ}}. As a result, empty
> {{Table.to_pandas()}} results use milliseconds instead of seconds for
> {{TIMESTAMP(0)}}, and microseconds instead of milliseconds for
> {{TIMESTAMP_LTZ(3)}}. This causes {{test_empty_to_pandas_with_timestamps}}
> and {{test_to_pandas_with_fixed_offset_timezones}} to fail in both batch and
> streaming modes.
> The fix preserves {{logical_type.getPrecision()}} for both timestamp types
> while retaining their nullability. These two changes match the implementation
> already introduced on master by
> [FLINK-40187|https://github.com/apache/flink/commit/0910fc925dbe28965f1e15aff8aba5faa651630a],
> which is missing from release-2.3.
> Regression coverage includes Python-Java-Python round trips for {{TIMESTAMP}}
> and {{TIMESTAMP_LTZ}}, precisions 0-9, and both nullable and non-nullable
> types (40 combinations). Local validation passed all 10
> {{DataTypeConvertTests}} and all 22 pandas conversion tests across batch and
> streaming modes, using a 2.3-SNAPSHOT runtime and 2.3.0 test helper JARs.
> PR: https://github.com/apache/flink/pull/29641
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
