[
https://issues.apache.org/jira/browse/FLINK-39724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Feng updated FLINK-39724:
--------------------------------
Summary: CsvReaderFormat produces a stream which fails get_type() and
assign_timestamps_and_watermarks() (was: CsvFormatReader produces a stream
which fails get_type() and assign_timestamps_and_watermarks())
> CsvReaderFormat produces a stream which fails get_type() and
> assign_timestamps_and_watermarks()
> -----------------------------------------------------------------------------------------------
>
> Key: FLINK-39724
> URL: https://issues.apache.org/jira/browse/FLINK-39724
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Affects Versions: 2.2.0
> Reporter: Andrew Feng
> Priority: Major
> Attachments: CsvFormatReader.tar.gz
>
>
> See good.py and bad.py attached.
> The only difference is good.py puts the stream through the identity map. This
> is highly counter-intuitive and seems to be undocumented. In v2.2
> documentation:
> {code:python}
> schema = CsvSchema.builder() \
> .add_number_column('id', number_type=DataTypes.BIGINT()) \
> .add_array_column('array', separator='#', element_type=DataTypes.INT()) \
> .set_column_separator(',') \
> .build()
> source = FileSource.for_record_stream_format(
> CsvReaderFormat.for_schema(schema), CSV_FILE_PATH).build()
> # the type of record will be Types.ROW_NAMED(['id', 'array'], [Types.LONG(),
> Types.LIST(Types.INT())])
> ds = env.from_source(source, WatermarkStrategy.no_watermarks(), 'csv-source')
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
