Jark Wu created FLINK-18303:
-------------------------------
Summary: Filesystem connector doesn't flush part files after
rolling interval
Key: FLINK-18303
URL: https://issues.apache.org/jira/browse/FLINK-18303
Project: Flink
Issue Type: Bug
Components: Connectors / FileSystem, Table SQL / Ecosystem
Reporter: Jark Wu
Assignee: Jark Wu
Fix For: 1.11.0
I have set "execution.checkpointing.interval" to "5s" and
"sink.rolling-policy.time-interval" to "2s". However, it still take 60 seconds
to see the first part file.
This can be reproduced by the following code in SQL CLI:
{code:sql}
CREATE TABLE CsvTable (
event_timestamp STRING,
`user` STRING,
message STRING,
duplicate_count BIGINT,
constant STRING
) WITH (
'connector' = 'filesystem',
'path' = '$RESULT',
'format' = 'csv',
'sink.rolling-policy.time-interval' = '2s'
);
INSERT INTO CsvTable -- read from Kafka Avro, and write into Filesystem Csv
SELECT AvroTable.*, RegReplace('Test constant folding.', 'Test', 'Success') AS
constant
FROM AvroTable;
{code}
This is found when I migrate SQLClientKafkaITCase to use DDL (FLINK-18086).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
