[
https://issues.apache.org/jira/browse/FLINK-24728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-24728:
-----------------------------------
Labels: pull-request-available (was: )
> Batch SQL file sink forgets to close the output stream
> ------------------------------------------------------
>
> Key: FLINK-24728
> URL: https://issues.apache.org/jira/browse/FLINK-24728
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Affects Versions: 1.11.5, 1.12.6, 1.15.0, 1.14.1, 1.13.4
> Reporter: Caizhi Weng
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.15.0, 1.14.1, 1.13.4
>
>
> I tried to write a large avro file into HDFS and discover that the displayed
> file size in HDFS is extremely small, but copying that file to local yields
> the correct size. If we create another Flink job and read that avro file from
> HDFS, the job will finish without outputting any record because the file size
> Flink gets from HDFS is the very small file size.
> This is because the output format created in
> {{FileSystemTableSink#createBulkWriterOutputFormat}} only finishes the
> {{BulkWriter}}. According to the java doc of {{BulkWriter#finish}} bulk
> writers should not close the output stream and should leave them to the
> framework.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
