[
https://issues.apache.org/jira/browse/FLINK-14170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kostas Kloudas closed FLINK-14170.
----------------------------------
Resolution: Fixed
Merged on master with f1c4ddf00a3c40dd451723d80e21adeb8b31fa2e
and on release-1.10 with 9c5313e1f634ade92d645dd20ea8b92552a0a3d0
> Support hadoop < 2.7 with StreamingFileSink.BulkFormatBuilder
> -------------------------------------------------------------
>
> Key: FLINK-14170
> URL: https://issues.apache.org/jira/browse/FLINK-14170
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 1.8.0, 1.8.1, 1.8.2, 1.9.0
> Reporter: Bhagavan
> Assignee: Bhagavan
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, StreamingFileSink is supported only with Hadoop >= 2.7
> irrespective of Row/bulk format builder. This restriction is due to truncate
> is not supported in Hadoop < 2.7
> However, BulkFormatBuilder does not use truncate method to restore the file.
> So the restricting StreamingFileSink.BulkFormatBuilder to be used only with
> Hadoop >= 2.7 is not necessary.
> So requested improvement is to remove the precondition on
> HadoopRecoverableWriter and allow BulkFormatBuilder (Parquet) to be used in
> Hadoop 2.6 ( Most of the enterprises still on CDH 5.x)
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
