Bhagavan created FLINK-14170:
--------------------------------

             Summary: Support hadoop < 2.7 with 
StreamingFileSink.BulkFormatBuilder
                 Key: FLINK-14170
                 URL: https://issues.apache.org/jira/browse/FLINK-14170
             Project: Flink
          Issue Type: Improvement
          Components: API / DataSet
    Affects Versions: 1.9.0, 1.8.2, 1.8.1, 1.8.0
            Reporter: Bhagavan


Currently, StreamingFileSink is supported only with Hadoop >= 2.7 irrespective 
of Row/bulk format builder. This restriction is due to truncate is not 
supported in  Hadoop < 2.7

However, BulkFormatBuilder does not use truncate method to restore the file. So 
the restricting StreamingFileSink.BulkFormatBuilder to be used only with Hadoop 
>= 2.7 is not necessary.

So requested improvement is to remove the precondition on 
HadoopRecoverableWriter and allow  BulkFormatBuilder (Parquet) to be used in 
Hadoop 2.6 ( Most of the enterprises still on CDH 5.x)

 

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to