>>> WriteMode, long)' does not work properly.
>>>>>> To be specific, data were not flushed by update frequency when write
>>>>>> to HDFS.
>>>>>>
>>>>>> what make it more disturbing is that, if I check the content with
>>>>>&g
;>>>> to HDFS.
>>>>>
>>>>> what make it more disturbing is that, if I check the content with
>>>>> 'hdfs dfs -cat xxx', sometimes I got partial records.
>>>>>
>>>>>
>>>>> I did a little digging in flink-0.9.1.
>&g
nk.streaming.api.functions.sink.FileSinkFunction.invoke(IN)'
>>>> does
>>>> is pushing data to
>>>> 'org.apache.flink.runtime.fs.hdfs.HadoopDataOutputStream'
>>>> which is a delegate of 'org.apache.hadoop.fs.FSDataOutputStream'.
>>>>
>>>> In this scenario, 'org.apache.hadoop.fs.FSDataOutputStream' is never
>>>> flushed.
>>>> Which result in data being held in local buffer, and 'hdfs dfs -cat
>>>> xxx' might return partial records.
>>>>
>>>>
>>>> Does 'DataStream.writeAsCsv' suppose to work like this? Or I messed up
>>>> somewhere?
>>>>
>>>>
>>>> Best regards and thanks for your time!
>>>>
>>>> Rex
>>>>
>>>
>>>
>>
>
fs -cat xxx'
> might return partial records.
>
>
> Does 'DataStream.writeAsCsv' suppose to work like this? Or I messed up
> somewhere?
>
>
> Best regards and thanks for your time!
>
> Rex
>
to 'org.apache.flink.runtime.fs.hdfs.HadoopDataOutputStream'
which is a delegate of 'org.apache.hadoop.fs.FSDataOutputStream'.
In this scenario, 'org.apache.hadoop.fs.FSDataOutputStream' is never
flushed.
Which result in data being held in local buffer, and 'hdfs dfs -cat xxx'
might return partial records.
Does 'DataStream.writeAsCsv' suppose to work
hich is a delegate of 'org.apache.hadoop.fs.FSDataOutputStream'.
>>
>> In this scenario, 'org.apache.hadoop.fs.FSDataOutputStream' is never
>> flushed.
>> Which result in data being held in local buffer, and 'hdfs dfs -cat xxx'
>> might return partial records.
>>
>>
>> Does 'DataStream.writeAsCsv' suppose to work like this? Or I messed up
>> somewhere?
>>
>>
>> Best regards and thanks for your time!
>>
>> Rex
>>
>
>
