Re: Check-pointing error

2018-12-02 Thread Congxian Qiu
Hi,Felipe Quirce Could you reproduce this in stand alone mode? or could you share your code? Best Congxian Felipe Quirce 于2018年11月30日周五 下午9:34写道: > Hi Chesnay, > > I tried with the version 1.7.0 and I had the same error. > > 2018-11-30 13:13:00,718 INFO

Re: Stream in loop and not getting to sink (Parquet writer )

2018-12-02 Thread Kostas Kloudas
Hi Avi, The ParquetAvroWriters cannot be used with the BucketingSink. In fact the StreamingFIleSink is the "evolution" of the BucketingSink and it supports all the functionality that the BucketingSink supports. Given this, why not using the StreamingFileSink? On Sat, Dec 1, 2018 at 7:56 AM Avi

Re: Stream in loop and not getting to sink (Parquet writer )

2018-12-02 Thread Avi Levi
Thanks Kostas. I will definitely look into that. but is the StreamingFileSink also support setting the batch size by size and/or by time interval like bucketing sink ? On Sun, Dec 2, 2018 at 5:09 PM Kostas Kloudas wrote: > Hi Avi, > > The ParquetAvroWriters cannot be used with the