sink with BucketingSink to S3 files override

2018-02-19 Thread galantaa
Hey, I have some kind of a concurrency problem with Bucketing sink when I write to S3. I use the AvroKeyValueSinkWriter. The problem is that when I send events the suppose to be written to the same directory, but to a different part file (because of different event types), the files override each

Too many open files on Bucketing sink

2018-03-13 Thread galantaa
Hey all, I'm using bucketing sink with a bucketer that creates partition per customer per day. I sink the files to s3. it suppose to work on around 500 files at the same time (according to my partitioning). I have a critical problem of 'Too many open files'. I've upload two taskmanagers, each

Re: Tentative release date for 1.6.3

2018-11-26 Thread galantaa
Hi vino, We want to upgrade flink version to 1.6 I searched but couldn't find, is there a release date to v1.7? Thanks! Alon -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/