[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-08-30 Thread joshfg
Github user joshfg commented on the issue: https://github.com/apache/flink/pull/2269 That's great, thanks Aljoscha! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-08-30 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/2269 I merged it, could you please close this PR if it's not closed automatically. Thanks for your work, and sorry again for taking so long to get it in! 👍 --- If your project is set up

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-08-26 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/2269 Hi, I'm very for the delays! I still have this sitting at the top of my list and I'm hoping to get this in by beginning of next week. --- If your project is set up for it, you can reply to

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-08-26 Thread joshfg
Github user joshfg commented on the issue: https://github.com/apache/flink/pull/2269 Hi Aljoscha, just wanted to remind you about this - any idea when the changes will be merged in? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-07-25 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/2269 Thanks for moving the tests! It should be ok to leave these other two ITCases. I'll merge the commits into one once I can find some time to look at the tests and merge it. --- If your

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-07-25 Thread joshfg
Github user joshfg commented on the issue: https://github.com/apache/flink/pull/2269 Ok I've migrated `BucketingSinkITCase` and `BucketingSinkMultipleActiveBucketsCase` over to `BucketingSinkTest` using the test harness with `TimeServiceProvider`. I've left the two fault tolerance IT

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-07-25 Thread joshfg
Github user joshfg commented on the issue: https://github.com/apache/flink/pull/2269 That works, thanks! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-07-25 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/2269 This is caused by `setInputType()` not being called on `SequenceFileWriter`. In the test, you can call `setInputType` on the `BucketingSink` once with the input `TypeInformation` and a `new

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-07-25 Thread joshfg
Github user joshfg commented on the issue: https://github.com/apache/flink/pull/2269 Ah I see, that makes sense. I've began refactoring the tests here:

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-07-25 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/2269 Not really, the reason for having an ITCase is just that they really exercise the sink embedded in a proper Flink job, which might bring up interactions that where overlooked when writing a test

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-07-25 Thread joshfg
Github user joshfg commented on the issue: https://github.com/apache/flink/pull/2269 Thanks! Oh nice, this looks like a better solution for checking for bucket inactivity... For the tests, is there any reason not to fold all of those tests into the new `BucketingSinkTest`?

[GitHub] flink issue #2269: [FLINK-4190] Generalise RollingSink to work with arbitrar...

2016-07-21 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/2269 Very good work! I know we discussed before whether to check for inactivity in a different thread or in `invoke()`. There's actually a third option that I'm showcasing in the PR I did