Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

Thanks for the reply Jacob. Please see my comment inline. On Mon, Jun 12, 2017 at 7:51 PM, Jacob Maes wrote: > > > > - For users that need partition expansion of the input streams for > stateful > > job, they have a really big headache in the sense that Samza does not >

Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

> > - For users that need partition expansion of the input streams for stateful > job, they have a really big headache in the sense that Samza does not allow > partition expansion for stateful job. SEP-5 addresses this headache for > them. > You are right that SEP-5 requires user to understand and

Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

Thanks Xinyu offering a solution. Yeah, we have actually listed it as the third rejected alternative in SEP-5 . I can move this to future work. I think it is actually a great idea to support more

Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

Hey Jacob, Thanks for the explanation. It seems that your biggest concern is with the generality of the proposal. Let me try to address this and other comments below. 1) ... it will cause headaches for Samza users ... I am not sure I understand why this proposal causes headache for Samza users.

Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

How about making the partition mapping function a pluggable component in the partition expansion? Mathematically, this is a mapping function which is able to map the new partitions to the old ones: *f (new partition) -> old partition* If the function is a surjective function (

[GitHub] samza pull request #223: SAMZA-<1324> :

GitHub user PawasChhokra opened a pull request: https://github.com/apache/samza/pull/223 SAMZA-<1324> : Added a metrics class for ZK based job coordinator that reports a few metrics. You can merge this pull request into a Git repository by running: $ git pull

Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

Hey Dong, I'm opposed (or a +0, at best) to this limited, Kafka-specific solution. I understand that the proposal is relatively simple to implement, but I think it will cause headaches for Samza users. They will not only have to understand all the limitations (increase only, double partitions

[GitHub] samza pull request #220: SAMZA-1330: Stand alone feature preview, known limi...

Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/220 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] samza pull request #217: SAMZA-1327: create zk namespace if does not exist

Github user sborya closed the pull request at: https://github.com/apache/samza/pull/217 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] samza pull request #218: SAMZA-1327: fail if namespace specified in the conn...

Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/218 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is