Re: [DISCUSS] Support Scala 2.12

Hi Maksim, Thanks for the PR comments and updates. Looks good to me too. - Prateek On Wed, Mar 29, 2017 at 12:17 PM, Prateek Maheshwari < pmaheshw...@linkedin.com> wrote: > Hi Maksim, > > I'm in favor of adding Scala 2.12 support as well, thanks for the PR. > I have a few questions about the

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

Yi, why add 'local' to the method name? Isn't the method called only by the StreamProcessor to get its own ID? Seems like both 1 & 2 belong in the method documentation. - Prateek On Thu, Mar 30, 2017 at 1:43 PM, Yi Pan wrote: > Talked w/ Navina offline and agreed upon: >

Re: Steps to Upgrading Samza (0.9 to 0.12)

FYI, I created https://issues.apache.org/jira/browse/SAMZA-1177 to track this. We will ensure that we have explicit upgrade documentation for upcoming releases. On Thu, Mar 30, 2017 at 2:21 PM, Jagadish Venkatraman < jagadish1...@gmail.com> wrote: > Hi Thomas and Maxim, > > Thank you for

Re: Steps to Upgrading Samza (0.9 to 0.12)

Hi Thomas and Maxim, Thank you for bringing up this concern and pointing out the gap in the documentation. We will ensure that this is documented in a separate upgrade page going forward. Best, Jagadish On Thu, Mar 30, 2017 at 12:02 PM, Navina Ramesh < nram...@linkedin.com.invalid> wrote: > Hi

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

Thanks for the answers Navina! +1 (non-binding) 2017-03-30 22:32 GMT+02:00 Navina Ramesh : > Hi Renato, > > > Having the big proposals documented on SEPs is really great to have a > good understanding on the system! > I agree. Our previous design process was not

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

Talked w/ Navina offline and agreed upon: 1) JobCoordinator.getLocalProcessorId() to be clear that we are getting the local processorId 2) Document the use case that there might be multiple StreamProcessors in the same JVM and ProcessorIdGenerator should implement a counter in this case. So, +1

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

Hi Renato, > Having the big proposals documented on SEPs is really great to have a good understanding on the system! I agree. Our previous design process was not being strictly enforced. We hope to enforce it going forward as there are major changes coming into the next release. > So this means

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

Hi Navina, Thanks for the great proposal! Having the big proposals documented on SEPs is really great to have a good understanding on the system! I have only a clarification question, the proposal states that every containerId is the same as the processorId. So this means that inside a container

[GitHub] samza pull request #105: SAMZA-1176: Make TestJoinOperator unit tests safe f...

GitHub user prateekm opened a pull request: https://github.com/apache/samza/pull/105 SAMZA-1176: Make TestJoinOperator unit tests safe for concurrent execution There are occasional failures like the following: ```joinRetainsMatchedMessagesReverse FAILED

Re: Steps to Upgrading Samza (0.9 to 0.12)

Hi everyone, Apologize for re-chiming in late on this issue. > I'm not sure I agree with the policy (removing migration code and wanting people to upgrade seem at odds to me), but minimally I think we should not assume people are upgrading to each new Samza version. I agree that we should not

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

Hi Yi, Good question. Three reasons: 1. In SAMZA-881, we came up with a set of responsibilities for the JobCoordinator. One of them was to generate/assign processorId. So, it makes sense to keep getProcessorId() within JobCoordinator interface. 2. StreamProcessor was initially introduced as a

Re: Steps to Upgrading Samza (0.9 to 0.12)

Is there some sort of document on how to upgrade Samza through various versions like the page here for Kafka: https://kafka.apache.org/documentation/#upgrade ? Having something like this would be ideal. On Thu, Mar 30, 2017 at 1:51 PM Thomas Becker wrote: > Thanks for the

Re: Steps to Upgrading Samza (0.9 to 0.12)

Thanks for the reply Yi, and I apologize if I came off a bit snarky. I'm not sure I agree with the policy (removing migration code and wanting people to upgrade seem at odds to me), but minimally I think we should not assume people are upgrading to each new Samza version. We have done so when

Re: Steps to Upgrading Samza (0.9 to 0.12)

Hi, Thomas, Sorry to hear that you were hit by the removal of migration in Samza 0.11. The reason we removed it is following a deprecate-removal policy in two versions. We are not aware that people still using 0.9 after we released 0.11 and were not expecting a direct upgrade from 0.9 to 0.12.

Re: [VOTE] SEP-1: Semantics of ProcessorId in Samza

@Navina, Sorry to chime in late. One question: 1. Why is it in JobCoordinator, and why not in StreamProcessor class? Because JobCoordinator provides coordination service across many processors, an interface getProcessorId() in JobCoordinator is confusing regarding to which processorId we are

Re: Steps to Upgrading Samza (0.9 to 0.12)

Yes, we were burned by this. The changelog mapping will be regenerated instead of migrated and the result will completely hose the job (because the mapping was not generated deterministically in previous versions of Samza). I don't understand why the migration code was removed but it was, and to