Re: [ANNOUNCE] Please welcome Bharath Kumarasubramanian to the Samza PMC
Congrats! Bharath, well deserved! -Yi On Thu, Feb 13, 2020 at 10:17 PM Wei Song wrote: > Congrats, Bharath, well deserved !!! > > > On 2/13/20, 8:51 PM, "Jagadish Venkatraman" > wrote: > > Congrats Bharath. Great work! Looking forward to continued > contributions! > > On Thursday, February 13, 2020, Yang Zhang wrote: > > > Congratulations, Bharath! Nice work and thanks for the contributions! > > > > Best, > > Yang > > > > On Thu, Feb 13, 2020 at 4:27 PM Xinyu Liu > wrote: > > > > > Hi all, > > > > > > I'm very pleased to announce that the Samza PMC has voted Bharath > > > Kumarasubramanian to be a Project Management Committee (PMC) > Member. The > > > PMC is responsible for the overall health of a project and for > voting in > > > new committers and PMC members, as well as voting on releases. > Over the > > > past few years, Bharath has been a valuable committer on the > project. > > > > > > Congrats Bharath! > > > > > > Thanks, > > > Xinyu > > > on behalf of the Samza PMC > > > > > > > > -- > Jagadish > > >
Re: [ANNOUNCE] Please welcome Bharath Kumarasubramanian to the Samza PMC
Congrats, Bharath, well deserved !!! On 2/13/20, 8:51 PM, "Jagadish Venkatraman" wrote: Congrats Bharath. Great work! Looking forward to continued contributions! On Thursday, February 13, 2020, Yang Zhang wrote: > Congratulations, Bharath! Nice work and thanks for the contributions! > > Best, > Yang > > On Thu, Feb 13, 2020 at 4:27 PM Xinyu Liu wrote: > > > Hi all, > > > > I'm very pleased to announce that the Samza PMC has voted Bharath > > Kumarasubramanian to be a Project Management Committee (PMC) Member. The > > PMC is responsible for the overall health of a project and for voting in > > new committers and PMC members, as well as voting on releases. Over the > > past few years, Bharath has been a valuable committer on the project. > > > > Congrats Bharath! > > > > Thanks, > > Xinyu > > on behalf of the Samza PMC > > > -- Jagadish
Re: [ANNOUNCE] Please welcome Bharath Kumarasubramanian to the Samza PMC
Congrats Bharath. Great work! Looking forward to continued contributions! On Thursday, February 13, 2020, Yang Zhang wrote: > Congratulations, Bharath! Nice work and thanks for the contributions! > > Best, > Yang > > On Thu, Feb 13, 2020 at 4:27 PM Xinyu Liu wrote: > > > Hi all, > > > > I'm very pleased to announce that the Samza PMC has voted Bharath > > Kumarasubramanian to be a Project Management Committee (PMC) Member. The > > PMC is responsible for the overall health of a project and for voting in > > new committers and PMC members, as well as voting on releases. Over the > > past few years, Bharath has been a valuable committer on the project. > > > > Congrats Bharath! > > > > Thanks, > > Xinyu > > on behalf of the Samza PMC > > > -- Jagadish
Re: [ANNOUNCE] Please welcome Bharath Kumarasubramanian to the Samza PMC
Congratulations, Bharath! Nice work and thanks for the contributions! Best, Yang On Thu, Feb 13, 2020 at 4:27 PM Xinyu Liu wrote: > Hi all, > > I'm very pleased to announce that the Samza PMC has voted Bharath > Kumarasubramanian to be a Project Management Committee (PMC) Member. The > PMC is responsible for the overall health of a project and for voting in > new committers and PMC members, as well as voting on releases. Over the > past few years, Bharath has been a valuable committer on the project. > > Congrats Bharath! > > Thanks, > Xinyu > on behalf of the Samza PMC >
[ANNOUNCE] Please welcome Bharath Kumarasubramanian to the Samza PMC
Hi all, I'm very pleased to announce that the Samza PMC has voted Bharath Kumarasubramanian to be a Project Management Committee (PMC) Member. The PMC is responsible for the overall health of a project and for voting in new committers and PMC members, as well as voting on releases. Over the past few years, Bharath has been a valuable committer on the project. Congrats Bharath! Thanks, Xinyu on behalf of the Samza PMC
Re: [DISCUSS] Samza 1.3.1 release
+1 thank you Hai for shepherding this release. On Thu, Feb 13, 2020 at 3:53 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > +1. > > Cheers, > Bharath > > On Thu, Feb 13, 2020 at 12:54 PM Yang Zhang wrote: > > > +1. Thanks! > > > > Best, > > Yang > > > > On Thu, Feb 13, 2020 at 12:06 PM Xinyu Liu > wrote: > > > > > Cool! Thanks for driving this release. > > > > > > Thanks, > > > Xinyu > > > > > > On Thu, Feb 13, 2020 at 10:28 AM Hai Lu wrote: > > > > > > > Hi all, > > > > > > > > We're going to make a 1.3.1 release to address some critical issues > > that > > > > were found in 1.3.0 > > > > > > > > 1.3.1 will be based off 1.3.0 but include the following additional > > > commits: > > > > > > > > SAMZA-2447: Checkpoint dir removal should only search in valid store > > dirs > > > > (#1261) > > > > SAMZA-2446: Invoke onCheckpoint only for registered SSPs (#1260) > > > > SAMZA-2431: Fix the checkpoint and changelog topic auto-creation. > > (#1251) > > > > SAMZA-2434: Fix the coordinator steam creation workflow > > > > SAMZA-2423: Heartbeat failure causes incorrect container shutdown > > (#1240) > > > > SAMZA-2305: Stream processor should ensure previous container is > > stopped > > > > during a rebalance (#1213) > > > > > > > > I'm going to create the release candidate soon. > > > > > > > > Thanks, > > > > Hai > > > > > > > > > > -- Jagadish V, Graduate Student, Department of Computer Science, Stanford University
Re: [DISCUSS] Samza 1.3.1 release
+1. Cheers, Bharath On Thu, Feb 13, 2020 at 12:54 PM Yang Zhang wrote: > +1. Thanks! > > Best, > Yang > > On Thu, Feb 13, 2020 at 12:06 PM Xinyu Liu wrote: > > > Cool! Thanks for driving this release. > > > > Thanks, > > Xinyu > > > > On Thu, Feb 13, 2020 at 10:28 AM Hai Lu wrote: > > > > > Hi all, > > > > > > We're going to make a 1.3.1 release to address some critical issues > that > > > were found in 1.3.0 > > > > > > 1.3.1 will be based off 1.3.0 but include the following additional > > commits: > > > > > > SAMZA-2447: Checkpoint dir removal should only search in valid store > dirs > > > (#1261) > > > SAMZA-2446: Invoke onCheckpoint only for registered SSPs (#1260) > > > SAMZA-2431: Fix the checkpoint and changelog topic auto-creation. > (#1251) > > > SAMZA-2434: Fix the coordinator steam creation workflow > > > SAMZA-2423: Heartbeat failure causes incorrect container shutdown > (#1240) > > > SAMZA-2305: Stream processor should ensure previous container is > stopped > > > during a rebalance (#1213) > > > > > > I'm going to create the release candidate soon. > > > > > > Thanks, > > > Hai > > > > > >
Re: [DISCUSS] Samza 1.3.1 release
+1. Thanks! Best, Yang On Thu, Feb 13, 2020 at 12:06 PM Xinyu Liu wrote: > Cool! Thanks for driving this release. > > Thanks, > Xinyu > > On Thu, Feb 13, 2020 at 10:28 AM Hai Lu wrote: > > > Hi all, > > > > We're going to make a 1.3.1 release to address some critical issues that > > were found in 1.3.0 > > > > 1.3.1 will be based off 1.3.0 but include the following additional > commits: > > > > SAMZA-2447: Checkpoint dir removal should only search in valid store dirs > > (#1261) > > SAMZA-2446: Invoke onCheckpoint only for registered SSPs (#1260) > > SAMZA-2431: Fix the checkpoint and changelog topic auto-creation. (#1251) > > SAMZA-2434: Fix the coordinator steam creation workflow > > SAMZA-2423: Heartbeat failure causes incorrect container shutdown (#1240) > > SAMZA-2305: Stream processor should ensure previous container is stopped > > during a rebalance (#1213) > > > > I'm going to create the release candidate soon. > > > > Thanks, > > Hai > > >
Re: [DISCUSS] Samza 1.3.1 release
Cool! Thanks for driving this release. Thanks, Xinyu On Thu, Feb 13, 2020 at 10:28 AM Hai Lu wrote: > Hi all, > > We're going to make a 1.3.1 release to address some critical issues that > were found in 1.3.0 > > 1.3.1 will be based off 1.3.0 but include the following additional commits: > > SAMZA-2447: Checkpoint dir removal should only search in valid store dirs > (#1261) > SAMZA-2446: Invoke onCheckpoint only for registered SSPs (#1260) > SAMZA-2431: Fix the checkpoint and changelog topic auto-creation. (#1251) > SAMZA-2434: Fix the coordinator steam creation workflow > SAMZA-2423: Heartbeat failure causes incorrect container shutdown (#1240) > SAMZA-2305: Stream processor should ensure previous container is stopped > during a rebalance (#1213) > > I'm going to create the release candidate soon. > > Thanks, > Hai >
[VOTE] Apache Samza 1.3.1 RC0
Hi, This is a call for a vote on a release of Apache Samza 1.3.1 to redress certain issues found in 1.3.0 The release candidate can be downloaded from here: http://home.apache.org/~lhaiesp/samza-1.3.1-rc0/ The release candidate is signed with pgp key 0x256F8FA2, which can be found here: https://keyserver.ubuntu.com/pks/lookup?search=0x256F8FA2=on=index or to directly see the public key here: https://keyserver.ubuntu.com/pks/lookup?op=get=0x9ebc0889d43fae16dd0d8f5ba2f50cf4256f8fa2 The git tag is release-1.3.1-rc0 and signed with the same pgp key above: https://gitbox.apache.org/repos/asf?p=samza.git;a=commit;h=7b849c047827587dec55ac169f41aac7321ce1cb Test binaries have been published to Maven's staging repository, and are available here: https://repository.apache.org/content/repositories/orgapachesamza-1074 The vote will be open for 128 hours (ending at 8:00 PM PST Tuesday, 2/18/2020). Please download the release candidate, check the hashes/signature, build it and test it, and then please vote: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) I ran check-all.sh and integration tests (both YARN and standalone). +1 (non-binding) from my side. Thanks, Hai
Re: [DISCUSS] SEP-22: Container Placements in Samza
Thanks for writing a very detailed design doc. My feedback is below: - The title of the SEP is Container Placements in Samza, which sounds like it's going to reimplement the container placements logic currently in ClusterBasedJobCoordinator. After reading the proposal, I think it should be renamed to be a much smaller scope, e.g. Container Relocation Tool. - Goal: the goal is to build a tool, right? I want to be crystal clear on that. After the solution, I think there will be an tool to do container relocation for Samza open source. Build the ability doesn't mean much to Samza open source users I believe. - Proposed Solution: after reading this I think the rejected the option is to write to metadastore, and the accepted one is to develop a container placement service. But then I read the architecture part, the proposal says the preferred approach is to use Samza metastore API to read and write the container placement metadata. It feels contradictory to me. Seems the metadatsstore is a must-have piece in this solution, so it might be cleaner to remove the first solution. - I don't understand the "service" part of the container placement. Is it a separate service that can be hosted somewhere? Is it based on jetty or netty? what's the communication protocol? From reading the proposal, it looks like a set of API instead of service. If that's the case, please remove all the usage of "container placement service" in this proposal. - key-value format: the description of the uuid of the ContainerPlacementRequestMessage has a typo I think (response -> request). - The diagram of the components interaction look nice, and definitely helps me understand what the solution will look like. I didn't find a description about writing the new locality information. I am interested in the ordering of that regarding to the ordering of reserving resource and running the container. I feel that part will be a bit complicated. - public interfaces: are these interfaces intended to by used by samza users? How do they use them? Is there an example? Are these interfaces going into samza-api? Are those interfaces stable? Why the uuid in the interface cannot be generated by Samza itself? Why ContainerPlacementMetadatstore is part of the API? should that be hidden from the users? If the goal is to provide a tool, I would rather make these classes internal in samza-core and also describe how the tool will look like here. Thanks, Xinyu On Wed, Jan 15, 2020 at 10:34 AM Sanil Jain wrote: > Hi Prateek, > > Thanks for the feedback, I have updated the SEP with your suggestions. > Please have a look. > > Thank you > Sanil > > On Mon, 6 Jan 2020 at 10:54, Sanil Jain wrote: > > > Hi all, > > > > Refreshing this back to see if there is any further feedback on this SEP. > > > > Thanks > > Sanil > > > > On Thu, 5 Dec 2019 at 16:25, Sanil Jain wrote: > > > >> Hi all, > >> > >> I have created SEP-22: Container Placements in Samza, which adds > >> abilities to move containers for a Samza job seamlessly from one host to > >> another without restarting jobs. > >> > >> Please find the SEP wiki below: > >> > https://cwiki.apache.org/confluence/display/SAMZA/SEP-22%3A+Container+Placements+in+Samza > >> > >> Please take a look and chime in your feedback. > >> > >> Thanks > >> Sanil > >> > > >
[DISCUSS] Samza 1.3.1 release
Hi all, We're going to make a 1.3.1 release to address some critical issues that were found in 1.3.0 1.3.1 will be based off 1.3.0 but include the following additional commits: SAMZA-2447: Checkpoint dir removal should only search in valid store dirs (#1261) SAMZA-2446: Invoke onCheckpoint only for registered SSPs (#1260) SAMZA-2431: Fix the checkpoint and changelog topic auto-creation. (#1251) SAMZA-2434: Fix the coordinator steam creation workflow SAMZA-2423: Heartbeat failure causes incorrect container shutdown (#1240) SAMZA-2305: Stream processor should ensure previous container is stopped during a rebalance (#1213) I'm going to create the release candidate soon. Thanks, Hai
