Re: [VOTE] SEP-11: Host affinity in standalone.
Hi, The vote of SEP-11 had been open for more than 72 hours and we got +1 (binding) x 3. Votes are as follows: +1 (binding) - Yi Pan, Xinyu Liu, Jagadish Venkatraman. SEP-11 had officially passed the VOTE! Thanks.
Re: [VOTE] SEP-11: Host affinity in standalone.
Hi Jagadish, Thanks for your feedback. I have added the suggestion test-case to the test plan. Thanks. On Mon, Apr 9, 2018 at 12:43 PM, Jagadish Venkatraman < jagadish1...@gmail.com> wrote: > Shanthoosh, > > As discussed earlier, I think we should add an explicit test-case for > verifying that we minimize partition movements during rolling upgrades. > Please update the proposal with this missing test-case. Other than this, +1 > (binding) from my side. > > Thanks! > > On Fri, Apr 6, 2018 at 6:30 PM, Jagadish Venkatraman < > jagadish1...@gmail.com > > wrote: > > > Let's extend the vote to Monday 11:59PM PST. > > > > On Thu, Apr 5, 2018 at 5:05 PM, xinyu liu wrote: > > > >> +1 (binding). Look forward to the implementation. > >> > >> Xinyu > >> > >> On Wed, Apr 4, 2018 at 2:43 PM, Yi Pan wrote: > >> > >> > +1 (binding). Thanks for the revisions! > >> > > >> > -Yi > >> > > >> > On Wed, Apr 4, 2018 at 2:39 PM, santhosh venkat < > >> > santhoshvenkat1...@gmail.com> wrote: > >> > > >> > > Hi, > >> > > > >> > > This is a voting thread for SEP-11: Host affinity in standalone. > >> > > > >> > > For reference, here is the wiki link: https://cwiki.apache.org > >> > > /confluence/pages/viewpage.action?pageId=75957309 > >> > > > >> > > Thanks. > >> > > > >> > > >> > > > > > > > > -- > > Jagadish V, > > Graduate Student, > > Department of Computer Science, > > Stanford University > > > > > > -- > Jagadish V, > Graduate Student, > Department of Computer Science, > Stanford University >
Re: [VOTE] SEP-11: Host affinity in standalone.
Shanthoosh, As discussed earlier, I think we should add an explicit test-case for verifying that we minimize partition movements during rolling upgrades. Please update the proposal with this missing test-case. Other than this, +1 (binding) from my side. Thanks! On Fri, Apr 6, 2018 at 6:30 PM, Jagadish Venkatraman wrote: > Let's extend the vote to Monday 11:59PM PST. > > On Thu, Apr 5, 2018 at 5:05 PM, xinyu liu wrote: > >> +1 (binding). Look forward to the implementation. >> >> Xinyu >> >> On Wed, Apr 4, 2018 at 2:43 PM, Yi Pan wrote: >> >> > +1 (binding). Thanks for the revisions! >> > >> > -Yi >> > >> > On Wed, Apr 4, 2018 at 2:39 PM, santhosh venkat < >> > santhoshvenkat1...@gmail.com> wrote: >> > >> > > Hi, >> > > >> > > This is a voting thread for SEP-11: Host affinity in standalone. >> > > >> > > For reference, here is the wiki link: https://cwiki.apache.org >> > > /confluence/pages/viewpage.action?pageId=75957309 >> > > >> > > Thanks. >> > > >> > >> > > > > -- > Jagadish V, > Graduate Student, > Department of Computer Science, > Stanford University > -- Jagadish V, Graduate Student, Department of Computer Science, Stanford University
Re: [VOTE] SEP-11: Host affinity in standalone.
Let's extend the vote to Monday 11:59PM PST. On Thu, Apr 5, 2018 at 5:05 PM, xinyu liu wrote: > +1 (binding). Look forward to the implementation. > > Xinyu > > On Wed, Apr 4, 2018 at 2:43 PM, Yi Pan wrote: > > > +1 (binding). Thanks for the revisions! > > > > -Yi > > > > On Wed, Apr 4, 2018 at 2:39 PM, santhosh venkat < > > santhoshvenkat1...@gmail.com> wrote: > > > > > Hi, > > > > > > This is a voting thread for SEP-11: Host affinity in standalone. > > > > > > For reference, here is the wiki link: https://cwiki.apache.org > > > /confluence/pages/viewpage.action?pageId=75957309 > > > > > > Thanks. > > > > > > -- Jagadish V, Graduate Student, Department of Computer Science, Stanford University
Re: [VOTE] SEP-11: Host affinity in standalone.
+1 (binding). Look forward to the implementation. Xinyu On Wed, Apr 4, 2018 at 2:43 PM, Yi Pan wrote: > +1 (binding). Thanks for the revisions! > > -Yi > > On Wed, Apr 4, 2018 at 2:39 PM, santhosh venkat < > santhoshvenkat1...@gmail.com> wrote: > > > Hi, > > > > This is a voting thread for SEP-11: Host affinity in standalone. > > > > For reference, here is the wiki link: https://cwiki.apache.org > > /confluence/pages/viewpage.action?pageId=75957309 > > > > Thanks. > > >
Re: [VOTE] SEP-11: Host affinity in standalone.
+1 (binding). Thanks for the revisions! -Yi On Wed, Apr 4, 2018 at 2:39 PM, santhosh venkat < santhoshvenkat1...@gmail.com> wrote: > Hi, > > This is a voting thread for SEP-11: Host affinity in standalone. > > For reference, here is the wiki link: https://cwiki.apache.org > /confluence/pages/viewpage.action?pageId=75957309 > > Thanks. >
[VOTE] SEP-11: Host affinity in standalone.
Hi, This is a voting thread for SEP-11: Host affinity in standalone. For reference, here is the wiki link: https://cwiki.apache.org /confluence/pages/viewpage.action?pageId=75957309 Thanks.
Re: [VOTE] SEP-11: Host affinity in standalone.
Yi, Thanks for your feedback. Replies inline. >> in YARN, are you sure that we persist the JobModel to coordinator stream? I checked the code and didn't find that. The JobModel was simply generated and served from the memory in the JobCoordinator. Yes, that’s right. Updated the SEP based upon this feedback. >> In YARN, we don't run stream processor yet. We are still running SamzaContainer as for now. I think it’s simpler to represent it logically as processor for both yarn and standalone. Choosing StreamProcessor(or SamzaContainer) for a deployment model is an implementation detail which doesn’t have to be mentioned in design doc. If you have strong opinions about this, let’s discuss offline. >> When describing the differences between YARN and standalone model, item 3 and 4 are not basic facts this design depends on. Instead, those seem to be detailed design choices already. Suggest to move to requirements in the design, not the difference between YARN and standalone model Done. Updated the SEP based upon this feedback. >> A further question on systems that provides VM and file system isolation (Locker G3): in this case, even two processors are on a single physical host, if they are in two different VMs, they can not share file system, which means that their state stores can not be shared. And in addition, if the system provides some common shared directory, then, what's the locationId in such case? LocationID represents the physical execution environment of the StreamProcessor. All the processors which run from a LocationId should be able to share(read/write)their local state stores. Any store created by a processor running from a locationId should be readable/writable by other processors running from the same locationId. If the containerized environment support shared data volume between VM's on the host, then locationId will be the physical hostname(Example: Kubernetes). If not, locationId will be a uniqueID identifying the VM amongst all the VM’s(Locker G3). For Locker G3, locationId will be the combination of sliceID and sliceInstanceId. I have updated the SEP with these details. Please take a look at this when you get a chance. Cancelling this vote for now. Thanks.
[CANCEL][VOTE] SEP-11: Host affinity in standalone.
Hi, This is an official CANCEL for the SEP-11: Host affinity in standalone vote. Thanks.
Re: [VOTE] SEP-11: Host affinity in standalone.
Hi, Santhoosh, Overall looks good to me now. There are still some minor issues / errors in the design doc - in YARN, are you sure that we persist the JobModel to coordinator stream? I checked the code and didn't find that. The JobModel was simply generated and served from the memory in the JobCoordinator. - In YARN, we don't run stream processor yet. We are still running SamzaContainer as for now. - When describing the differences between YARN and standalone model, item 3 and 4 are not basic facts this design depends on. Instead, those seem to be detailed design choices already. Suggest to move to requirements in the design, not the difference between YARN and standalone model - A further question on systems that provides VM and file system isolation (Locker G3): in this case, even two processors are on a single physical host, if they are in two different VMs, they can not share file system, which means that their state stores can not be shared. And in addition, if the system provides some common shared directory, then, what's the locationId in such case? -Yi On Tue, Feb 20, 2018 at 6:32 PM, santhosh venkat < santhoshvenkat1...@gmail.com> wrote: > Hi everyone, > > This is a voting thread for SEP-11: Host affinity in standalone. > For reference, here is the wiki link: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957309 > > Link to discussion mail thread: > http://mail-archives.apache.org/mod_mbox/samza-dev/201802.mbox/% > 3CCAFvExu2hrUhnjknWoH3er=q4agh3pzqjtbqwrp9xpdijawv...@mail.gmail.com%3E > > Thanks. >
[VOTE] SEP-11: Host affinity in standalone.
Hi everyone, This is a voting thread for SEP-11: Host affinity in standalone. For reference, here is the wiki link: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957309 Link to discussion mail thread: http://mail-archives.apache.org/mod_mbox/samza-dev/201802.mbox/%3CCAFvExu2hrUhnjknWoH3er=q4agh3pzqjtbqwrp9xpdijawv...@mail.gmail.com%3E Thanks.
