Hi Jeremiah, If you're using YARN, it requires ZK for itself. You'll also need ZK if you're using Standalone with the ZKJobCoordinator. Other than that the framework does not rely on ZK for anything.
Hope that helps. - Prateek On Tue, Mar 12, 2019 at 7:23 AM Jeremiah Adams <jad...@helixeducation.com> wrote: > Is zookeeper required for the tech stack? If samza does all of its > checkpointing on kafka, I may be able to save us some money by eliminating > the ZK cluster. > > > > > > > * Jeremiah Adams* > > Software Engineer > > www.helixeducation.com > > Blog <http://www.helixeducation.com/blog/> | Twitter > <https://twitter.com/HelixEducation> | Facebook > <https://www.facebook.com/HelixEducation> | LinkedIn > <http://www.linkedin.com/company/3609946> > > > > > > *From: *Prateek Maheshwari <prateek...@gmail.com> > *Date: *Monday, March 11, 2019 at 6:31 PM > *To: *"dev@samza.apache.org" <dev@samza.apache.org>, Jeremiah Adams < > jad...@helixeducation.com> > *Subject: *Re: Backing Kafka/Yarn/Zookeeper version for Samza 1.0.0? > > > > Hi Jeremiah, > > > > We're in the process of upgrading Samza to use Kafka clients version 2.0, > and YARN client version 2.9. This should be available in the next release > (version 1.2). In the mean time, Kafka 0.11 and YARN 2.6 / 2.7 are the > recommended versions. > > > > Can you clarify what you mean about hard requirements for Zookeeper since > checkpointing? > > > > Thanks, > Prateek > > > > On Fri, Mar 8, 2019 at 11:27 AM Jeremiah Adams <jad...@helixeducation.com> > wrote: > > I am in the process of updating our stack. I’m not seeing documentation on > what versions of Kafka, Zookeeper and Yarn Samza 1.0.0 should be run on. > Best I can tell is a 2.6.1 dependency on Hadoop jars and 0.11.0.2 kafka > jars. If these jars are meant to match versions of the software, they are > dated. Hadoop is up to version 3.2.0 Kafka is up to version 2.1.1 > > Any advice concerning versions of the components? Also we are planning on > moving all Samza jobs to standalone and eliminating the Yarn dependencies. > I prefer taking iterative steps so want to get the updates done before > moving to containers. > > Also, does Samza have any hard requirements for Zookeeper since > checkpointing was implemented? > > > Jeremiah Adams > Software Engineer > www.helixeducation.com > <https://url.emailprotection.link/?basKr9vk92a8vVw0XMnK5bnWsxM_w6KChRx8CY_UgrU5RmcwzgGL3Po63B7rJIXeNyMBLKYpptY6Rl-f5kb6p2A~~> > <http://www.helixeducation.com/ > <https://url.emailprotection.link/?basKr9vk92a8vVw0XMnK5bmaSKuBc0AuEZ7YasYc7Df8YVt3SYmcjmLWdKMWzAAINWlUUA33ebGI7pSoTl9cg1g~~> > > > Blog<http://www.helixeducation.com/blog/ > <https://url.emailprotection.link/?basKr9vk92a8vVw0XMnK5bmaSKuBc0AuEZ7YasYc7Df-lAcqG1fqHPpNw-wd9z7HtUJeCG5_8UjCf2mHtn6C_zQ~~>> > | Twitter<https://twitter.com/HelixEducation > <https://url.emailprotection.link/?bVO2q0UXR235wN_yOnM0FjqITPdBYMD3reLGNddq-zPV5ChMQK9JwV4Be-QnrbRoXpJl8IcknAqKzYtA3RABKww~~>> > | Facebook<https://www.facebook.com/HelixEducation > <https://url.emailprotection.link/?bUU7m4NfMS_EWGtH1yojBHX9sWZ6uxVdT1eQUkmU5vWY01WFZiS2KJ-c9iLIncdHB7Uw1lRYCprEEpPPQCdiK6Q~~>> > | LinkedIn<http://www.linkedin.com/company/3609946 > <https://url.emailprotection.link/?b0ZQfJ1pZYnASyoShs9MJI46-r1lxPhA-JS5VSkR7so-DFP0_HxbOo2LsajGOaoYXxb1ZCOMAu7hZscPCnIKWpXz0cpgQ386SnNHjPcwsu4z90mzBkuwoZc6YxOCzMGA0> > > > >
