Re: Samza consumer without Zookeeper
Thanks Prateek and Yi. We were able to find the reference in the application. After removing it, we are able to produce and consume without zookeeper. On 2/10/20, 3:46 PM, "Prateek Maheshwari" wrote: Robert, from the stacktrace this looks like something in the application logic is looking for that key (i.e. in the HelixApp constructor). Thanks, Prateek On Mon, Feb 10, 2020 at 1:14 PM Robert Wigginton wrote: > Yi, > > I think I am getting ahead of myself. We are not getting past the job > coordinator portion of the config. Running 1.3 and defining > job.coordinator.factory=org.apache.samza.standalone.PassthroughJobCoordinatorFactory > in the properties file we still get the error below. > > > Exception in thread "main" org.apache.samza.config.ConfigException: > Missing key job.coordinator.zk.connect. > at org.apache.samza.config.Config.getList(Config.java:168) > at com.helixeducation.porter.HelixApp.(HelixApp.java:29) > at > com.helixeducation.porter.HelixRocksDBApp.(HelixRocksDBApp.java:16) > at > com.helixeducation.porter.inquirySubmission.InquirySubmissionApp.(InquirySubmissionApp.java:25) > at > com.helixeducation.porter.inquirySubmission.InquirySubmissionTaskRunner.main(InquirySubmissionTaskRunner.java:19) > > On 2/5/20, 6:32 PM, "Yi Pan" wrote: > > Hi, Robert, > > Thanks! I believe since Samza 1.2, the ZK dependency from Kafka > consumer is > removed from Samza. I briefly checked the code base from master and > there > is no reference to KafkaConsumerConfig.getZkConnect(). Do you see any > error > messages when you remove the systems.x.consumer.zookeeper.connect from > your > config? > > -Yi > > On Wed, Feb 5, 2020 at 3:12 PM Robert Wigginton > wrote: > > > Currently 1.0.0 but are in the process of upgrading to 1.3.0. > > > > On Wed, 2020-02-05 at 15:01 -0800, Yi Pan wrote: > > > Hi, Robert, > > > > > > Which version of Samza are you using? > > > > > > -Yi > > > > > > On Tue, Feb 4, 2020 at 9:17 AM Robert Wigginton > > > wrote: > > > > > > > Hello, > > > > > > > > We are currently evaluating moving from an on prem Kafka > deployment > > > > to > > > > Confluent Cloud. Confluent Cloud does not expose Zookeeper to > > > > user. > > > > Is it possible to setup a Samza consumer without the zookeeper > > > > connect > > > > option? > > > > > > > > Thanks, > > > > > > > > Robert > > > > > > > > >
Re: Samza consumer without Zookeeper
Robert, from the stacktrace this looks like something in the application logic is looking for that key (i.e. in the HelixApp constructor). Thanks, Prateek On Mon, Feb 10, 2020 at 1:14 PM Robert Wigginton wrote: > Yi, > > I think I am getting ahead of myself. We are not getting past the job > coordinator portion of the config. Running 1.3 and defining > job.coordinator.factory=org.apache.samza.standalone.PassthroughJobCoordinatorFactory > in the properties file we still get the error below. > > > Exception in thread "main" org.apache.samza.config.ConfigException: > Missing key job.coordinator.zk.connect. > at org.apache.samza.config.Config.getList(Config.java:168) > at com.helixeducation.porter.HelixApp.(HelixApp.java:29) > at > com.helixeducation.porter.HelixRocksDBApp.(HelixRocksDBApp.java:16) > at > com.helixeducation.porter.inquirySubmission.InquirySubmissionApp.(InquirySubmissionApp.java:25) > at > com.helixeducation.porter.inquirySubmission.InquirySubmissionTaskRunner.main(InquirySubmissionTaskRunner.java:19) > > On 2/5/20, 6:32 PM, "Yi Pan" wrote: > > Hi, Robert, > > Thanks! I believe since Samza 1.2, the ZK dependency from Kafka > consumer is > removed from Samza. I briefly checked the code base from master and > there > is no reference to KafkaConsumerConfig.getZkConnect(). Do you see any > error > messages when you remove the systems.x.consumer.zookeeper.connect from > your > config? > > -Yi > > On Wed, Feb 5, 2020 at 3:12 PM Robert Wigginton > wrote: > > > Currently 1.0.0 but are in the process of upgrading to 1.3.0. > > > > On Wed, 2020-02-05 at 15:01 -0800, Yi Pan wrote: > > > Hi, Robert, > > > > > > Which version of Samza are you using? > > > > > > -Yi > > > > > > On Tue, Feb 4, 2020 at 9:17 AM Robert Wigginton > > > wrote: > > > > > > > Hello, > > > > > > > > We are currently evaluating moving from an on prem Kafka > deployment > > > > to > > > > Confluent Cloud. Confluent Cloud does not expose Zookeeper to > > > > user. > > > > Is it possible to setup a Samza consumer without the zookeeper > > > > connect > > > > option? > > > > > > > > Thanks, > > > > > > > > Robert > > > > > > > > >
Re: Samza consumer without Zookeeper
Yi, I think I am getting ahead of myself. We are not getting past the job coordinator portion of the config. Running 1.3 and defining job.coordinator.factory=org.apache.samza.standalone.PassthroughJobCoordinatorFactory in the properties file we still get the error below. Exception in thread "main" org.apache.samza.config.ConfigException: Missing key job.coordinator.zk.connect. at org.apache.samza.config.Config.getList(Config.java:168) at com.helixeducation.porter.HelixApp.(HelixApp.java:29) at com.helixeducation.porter.HelixRocksDBApp.(HelixRocksDBApp.java:16) at com.helixeducation.porter.inquirySubmission.InquirySubmissionApp.(InquirySubmissionApp.java:25) at com.helixeducation.porter.inquirySubmission.InquirySubmissionTaskRunner.main(InquirySubmissionTaskRunner.java:19) On 2/5/20, 6:32 PM, "Yi Pan" wrote: Hi, Robert, Thanks! I believe since Samza 1.2, the ZK dependency from Kafka consumer is removed from Samza. I briefly checked the code base from master and there is no reference to KafkaConsumerConfig.getZkConnect(). Do you see any error messages when you remove the systems.x.consumer.zookeeper.connect from your config? -Yi On Wed, Feb 5, 2020 at 3:12 PM Robert Wigginton wrote: > Currently 1.0.0 but are in the process of upgrading to 1.3.0. > > On Wed, 2020-02-05 at 15:01 -0800, Yi Pan wrote: > > Hi, Robert, > > > > Which version of Samza are you using? > > > > -Yi > > > > On Tue, Feb 4, 2020 at 9:17 AM Robert Wigginton > > wrote: > > > > > Hello, > > > > > > We are currently evaluating moving from an on prem Kafka deployment > > > to > > > Confluent Cloud. Confluent Cloud does not expose Zookeeper to > > > user. > > > Is it possible to setup a Samza consumer without the zookeeper > > > connect > > > option? > > > > > > Thanks, > > > > > > Robert > > > >
Re: Samza consumer without Zookeeper
Hi, Robert, Thanks! I believe since Samza 1.2, the ZK dependency from Kafka consumer is removed from Samza. I briefly checked the code base from master and there is no reference to KafkaConsumerConfig.getZkConnect(). Do you see any error messages when you remove the systems.x.consumer.zookeeper.connect from your config? -Yi On Wed, Feb 5, 2020 at 3:12 PM Robert Wigginton wrote: > Currently 1.0.0 but are in the process of upgrading to 1.3.0. > > On Wed, 2020-02-05 at 15:01 -0800, Yi Pan wrote: > > Hi, Robert, > > > > Which version of Samza are you using? > > > > -Yi > > > > On Tue, Feb 4, 2020 at 9:17 AM Robert Wigginton > > wrote: > > > > > Hello, > > > > > > We are currently evaluating moving from an on prem Kafka deployment > > > to > > > Confluent Cloud. Confluent Cloud does not expose Zookeeper to > > > user. > > > Is it possible to setup a Samza consumer without the zookeeper > > > connect > > > option? > > > > > > Thanks, > > > > > > Robert > > > >
Re: Samza consumer without Zookeeper
Currently 1.0.0 but are in the process of upgrading to 1.3.0. On Wed, 2020-02-05 at 15:01 -0800, Yi Pan wrote: > Hi, Robert, > > Which version of Samza are you using? > > -Yi > > On Tue, Feb 4, 2020 at 9:17 AM Robert Wigginton > wrote: > > > Hello, > > > > We are currently evaluating moving from an on prem Kafka deployment > > to > > Confluent Cloud. Confluent Cloud does not expose Zookeeper to > > user. > > Is it possible to setup a Samza consumer without the zookeeper > > connect > > option? > > > > Thanks, > > > > Robert > >
Re: Samza consumer without Zookeeper
Hi, Robert, Which version of Samza are you using? -Yi On Tue, Feb 4, 2020 at 9:17 AM Robert Wigginton wrote: > Hello, > > We are currently evaluating moving from an on prem Kafka deployment to > Confluent Cloud. Confluent Cloud does not expose Zookeeper to user. > Is it possible to setup a Samza consumer without the zookeeper connect > option? > > Thanks, > > Robert >
Samza consumer without Zookeeper
Hello, We are currently evaluating moving from an on prem Kafka deployment to Confluent Cloud. Confluent Cloud does not expose Zookeeper to user. Is it possible to setup a Samza consumer without the zookeeper connect option? Thanks, Robert
