If this was fixed, why is the JIRA still open? On Wed, Dec 7, 2016 at 10:58 PM, Chaitanya Chebolu < [email protected]> wrote:
> There was a bug in Malhar-3.4.0 and is fixed in Malhar-3.6.0. > JIRA details of this issue is here > <https://issues.apache.org/jira/browse/APEXMALHAR-2241>. > > Regards, > Chaitanya > > On Wed, Dec 7, 2016 at 11:24 PM, Sanjay Pujare <[email protected]> > wrote: > >> Just out of curiosity – what was the problem? Why were the ssl.* project >> properties not seen by the Kafka consumer? >> >> >> >> *From: *"Raja.Aravapalli" <[email protected]> >> *Reply-To: *<[email protected]> >> *Date: *Wednesday, December 7, 2016 at 7:56 AM >> >> *To: *"[email protected]" <[email protected]> >> *Subject: *Re: [EXTERNAL] Re: KafkaSinglePortInputOperator >> >> >> >> >> >> Thanks a tonnn for the support today Chaitanya!! >> >> >> >> We were able to successfully download messages from Kafka SSL Secured >> Topics!! >> >> >> >> Thanks you very much!! >> >> >> >> >> >> Regards, >> >> Raja. >> >> >> >> *From: *Chaitanya Chebolu <[email protected]> >> *Reply-To: *"[email protected]" <[email protected]> >> *Date: *Wednesday, December 7, 2016 at 11:28 AM >> *To: *"[email protected]" <[email protected]> >> *Subject: *Re: [EXTERNAL] Re: KafkaSinglePortInputOperator >> >> >> >> Raja, >> >> >> >> Issue is the SSL properties(ssl.*.*) are not reflected to Kafka >> consumer. >> >> Could you please share the complete project ? >> >> >> >> Thanks, >> >> Chaitanya >> >> >> >> >> >> On Wed, Dec 7, 2016 at 7:39 AM, Raja.Aravapalli < >> [email protected]> wrote: >> >> Hi Chaitanya, >> >> >> >> Any other thoughts on how I can fix this ?? >> >> >> >> Are Apex doesn’t yet support SSL secured topics ? >> >> >> >> >> >> Thanks a lot. >> >> >> >> Regards, >> >> Raja. >> >> >> >> *From: *"Raja.Aravapalli" <[email protected]> >> *Reply-To: *"[email protected]" <[email protected]> >> *Date: *Tuesday, December 6, 2016 at 5:32 PM >> >> >> *To: *"[email protected]" <[email protected]> >> *Subject: *Re: [EXTERNAL] Re: KafkaSinglePortInputOperator >> >> >> >> >> >> >> >> I added the below line as said… I cannot see any exceptions also!!!! >> >> >> >> Still nothing is happening L >> >> >> >> I am not sure, why these below as always showing as null… even though I >> set them in my Application.java class!! Any help on how to set these >> properties ??? >> >> >> >> ssl.keystore.location = null >> >> ssl.truststore.location = null >> >> ssl.keystore.password = null >> >> >> >> >> >> Thanks a lot in advance. >> >> >> >> Regards, >> >> Raja. >> >> >> >> *From: *Chaitanya Chebolu <[email protected]> >> *Reply-To: *"[email protected]" <[email protected]> >> *Date: *Tuesday, December 6, 2016 at 5:17 PM >> *To: *"[email protected]" <[email protected]> >> *Subject: *Re: [EXTERNAL] Re: KafkaSinglePortInputOperator >> >> >> >> Raja, >> >> >> >> Please set the consumerProps to the KafkaSinglePortInputOperator. >> >> Add the below line in your application: >> >> KafkaSinglePortInputOperator in = dag.addOperator("in", new >> KafkaSinglePortInputOperator()); >> >> ------ >> >> in.setConsumerProps(props); >> >> >> >> Please let me know, if you are still facing issues. >> >> >> >> Regards, >> >> Chaitanya >> >> >> >> >> >> >> >> On Tue, Dec 6, 2016 at 5:00 PM, Raja.Aravapalli < >> [email protected]> wrote: >> >> >> >> Find below the log I am observing: >> >> >> >> 2016-12-06 05:17:37,264 INFO kafka.AbstractKafkaInputOperator >> (AbstractKafkaInputOperator.java:initPartitioner(311)) - Initialize >> Partitioner >> >> 2016-12-06 05:17:37,265 INFO kafka.AbstractKafkaInputOperator >> (AbstractKafkaInputOperator.java:initPartitioner(324)) - Actual >> Partitioner is class org.apache.apex.malhar.kafka.OneToOnePartitioner >> >> 2016-12-06 05:17:37,280 INFO consumer.ConsumerConfig >> (AbstractConfig.java:logAll(165)) - ConsumerConfig values: >> >> metric.reporters = [] >> >> metadata.max.age.ms = 300000 >> >> value.deserializer = class org.apache.kafka.common.serial >> ization.ByteArrayDeserializer >> >> group.id = org.apache.apex.malhar.kafka.A >> bstractKafkaInputOperatorMETA_GROUP >> >> partition.assignment.strategy = >> [org.apache.kafka.clients.consumer.RangeAssignor] >> >> reconnect.backoff.ms = 50 >> >> sasl.kerberos.ticket.renew.window.factor = 0.8 >> >> max.partition.fetch.bytes = 1048576 >> >> bootstrap.servers = [10.66.137.116:9093] >> >> retry.backoff.ms = 100 >> >> sasl.kerberos.kinit.cmd = /usr/bin/kinit >> >> sasl.kerberos.service.name = null >> >> sasl.kerberos.ticket.renew.jitter = 0.05 >> >> ssl.keystore.type = JKS >> >> ssl.trustmanager.algorithm = PKIX >> >> enable.auto.commit = false >> >> ssl.key.password = null >> >> fetch.max.wait.ms = 500 >> >> sasl.kerberos.min.time.before.relogin = 60000 >> >> connections.max.idle.ms = 540000 >> >> ssl.truststore.password = null >> >> session.timeout.ms = 30000 >> >> metrics.num.samples = 2 >> >> client.id = >> >> ssl.endpoint.identification.algorithm = null >> >> key.deserializer = class org.apache.kafka.common.serial >> ization.ByteArrayDeserializer >> >> ssl.protocol = TLS >> >> check.crcs = true >> >> request.timeout.ms = 40000 >> >> ssl.provider = null >> >> ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] >> >> ssl.keystore.location = null >> >> heartbeat.interval.ms = 3000 >> >> auto.commit.interval.ms = 5000 >> >> receive.buffer.bytes = 32768 >> >> ssl.cipher.suites = null >> >> ssl.truststore.type = JKS >> >> security.protocol = PLAINTEXT >> >> ssl.truststore.location = null >> >> ssl.keystore.password = null >> >> ssl.keymanager.algorithm = SunX509 >> >> metrics.sample.window.ms = 30000 >> >> fetch.min.bytes = 1024 >> >> send.buffer.bytes = 131072 >> >> auto.offset.reset = latest >> >> >> >> 2016-12-06 05:17:37,385 INFO utils.AppInfoParser >> (AppInfoParser.java:<init>(82)) - Kafka version : 0.9.0.0 >> >> 2016-12-06 05:17:37,385 INFO utils.AppInfoParser >> (AppInfoParser.java:<init>(83)) - Kafka commitId : fc7243c2af4b2b4a >> >> >> >> >> >> >> >> Regards, >> >> Raja. >> >> >> >> *From: *Chaitanya Chebolu <[email protected]> >> *Reply-To: *"[email protected]" <[email protected]> >> *Date: *Tuesday, December 6, 2016 at 4:28 PM >> *To: *"[email protected]" <[email protected]> >> *Subject: *[EXTERNAL] Re: KafkaSinglePortInputOperator >> >> >> >> Hi Raja, >> >> >> >> Could you please share the Application Master logs and Kafka operator >> container logs. >> >> >> >> Regards, >> >> Chaitanya >> >> >> >> On Tue, Dec 6, 2016 at 4:17 PM, Raja.Aravapalli < >> [email protected]> wrote: >> >> >> >> Hi Team, >> >> >> >> I am using “KafkaSinglePortInputOperator” to connect to a SSL Secured >> topic in Kafka 0.9!! >> >> >> >> Unfortunately… my apex application is not going to “RUNNING” state…!! >> Only staying in ACCEPTED State and then going into FAILED statie!! I don’t >> see much information in the logs…!! L >> >> >> >> Can someone please help fix the issue…. We have immediate need to read >> messages from kafka 0.9 SSL configured topics… >> >> >> >> Please advise! >> >> >> >> >> >> Thanks very much in advance. >> >> >> >> >> >> Regards, >> >> Raja. >> >> >> >> >> >> >> > >
