This issue indicates that your certificates don't have a path between the worker and the schema registry. You need to make sure your certs know that certification path, though I don't actually know how to ensure that.
On Mon, Oct 31, 2022 at 6:13 PM Ahmet Altay <[email protected]> wrote: > (moving this to the user list, dev list to bcc.) > > Adding relevant people: @John Casey <[email protected]>. > > (Keshav, for Dataflow issues you could also reach out to Dataflow support: > https://cloud.google.com/dataflow/docs/support) > > On Mon, Oct 31, 2022 at 1:23 PM Chennakeshavlu Maddela < > [email protected]> wrote: > >> Hi Team, >> >> >> >> We are setting up avro write to a kafka topic with confluent schema >> registry on SSL, its throwing (below) error. >> >> >> >> We are using SASL_SSL with PEM certificate for connecting Kafka broker, >> which is working fine with non-avro kafka topics. Can you please help us >> with configuring SSL for schema registry? (we are using dataflow runner) >> >> >> >> Please let me know if you need more details. >> >> >> >> Thank you, >> >> Keshav >> >> >> >> *Exception:* >> >> Failed to send HTTP request to endpoint: >> https://confluent-schemaregistry-xxxx.com/subjects/topic-value?deleted=false >> >> >> >> javax.net.ssl.SSLHandshakeException: PKIX path building failed: >> sun.security.provider.certpath.SunCertPathBuilderException: unable to find >> valid certification path to requested target >> >> at >> java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) >> >> at >> java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:350) >> >> at >> java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:293) >> >> at >> java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:288) >> >> at >> java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1356) >> >> at >> java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1231) >> >> at >> java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1174) >> >> at >> java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392) >> >> at >> java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444) >> >> at >> java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:422) >> >> at >> java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:183) >> >> at >> java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:171) >> >> at >> java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1408) >> >> at >> java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1314) >> >> at >> java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440) >> >> at >> java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:411) >> >> at >> java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567) >> >> at >> java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) >> >> at >> java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1367) >> >> at >> java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1342) >> >> at >> java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:246) >> >> at >> io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:199) >> >> at >> io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:256) >> >> at >> io.confluent.kafka.schemaregistry.client.rest.RestService.lookUpSubjectVersion(RestService.java:323) >> >> at >> io.confluent.kafka.schemaregistry.client.rest.RestService.lookUpSubjectVersion(RestService.java:311) >> >> at >> io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getIdFromRegistry(CachedSchemaRegistryClient.java:191) >> >> at >> io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getId(CachedSchemaRegistryClient.java:323) >> >> at >> io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:73) >> >> at >> io.confluent.kafka.serializers.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:53) >> >> at >> org.apache.kafka.common.serialization.Serializer.serialize(Serializer.java:62) >> >> at >> org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:952) >> >> at >> org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:912) >> >> at >> com.davita.cwow.pmt.transformations.PmtAvroKafkaWriter$KafkaWriteEvaluationFn.processElement(PmtAvroKafkaWriter.java:86) >> >> at >> com.davita.cwow.pmt.transformations.PmtAvroKafkaWriter$KafkaWriteEvaluationFn$DoFnInvoker.invokeProcessElement(Unknown >> Source) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:285) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:275) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:85) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:423) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:411) >> >> at >> com.davita.cwow.pmt.transformations.PmtAvroKafkaWriter$EvaluationAvroToProducerRecordFn.processElement(PmtAvroKafkaWriter.java:67) >> >> at >> com.davita.cwow.pmt.transformations.PmtAvroKafkaWriter$EvaluationAvroToProducerRecordFn$DoFnInvoker.invokeProcessElement(Unknown >> Source) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:285) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:275) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:85) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:423) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:411) >> >> at >> com.davita.cwow.pmt.transformations.BatchEvaluationTransform$2.process(BatchEvaluationTransform.java:100) >> >> at >> com.davita.cwow.pmt.transformations.BatchEvaluationTransform$2$DoFnInvoker.invokeProcessElement(Unknown >> Source) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:285) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:275) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:85) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:423) >> >> at >> com.davita.cwow.pmt.transformations.PatientProtocolEvaluationResultMutationTransform$ToEntityFn.processElement(PatientProtocolEvaluationResultMutationTransform.java:194) >> >> at >> com.davita.cwow.pmt.transformations.PatientProtocolEvaluationResultMutationTransform$ToEntityFn$DoFnInvoker.invokeProcessElement(Unknown >> Source) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:185) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:285) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:275) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:85) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:423) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:411) >> >> at >> java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) >> >> at >> java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) >> >> at >> com.davita.cwow.pmt.transformations.ProtocolEvaluatorTransform$5.process(ProtocolEvaluatorTransform.java:248) >> >> at >> com.davita.cwow.pmt.transformations.ProtocolEvaluatorTransform$5$DoFnInvoker.invokeProcessElement(Unknown >> Source) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:285) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:275) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:85) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:423) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:411) >> >> at >> org.apache.beam.sdk.transforms.join.CoGroupByKey$ConstructCoGbkResultFn.processElement(CoGroupByKey.java:192) >> >> at >> org.apache.beam.sdk.transforms.join.CoGroupByKey$ConstructCoGbkResultFn$DoFnInvoker.invokeProcessElement(Unknown >> Source) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) >> >> at >> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) >> >> at >> org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) >> >> at >> org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185) >> >> at >> org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108) >> >> at >> org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128) >> >> at >> org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56) >> >> at >> org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) >> >> at >> org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) >> >> at >> org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169) >> >> at >> org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83) >> >> at >> org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) >> >> at >> org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) >> >> at >> org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) >> >> at >> org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140) >> >> at >> org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120) >> >> at >> org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107) >> >> at >> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) >> >> at >> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) >> >> at >> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) >> >> at java.base/java.lang.Thread.run(Thread.java:834) >> >> Caused by: sun.security.validator.ValidatorException: PKIX path building >> failed: sun.security.provider.certpath.SunCertPathBuilderException: unable >> to find valid certification path to requested target >> >> at >> java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439) >> >> at >> java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) >> >> at >> java.base/sun.security.validator.Validator.validate(Validator.java:264) >> >> at >> java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313) >> >> at >> java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:222) >> >> at >> java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129) >> >> at >> java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1340) >> >> ... 117 more >> >> Caused by: sun.security.provider.certpath.SunCertPathBuilderException: >> unable to find valid certification path to requested target >> >> at >> java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) >> >> at >> java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) >> >> at >> java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297) >> >> at >> java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434) >> >> ... 123 more >> >> CONFIDENTIALITY NOTICE: THIS MESSAGE IS CONFIDENTIAL, INTENDED FOR THE >> NAMED RECIPIENT(S) AND MAY CONTAIN INFORMATION THAT IS (I) PROPRIETARY TO >> THE SENDER, AND/OR, (II) PRIVILEGED, CONFIDENTIAL, AND/OR OTHERWISE EXEMPT >> FROM DISCLOSURE UNDER APPLICABLE STATE AND FEDERAL LAW, INCLUDING, BUT NOT >> LIMITED TO, PRIVACY STANDARDS IMPOSED PURSUANT TO THE FEDERAL HEALTH >> INSURANCE PORTABILITY AND ACCOUNTABILITY ACT OF 1996 ("HIPAA"). IF YOU ARE >> NOT THE INTENDED RECIPIENT, OR THE EMPLOYEE OR AGENT RESPONSIBLE FOR >> DELIVERING THE MESSAGE TO THE INTENDED RECIPIENT, YOU ARE HEREBY NOTIFIED >> THAT ANY DISSEMINATION, DISTRIBUTION OR COPYING OF THIS COMMUNICATION IS >> STRICTLY PROHIBITED. IF YOU HAVE RECEIVED THIS TRANSMISSION IN ERROR, >> PLEASE (I) NOTIFY US IMMEDIATELY BY REPLY E-MAIL OR BY TELEPHONE AT >> (855.472.9822 <(855)%20472-9822>), (II) REMOVE IT FROM YOUR SYSTEM, AND >> (III) DESTROY THE ORIGINAL TRANSMISSION AND ITS ATTACHMENTS WITHOUT READING >> OR SAVING THEM. THANK YOU. >> >> -DaVita Inc- >> >
