Hello! Hoping someone can point me in the right direction. I am trying to use PEM certificates for authenticating to Kafka topic using Python and Java Expansion Service. However I am running into an error:
Caused by: java.security.NoSuchAlgorithmException: PEM KeyStore not > available I am using FlinkRunner and starting expansion service myself with the expansion service jar taken from Maven. I suspect that the issue is older version of Java-s kafka-client that has been packaged into expansion service jar (and job server?). From Apache Beam docs page on Java SDK dependencies, it seems that default libraries that get used during building jars are set in https://github.com/apache/beam/blob/867d70c1689bc4b16141043616f912ce0b10b74c/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L626 and kafka-client version there is quite old (2.4.1). PEM support is supposed to be there after 2.7. With that I wonder what would be a good way to specify Java dependencies for multi language pipeline, eg using KafkaIO with newer kafka-client from Python? PS: I tried updating the kafka-client version in BeamModulePlugin.groovy and creating expansion-service jar and packaging into my python beam image, however it seems that the error still remained. It might be skill issue, I am not very familiar with java build tooling (maven, gradle) and might have missed something. Beam version I am using is 2.61.0. Thank you, Kaur Jaanson
