Hi,

This is my first email to this list, hello everyone!

I have a question regarding the confluent version dependency that beam, or
more specifically kafka io extension, is currently using. Tl;dr is that I'd
like to update the confluent library version to something more recent, and
was wondering if I could just open a PR and maybe create an issue for that
in github.

More context:
Right now the confluent dependency version is set at 5.3.2, which is quite
old (5 years or so) - one feature that we recently discovered is missing in
this version is the support for configuring schema registry client's ssl
configuration - according to the (current) docs (
https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html)
there's a whole bunch of configuration keys starting with a "ssl." prefix
that dictate how a schema registry client might access the schema registry
- the issue is that support for those config keys on the client side had
only been added starting from version 5.4.0 IIUC (
https://github.com/confluentinc/schema-registry/pull/957). Without it, the
ssl has to be configured using standard JVM classes such as SSLContext or
HttpsURLConnection, which on top of being slightly unintuitive is also
undocumented - therefore I'd like to suggest updating the confluent libs
version to something more recent. I have already managed to update them to
a version as high as 7.6.0 (which is the most recent one, from what I
gather), and successfully run tests and build the extension - apart from
updating the confluent library version itself I had to additionally update
the avro code generation plugin as well as the avro dependency.

Let me know what you think.

Thanks,
Maciej

Reply via email to