This sounds great! Feel free to create an issue to track this work! On Mon, Mar 11, 2024 at 9:36 AM Maciej Szwaja via dev <dev@beam.apache.org> wrote:
> Hi, > > This is my first email to this list, hello everyone! > > I have a question regarding the confluent version dependency that beam, or > more specifically kafka io extension, is currently using. Tl;dr is that I'd > like to update the confluent library version to something more recent, and > was wondering if I could just open a PR and maybe create an issue for that > in github. > > More context: > Right now the confluent dependency version is set at 5.3.2, which is quite > old (5 years or so) - one feature that we recently discovered is missing in > this version is the support for configuring schema registry client's ssl > configuration - according to the (current) docs ( > https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html) > there's a whole bunch of configuration keys starting with a "ssl." prefix > that dictate how a schema registry client might access the schema registry > - the issue is that support for those config keys on the client side had > only been added starting from version 5.4.0 IIUC ( > https://github.com/confluentinc/schema-registry/pull/957). Without it, > the ssl has to be configured using standard JVM classes such as SSLContext > or HttpsURLConnection, which on top of being slightly unintuitive is also > undocumented - therefore I'd like to suggest updating the confluent libs > version to something more recent. I have already managed to update them to > a version as high as 7.6.0 (which is the most recent one, from what I > gather), and successfully run tests and build the extension - apart from > updating the confluent library version itself I had to additionally update > the avro code generation plugin as well as the avro dependency. > > Let me know what you think. > > Thanks, > Maciej >
