michaeljmarshall opened a new pull request, #20863: URL: https://github.com/apache/pulsar/pull/20863
Fixes #20862 ### Motivation See the issue for detailed motivation. Briefly, the current solution for configuring sinks and sources requires each sink and source to implement calls to materialize secrets using the `SecretsProvider`. Instead of relying on each connector to implement this feature, this PR introduces an option to pass the configuration into the sink and source `open` method call, so that the secrets are available without needing them to be passed in as plain text. One major benefit of this solution is that it integrates with the existing functionality to pass in secrets. In order to be backwards compatible, I followed the approach used by https://github.com/apache/pulsar/pull/20116 to add a configuration to the function worker that will apply to all sinks/sources created by the function worker. ### Modifications * Add the option to merge a connector's `secrets` map into the config map that is subsequently passed to the sink or source when calling `open`. * Add a new configuration to the function worker (`mergeSecretsIntoConfigMap`) and an associated new parameter to the `JavaInstanceStarter` (`--merge_secrets_into_config_map`) to enable this feature. The default value keeps the current behavior. ### Verifying this change This change added tests. ### Does this pull request potentially affect one of the following parts: Adds a new configuration option. It is backwards compatible and defaults to use the old behavior. ### Documentation - [x] `doc-required` I'll need a PR to add docs for this feature. ### Matching PR in forked repository PR in forked repository: https://github.com/michaeljmarshall/pulsar/pull/53 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
