fmiguelez opened a new issue #9867:
URL: https://github.com/apache/pulsar/issues/9867
**Describe the bug**
Script `gen-yml-from-env.py` is typically used when separate function worker
instances (e.g. on a Docker cluster) are configured. This script takes
environment variables starting with `PF_` prefix and assigns their values to
`conf/functions_worker.yml`.
Some properties in YAML file take the form of an array. If we set one of
those properties with an environment variable, e.g.
`PF_authenticationProviders` as we would do to configure the broker or proxy
(in this case the variable would be `authenticationProviders` without the
prefix) the script will produce an invalid YAML file that will fail with
following error:
> Exception in thread "main"
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize
instance of `java.util.HashSet<java.lang.Object>` out of VALUE_STRING token
> at [Source: (File); line: 7, column: 26] (through reference chain:
org.apache.pulsar.functions.worker.WorkerConfig["authenticationProviders"])
> at
com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
> at
com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1468)
> at
com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1242)
> at
com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1148)
> at
com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.handleNonArray(StringCollectionDeserializer.java:274)
> at
com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.deserialize(StringCollectionDeserializer.java:183)
> at
com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.deserialize(StringCollectionDeserializer.java:173)
> at
com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.deserialize(StringCollectionDeserializer.java:21)
> at
com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
> at
com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:293)
> at
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:156)
> at
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4482)
> at
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3299)
> at
org.apache.pulsar.functions.worker.WorkerConfig.load(WorkerConfig.java:491)
> at
org.apache.pulsar.functions.worker.FunctionWorkerStarter.main(FunctionWorkerStarter.java:62)
We can define indicated variable `PF_authenticationProviders` taking value
`org.apache.pulsar.broker.authentication.AuthenticationProviderTls` or
`org.apache.pulsar.broker.authentication.AuthenticationProviderTls,org.apache.pulsar.broker.authentication.AuthenticationProviderAthenz`.
**To Reproduce**
We can execute following docker command to reproduce the issue:
```shell
docker run -e
PF_authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
apachepulsar/pulsar:2.7.0 bash -c "bin/gen-yml-from-env.py
conf/functions_worker.yml; bin/pulsar functions-worker;"
```
**Expected behavior**
Produced configuration file must be valid for function workers.
**Screenshots**
**Desktop (please complete the following information):**
- OS: Tested on Windows with Docker Desktop
**Additional context**
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
