evanhlavaty opened a new issue #4787: Sink ignores inputSpecs in configuration 
yaml on create
URL: https://github.com/apache/pulsar/issues/4787
 
 
   **Describe the bug**
   When creating a sink with an accompanying configuration yaml, I want to 
increase the receiverQueueSize so I have the following set in my configuration 
yaml
   
   ```
   inputSpecs:
     sample_topic_name:
       receiverQueueSize: 1000000
   ```
   
   however, my inputSpecs configuration settings are ignored after creation of 
the sink and receiverQueueSize is set to the default of 1000
   
   **To Reproduce**
   - **Create sink with pulsar admin cli**
   
[solr_kerberos_io-2.3.2.yml](https://github.com/apache/pulsar/files/3423473/sink_config.txt)
   ```
   pulsar-admin --admin-url https://sample-admin:6751 sink create \
   --sink-config-file /data/connectors/solr_kerberos_io-2.3.2.yml
   ```
   
   `"Created Successfully"`
   
   - **Confirm sink settings**
   
   ```
   pulsar-admin --admin-url https://sample-admin:6751 sink get \
   --tenant public \
   --namespace default \
   --name solr_kerberos
   
   {
     "tenant": "public",
     "namespace": "default",
     "name": "solr_kerberos",
     "className": "org.apache.pulsar.io.solrcloud.SolrCloudSink",
     "inputSpecs": {
       "sample_topic_name": {
         "isRegexPattern": false
       }
     },
     "configs": {
       "solrUrl": "http://sample-solr:8983/solr";,
       "loginConfigurationLocation": 
"/data/connectors/solr_kerberos_io_jaas.conf",
       "collectionName": "sample_collection",
       "documentQueueSize": "5000.0",
       "concurrentThreadCount": "25.0"
     },
     "parallelism": 3,
     "processingGuarantees": "ATLEAST_ONCE",
     "retainOrdering": false,
     "autoAck": true,
     "archive": "builtin://solr_kerberos"
   }
   ```
   
   
   **Expected behavior**
   I expect the receiverQueueSize to be set to what I define it to be in the 
configuration yaml and not be overridden and/or ignored.
   
   **Desktop (please complete the following information):**
   NAME="Oracle Linux Server"
   VERSION="7.6"
   ID="ol"
   VARIANT="Server"
   VARIANT_ID="server"
   VERSION_ID="7.6"
   PRETTY_NAME="Oracle Linux Server 7.6"
   
   
   **Additional context**
   Pulsar Version 2.3.2. Sink is running on a cluster of function 
workers/brokers/proxies. When creating a function using the same inputSpecs 
yaml entry the receiverQueueSize is set correctly. This issue only seems to be 
related to sinks
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


With regards,
Apache Git Services

Reply via email to