[
https://issues.apache.org/jira/browse/NIFI-13544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul-Adrien Cordonnier updated NIFI-13544:
------------------------------------------
Summary: Add property to set the size of the threadpool of pubsub publish
(was: Add configuration to set the size of the threadpool of pubsub publish)
> Add property to set the size of the threadpool of pubsub publish
> ----------------------------------------------------------------
>
> Key: NIFI-13544
> URL: https://issues.apache.org/jira/browse/NIFI-13544
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.27.0
> Reporter: Paul-Adrien Cordonnier
> Priority: Major
>
> The GCP java-pubsub library has a default behaviour of creating a threadpool
> of 5 threads per processor on the machine. On a typical production server it
> can be 200 threads per processor, mutliplied by the number of GCP publish
> processor it can very well overwhelm a JVM.
> [Link to the gcp library
> snippet|https://github.com/googleapis/java-pubsub/blob/4e17c34880d0498d2bd891ea7caa8fe095054016/google-cloud-pubsub/src/main/java/com/google/cloud/pubsub/v1/Publisher.java#L712-L716]
> Similar to whats its done with
> [putkudu|https://github.com/apache/nifi/blob/support/nifi-1.x/nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-processors/src/main/java/org/apache/nifi/processors/kudu/AbstractKuduProcessor.java#L148].
> I think we should be able to set the size of the threadpool as a property of
> the processor. Default behaviour being the one in pubsub-library so that
> there are no impact on performance after upgrading.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
