[
https://issues.apache.org/jira/browse/NIFI-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18020634#comment-18020634
]
Pierre Villard commented on NIFI-14977:
---------------------------------------
Thanks for reporting the issue [~vvpete]. While I can see multiple approaches
here, I think the most efficient/immediate option is to expose the possibility
of configuring the HTTP version used in the standard web client service
provider controller service. I have filed NIFI-14977 for this improvement. We
could also consider additional improvements at NiFi startup to not sync with
registry all process groups at the same time but that would need more
discussions.
> NiFi 2.4 Fails to Start Due to Bitbucket Flow Registry Client – “Too
> Many Concurrent Streams” Errors
> -----------------------------------------------------------------------------------------------------
>
> Key: NIFI-14977
> URL: https://issues.apache.org/jira/browse/NIFI-14977
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning, NiFi API
> Affects Versions: 2.4.0
> Environment: Kubernetes, NiFiKop helm chart
> Reporter: Pete Vlad
> Priority: Major
>
> We are running *Apache NiFi 2.4* (NifiKop Helm chart) in a clustered
> environment.
> When NiFi attempts to start up and synchronize with {*}Bitbucket Cloud
> Registry Client{*}, the application fails to fully initialize.
> We have:
> *
> ** *72 process groups* under version control (Bitbucket Cloud registry).
> *
> ** nifi.web.jetty.threads=30 (max web threads).
> *
> ** nifi-atlassian-nar-2.5.0-SNAPSHOT.
> We are observing inconsistent startup behavior:In some cases, NiFi {*}starts
> successfully{*}, but the logs contain repeated errors:
> *java.io.IOException: too many concurrent streams* (from the JDK HTTP/2
> client used by the Bitbucket registry integration).
> Even though these errors appear, the cluster eventually loads, and the UI
> becomes available.
> In other cases, NiFi {*}fails to start entirely{*}. It gets stuck ( The UI is
> visible for a few minutes but then it appear like an empty canvas where you
> cannot change or create anything) at the following log entry and does not
> progress further:
> {code:java}
> 2025-09-04 13:57:41,246 INFO [Framework Task Thread-27]
> o.a.n.a.b.BitbucketFlowRegistryClient BitbucketFlowRegistryClient[id=XXXXXX]
> Initializing repository client{code}
> When this occurs, the Controller Settings → Registry Clients page cannot
> load, and NiFi remains blocked.
> If we disable the Bitbucket registry (by modifying the registry client URL so
> it cannot connect), NiFi starts consistently without issue.
> We need guidance on:
> # Whether this is a {*}known issue in NiFi 2.4’s Bitbucket Flow Registry
> Client{*}.
> # Whether there is a *configuration option* to force HTTP/1.1 or limit
> HTTP/2 concurrency for registry communications. (Not sure if this is the case)
> # Whether there is a *supported workaround* to prevent registry client
> initialization from blocking NiFi startup when many versioned process groups
> are present.
> h3. *Logs when NiFi Starts Successfully (but with Errors)*
> Example log with *“too many concurrent streams”* during startup:
> {code:java}
> at
> org.apache.nifi.web.client.StandardWebClientService$StandardHttpRequestBodySpec.getResponse(StandardWebClientService.java:366)
> at
> org.apache.nifi.web.client.StandardWebClientService$StandardHttpRequestBodySpec.retrieve(StandardWebClientService.java:349)
> at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:251)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:237)
> at jdk.proxy26/jdk.proxy26.$Proxy234.retrieve(Unknown Source)
> at
> org.apache.nifi.atlassian.bitbucket.BitbucketRepositoryClient.getPagedResponseValues(BitbucketRepositoryClient.java:373)
> at
> org.apache.nifi.atlassian.bitbucket.BitbucketRepositoryClient.getListCommits(BitbucketRepositoryClient.java:362)
> at
> org.apache.nifi.atlassian.bitbucket.BitbucketRepositoryClient.getLatestCommit(BitbucketRepositoryClient.java:403)
> at
> org.apache.nifi.atlassian.bitbucket.BitbucketRepositoryClient.getContentFromBranch(BitbucketRepositoryClient.java:244)
> at
> org.apache.nifi.registry.flow.git.AbstractGitFlowRegistryClient.getSnapshot(AbstractGitFlowRegistryClient.java:491)
> at
> org.apache.nifi.registry.flow.git.AbstractGitFlowRegistryClient.getFlow(AbstractGitFlowRegistryClient.java:279)
> at
> org.apache.nifi.registry.flow.StandardFlowRegistryClientNode.lambda$getFlow$6(StandardFlowRegistryClientNode.java:231)
> at
> org.apache.nifi.registry.flow.StandardFlowRegistryClientNode.execute(StandardFlowRegistryClientNode.java:314)
> at
> org.apache.nifi.registry.flow.StandardFlowRegistryClientNode.getFlow(StandardFlowRegistryClientNode.java:231)
> at
> org.apache.nifi.groups.StandardProcessGroup.synchronizeWithFlowRegistry(StandardProcessGroup.java:3769)
> at
> org.apache.nifi.groups.StandardProcessGroup.lambda$setVersionControlInformation$27(StandardProcessGroup.java:3575)
> at
> org.apache.nifi.controller.scheduling.StandardProcessScheduler.lambda$wrapTask$1(StandardProcessScheduler.java:177)
> at java.base/java.lang.VirtualThread.run(VirtualThread.java:329)
> Caused by: java.io.IOException: too many concurrent streams
> at
> java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:967)
> at
> java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:133)
> at
> org.apache.nifi.web.client.StandardWebClientService$StandardHttpRequestBodySpec.getResponse(StandardWebClientService.java:364)
> ... 19 common frames omitted
> Caused by: java.io.IOException: too many concurrent streams
> at
> java.net.http/jdk.internal.net.http.Http2Connection.reserveStream0(Http2Connection.java:616)
> at
> java.net.http/jdk.internal.net.http.Http2Connection.reserveStream(Http2Connection.java:583)
> at
> java.net.http/jdk.internal.net.http.Http2ClientImpl.getConnectionFor(Http2ClientImpl.java:109)
> at java.net.http/jdk.internal.net.http.ExchangeImpl.get(ExchangeImpl.java:94)
> at
> java.net.http/jdk.internal.net.http.Exchange.establishExchange(Exchange.java:391)
> at
> java.net.http/jdk.internal.net.http.Exchange.responseAsyncImpl0(Exchange.java:584)
> at
> java.net.http/jdk.internal.net.http.Exchange.responseAsyncImpl(Exchange.java:428)
> at
> java.net.http/jdk.internal.net.http.Exchange.responseAsync(Exchange.java:420)
> at
> java.net.http/jdk.internal.net.http.MultiExchange.responseAsyncImpl(MultiExchange.java:414)
> at
> java.net.http/jdk.internal.net.http.MultiExchange.lambda$responseAsync0$2(MultiExchange.java:347)
> at
> java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1150)
> at
> java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
> at
> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1773)
> at
> java.net.http/jdk.internal.net.http.HttpClientImpl$DelegatingExecutor.execute(HttpClientImpl.java:177)
> at
> java.base/java.util.concurrent.CompletableFuture.completeAsync(CompletableFuture.java:2719)
> at
> java.net.http/jdk.internal.net.http.MultiExchange.responseAsync(MultiExchange.java:300)
> at
> java.net.http/jdk.internal.net.http.HttpClientImpl.sendAsync(HttpClientImpl.java:1050)
> at
> java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:931)
> ... 21 common frames omitted {code}
> What worked was creating a StandardWebClientProvider for each Bitbucket
> registry. However, I’m now running into the same issue again: when too many
> calls are made too quickly to the NiFi API, I hit the *“too many concurrent
> streams”* error. This causes the UI to become unusable and displays the
> following error:
> {code:java}
> Failed to load Flow Configuration. -
> [org.apache.nifi.web.client.api.WebClientServiceException: Request execution
> failed HTTP Method [GET] URI
> [https://nifi-cluster-3-node.nifi-cluster-headless.nifi2.svc.cluster.local:8443/nifi-api/flow/config]]
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
