Hello all,
Somebody has already done a site2site configuration between nifi intranet and nifi gcp ?
From nifi intranet we use a proxy to try to communicate with nifi gcp.
In the gcp console, we have created a VM with loadbalancer for public DNS entry, and setup the port we need to use.
There are some warning and errors.
There are some warning and errors.
The handshake seems ok like there is blue circle "remote process group" but there is no data sending from nifi gcp to nifi intranet
- Somebody already configure the VM gcp with loadbalancer in GCP and could help me about the network configuration ?
- Are we need to some specific configuration from nifi.properties side ?
In attachment, the logs we got for more details
Thanks for you help
Minh
LOGS NIFI GCP 2025-03-24 15:04:09,229 INFO [NiFi Web Server-28] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 35.191.x.x [CN=server.nifi01.fr, OU=Dataflow, C=FR, O=man] POST https://ope-nifi-gcp.nifi.com:9091/nifi-api/data-transfer/output-ports/b75d4b0f-0194-1000-1f5f-562d1892561b/transactions 2025-03-24 15:04:09,229 INFO [NiFi Web Server-28] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [CN=server.nifi01.fr, OU=Dataflow, C=FR, O=man] 35.191.x.x POST https://ope-nifi-gcp.nifi.com:9091/nifi-api/data-transfer/output-ports/b75d4b0f-0194-1000-1f5f-562d1892561b/transactions
2025-03-24 15:04:09,357 INFO [NiFi Web Server-28] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 35.191.y.y [CN=server.nifi01.fr, OU=Dataflow, C=FR, O=man] GET https://ope-nifi-gcp.nifi.com:9091/nifi-api/data-transfer/output-ports/b75d4b0f-0194-1000-1f5f-562d1892561b/transactions/da9e9c59-fd3f-4530-938f-355cef8d476f/flow-files 2025-03-24 15:04:09,357 INFO [NiFi Web Server-28] o.a.n.w.s.NiFiAuthenticationFilter Authentication Success [CN=server.nifi01.fr, OU=Dataflow, C=FR, O=man] 35.191.y.y GET https://ope-nifi-gcp.nifi.com:9091/nifi-api/data-transfer/output-ports/b75d4b0f-0194-1000-1f5f-562d1892561b/transactions/da9e9c59-fd3f-4530-938f-355cef8d476f/flow-files 2025-03-24 15:04:09,602 WARN [NiFi Web Server-28] o.a.n.w.a.c.WebApplicationExceptionMapper javax.ws.rs.WebApplicationException: HTTP 200 OK. Returning 200 response. javax.ws.rs.WebApplicationException: HTTP 200 OK at org.apache.nifi.web.api.DataTransferResource$1.write(DataTransferResource.java:670) at org.glassfish.jersey.message.internal.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:55) at org.glassfish.jersey.message.internal.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:37) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) at org.glassfish.jersey.spi.ContentEncoder.aroundWriteTo(ContentEncoder.java:113) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116) at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:649) at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:380) at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:370) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:259) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:352) at org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) at org.apache.nifi.web.security.log.AuthenticationUserFilter.doFilterInternal(AuthenticationUserFilter.java:57) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) at org.springframework.security.oauth2.server.resource.web.authentication.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:132) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) at org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:94) at org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:56) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:118) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) at org.apache.nifi.web.security.csrf.SkipReplicatedCsrfFilter.doFilterInternal(SkipReplicatedCsrfFilter.java:59) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:361) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:225) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:190) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.apache.nifi.web.filter.ExceptionFilter.doFilter(ExceptionFilter.java:44) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.apache.nifi.web.server.filter.DataTransferExcludedDoSFilter.doFilterChain(DataTransferExcludedDoSFilter.java:49) at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:320) at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:285) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.apache.nifi.web.server.log.RequestAuthenticationFilter.doFilterInternal(RequestAuthenticationFilter.java:59) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Thread.java:829) LOG NIFI INTRANET Message in the nifi console "Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance" 2025-03-24 15:38:43,516 WARN [Timer-Driven Process Thread-9] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:43,516 WARN [Timer-Driven Process Thread-9] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:38:43,571 WARN [Timer-Driven Process Thread-9] o.a.nifi.remote.client.http.HttpClient Penalizing a peer Peer[url=https://ope-nifi-gcp.nifi.com:9091/nifi-api] due to java.io.IOException: Unexpected response code: 405 errCode:Abort errMessage:{ "servlet":"org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-97eb156", "message":"HTTP method POST is not supported by this URL", "url":"/nifi-api/data-transfer/output-ports/b75d4b0f-0194-1000-1f5f-562d1892561b/transactions", "status":"405" } 2025-03-24 15:38:43,630 WARN [Timer-Driven Process Thread-9] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:43,630 WARN [Timer-Driven Process Thread-9] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:38:43,631 INFO [Timer-Driven Process Thread-9] o.a.nifi.remote.client.http.HttpClient Couldn't find a valid peer to communicate with. 2025-03-24 15:38:45,090 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:45,090 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:38:45,955 INFO [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Successfully refreshed peer status cache; remote group consists of 1 peers 2025-03-24 15:38:47,167 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:47,168 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 5-03-24 15:38:50,068 WARN [Timer-Driven Process Thread-23] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from https://ope-nifi-gcp.nifi.com:9091/nifi-api due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:50,068 WARN [Timer-Driven Process Thread-23] o.apache.nifi.controller.FlowController Unable to communicate with remote instance RemoteProcessGroup[https://ope-nifi-gcp.nifi.com:9091/nifi/] due to org.apache.nifi.controller.exception.CommunicationsException: org.apache.nifi.controller.exception.CommunicationsException: Unable to communicate with Remote NiFi at URI https://ope-nifi-gcp.nifi.com:9091/nifi/ due to: response code 404:Not Found with explanation: null 2025-03-24 15:38:50,169 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:50,170 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:38:52,244 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:52,244 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:38:53,718 WARN [Timer-Driven Process Thread-15] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:53,718 WARN [Timer-Driven Process Thread-15] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:38:53,787 WARN [Timer-Driven Process Thread-15] o.a.nifi.remote.client.http.HttpClient Penalizing a peer Peer[url=https://ope-nifi-gcp.nifi.com:9091/nifi-api] due to java.io.IOException: Unexpected response code: 405 errCode:Abort errMessage:{ "servlet":"org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-57fa6d84", "message":"HTTP method POST is not supported by this URL", "url":"/nifi-api/data-transfer/output-ports/b75d4b0f-0194-1000-1f5f-562d1892561b/transactions", "status":"405" } 2025-03-24 15:38:53,866 WARN [Timer-Driven Process Thread-15] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:53,866 WARN [Timer-Driven Process Thread-15] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:38:53,866 INFO [Timer-Driven Process Thread-15] o.a.nifi.remote.client.http.HttpClient Couldn't find a valid peer to communicate with. 2025-03-24 15:38:54,760 INFO [Timer-Driven Process Thread-30] o.a.n.r.SiteToSiteBulletinReportingTask SiteToSiteBulletinReportingTask[id=313fc12e-0186-1000-ca30-28b1002ce906] Successfully sent 10 Bulletins to destination in 130 ms; Transaction ID = 32c49944-3e40-4122-be13-26b6be305d7b; First Event ID = 1099 2025-03-24 15:38:55,047 INFO [NiFi Site-to-Site Connection Pool Maintenance] o.apache.nifi.remote.client.PeerSelector Successfully refreshed peer status cache; remote group consists of 1 peers 2025-03-24 15:38:55,217 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:55,217 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:38:57,289 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:38:57,289 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:38:59,580 INFO [Framework Task Thread Thread-1] o.a.n.p.store.WriteAheadStorePartition Successfully rolled over Event Writer for Provenance Event Store Partition[directory=/data/nifi/provenance_repository] due to MAX_TIME_REACHED. Event File was 4.35 MB and contained 3623 events. 2025-03-24 15:39:00,331 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:39:00,331 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:39:02,362 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:39:02,362 WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:39:04,024 WARN [Timer-Driven Process Thread-26] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null 2025-03-24 15:39:04,024 WARN [Timer-Driven Process Thread-26] o.apache.nifi.remote.client.PeerSelector Unable to refresh remote group peers due to: Unable to retrieve nodes from remote instance 2025-03-24 15:39:04,094 WARN [Timer-Driven Process Thread-26] o.a.nifi.remote.client.http.HttpClient Penalizing a peer Peer[url=https://ope-nifi-gcp.nifi.com:9091/nifi-api] due to java.io.IOException: Unexpected response code: 405 errCode:Abort errMessage:{ "servlet":"org.eclipse.jetty.servlet.ServletHandler$Default404Servlet-57fa6d84", "message":"HTTP method POST is not supported by this URL", "url":"/nifi-api/data-transfer/output-ports/b75d4b0f-0194-1000-1f5f-562d1892561b/transactions", "status":"405" } 2025-03-24 15:39:04,227 WARN [Timer-Driven Process Thread-26] o.apache.nifi.remote.client.PeerSelector Could not communicate with ope-nifi-gcp.nifi.com:9091 to determine which node(s) exist in the remote NiFi instance, due to org.apache.nifi.remote.util.SiteToSiteRestApiClient$HttpGetFailedException: response code 404:Not Found with explanation: null