[
https://issues.apache.org/jira/browse/NIFI-11794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742534#comment-17742534
]
Pierre Villard commented on NIFI-11794:
---------------------------------------
The issue only happens when clearing a local state for a component in a NiFi
cluster. Even though the state is local, each node of the cluster will receive
the request to clear the state for this component at cluster scope, and this
causes the Redis State Provider to be called. When all nodes are trying to
clear the state at the same time, only one node succeeds and the transaction is
aborted for the other nodes, causing the NPE. The fix, prevents the NPE and
makes the maxAttempts configurable. The reason being that one node will need as
many attempts as there are nodes in the cluster to successfully clear the state.
First node:
2023-07-12 17:12:42,428 DEBUG
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1:
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@5c4008ef
Attempt # 1 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372
was successful
Second node:
2023-07-12 17:12:42,430 DEBUG
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1:
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@15ede30f
Attempt # 1 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372
was unsuccessful
2023-07-12 17:12:42,433 DEBUG
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1:
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@15ede30f
Attempt # 2 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372
was successful
Third node:
2023-07-12 17:12:42,429 DEBUG
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1:
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@6ed23ad8
Attempt # 1 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372
was unsuccessful
2023-07-12 17:12:42,434 DEBUG
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1:
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@6ed23ad8
Attempt # 2 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372
was unsuccessful
2023-07-12 17:12:42,437 DEBUG
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1:
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1@6ed23ad8
Attempt # 3 to clear state for component 46025cdf-0189-1000-ffff-ffffc11ae372
was successful
It means that the number should be at least the number of nodes in the NiFi
cluster. The default value is 20.
> RedisStateProvider failing when used with GCP Memorystore
> ---------------------------------------------------------
>
> Key: NIFI-11794
> URL: https://issues.apache.org/jira/browse/NIFI-11794
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Pierre Villard
> Assignee: Pierre Villard
> Priority: Major
> Fix For: 1.latest, 2.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Trying to configure the NiFi cluster state provider with the Redis
> implementation against a Google Cloud Platform Memorystore instance.
> I can see that there is no state saved in the redis instance.
> When trying to clear state of a processor:
> {code:java}
> 2023-07-11 17:57:39,007 ERROR [NiFi Web Server-22]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.redis.state.RedisStateProvider.lambda$replace$1(RedisStateProvider.java:248)
> at
> org.apache.nifi.redis.state.RedisStateProvider.withConnection(RedisStateProvider.java:313)
> at
> org.apache.nifi.redis.state.RedisStateProvider.replace(RedisStateProvider.java:207)
> at
> org.apache.nifi.redis.state.RedisStateProvider.clear(RedisStateProvider.java:263)
> at
> org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1.clear(StandardStateManagerProvider.java:395)
> at
> org.apache.nifi.controller.state.StandardStateManager.clear(StandardStateManager.java:85)
> at
> org.apache.nifi.web.dao.impl.StandardComponentStateDAO.clearState(StandardComponentStateDAO.java:58)
> at
> org.apache.nifi.web.dao.impl.StandardComponentStateDAO.clearState(StandardComponentStateDAO.java:72)
> at
> org.apache.nifi.web.dao.impl.StandardComponentStateDAO$$FastClassBySpringCGLIB$$51589743.invoke(<generated>)
> {code}
> However this action results in creating the key in the Redis instance:
> {code:java}
> # redis-cli -h ... -p ... -a ... --tls --cacert /tmp/server-ca.pem
> ...:6378> KEYS *
> 1) "nifi/components/46025cdf-0189-1000-ffff-ffffc11ae372"
> ...:6378> GET "nifi/components/46025cdf-0189-1000-ffff-ffffc11ae372"
> "{\"version\":0,\"encodingVersion\":1,\"stateValues\":{}}"
> {code}
> The configuration in state-management.xml is looking like:
> {code:java}
> <cluster-provider>
> <id>redis-provider</id>
> <class>org.apache.nifi.redis.state.RedisStateProvider</class>
> <property name="Redis Mode">Standalone</property>
> <property name="Connection String">...:6378</property>
> <property name="Password">...</property>
> <property name="Enable TLS">true</property>
> </cluster-provider> {code}
> CA cert has been added in the NiFi truststore.
> Still debugging the code to figure out the issue and will also add additional
> logs.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
