[jira] [Commented] (NIFI-5830) RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment
[
https://issues.apache.org/jira/browse/NIFI-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712106#comment-16712106
]
ASF GitHub Bot commented on NIFI-5830:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/3176
> RedisConnectionPoolService does not work with Standalone Redis using
> non-localhost deployment
> -
>
> Key: NIFI-5830
> URL: https://issues.apache.org/jira/browse/NIFI-5830
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: Ubuntu 16 LTS, NiFi 1.8.0
>Reporter: Alexander Bukarev
>Priority: Major
> Fix For: 1.9.0
>
>
> The controller service {{RedisConnectionPoolService}} does not work with
> Standalone Redis which is deployed on a host other than {{localhost}} (or if
> Redis uses the port other than {{6379}}). So the only way to use
> {{RedisConnectionPoolService}} is to deploy Redis on {{localhost}} and run it
> with default port {{6379}}.
> *Steps*:
> Let's assume our Redis is deployed on host *redis* (not {{localhost}}) and it
> listens port 6379 (I use docker for that)
> # Create a {{PutDistributedMapCache}} processor
> # Configure the processor with {{RedisDistributedMapCacheClientService}}
> # Create a new controller service: {{RedisConnectionPoolService}}
> #* Choose *Standalone* Redis Mode
> #* Use *redis:6379* as a Connection String
> # Connect some incoming flow to the {{PutDistributedMapCache}} processor
> (I've used {{GetFile}} as a producer) and run the whole flow. Allow
> {{GetFile}} to consume some file (if you use {{GetFile}} ro reproduce) and
> wait some time till the {{PutDistributedMapCache}} will be schedulled.
> Result:
> The processor is failed to run. And we can see the error in logs:
> {panel}2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
> tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught
> Exception: org.springframework.data.redis.RedisConnectionFailureExce
> ption: Cannot get Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource
> from the pool
> org.springframework.data.redis.RedisConnectionFailureException: Cannot get
> Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource from the pool
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:281)
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:464)
> at
> org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(RedisConnectionPoolService.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy128.getConnection(Unknown Source)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:343)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(RedisDistributedMapCacheClientService.java:189)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy124.put(Unknown Source)
> at
> org.apache.nifi.processors.standard.PutDistributedMapCache.onTrigger(PutDistributedMapCache.java:202){panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5830) RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment
[
https://issues.apache.org/jira/browse/NIFI-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712105#comment-16712105
]
ASF subversion and git services commented on NIFI-5830:
---
Commit 84c32f913780d585ba16960c3f23e83313e1bcab in nifi's branch
refs/heads/master from [~javajefe]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=84c32f9 ]
NIFI-5830 - RedisConnectionPoolService does not work with Standalone Redis
using non-localhost deployment
Signed-off-by: Pierre Villard
This closes #3176.
> RedisConnectionPoolService does not work with Standalone Redis using
> non-localhost deployment
> -
>
> Key: NIFI-5830
> URL: https://issues.apache.org/jira/browse/NIFI-5830
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: Ubuntu 16 LTS, NiFi 1.8.0
>Reporter: Alexander Bukarev
>Priority: Major
>
> The controller service {{RedisConnectionPoolService}} does not work with
> Standalone Redis which is deployed on a host other than {{localhost}} (or if
> Redis uses the port other than {{6379}}). So the only way to use
> {{RedisConnectionPoolService}} is to deploy Redis on {{localhost}} and run it
> with default port {{6379}}.
> *Steps*:
> Let's assume our Redis is deployed on host *redis* (not {{localhost}}) and it
> listens port 6379 (I use docker for that)
> # Create a {{PutDistributedMapCache}} processor
> # Configure the processor with {{RedisDistributedMapCacheClientService}}
> # Create a new controller service: {{RedisConnectionPoolService}}
> #* Choose *Standalone* Redis Mode
> #* Use *redis:6379* as a Connection String
> # Connect some incoming flow to the {{PutDistributedMapCache}} processor
> (I've used {{GetFile}} as a producer) and run the whole flow. Allow
> {{GetFile}} to consume some file (if you use {{GetFile}} ro reproduce) and
> wait some time till the {{PutDistributedMapCache}} will be schedulled.
> Result:
> The processor is failed to run. And we can see the error in logs:
> {panel}2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
> tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught
> Exception: org.springframework.data.redis.RedisConnectionFailureExce
> ption: Cannot get Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource
> from the pool
> org.springframework.data.redis.RedisConnectionFailureException: Cannot get
> Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource from the pool
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:281)
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:464)
> at
> org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(RedisConnectionPoolService.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy128.getConnection(Unknown Source)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:343)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(RedisDistributedMapCacheClientService.java:189)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy124.put(Unknown Source)
> at
> org.apache.nifi.processors.standard.PutDistributedMapCache.onTrigger(PutDistributedMapCache.java:202){panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5830) RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment
[
https://issues.apache.org/jira/browse/NIFI-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712104#comment-16712104
]
ASF GitHub Bot commented on NIFI-5830:
--
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/3176
Changes make sense to me, reproduced the issue with a Docker instance and
confirmed this PR fixes the issue when using a standalone Redis instance. +1,
merging to master, thanks @javajefe
> RedisConnectionPoolService does not work with Standalone Redis using
> non-localhost deployment
> -
>
> Key: NIFI-5830
> URL: https://issues.apache.org/jira/browse/NIFI-5830
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: Ubuntu 16 LTS, NiFi 1.8.0
>Reporter: Alexander Bukarev
>Priority: Major
>
> The controller service {{RedisConnectionPoolService}} does not work with
> Standalone Redis which is deployed on a host other than {{localhost}} (or if
> Redis uses the port other than {{6379}}). So the only way to use
> {{RedisConnectionPoolService}} is to deploy Redis on {{localhost}} and run it
> with default port {{6379}}.
> *Steps*:
> Let's assume our Redis is deployed on host *redis* (not {{localhost}}) and it
> listens port 6379 (I use docker for that)
> # Create a {{PutDistributedMapCache}} processor
> # Configure the processor with {{RedisDistributedMapCacheClientService}}
> # Create a new controller service: {{RedisConnectionPoolService}}
> #* Choose *Standalone* Redis Mode
> #* Use *redis:6379* as a Connection String
> # Connect some incoming flow to the {{PutDistributedMapCache}} processor
> (I've used {{GetFile}} as a producer) and run the whole flow. Allow
> {{GetFile}} to consume some file (if you use {{GetFile}} ro reproduce) and
> wait some time till the {{PutDistributedMapCache}} will be schedulled.
> Result:
> The processor is failed to run. And we can see the error in logs:
> {panel}2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
> tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught
> Exception: org.springframework.data.redis.RedisConnectionFailureExce
> ption: Cannot get Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource
> from the pool
> org.springframework.data.redis.RedisConnectionFailureException: Cannot get
> Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource from the pool
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:281)
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:464)
> at
> org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(RedisConnectionPoolService.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy128.getConnection(Unknown Source)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:343)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(RedisDistributedMapCacheClientService.java:189)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy124.put(Unknown Source)
> at
> org.apache.nifi.processors.standard.PutDistributedMapCache.onTrigger(PutDistributedMapCache.java:202){panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5830) RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment
[
https://issues.apache.org/jira/browse/NIFI-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16709213#comment-16709213
]
ASF GitHub Bot commented on NIFI-5830:
--
Github user javajefe commented on the issue:
https://github.com/apache/nifi/pull/3176
Done!
However, I see Travis-CI build failed and I can't believe that's because of
my PR (the problem is in org.apache.nifi.dbcp.DBCPServiceTest and it looks like
an integration test rather than unit test, anyway there is no relation between
DBCPService and RedisUtils)
> RedisConnectionPoolService does not work with Standalone Redis using
> non-localhost deployment
> -
>
> Key: NIFI-5830
> URL: https://issues.apache.org/jira/browse/NIFI-5830
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: Ubuntu 16 LTS, NiFi 1.8.0
>Reporter: Alexander Bukarev
>Priority: Major
>
> The controller service {{RedisConnectionPoolService}} does not work with
> Standalone Redis which is deployed on a host other than {{localhost}} (or if
> Redis uses the port other than {{6379}}). So the only way to use
> {{RedisConnectionPoolService}} is to deploy Redis on {{localhost}} and run it
> with default port {{6379}}.
> *Steps*:
> Let's assume our Redis is deployed on host *redis* (not {{localhost}}) and it
> listens port 6379 (I use docker for that)
> # Create a {{PutDistributedMapCache}} processor
> # Configure the processor with {{RedisDistributedMapCacheClientService}}
> # Create a new controller service: {{RedisConnectionPoolService}}
> #* Choose *Standalone* Redis Mode
> #* Use *redis:6379* as a Connection String
> # Connect some incoming flow to the {{PutDistributedMapCache}} processor
> (I've used {{GetFile}} as a producer) and run the whole flow. Allow
> {{GetFile}} to consume some file (if you use {{GetFile}} ro reproduce) and
> wait some time till the {{PutDistributedMapCache}} will be schedulled.
> Result:
> The processor is failed to run. And we can see the error in logs:
> {panel}2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
> tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught
> Exception: org.springframework.data.redis.RedisConnectionFailureExce
> ption: Cannot get Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource
> from the pool
> org.springframework.data.redis.RedisConnectionFailureException: Cannot get
> Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource from the pool
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:281)
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:464)
> at
> org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(RedisConnectionPoolService.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy128.getConnection(Unknown Source)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:343)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(RedisDistributedMapCacheClientService.java:189)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy124.put(Unknown Source)
> at
> org.apache.nifi.processors.standard.PutDistributedMapCache.onTrigger(PutDistributedMapCache.java:202){panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5830) RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment
[
https://issues.apache.org/jira/browse/NIFI-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706896#comment-16706896
]
ASF GitHub Bot commented on NIFI-5830:
--
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/3176
Hey @javajefe - I just merged #3135, could you rebase this PR against
master?
> RedisConnectionPoolService does not work with Standalone Redis using
> non-localhost deployment
> -
>
> Key: NIFI-5830
> URL: https://issues.apache.org/jira/browse/NIFI-5830
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: Ubuntu 16 LTS, NiFi 1.8.0
>Reporter: Alexander Bukarev
>Priority: Major
>
> The controller service {{RedisConnectionPoolService}} does not work with
> Standalone Redis which is deployed on a host other than {{localhost}} (or if
> Redis uses the port other than {{6379}}). So the only way to use
> {{RedisConnectionPoolService}} is to deploy Redis on {{localhost}} and run it
> with default port {{6379}}.
> *Steps*:
> Let's assume our Redis is deployed on host *redis* (not {{localhost}}) and it
> listens port 6379 (I use docker for that)
> # Create a {{PutDistributedMapCache}} processor
> # Configure the processor with {{RedisDistributedMapCacheClientService}}
> # Create a new controller service: {{RedisConnectionPoolService}}
> #* Choose *Standalone* Redis Mode
> #* Use *redis:6379* as a Connection String
> # Connect some incoming flow to the {{PutDistributedMapCache}} processor
> (I've used {{GetFile}} as a producer) and run the whole flow. Allow
> {{GetFile}} to consume some file (if you use {{GetFile}} ro reproduce) and
> wait some time till the {{PutDistributedMapCache}} will be schedulled.
> Result:
> The processor is failed to run. And we can see the error in logs:
> {panel}2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
> tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught
> Exception: org.springframework.data.redis.RedisConnectionFailureExce
> ption: Cannot get Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource
> from the pool
> org.springframework.data.redis.RedisConnectionFailureException: Cannot get
> Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource from the pool
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:281)
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:464)
> at
> org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(RedisConnectionPoolService.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy128.getConnection(Unknown Source)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:343)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(RedisDistributedMapCacheClientService.java:189)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy124.put(Unknown Source)
> at
> org.apache.nifi.processors.standard.PutDistributedMapCache.onTrigger(PutDistributedMapCache.java:202){panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5830) RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment
[
https://issues.apache.org/jira/browse/NIFI-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706741#comment-16706741
]
ASF GitHub Bot commented on NIFI-5830:
--
Github user javajefe commented on the issue:
https://github.com/apache/nifi/pull/3176
Hi, I'm just checking should I fix/improve/explain anything more or not? Thx
> RedisConnectionPoolService does not work with Standalone Redis using
> non-localhost deployment
> -
>
> Key: NIFI-5830
> URL: https://issues.apache.org/jira/browse/NIFI-5830
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: Ubuntu 16 LTS, NiFi 1.8.0
>Reporter: Alexander Bukarev
>Priority: Major
>
> The controller service {{RedisConnectionPoolService}} does not work with
> Standalone Redis which is deployed on a host other than {{localhost}} (or if
> Redis uses the port other than {{6379}}). So the only way to use
> {{RedisConnectionPoolService}} is to deploy Redis on {{localhost}} and run it
> with default port {{6379}}.
> *Steps*:
> Let's assume our Redis is deployed on host *redis* (not {{localhost}}) and it
> listens port 6379 (I use docker for that)
> # Create a {{PutDistributedMapCache}} processor
> # Configure the processor with {{RedisDistributedMapCacheClientService}}
> # Create a new controller service: {{RedisConnectionPoolService}}
> #* Choose *Standalone* Redis Mode
> #* Use *redis:6379* as a Connection String
> # Connect some incoming flow to the {{PutDistributedMapCache}} processor
> (I've used {{GetFile}} as a producer) and run the whole flow. Allow
> {{GetFile}} to consume some file (if you use {{GetFile}} ro reproduce) and
> wait some time till the {{PutDistributedMapCache}} will be schedulled.
> Result:
> The processor is failed to run. And we can see the error in logs:
> {panel}2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
> tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught
> Exception: org.springframework.data.redis.RedisConnectionFailureExce
> ption: Cannot get Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource
> from the pool
> org.springframework.data.redis.RedisConnectionFailureException: Cannot get
> Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource from the pool
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:281)
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:464)
> at
> org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(RedisConnectionPoolService.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy128.getConnection(Unknown Source)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:343)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(RedisDistributedMapCacheClientService.java:189)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy124.put(Unknown Source)
> at
> org.apache.nifi.processors.standard.PutDistributedMapCache.onTrigger(PutDistributedMapCache.java:202){panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5830) RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment
[
https://issues.apache.org/jira/browse/NIFI-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691491#comment-16691491
]
Alexander Bukarev commented on NIFI-5830:
-
Just to explain more: if you'll check {{spring-data-redis:2.1.0.RELEASE}} code
you can see {code:java}Assert.notNull(option, "Option must not be
null!");{code} at the beggining of each of implementation methods {{Boolean
set(byte[] key, byte[] value, Expiration expiration, SetOption option)}}, I
mean
- JedisStringCommands implements RedisStringCommands
- JedisClusterStringCommands implements RedisStringCommands
- LettuceStringCommands implements RedisStringCommands
So any implementation fails to execute current line
{code:java}redisConnection.set(kv.getKey(), kv.getValue(),
Expiration.seconds(ttl), null);{code}
So I've decided to fix it...
> RedisConnectionPoolService does not work with Standalone Redis using
> non-localhost deployment
> -
>
> Key: NIFI-5830
> URL: https://issues.apache.org/jira/browse/NIFI-5830
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: Ubuntu 16 LTS, NiFi 1.8.0
>Reporter: Alexander Bukarev
>Priority: Major
>
> The controller service {{RedisConnectionPoolService}} does not work with
> Standalone Redis which is deployed on a host other than {{localhost}} (or if
> Redis uses the port other than {{6379}}). So the only way to use
> {{RedisConnectionPoolService}} is to deploy Redis on {{localhost}} and run it
> with default port {{6379}}.
> *Steps*:
> Let's assume our Redis is deployed on host *redis* (not {{localhost}}) and it
> listens port 6379 (I use docker for that)
> # Create a {{PutDistributedMapCache}} processor
> # Configure the processor with {{RedisDistributedMapCacheClientService}}
> # Create a new controller service: {{RedisConnectionPoolService}}
> #* Choose *Standalone* Redis Mode
> #* Use *redis:6379* as a Connection String
> # Connect some incoming flow to the {{PutDistributedMapCache}} processor
> (I've used {{GetFile}} as a producer) and run the whole flow. Allow
> {{GetFile}} to consume some file (if you use {{GetFile}} ro reproduce) and
> wait some time till the {{PutDistributedMapCache}} will be schedulled.
> Result:
> The processor is failed to run. And we can see the error in logs:
> {panel}2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
> tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught
> Exception: org.springframework.data.redis.RedisConnectionFailureExce
> ption: Cannot get Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource
> from the pool
> org.springframework.data.redis.RedisConnectionFailureException: Cannot get
> Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource from the pool
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:281)
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:464)
> at
> org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(RedisConnectionPoolService.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy128.getConnection(Unknown Source)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:343)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(RedisDistributedMapCacheClientService.java:189)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at
[jira] [Commented] (NIFI-5830) RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment
[
https://issues.apache.org/jira/browse/NIFI-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691017#comment-16691017
]
ASF GitHub Bot commented on NIFI-5830:
--
Github user javajefe commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3176#discussion_r234459335
--- Diff:
nifi-nar-bundles/nifi-redis-bundle/nifi-redis-extensions/src/main/java/org/apache/nifi/redis/service/RedisDistributedMapCacheClientService.java
---
@@ -188,7 +189,7 @@ public void onDisabled() {
public void put(final K key, final V value, final Serializer
keySerializer, final Serializer valueSerializer) throws IOException {
withConnection(redisConnection -> {
final Tuple kv = serialize(key, value,
keySerializer, valueSerializer);
-redisConnection.set(kv.getKey(), kv.getValue(),
Expiration.seconds(ttl), null);
+redisConnection.set(kv.getKey(), kv.getValue(),
Expiration.seconds(ttl), RedisStringCommands.SetOption.UPSERT);
--- End diff --
You’re right, this solves NPE you ‘ll see disregarding to the rest of
changes. But as the root cause of the issue is the same (migration to newer
version of Spring Data Redis 2.x) and the result of RedisUtils fix is
incomplete (you can’t use non-localhost standalone Redis) I’ve decided to fix
it in the same patch.
> RedisConnectionPoolService does not work with Standalone Redis using
> non-localhost deployment
> -
>
> Key: NIFI-5830
> URL: https://issues.apache.org/jira/browse/NIFI-5830
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: Ubuntu 16 LTS, NiFi 1.8.0
>Reporter: Alexander Bukarev
>Priority: Major
>
> The controller service {{RedisConnectionPoolService}} does not work with
> Standalone Redis which is deployed on a host other than {{localhost}} (or if
> Redis uses the port other than {{6379}}). So the only way to use
> {{RedisConnectionPoolService}} is to deploy Redis on {{localhost}} and run it
> with default port {{6379}}.
> *Steps*:
> Let's assume our Redis is deployed on host *redis* (not {{localhost}}) and it
> listens port 6379 (I use docker for that)
> # Create a {{PutDistributedMapCache}} processor
> # Configure the processor with {{RedisDistributedMapCacheClientService}}
> # Create a new controller service: {{RedisConnectionPoolService}}
> #* Choose *Standalone* Redis Mode
> #* Use *redis:6379* as a Connection String
> # Connect some incoming flow to the {{PutDistributedMapCache}} processor
> (I've used {{GetFile}} as a producer) and run the whole flow. Allow
> {{GetFile}} to consume some file (if you use {{GetFile}} ro reproduce) and
> wait some time till the {{PutDistributedMapCache}} will be schedulled.
> Result:
> The processor is failed to run. And we can see the error in logs:
> {panel}2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
> tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught
> Exception: org.springframework.data.redis.RedisConnectionFailureExce
> ption: Cannot get Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource
> from the pool
> org.springframework.data.redis.RedisConnectionFailureException: Cannot get
> Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource from the pool
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:281)
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:464)
> at
> org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(RedisConnectionPoolService.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy128.getConnection(Unknown Source)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:343)
> at
>
[jira] [Commented] (NIFI-5830) RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment
[
https://issues.apache.org/jira/browse/NIFI-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691006#comment-16691006
]
ASF GitHub Bot commented on NIFI-5830:
--
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3176#discussion_r234457408
--- Diff:
nifi-nar-bundles/nifi-redis-bundle/nifi-redis-extensions/src/main/java/org/apache/nifi/redis/service/RedisDistributedMapCacheClientService.java
---
@@ -188,7 +189,7 @@ public void onDisabled() {
public void put(final K key, final V value, final Serializer
keySerializer, final Serializer valueSerializer) throws IOException {
withConnection(redisConnection -> {
final Tuple kv = serialize(key, value,
keySerializer, valueSerializer);
-redisConnection.set(kv.getKey(), kv.getValue(),
Expiration.seconds(ttl), null);
+redisConnection.set(kv.getKey(), kv.getValue(),
Expiration.seconds(ttl), RedisStringCommands.SetOption.UPSERT);
--- End diff --
Not sure that change is related to the reported issue, no?
> RedisConnectionPoolService does not work with Standalone Redis using
> non-localhost deployment
> -
>
> Key: NIFI-5830
> URL: https://issues.apache.org/jira/browse/NIFI-5830
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: Ubuntu 16 LTS, NiFi 1.8.0
>Reporter: Alexander Bukarev
>Priority: Major
>
> The controller service {{RedisConnectionPoolService}} does not work with
> Standalone Redis which is deployed on a host other than {{localhost}} (or if
> Redis uses the port other than {{6379}}). So the only way to use
> {{RedisConnectionPoolService}} is to deploy Redis on {{localhost}} and run it
> with default port {{6379}}.
> *Steps*:
> Let's assume our Redis is deployed on host *redis* (not {{localhost}}) and it
> listens port 6379 (I use docker for that)
> # Create a {{PutDistributedMapCache}} processor
> # Configure the processor with {{RedisDistributedMapCacheClientService}}
> # Create a new controller service: {{RedisConnectionPoolService}}
> #* Choose *Standalone* Redis Mode
> #* Use *redis:6379* as a Connection String
> # Connect some incoming flow to the {{PutDistributedMapCache}} processor
> (I've used {{GetFile}} as a producer) and run the whole flow. Allow
> {{GetFile}} to consume some file (if you use {{GetFile}} ro reproduce) and
> wait some time till the {{PutDistributedMapCache}} will be schedulled.
> Result:
> The processor is failed to run. And we can see the error in logs:
> {panel}2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
> tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught
> Exception: org.springframework.data.redis.RedisConnectionFailureExce
> ption: Cannot get Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource
> from the pool
> org.springframework.data.redis.RedisConnectionFailureException: Cannot get
> Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource from the pool
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:281)
> at
> org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:464)
> at
> org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(RedisConnectionPoolService.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
> at com.sun.proxy.$Proxy128.getConnection(Unknown Source)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:343)
> at
> org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(RedisDistributedMapCacheClientService.java:189)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
[jira] [Commented] (NIFI-5830) RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment
[
https://issues.apache.org/jira/browse/NIFI-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690426#comment-16690426
]
ASF GitHub Bot commented on NIFI-5830:
--
GitHub user javajefe opened a pull request:
https://github.com/apache/nifi/pull/3176
NIFI-5830 RedisConnectionPoolService does not work with Standal…
…one Redis using non-localhost deployment
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [Y] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message? NIFI-5830
- [Y] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [Y] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [Y] Is your initial contribution a single, squashed commit?
### For code changes:
- [Y] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [N] Have you written or updated unit tests to verify your changes?
- [N/A] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [N/A] If applicable, have you updated the LICENSE file, including the
main LICENSE file under nifi-assembly?
- [N/A] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [N/A] If adding new Properties, have you added .displayName in addition
to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [Y] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/javajefe/nifi NIFI-5830
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/3176.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3176
commit 979da56175d2943813c866afb52bca6704d91f35
Author: Alexander Bukarev
Date: 2018-11-17T06:39:52Z
Fixed NIFI-5830 RedisConnectionPoolService does not work with Standalone
Redis using non-localhost deployment
> RedisConnectionPoolService does not work with Standalone Redis using
> non-localhost deployment
> -
>
> Key: NIFI-5830
> URL: https://issues.apache.org/jira/browse/NIFI-5830
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: Ubuntu 16 LTS, NiFi 1.8.0
>Reporter: Alexander Bukarev
>Priority: Major
>
> The controller service {{RedisConnectionPoolService}} does not work with
> Standalone Redis which is deployed on a host other than {{localhost}} (or if
> Redis uses the port other than {{6379}}). So the only way to use
> {{RedisConnectionPoolService}} is to deploy Redis on {{localhost}} and run it
> with default port {{6379}}.
> *Steps*:
> Let's assume our Redis is deployed on host *redis* (not {{localhost}}) and it
> listens port 6379 (I use docker for that)
> # Create a {{PutDistributedMapCache}} processor
> # Configure the processor with {{RedisDistributedMapCacheClientService}}
> # Create a new controller service: {{RedisConnectionPoolService}}
> #* Choose *Standalone* Redis Mode
> #* Use *redis:6379* as a Connection String
> # Connect some incoming flow to the {{PutDistributedMapCache}} processor
> (I've used {{GetFile}} as a producer) and run the whole flow. Allow
> {{GetFile}} to consume some file (if you use {{GetFile}} ro reproduce) and
> wait some time till the {{PutDistributedMapCache}} will be schedulled.
> Result:
> The processor is failed to run. And we can see the error in logs:
> {panel}2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
> tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught
> Exception: org.springframework.data.redis.RedisConnectionFailureExce
> ption: Cannot get Jedis connection; nested exception is
> redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
> resource
