Hello,
I'm trying to set up the
RedisConnectionPoolService/RedisDistributedMapCacheClientService.
Some basic observations:
* This is a standalone Nifi 1.8.0 server
* SELinux is disabled on the server
* There are no iptables rules configured for blocking on the server
* I am able to resolve the hostname of the Redis server to an IP
address on the Nifi server
* I can connect to the Redis server to the Nifi server using telnet
The stack trace I see when the services are started is:
2018-11-29 21:16:03,527 WARN [Timer-Driven Process Thread-8]
o.a.n.controller.tasks.ConnectableTask Administratively Yielding
PutDistributedMapCache[id=0167105c-4a54-1adf-cb8d-1b45de7f0c99] due to
uncaught Exception: org.springframework.data.redis.RedisConnection
FailureException: Cannot get Jedis connection; nested exception is
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
resource from the pool
org.springframework.data.redis.RedisConnectionFailureException: Cannot get
Jedis connection; nested exception is
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a
resource from the pool
at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetch
JedisConnector(JedisConnectionFactory.java:281)
at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getCo
nnection(JedisConnectionFactory.java:464)
at
org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(Redis
ConnectionPoolService.java:89)
at sun.reflect.GeneratedMethodAccessor580.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandle
r.invoke(StandardControllerServiceInvocationHandler.java:84)
at com.sun.proxy.$Proxy98.getConnection(Unknown Source)
at
org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConn
ection(RedisDistributedMapCacheClientService.java:343)
at
org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(Redi
sDistributedMapCacheClientService.java:189)
at sun.reflect.GeneratedMethodAccessor579.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandle
r.invoke(StandardControllerServiceInvocationHandler.java:84)
at com.sun.proxy.$Proxy96.put(Unknown Source)
at
org.apache.nifi.processors.standard.PutDistributedMapCache.onTrigger(PutDist
ributedMapCache.java:202)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java
:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessor
Node.java:1165)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java
:203)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(Timer
DrivenSchedulingAgent.java:117)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$
301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Sch
eduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
49)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
24)
at java.lang.Thread.run(Thread.java:748)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: Could
not get a resource from the pool
at redis.clients.util.Pool.getResource(Pool.java:53)
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:226)
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:16)
at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetch
JedisConnector(JedisConnectionFactory.java:271)
... 26 common frames omitted
Caused by: redis.clients.jedis.exceptions.JedisConnectionException:
java.net.ConnectException: Connection refused (Connection refused)
at redis.clients.jedis.Connection.connect(Connection.java:207)
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93)
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767)
at
redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106)
at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.jav
a:868)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPo
ol.java:435)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPo
ol.java:363)
at redis.clients.util.Pool.getResource(Pool.java:49)
... 29 common frames omitted
Caused by: java.net.ConnectException: Connection refused (Connection
refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.ja
va:206)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at redis.clients.jedis.Connection.connect(Connection.java:184)
... 36 common frames omitted
A tcpdump shows an attempt to connect to localhost:6379 rather than the
Redis server ip address, with the result being a TCP RST. (Thus the
"Connection refused" seen in the stack dump.)
[root@REDACTED logs]# tcpdump -i any -s 1500 -X port 6379
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 1500
bytes
21:16:01.523226 IP localhost.39300 > localhost.6379: Flags [S], seq
2608543115, win 43690, options [mss 65495,sackOK,TS val 2641616382 ecr
0,nop,wscale 7], length 0
0x0000: 4500 003c 2df9 4000 ff06 4fc0 7f00 0001 E..<[email protected].....
0x0010: 7f00 0001 9984 18eb 9b7b 358b 0000 0000 .........{5.....
0x0020: a002 aaaa fe30 0000 0204 ffd7 0402 080a .....0..........
0x0030: 9d73 ddfe 0000 0000 0103 0307 0000 0000 .s..............
0x0040: 0000 0000 0000 0000 0000 0000 ............
21:16:01.523234 IP localhost.6379 > localhost.39300: Flags [R.], seq 0, ack
2608543116, win 0, length 0
0x0000: 4500 0028 0000 4000 ff06 7dcd 7f00 0001 E..(..@...}.....
0x0010: 7f00 0001 18eb 9984 0000 0000 9b7b 358c .............{5.
0x0020: 5014 0000 2e57 0000 0000 0000 0000 0000 P....W..........
0x0030: 0000 0000 0000 0000 ........
This is the configuration for the RedisConnectionPoolService:
<controllerService>
<id>01671039-4a54-1adf-4b56-4ab9dcfaebb6</id>
<name>RedisConnectionPoolService</name>
<comment/>
<class>org.apache.nifi.redis.service.RedisConnectionPoolService</class>
<bundle>
<group>org.apache.nifi</group>
<artifact>nifi-redis-nar</artifact>
<version>1.8.0</version>
</bundle>
<enabled>true</enabled>
<property>
<name>Redis Mode</name>
<value>Standalone</value>
</property>
<property>
<name>Connection String</name>
<value>REDACTED.use1.cache.amazonaws.com:6379</value>
</property>
<property>
<name>Database Index</name>
<value>0</value>
</property>
<property>
<name>Communication Timeout</name>
<value>10 seconds</value>
</property>
<property>
<name>Cluster Max Redirects</name>
<value>5</value>
</property>
<property>
<name>Sentinel Master</name>
</property>
<property>
<name>Password</name>
</property>
<property>
<name>Pool - Max Total</name>
<value>4</value>
</property>
<property>
<name>Pool - Max Idle</name>
<value>2</value>
</property>
<property>
<name>Pool - Min Idle</name>
<value>1</value>
</property>
<property>
<name>Pool - Block When Exhausted</name>
<value>true</value>
</property>
<property>
<name>Pool - Max Wait Time</name>
<value>10 seconds</value>
</property>
<property>
<name>Pool - Min Evictable Idle Time</name>
<value>60 seconds</value>
</property>
<property>
<name>Pool - Time Between Eviction Runs</name>
<value>30 seconds</value>
</property>
<property>
<name>Pool - Num Tests Per Eviction Run</name>
<value>-1</value>
</property>
<property>
<name>Pool - Test On Create</name>
<value>true</value>
</property>
<property>
<name>Pool - Test On Borrow</name>
<value>false</value>
</property>
<property>
<name>Pool - Test On Return</name>
<value>false</value>
</property>
<property>
<name>Pool - Test While Idle</name>
<value>true</value>
</property>
</controllerService>
I've seen where other folks have RedisDistributedMapCacheClientService
working in their environments, so this seems to be something particular to
this environment and not necessarily Nifi. Does anyone have an idea as to
what this might be or other items to check?
Warm regards,
<https://www.alertlogic.com/>
Jim Williams | Principal Database Developer
O: +1 713.341.7812 | C: +1 919.523.8767 | [email protected] |
<http://www.alertlogic.com/> alertlogic.com
<https://twitter.com/alertlogic>
<https://www.linkedin.com/company/alert-logic>
smime.p7s
Description: S/MIME cryptographic signature
