[ 
https://issues.apache.org/jira/browse/GEODE-4451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350828#comment-16350828
 ] 

Diane Hardman commented on GEODE-4451:
--------------------------------------

If the gateway sender fails authentication with the receiver, the exception 
should be thrown and the gateway sender should be destroyed.

> When a gateway sender is created using gfsh and fails to connect due to 
> AuthenticationFailedException, it is left in an inconsistent state
> ------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: GEODE-4451
>                 URL: https://issues.apache.org/jira/browse/GEODE-4451
>             Project: Geode
>          Issue Type: Bug
>          Components: wan
>            Reporter: Barry Oglesby
>            Priority: Major
>
> If I attempt to create a gateway sender with invalid credentials using gfsh, 
> it fails like:
> {noformat}
> (2) Executing - create gateway-sender --remote-distributed-system-id=1 
> --id=ny --dispatcher-threads=1 --parallel=true
> Member | Status
> ------ | 
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> ln-1 | ERROR: Could not start a gateway sender ny because of exception Could 
> not start a gateway sender 84 because of exception 
> org.apache.geode.security.AuthenticationFailedException: Authentication 
> error. Please check your credentials.
> {noformat}
> The 84 should be the gateway sender id; instead its the thread id (from 
> \{{ConcurrentParallelGatewaySenderEventProcessor.waitForRunningStatus}}.
> In the server log, I see 5 AuthenticationFailedException stacks like (one per 
> dispatcher-thread):
> {noformat}
> [severe 2018/01/31 16:13:12.638 PST ln-1 <Event Processor for 
> GatewaySender_ny_0> tid=0x55] Message dispatch failed due to unexpected 
> exception..
> org.apache.geode.internal.cache.wan.GatewaySenderException: 
> org.apache.geode.security.AuthenticationFailedException: Authentication 
> error. Please check your credentials., caused by 
> org.apache.geode.security.AuthenticationFailedException: Authentication 
> error. Please check your credentials.
>  at 
> org.apache.geode.internal.cache.wan.GatewaySenderEventRemoteDispatcher.initializeConnection(GatewaySenderEventRemoteDispatcher.java:432)
>  at 
> org.apache.geode.internal.cache.wan.GatewaySenderEventRemoteDispatcher.<init>(GatewaySenderEventRemoteDispatcher.java:75)
>  at 
> org.apache.geode.internal.cache.wan.parallel.RemoteParallelGatewaySenderEventProcessor.initializeEventDispatcher(RemoteParallelGatewaySenderEventProcessor.java:74)
>  at 
> org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.setRunningStatus(AbstractGatewaySenderEventProcessor.java:1078)
>  at 
> org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1050)
> Caused by: org.apache.geode.security.AuthenticationFailedException: 
> Authentication error. Please check your credentials.
>  at 
> org.apache.geode.internal.cache.tier.sockets.HandShake.readMessage(HandShake.java:1397)
>  at 
> org.apache.geode.internal.cache.tier.sockets.HandShake.handshakeWithServer(HandShake.java:1253)
>  at 
> org.apache.geode.cache.client.internal.ConnectionImpl.connect(ConnectionImpl.java:118)
>  at 
> org.apache.geode.cache.client.internal.ConnectionFactoryImpl.createClientToServerConnection(ConnectionFactoryImpl.java:136)
>  at 
> org.apache.geode.cache.client.internal.ConnectionFactoryImpl.createClientToServerConnection(ConnectionFactoryImpl.java:259)
>  at 
> org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl.borrowConnection(ConnectionManagerImpl.java:206)
>  at 
> org.apache.geode.cache.client.internal.PoolImpl.acquireConnection(PoolImpl.java:902)
>  at 
> org.apache.geode.internal.cache.wan.GatewaySenderEventRemoteDispatcher.initializeConnection(GatewaySenderEventRemoteDispatcher.java:388)
>  ... 4 more
> {noformat}
> A thread dump shows the Event Processor threads are gone, but 5 
> BatchRemovalThreads like this exist:
> {noformat}
> "BatchRemovalThread" #96 daemon prio=10 os_prio=31 tid=0x00007fc1dd009000 
> nid=0xcb03 waiting on condition [0x000070000e245000]
>  java.lang.Thread.State: TIMED_WAITING (parking)
>  at sun.misc.Unsafe.park(Native Method)
>  - parking to wait for <0x00000007a04043c8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>  at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
>  at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
>  at 
> org.apache.geode.internal.util.concurrent.StoppableCondition.await(StoppableCondition.java:61)
>  at 
> org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderQueue$BatchRemovalThread.run(ParallelGatewaySenderQueue.java:1613)
> {noformat}
> list gateways shows the sender in the \{{Not Running}} state:
> {noformat}
> GatewaySender Id | Member | Remote Cluster Id | Type | Status | Queued Events 
> | Receiver Location
> ---------------- | -------------------------------- | ----------------- | 
> -------- | ----------- | ------------- | -----------------
> ny | 192.168.2.4(ln-1:13883)<v3>:1027 | 1 | Parallel | Not Running | 0 | null
> {noformat}
> The sender is left in an inconsistent state. It should either be completely 
> gone or retrying to connect.
> {\{GatewaySenderEventRemoteDispatcher.<init>}} catches the 
> \{{GatewaySenderException}} attempting to \{{initializeConnection}} and 
> rethrows it if is caused by \{{GemFireSecurityException}}:
> {noformat}
> public GatewaySenderEventRemoteDispatcher(AbstractGatewaySenderEventProcessor 
> eventProcessor) {
>  this.processor = eventProcessor;
>  this.sender = eventProcessor.getSender();
>  // this.ackReaderThread = new AckReaderThread(sender);
>  try {
>  initializeConnection();
>  } catch (GatewaySenderException e) {
>  if (e.getCause() instanceof GemFireSecurityException) {
>  throw e;
>  }
>  }
> }
> {noformat}
> {\{AbstractGatewaySenderEventProcessor.run}} catches that exception and logs 
> severe message and stack above.
> If I change this area of code to log the warning instead of stopping the 
> Event Processor threads, I see this exception after puts are done:
> {noformat}
> [severe 2018/01/31 17:37:16.324 PST ln-1 <Event Processor for 
> GatewaySender_ny_0> tid=0x49] Stopping the processor because the following 
> exception occurred while processing a batch:
> org.apache.geode.internal.cache.wan.GatewaySenderException: 
> org.apache.geode.security.AuthenticationFailedException: Authentication 
> error. Please check your credentials., caused by 
> org.apache.geode.security.AuthenticationFailedException: Authentication 
> error. Please check your credentials.
>  at 
> org.apache.geode.internal.cache.wan.GatewaySenderEventRemoteDispatcher.initializeConnection(GatewaySenderEventRemoteDispatcher.java:431)
>  at 
> org.apache.geode.internal.cache.wan.GatewaySenderEventRemoteDispatcher.getConnection(GatewaySenderEventRemoteDispatcher.java:320)
>  at 
> org.apache.geode.internal.cache.wan.GatewaySenderEventRemoteDispatcher._dispatchBatch(GatewaySenderEventRemoteDispatcher.java:198)
>  at 
> org.apache.geode.internal.cache.wan.GatewaySenderEventRemoteDispatcher.dispatchBatch(GatewaySenderEventRemoteDispatcher.java:147)
>  at 
> org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
>  at 
> org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
> Caused by: org.apache.geode.security.AuthenticationFailedException: 
> Authentication error. Please check your credentials.
>  at 
> org.apache.geode.internal.cache.tier.sockets.HandShake.readMessage(HandShake.java:1397)
>  at 
> org.apache.geode.internal.cache.tier.sockets.HandShake.handshakeWithServer(HandShake.java:1253)
>  at 
> org.apache.geode.cache.client.internal.ConnectionImpl.connect(ConnectionImpl.java:118)
>  at 
> org.apache.geode.cache.client.internal.ConnectionFactoryImpl.createClientToServerConnection(ConnectionFactoryImpl.java:136)
>  at 
> org.apache.geode.cache.client.internal.ConnectionFactoryImpl.createClientToServerConnection(ConnectionFactoryImpl.java:259)
>  at 
> org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl.borrowConnection(ConnectionManagerImpl.java:206)
>  at 
> org.apache.geode.cache.client.internal.PoolImpl.acquireConnection(PoolImpl.java:902)
>  at 
> org.apache.geode.internal.cache.wan.GatewaySenderEventRemoteDispatcher.initializeConnection(GatewaySenderEventRemoteDispatcher.java:387)
>  ... 5 more
> {noformat}
> That is coming from \{{GatewaySenderEventRemoteDispatcher.dispatchBatch}}:
> {noformat}
> } catch (GatewaySenderException ge) {
>  if (...) {
>  ...
>  } else {
>  logger.fatal(
>  LocalizedMessage.create(
>  
> LocalizedStrings.GatewayEventRemoteDispatcher_STOPPING_THE_PROCESSOR_BECAUSE_THE_FOLLOWING_EXCEPTION_OCCURRED_WHILE_PROCESSING_A_BATCH),
>  ge);
>  this.processor.setIsStopped(true);
>  }
> {noformat}
> So, \{{AuthenticationFailedExceptions}} are definitely meant to stop the 
> sender.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to