boglesby commented on pull request #5600:
URL: https://github.com/apache/geode/pull/5600#issuecomment-754953043
I'm still reviewing these changes, but here are a few things I see.
I can run a test like:
1. Start server 1 with a gateway sender with
enforce-threads-connect-same-receiver=true
2. Start server 1 with a gateway sender with
enforce-threads-connect-same-receiver=false
The servers successfully start up in this case, but I don't think they
should with these conflicting property values.
The gfsh commands look like:
```
start server --name=server-A-1 --group=server-A-1
--locators=localhost[10331] --mcast-port=0 --server-port=0
start server --name=server-A-2 --group=server-A-2
--locators=localhost[10331] --mcast-port=0 --server-port=0
create gateway-sender --id=B --group=server-A-1
--remote-distributed-system-id=2 --enforce-threads-connect-same-receiver=true
create gateway-sender --id=B --group=server-A-2
--remote-distributed-system-id=2 --enforce-threads-connect-same-receiver=false
```
The generated xml looks like:
server-A-1:
```
<cache ...>
<gateway-sender enforce-threads-connect-same-receiver="true" id="B" .../>
</cache>
```
server-A-2:
```
<cache ...>
<gateway-sender enforce-threads-connect-same-receiver="false" id="B" .../>
</cache>
```
Here is an example where conflicting gateway sender parameters cause the
startup to fail:
```
create gateway-sender --id=B --group=server-A-1
--remote-distributed-system-id=2 --parallel=true
create gateway-sender --id=B --group=server-A-2
--remote-distributed-system-id=2 --parallel=false
```
This error message is logged:
```
Executing - create gateway-sender --id=B --group=server-A-2
--remote-distributed-system-id=2 --parallel=false
Member | Status | Message
---------- | ------ |
--------------------------------------------------------------------------------------------------------------------------------------------------------------
server-A-2 | ERROR | java.lang.IllegalStateException: Cannot create
Gateway Sender B as serial gateway sender because another cache has the same
sender as parallel gateway sender
And this error message is in the logs:
[error 2021/01/05 12:05:51.385 PST server-A-2 <Function Execution
Processor2> tid=0x3f] Cannot create Gateway Sender B as serial gateway sender
because another cache has the same sender as parallel gateway sender
java.lang.IllegalStateException: Cannot create Gateway Sender B as serial
gateway sender because another cache has the same sender as parallel gateway
sender
at Remote Member '10.166.145.67(server-A-1:89588)<v1>:41001' in
org.apache.geode.internal.cache.wan.GatewaySenderAdvisor.checkCompatibility(GatewaySenderAdvisor.java:125)
at Remote Member '10.166.145.67(server-A-1:89588)<v1>:41001' in
org.apache.geode.internal.cache.wan.GatewaySenderAdvisor.profileCreated(GatewaySenderAdvisor.java:105)
at Remote Member '10.166.145.67(server-A-1:89588)<v1>:41001' in
org.apache.geode.distributed.internal.DistributionAdvisor.doPutProfile(DistributionAdvisor.java:625)
at Remote Member '10.166.145.67(server-A-1:89588)<v1>:41001' in
org.apache.geode.distributed.internal.DistributionAdvisor.putProfile(DistributionAdvisor.java:523)
at Remote Member '10.166.145.67(server-A-1:89588)<v1>:41001' in
org.apache.geode.distributed.internal.DistributionAdvisor.putProfile(DistributionAdvisor.java:518)
at Remote Member '10.166.145.67(server-A-1:89588)<v1>:41001' in
org.apache.geode.distributed.internal.DistributionAdvisor$Profile.handleDistributionAdvisee(DistributionAdvisor.java:1537)
at Remote Member '10.166.145.67(server-A-1:89588)<v1>:41001' in
org.apache.geode.internal.cache.wan.GatewaySenderAdvisor$GatewaySenderProfile.processIncoming(GatewaySenderAdvisor.java:708)
at
org.apache.geode.distributed.internal.ReplyException.handleCause(ReplyException.java:86)
at
org.apache.geode.internal.cache.UpdateAttributesProcessor.waitForProfileResponse(UpdateAttributesProcessor.java:113)
at
org.apache.geode.internal.cache.UpdateAttributesProcessor.distribute(UpdateAttributesProcessor.java:99)
at
org.apache.geode.internal.cache.GemFireCacheImpl.addGatewaySender(GemFireCacheImpl.java:3732)
at
org.apache.geode.internal.cache.wan.GatewaySenderFactoryImpl.create(GatewaySenderFactoryImpl.java:300)
at
org.apache.geode.management.internal.cli.functions.GatewaySenderCreateFunction.createGatewaySender(GatewaySenderCreateFunction.java:183)
at
org.apache.geode.management.internal.cli.functions.GatewaySenderCreateFunction.execute(GatewaySenderCreateFunction.java:60)
```
Something similar should happen with the
enforce-threads-connect-same-receiver property.
This check and error message are coming from
GatewaySenderAdvisor.checkCompatibility.
GatewaySenderProfile needs to be updated to include
enforceThreadsConnectSameReceiver. Here are some changes that do this:
```
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/GatewaySenderAdvisor.java
b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/GatewaySenderAdvisor.java
index adf80cb1e5..dc70a0ae6b 100644
---
a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/GatewaySenderAdvisor.java
+++
b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/GatewaySenderAdvisor.java
@@ -232,6 +232,14 @@ public class GatewaySenderAdvisor extends
DistributionAdvisor {
"Cannot create Gateway Sender %s with isDiskSynchronous %s
because another cache has the same Gateway Sender defined with
isDiskSynchronous %s",
sp.Id, sp.isDiskSynchronous, sender.isDiskSynchronous()));
}
+
+ if (sp.enforceThreadsConnectSameReceiver !=
sender.getEnforceThreadsConnectSameReceiver()) {
+ throw new IllegalStateException(
+ String.format(
+ "Cannot create Gateway Sender %s with
enforceThreadsConnectSameReceiver %s because another cache has the same Gateway
Sender defined with enforceThreadsConnectSameReceiver %s",
+ sp.Id, sp.enforceThreadsConnectSameReceiver,
+ sender.getEnforceThreadsConnectSameReceiver()));
+ }
}
/**
@@ -532,6 +540,8 @@ public class GatewaySenderAdvisor extends
DistributionAdvisor {
public ServerLocation serverLocation;
+ public boolean enforceThreadsConnectSameReceiver = false;
+
public GatewaySenderProfile(InternalDistributedMember memberId, int
version) {
super(memberId, version);
}
@@ -578,6 +588,7 @@ public class GatewaySenderAdvisor extends
DistributionAdvisor {
this.serverLocation = new ServerLocation();
InternalDataSerializer.invokeFromData(this.serverLocation, in);
}
+ this.enforceThreadsConnectSameReceiver = in.readBoolean();
}
@Override
@@ -617,6 +628,7 @@ public class GatewaySenderAdvisor extends
DistributionAdvisor {
if (serverLocationFound) {
InternalDataSerializer.invokeToData(serverLocation, out);
}
+ out.writeBoolean(enforceThreadsConnectSameReceiver);
}
public void fromDataPre_GFE_8_0_0_0(DataInput in,
DeserializationContext context)
diff --git
a/geode-wan/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderImpl.java
b/geode-wan/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderImpl.java
index 7c93836667..1b5206b532 100644
---
a/geode-wan/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderImpl.java
+++
b/geode-wan/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderImpl.java
@@ -230,6 +230,7 @@ public class SerialGatewaySenderImpl extends
AbstractRemoteGatewaySender {
pf.dispatcherThreads = getDispatcherThreads();
pf.orderPolicy = getOrderPolicy();
pf.serverLocation = this.getServerLocation();
+ pf.enforceThreadsConnectSameReceiver =
getEnforceThreadsConnectSameReceiver();
}
@Override
```
With these changes, gfsh logs this error message:
```
Executing - create gateway-sender --id=B --group=server-A-2
--remote-distributed-system-id=2 --enforce-threads-connect-same-receiver=false
Member | Status | Message
---------- | ------ |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
server-A-2 | ERROR | java.lang.IllegalStateException: Cannot create
Gateway Sender B with enforceThreadsConnectSameReceiver false because another
cache has the same Gateway Sender defined with
enforceThreadsConnectSameReceiver true
```
There might be a problem with these changes as written though. gfsh prevents
gateway senders from being created while all the members are not the current
version, but thats not true for GemFire xml.
If I run a test like this with these changes:
1. Start locator with an older version
2. Start several servers with GemFire xml
3. Bounce locator to a newer version
4. Start server with the newer version
In this case, I see an exception like this in the new server when it tries
to deserialize a GatewaySenderProfile from an older member:
```
[fatal 2021/01/05 14:01:51.543 PST server-A-new <P2P message reader for
10.166.145.67(server-A-2:97374)<v2>:41002(version:GEODE 1.13.0) shared
unordered uid=12 local port=41102 remote port=63358> tid=0x2b] Error
deserializing message
java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:500)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:249)
at
org.apache.geode.internal.tcp.ByteBufferInputStream$ByteBufferByteSource.get(ByteBufferInputStream.java:206)
at
org.apache.geode.internal.tcp.ByteBufferInputStream.readBoolean(ByteBufferInputStream.java:878)
at
org.apache.geode.internal.cache.wan.GatewaySenderAdvisor$GatewaySenderProfile.fromData(GatewaySenderAdvisor.java:592)
```
I'm not sure if this case needs to be supported. If so, then the
GatewaySenderProfile toData/fromData will have to handle versioning.
btw - this same change might need to be made for the groupTransactionEvents
property.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
