[geode] branch develop updated (8518348 -> 55921a4)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a change to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git.
from 8518348 GEODE-9155: change frequency of passive expiration (#6325)
add 55921a4 GEODE-9139 SSLException in starting up a Locator (#6308)
No new revisions were added by this update.
Summary of changes:
.../client/sni/ClientSNICQAcceptanceTest.java | 18 ++--
.../client/sni/DualServerSNIAcceptanceTest.java| 28 +-
.../client/sni/GenerateSNIKeyAndTrustStores.java | 7 +-
.../sni/dual-server-docker-compose.yml}| 45 +-
.../sni/{haproxy.cfg => dual-server-haproxy.cfg} | 6 +-
.../sni/geode-config/locator-maeve-keystore.jks| Bin 3529 -> 3514 bytes
.../geode-config/server-clementine-keystore.jks| Bin 3537 -> 3520 bytes
.../sni/geode-config/server-dolores-keystore.jks | Bin 3533 -> 3515 bytes
.../geode/client/sni/geode-config/truststore.jks | Bin 1129 -> 1129 bytes
.../geode/client/sni/scripts/create-regions.gfsh | 7 +-
.../geode/client/sni/scripts/geode-starter-2.gfsh | 23 -
.../geode/client/sni/scripts/geode-starter.gfsh| 2 +-
.../geode/client/sni/scripts/locator-maeve.gfsh| 5 +-
.../client/sni/scripts/server-clementine.gfsh | 5 +-
.../geode/client/sni/scripts/server-dolores.gfsh | 5 +-
.../StartLocatorCommandIntegrationTest.java| 8 +-
...ReconnectWithClusterConfigurationDUnitTest.java | 3 +-
.../internal/InternalLocatorIntegrationTest.java | 8 +-
.../internal/membership/MembershipJUnitTest.java | 9 +-
.../java/org/apache/geode/distributed/Locator.java | 39 +---
.../apache/geode/distributed/LocatorLauncher.java | 78 +---
.../internal/InternalDistributedSystem.java| 6 +-
.../distributed/internal/InternalLocator.java | 66 +-
.../geode/distributed/internal/ServerLocator.java | 7 +-
.../internal/membership/adapter/ServiceConfig.java | 5 +-
.../apache/geode/internal/DistributionLocator.java | 30 +++
.../admin/remote/DistributionLocatorId.java| 7 +-
.../apache/geode/internal/net/SocketCreator.java | 12 ---
.../org/apache/geode/internal/tcp/Connection.java | 8 +-
.../distributed/internal/InternalLocatorTest.java | 3 +-
.../internal/cli/commands/StartLocatorCommand.java | 9 +-
.../internal/cli/shell/JmxOperationInvoker.java| 3 +-
.../locator/GMSLocatorRecoveryIntegrationTest.java | 5 +-
.../gms/membership/GMSJoinLeaveJUnitTest.java | 2 +-
.../membership/api/MembershipLocatorBuilder.java | 4 +-
.../internal/membership/gms/GMSMemberData.java | 2 -
.../gms/MembershipLocatorBuilderImpl.java | 6 +-
.../membership/gms/locator/GMSLocator.java | 7 +-
.../gms/locator/MembershipLocatorImpl.java | 8 +-
.../membership/gms/messenger/JGroupsMessenger.java | 7 +-
.../internal/tcpserver/HostAddress.java| 56
.../internal/tcpserver/HostAndPort.java| 81 +++--
.../internal/tcpserver/InetSocketWrapper.java | 100 +
.../geode/internal/cache/wan/WANTestBase.java | 3 +-
44 files changed, 442 insertions(+), 291 deletions(-)
copy
geode-assembly/src/acceptanceTest/resources/org/apache/geode/{cache/wan/docker-compose.yml
=> client/sni/dual-server-docker-compose.yml} (73%)
copy
geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/{haproxy.cfg
=> dual-server-haproxy.cfg} (92%)
copy ci/resource-types/gce-instances-resource/Dockerfile =>
geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/create-regions.gfsh
(74%)
delete mode 100644
geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/geode-starter-2.gfsh
copy ci/resource-types/gce-instances-resource/build =>
geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/locator-maeve.gfsh
(62%)
mode change 100755 => 100644
copy ci/resource-types/gce-instances-resource/build =>
geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/server-clementine.gfsh
(61%)
mode change 100755 => 100644
copy ci/resource-types/gce-instances-resource/build =>
geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/server-dolores.gfsh
(62%)
mode change 100755 => 100644
create mode 100644
geode-tcp-server/src/main/java/org/apache/geode/distributed/internal/tcpserver/HostAddress.java
create mode 100644
geode-tcp-server/src/main/java/org/apache/geode/distributed/internal/tcpserver/InetSocketWrapper.java
[geode] branch develop updated (3b4d4ac -> 5567fe2)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 3b4d4ac GEODE-9153: Fix alpine-tools docker image (#6321) add 5567fe2 GEODE-9145: update CODEOWNERS (#6316) No new revisions were added by this update. Summary of changes: CODEOWNERS | 30 +++--- 1 file changed, 15 insertions(+), 15 deletions(-)
[geode] branch support/1.13 updated: GEODE-9128: Remove host name look-up from JGAddress (#6298)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 9371dc8 GEODE-9128: Remove host name look-up from JGAddress (#6298)
9371dc8 is described below
commit 9371dc8e071251412fd6633b1ad4d12c25305440
Author: Bruce Schuchardt
AuthorDate: Tue Apr 13 14:06:27 2021 -0700
GEODE-9128: Remove host name look-up from JGAddress (#6298)
toString() was including the host name of the address, which is overkill
for the limited use of JGAddress.
JGroupsMessenger's logging that included this class wasn't checking the
log level before invoking logDebug() and other methods. I've added
that.
(cherry picked from commit 8e1c7cdbab0fe806de14355af4dae938f8ddedda)
---
.../membership/gms/messenger/JGAddress.java| 4 +-
.../membership/gms/messenger/JGroupsMessenger.java | 53 --
2 files changed, 42 insertions(+), 15 deletions(-)
diff --git
a/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/messenger/JGAddress.java
b/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/messenger/JGAddress.java
index 2f75b7a..083033e 100755
---
a/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/messenger/JGAddress.java
+++
b/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/messenger/JGAddress.java
@@ -91,9 +91,9 @@ public class JGAddress extends UUID {
StringBuilder sb = new StringBuilder();
if (ip_addr == null)
- sb.append("");
+ sb.append("");
else {
- sb.append(ip_addr.getHostName());
+ sb.append(ip_addr);
}
if (vmViewId >= 0) {
sb.append("');
diff --git
a/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/messenger/JGroupsMessenger.java
b/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/messenger/JGroupsMessenger.java
index da67d36..a695a2d 100644
---
a/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/messenger/JGroupsMessenger.java
+++
b/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/messenger/JGroupsMessenger.java
@@ -333,7 +333,9 @@ public class JGroupsMessenger
implements Messenger<
try {
Object oldDSMembershipInfo = services.getConfig().getOldMembershipInfo();
if (oldDSMembershipInfo != null) {
-logger.debug("Reusing JGroups channel from previous system",
properties);
+if (logger.isDebugEnabled()) {
+ logger.debug("Reusing JGroups channel from previous system",
properties);
+}
MembershipInformationImpl oldInfo = (MembershipInformationImpl)
oldDSMembershipInfo;
myChannel = oldInfo.getChannel();
queuedMessagesFromReconnect = oldInfo.getQueuedMessages();
@@ -358,7 +360,9 @@ public class JGroupsMessenger
implements Messenger<
}
reconnecting = true;
} else {
-logger.debug("JGroups configuration: {}", properties);
+if (logger.isDebugEnabled()) {
+ logger.debug("JGroups configuration: {}", properties);
+}
checkForIPv6();
InputStream is = new
ByteArrayInputStream(properties.getBytes("UTF-8"));
@@ -410,7 +414,10 @@ public class JGroupsMessenger
implements Messenger<
private void checkForIPv6() throws Exception {
boolean preferIpV6Addr =
Boolean.getBoolean("java.net.preferIPv6Addresses");
if (!preferIpV6Addr) {
- logger.debug("forcing JGroups to think IPv4 is being used so it will
choose an IPv4 address");
+ if (logger.isDebugEnabled()) {
+logger
+.debug("forcing JGroups to think IPv4 is being used so it will
choose an IPv4 address");
+ }
Field m = org.jgroups.util.Util.class.getDeclaredField("ip_stack_type");
m.setAccessible(true);
m.set(null, org.jgroups.util.StackType.IPv4);
@@ -459,7 +466,9 @@ public class JGroupsMessenger
implements Messenger<
List mbrs =
v.getMembers().stream().map(JGAddress::new).collect(Collectors.toList());
ViewId vid = new ViewId(new JGAddress(v.getCoordinator()), v.getViewId());
View jgv = new View(vid, new ArrayList<>(mbrs));
-logger.trace("installing view into JGroups stack: {}", jgv);
+if (logger.isTraceEnabled()) {
+ logger.trace("installing view into JGroups stack: {}", jgv);
+}
this.myChannel.down(new Event(Event.VIEW_CHANGE, jgv));
addressesWithIoExceptionsProcessed.clear();
@@ -698,7 +707,9 @@ public class JGroupsMessenger
implements Messenger<
[geode] branch develop updated (dd956a9 -> 8e1c7cd)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from dd956a9 GEODE-9065 - Introduce ClasspathService abstraction and create LegacyClasspathServiceImpl. (#6223) add 8e1c7cd GEODE-9128: Remove host name look-up from JGAddress (#6298) No new revisions were added by this update. Summary of changes: .../membership/gms/messenger/JGAddress.java| 4 +- .../membership/gms/messenger/JGroupsMessenger.java | 53 -- 2 files changed, 42 insertions(+), 15 deletions(-)
[geode] branch develop updated: GEODE-8972: remove shunnedMembers collection from GMSMembership (#6089)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new c2fc107 GEODE-8972: remove shunnedMembers collection from
GMSMembership (#6089)
c2fc107 is described below
commit c2fc107bad1de221157072470e2a6ad426533f20
Author: Kamilla Aslami
AuthorDate: Tue Mar 9 09:59:13 2021 -0600
GEODE-8972: remove shunnedMembers collection from GMSMembership (#6089)
* GEODE-8972: remove shunnedMembers collection from GMSMembership
* Changes after the code review
---
.../internal/membership/api/Membership.java| 7 -
.../internal/membership/gms/GMSMembership.java | 168 ++---
2 files changed, 12 insertions(+), 163 deletions(-)
diff --git
a/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/api/Membership.java
b/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/api/Membership.java
index 91583d2..dfaa045 100644
---
a/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/api/Membership.java
+++
b/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/api/Membership.java
@@ -243,13 +243,6 @@ public interface Membership {
*/
Throwable getShutdownCause();
- /**
- * If this member is shunned, ensure that a warning is generated at least
once.
- *
- * @param mbr the member that may be shunned
- */
- void warnShun(ID mbr);
-
boolean addSurpriseMember(ID mbr);
/**
diff --git
a/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/GMSMembership.java
b/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/GMSMembership.java
index 3c05988..a119c02 100644
---
a/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/GMSMembership.java
+++
b/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/GMSMembership.java
@@ -258,19 +258,6 @@ public class GMSMembership
implements Membership shunnedMembers = new ConcurrentHashMap<>();
-
- /**
* Members that have sent a shutdown message. This is used to suppress
suspect processing that
* otherwise becomes pretty aggressive when a member is shutting down.
*/
@@ -286,15 +273,6 @@ public class GMSMembership
implements Membership shunnedAndWarnedMembers =
- Collections.newSetFromMap(new ConcurrentHashMap<>());
- /**
* The identities and birth-times of others that we have allowed into
membership at the
* distributed system level, but have not yet appeared in a view.
*
@@ -323,14 +301,6 @@ public class GMSMembership
implements Membership suspectedMembers = new ConcurrentHashMap<>();
/**
- * Length of time, in seconds, that a member is retained in the zombie set
- *
- * @see #shunnedMembers
- */
- private static final int SHUNNED_SUNSET = Integer
- .getInteger(GeodeGlossary.GEMFIRE_PREFIX + "shunned-member-timeout",
300).intValue();
-
- /**
* Set to true when the service should stop.
*/
private volatile boolean shutdownInProgress = false;
@@ -442,14 +412,7 @@ public class GMSMembership
implements Membership", m);
@@ -646,11 +609,10 @@ public class GMSMembership
implements Membership
implements Membership
implements Membership
implements Membership", m);
- }
-
/**
* Logic for processing a distribution message.
*
@@ -874,9 +820,7 @@ public class GMSMembership
implements Membership
implements Membership", m);
if (logger.isTraceEnabled()) {
logger.trace("Membership: Ignoring message from shunned member
<{}>:{}", m, msg);
}
@@ -1427,14 +1371,6 @@ public class GMSMembership
implements Membership
implements Membership now) {
- return true;
-}
-
-// Oh, it _is_ stale. Remove it while we're here.
-endShun(m);
-return false;
+final MembershipView view = latestView;
+return m.getVmViewId() <= view.getViewId() && !view.contains(m);
}
private boolean isShunnedOrNew(final ID m) {
+final MembershipView view = latestView;
+if (m.getVmViewId() <= view.getViewId() && view.contains(m)) {
+ return false;
+}
latestViewReadLock.lock();
try {
- return shunnedMembers.containsKey(m) || isNew(m);
+ return isShunned(m) || isNew(m);
} finally { // synchronized
latestViewReadLock.unlock();
}
@@ -1483,11 +1408,9 @@ public class GMSMembership
implements Membership
- * Unlike isShunned, this method will not cause expiry of a surprise member.
That must be done
+ * This method will not cause expiry of a surprise member. That must be done
* during view processing.
*
- * Like isShunned, this method holds the view lock
[geode] branch develop updated (b2b31ce -> 52e7411)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from b2b31ce GEODE-8864:finish implementation of Redis HScan Command (#5954) add 52e7411 GEODE-8979: CI Failure: SSLSocketHostNameVerificationIntegrationTest (#6079) No new revisions were added by this update. Summary of changes: ...LSocketHostNameVerificationIntegrationTest.java | 55 +++--- 1 file changed, 27 insertions(+), 28 deletions(-)
[geode] branch develop updated (57666f6 -> dc3ed8c)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 57666f6 GEODE-8986: bump benchmark baseline version (#6072) add dc3ed8c GEODE-8963: separate client/server compatibility from server/server version compatibility (#6050) No new revisions were added by this update. Summary of changes: .../client/internal/ClientSideHandshakeImpl.java | 5 +- .../geode/internal/cache/tier/ConnectionProxy.java | 11 +-- .../cache/tier/sockets/CommandInitializer.java | 4 +- .../cache/tier/sockets/ServerConnection.java | 8 ++- .../tier/sockets/ServerSideHandshakeFactory.java | 31 .../cache/tier/sockets/CommandInitializerTest.java | 17 +++-- .../tier/sockets/ClientServerMiscBCDUnitTest.java | 47 +++- .../geode/internal/serialization/KnownVersion.java | 84 ++ .../serialization/KnownVersionJUnitTest.java | 27 +++ 9 files changed, 164 insertions(+), 70 deletions(-)
[geode] branch develop updated (fc2fc9f -> 63a0574)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from fc2fc9f GEODE-8920: Modify debug logging to make it easier to trace a message (#6041) add 63a0574 Removing myself from a few code-owner directories (#6043) No new revisions were added by this update. Summary of changes: CODEOWNERS | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-)
[geode] branch develop updated (4ffaa38 -> fc2fc9f)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 4ffaa38 GEODE-8934: add incrby to supported commands (#6022) add fc2fc9f GEODE-8920: Modify debug logging to make it easier to trace a message (#6041) No new revisions were added by this update. Summary of changes: .../geode/distributed/internal/direct/DirectChannel.java | 10 +- .../main/java/org/apache/geode/internal/tcp/Connection.java| 2 +- 2 files changed, 6 insertions(+), 6 deletions(-)
[geode] branch develop updated (9eb0f74 -> d904c92)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 9eb0f74 GEODE-8924: Add VM restart tests for Redis and Spring sessions (#6011) add d904c92 GEODE-8922: Remove ProductUseLog (#6023) No new revisions were added by this update. Summary of changes: .../apache/geode/cache30/ReconnectDUnitTest.java | 19 --- .../internal/ProductUseLogDUnitTest.java | 147 .../internal/ProductUseLogJUnitTest.java | 107 --- .../distributed/internal/InternalLocator.java | 29 +--- .../geode/distributed/internal/ProductUseLog.java | 149 - .../geode/distributed/internal/ServerLocator.java | 52 +-- 6 files changed, 3 insertions(+), 500 deletions(-) delete mode 100644 geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ProductUseLogDUnitTest.java delete mode 100644 geode-core/src/integrationTest/java/org/apache/geode/distributed/internal/ProductUseLogJUnitTest.java delete mode 100644 geode-core/src/main/java/org/apache/geode/distributed/internal/ProductUseLog.java
[geode] branch develop updated: GEODE-8817: server hangs in cache.close() (#5987)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new 3ef1905 GEODE-8817: server hangs in cache.close() (#5987)
3ef1905 is described below
commit 3ef1905e8b88d99ac3071f94d830ae23010e3d2c
Author: Bruce Schuchardt
AuthorDate: Tue Feb 9 07:56:45 2021 -0800
GEODE-8817: server hangs in cache.close() (#5987)
* possible fix
* Revert "possible fix"
This reverts commit f8451da33d3568c1900bda2b7a4bb3f4fef660ba.
Testing showed that SSLSocket.close() still hangs when there is a
network failure and another thread is reading from the socket.
* revert socket timeout and introduce a SocketCloser executor
* SocketCloser cleanup
* change cleanup to be performed after the socket is closed
* bill's comments addressed
* bill's new comments
* removed useless method annotation
---
.../cache/tier/sockets/CacheClientProxyTest.java | 2 +-
.../internal/net/SocketCloserIntegrationTest.java | 28 +++--
.../internal/LonerDistributionManager.java | 137 -
.../apache/geode/internal/cache/tier/Acceptor.java | 3 +
.../internal/cache/tier/sockets/AcceptorImpl.java | 9 ++
.../cache/tier/sockets/CacheClientProxy.java | 4 +-
.../cache/tier/sockets/ServerConnection.java | 20 ++-
.../apache/geode/internal/net/SocketCloser.java| 72 ++-
8 files changed, 92 insertions(+), 183 deletions(-)
diff --git
a/geode-core/src/integrationTest/java/org/apache/geode/internal/cache/tier/sockets/CacheClientProxyTest.java
b/geode-core/src/integrationTest/java/org/apache/geode/internal/cache/tier/sockets/CacheClientProxyTest.java
index 0158946..079c54e 100644
---
a/geode-core/src/integrationTest/java/org/apache/geode/internal/cache/tier/sockets/CacheClientProxyTest.java
+++
b/geode-core/src/integrationTest/java/org/apache/geode/internal/cache/tier/sockets/CacheClientProxyTest.java
@@ -65,7 +65,7 @@ public class CacheClientProxyTest {
final InetAddress address = mock(InetAddress.class);
when(socket.getInetAddress()).thenReturn(address);
when(address.getHostAddress()).thenReturn("localhost");
-doNothing().when(sc).asyncClose(any(), eq("localhost"), eq(null));
+doNothing().when(sc).asyncClose(any(), eq("localhost"),
any(Runnable.class));
final ClientProxyMembershipID proxyID =
mock(ClientProxyMembershipID.class);
final DistributedMember member =
cache.getDistributedSystem().getDistributedMember();
diff --git
a/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SocketCloserIntegrationTest.java
b/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SocketCloserIntegrationTest.java
index b2db1b8..ee7ac5c 100644
---
a/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SocketCloserIntegrationTest.java
+++
b/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SocketCloserIntegrationTest.java
@@ -113,13 +113,17 @@ public class SocketCloserIntegrationTest {
* Verify that requesting an asyncClose on an already closed socket is a
noop.
*/
@Test
- public void testClosedSocket() throws Exception {
-final AtomicBoolean runnableCalled = new AtomicBoolean();
+ public void testOpenSocketCloser() {
+final AtomicBoolean beforeSocketCloseRunnableWasCalled = new
AtomicBoolean();
+final AtomicBoolean afterSocketCloseRunnableWasCalled = new
AtomicBoolean();
-Socket s = createClosableSocket();
-s.close();
-this.socketCloser.asyncClose(s, "A", () -> runnableCalled.set(true));
-await().until(() -> !runnableCalled.get());
+final Socket closableSocket = createClosableSocket();
+this.socketCloser.asyncClose(closableSocket, "A",
+() -> beforeSocketCloseRunnableWasCalled.set(true),
+() -> afterSocketCloseRunnableWasCalled.set(true));
+await().until(() -> beforeSocketCloseRunnableWasCalled.get());
+await().until(() -> closableSocket.isClosed());
+await().until(() -> afterSocketCloseRunnableWasCalled.get());
}
/**
@@ -127,12 +131,16 @@ public class SocketCloserIntegrationTest {
*/
@Test
public void testClosedSocketCloser() {
-final AtomicBoolean runnableCalled = new AtomicBoolean();
+final AtomicBoolean beforeSocketCloseRunnableWasCalled = new
AtomicBoolean();
+final AtomicBoolean afterSocketCloseRunnableWasCalled = new
AtomicBoolean();
final Socket closableSocket = createClosableSocket();
this.socketCloser.close();
-this.socketCloser.asyncClose(closableSocket, "A", () ->
runnableCalled.set(true));
-await()
-.until(() -> runnableCalled.get() && closableSocket.isClosed());
+
[geode] branch develop updated: GEODE-8767: NullPointerException in TCPConduit.getBufferPool due to conTable being null (#5962)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new f3f07a0 GEODE-8767: NullPointerException in TCPConduit.getBufferPool
due to conTable being null (#5962)
f3f07a0 is described below
commit f3f07a006c30f54fd4ae84309c4c4ede4af22f36
Author: Bruce Schuchardt
AuthorDate: Fri Jan 29 08:07:17 2021 -0800
GEODE-8767: NullPointerException in TCPConduit.getBufferPool due to
conTable being null (#5962)
A NPE was being thrown in TCPConduit.getBufferPool() because the conTable
instance
variable is nulled out when the TCPConduit is stopped. This commit
moves the buffer pool from TCPConduit's conTable object up to the enclosing
DirectChannel object. This removes the need for DirectChannel to ask
TCPConduit for the buffer pool.
---
.../distributed/internal/direct/DirectChannel.java | 22 ++-
.../apache/geode/internal/tcp/ConnectionTable.java | 2 +-
.../org/apache/geode/internal/tcp/TCPConduit.java | 11 +---
.../apache/geode/internal/tcp/TCPConduitTest.java | 31 +-
4 files changed, 48 insertions(+), 18 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/direct/DirectChannel.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/direct/DirectChannel.java
index c195655..fc8800f 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/direct/DirectChannel.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/direct/DirectChannel.java
@@ -49,6 +49,7 @@ import
org.apache.geode.distributed.internal.membership.api.MessageListener;
import org.apache.geode.internal.cache.DirectReplyMessage;
import org.apache.geode.internal.inet.LocalHostUtil;
import org.apache.geode.internal.logging.log4j.LogMarker;
+import org.apache.geode.internal.net.BufferPool;
import org.apache.geode.internal.tcp.BaseMsgStreamer;
import org.apache.geode.internal.tcp.ConnectExceptions;
import org.apache.geode.internal.tcp.Connection;
@@ -71,6 +72,8 @@ public class DirectChannel {
/** this is the conduit used for communications */
private final transient TCPConduit conduit;
private final ClusterDistributionManager dm;
+ private final DMStats stats;
+ private final BufferPool bufferPool;
private volatile boolean disconnected = true;
@@ -112,6 +115,8 @@ public class DirectChannel {
throws ConnectionException {
this.receiver = listener;
this.dm = dm;
+this.stats = dm.getStats();
+this.bufferPool = new BufferPool(stats);
DistributionConfig dc = dm.getConfig();
this.address = initAddress(dc);
@@ -137,7 +142,7 @@ public class DirectChannel {
props.setProperty("membership_port_range_start", "" + range[0]);
props.setProperty("membership_port_range_end", "" + range[1]);
- this.conduit = new TCPConduit(mgr, port, address, isBindAddress, this,
props);
+ this.conduit = new TCPConduit(mgr, port, address, isBindAddress, this,
bufferPool, props);
disconnected = false;
disconnectCompleted = false;
logger.info("GemFire P2P Listener started on {}",
@@ -184,6 +189,13 @@ public class DirectChannel {
/**
+ * Returns the buffer pool used for direct-memory byte buffers in this
DirectChannel
+ */
+ public BufferPool getBufferPool() {
+return bufferPool;
+ }
+
+ /**
* Sends a msg to a list of destinations. This code does some special
optimizations to stream
* large messages
*
@@ -295,7 +307,7 @@ public class DirectChannel {
List sentCons; // used for cons we sent to this time
final BaseMsgStreamer ms =
-MsgStreamer.create(cons, msg, directReply, stats,
getConduit().getBufferPool());
+MsgStreamer.create(cons, msg, directReply, stats, bufferPool);
try {
startTime = 0;
if (ackTimeout > 0) {
@@ -520,11 +532,7 @@ public class DirectChannel {
* Returns null if no stats available.
*/
public DMStats getDMStats() {
-if (dm != null) {
- return dm.getStats(); // fix for bug#34004
-} else {
- return null;
-}
+return stats;
}
/**
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/tcp/ConnectionTable.java
b/geode-core/src/main/java/org/apache/geode/internal/tcp/ConnectionTable.java
index df9a4a5..614b502 100644
---
a/geode-core/src/main/java/org/apache/geode/internal/tcp/ConnectionTable.java
+++
b/geode-core/src/main/java/org/apache/geode/internal/tcp/ConnectionTable.java
@@ -207,7 +207,7 @@ public class ConnectionTable {
threadConnectionMap = new ConcurrentHashMap();
p2pReaderThreadPool =
createThreadPoolForIO(conduit.getDM().getSystem().isShareSockets());
socketCloser =
[geode] branch develop updated (86693d5 -> 089c1ba)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 86693d5 GEODE-8799: Increase defaults for MAX_THREADS and MAX_PR_THREADS (#5862) add 089c1ba GEODE-7861: Improve error reporting in GMSJoinLeave.join() (#5839) No new revisions were added by this update. Summary of changes: .../apache/geode/distributed/LocatorDUnitTest.java | 3 +- .../gms/membership/GMSJoinLeaveJUnitTest.java | 100 - .../internal/membership/gms/GMSMembership.java | 7 +- .../membership/gms/interfaces/JoinLeave.java | 12 ++- .../membership/gms/membership/GMSJoinLeave.java| 76 ++-- 5 files changed, 143 insertions(+), 55 deletions(-)
[geode] branch support/1.12 updated: GEODE-5922: concurrency problems in SerialGatewaySenderQueue (#5870)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.12 by this push:
new ec2d79e GEODE-5922: concurrency problems in SerialGatewaySenderQueue
(#5870)
ec2d79e is described below
commit ec2d79ebfd0b85fc90203c3c114c4b3eec37602f
Author: Bruce Schuchardt
AuthorDate: Wed Jan 6 09:56:54 2021 -0800
GEODE-5922: concurrency problems in SerialGatewaySenderQueue (#5870)
reverting 3ed37a754d789bb52cf190db23088e819955fd58
(cherry picked from commit ab16f68c7c3b121af00c3aca64a92d9809cb6019)
---
.../cache/wan/serial/SerialGatewaySenderQueue.java | 124 -
1 file changed, 49 insertions(+), 75 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueue.java
b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueue.java
index 0e19ebb..437455b 100644
---
a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueue.java
+++
b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueue.java
@@ -26,7 +26,6 @@ import java.util.Set;
import java.util.concurrent.LinkedBlockingDeque;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
-import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.apache.logging.log4j.Logger;
@@ -141,13 +140,6 @@ public class SerialGatewaySenderQueue implements
RegionQueue {
private boolean isDiskSynchronous;
/**
- * The writeLock of this concurrent lock is used to protect access to the
queue.
- * It is implemented as a fair lock to ensure FIFO ordering of queueing
attempts.
- * Otherwise threads can be unfairly delayed.
- */
- private ReentrantReadWriteLock lock = new ReentrantReadWriteLock(true);
-
- /**
* The Map mapping the regionName->key to the queue key. This
index allows fast
* updating of entries in the queue for conflation.
*/
@@ -219,23 +211,18 @@ public class SerialGatewaySenderQueue implements
RegionQueue {
}
@Override
- public boolean put(Object event) throws CacheException {
-lock.writeLock().lock();
-try {
- GatewaySenderEventImpl eventImpl = (GatewaySenderEventImpl) event;
- final Region r = eventImpl.getRegion();
- final boolean isPDXRegion =
- (r instanceof DistributedRegion &&
r.getName().equals(PeerTypeRegistration.REGION_NAME));
- final boolean isWbcl =
-
this.regionName.startsWith(AsyncEventQueueImpl.ASYNC_EVENT_QUEUE_PREFIX);
- if (!(isPDXRegion && isWbcl)) {
-putAndGetKey(event);
-return true;
- }
- return false;
-} finally {
- lock.writeLock().unlock();
+ public synchronized boolean put(Object event) throws CacheException {
+GatewaySenderEventImpl eventImpl = (GatewaySenderEventImpl) event;
+final Region r = eventImpl.getRegion();
+final boolean isPDXRegion =
+(r instanceof DistributedRegion &&
r.getName().equals(PeerTypeRegistration.REGION_NAME));
+final boolean isWbcl =
+
this.regionName.startsWith(AsyncEventQueueImpl.ASYNC_EVENT_QUEUE_PREFIX);
+if (!(isPDXRegion && isWbcl)) {
+ putAndGetKey(event);
+ return true;
}
+return false;
}
private long putAndGetKey(Object object) throws CacheException {
@@ -259,6 +246,7 @@ public class SerialGatewaySenderQueue implements
RegionQueue {
return key.longValue();
}
+
@Override
public AsyncEvent take() throws CacheException {
// Unsupported since we have no callers.
@@ -280,49 +268,44 @@ public class SerialGatewaySenderQueue implements
RegionQueue {
* have peeked. If the entry was not peeked, this method will silently
return.
*/
@Override
- public void remove() throws CacheException {
-lock.writeLock().lock();
+ public synchronized void remove() throws CacheException {
+if (this.peekedIds.isEmpty()) {
+ return;
+}
+Long key = this.peekedIds.remove();
try {
- if (this.peekedIds.isEmpty()) {
-return;
- }
- Long key = this.peekedIds.remove();
- try {
-// Increment the head key
-updateHeadKey(key.longValue());
-removeIndex(key);
-// Remove the entry at that key with a callback arg signifying it is
-// a WAN queue so that AbstractRegionEntry.destroy can get the value
-// even if it has been evicted to disk. In the normal case, the
-// AbstractRegionEntry.destroy only gets the value in the VM.
-this.region.localDestroy(key, WAN_QUEUE_TOKEN);
-this.stats.decQueueSize();
-
- } catch (EntryNotFoundException ok) {
-// this is acceptable because the conflation can remove entries
[geode] branch support/1.13 updated: GEODE-5922: concurrency problems in SerialGatewaySenderQueue (#5870)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 2dfef38 GEODE-5922: concurrency problems in SerialGatewaySenderQueue
(#5870)
2dfef38 is described below
commit 2dfef383298035d6a65b939c5d0edef5060bcd46
Author: Bruce Schuchardt
AuthorDate: Wed Jan 6 09:56:54 2021 -0800
GEODE-5922: concurrency problems in SerialGatewaySenderQueue (#5870)
reverting 3ed37a754d789bb52cf190db23088e819955fd58
(cherry picked from commit ab16f68c7c3b121af00c3aca64a92d9809cb6019)
---
.../cache/wan/serial/SerialGatewaySenderQueue.java | 124 -
1 file changed, 49 insertions(+), 75 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueue.java
b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueue.java
index 243fe73..42ff881 100644
---
a/geode-core/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueue.java
+++
b/geode-core/src/main/java/org/apache/geode/internal/cache/wan/serial/SerialGatewaySenderQueue.java
@@ -25,7 +25,6 @@ import java.util.Set;
import java.util.concurrent.LinkedBlockingDeque;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
-import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.apache.logging.log4j.Logger;
@@ -141,13 +140,6 @@ public class SerialGatewaySenderQueue implements
RegionQueue {
private boolean isDiskSynchronous;
/**
- * The writeLock of this concurrent lock is used to protect access to the
queue.
- * It is implemented as a fair lock to ensure FIFO ordering of queueing
attempts.
- * Otherwise threads can be unfairly delayed.
- */
- private ReentrantReadWriteLock lock = new ReentrantReadWriteLock(true);
-
- /**
* The Map mapping the regionName->key to the queue key. This
index allows fast
* updating of entries in the queue for conflation.
*/
@@ -219,23 +211,18 @@ public class SerialGatewaySenderQueue implements
RegionQueue {
}
@Override
- public boolean put(Object event) throws CacheException {
-lock.writeLock().lock();
-try {
- GatewaySenderEventImpl eventImpl = (GatewaySenderEventImpl) event;
- final Region r = eventImpl.getRegion();
- final boolean isPDXRegion =
- (r instanceof DistributedRegion &&
r.getName().equals(PeerTypeRegistration.REGION_NAME));
- final boolean isWbcl =
-
this.regionName.startsWith(AsyncEventQueueImpl.ASYNC_EVENT_QUEUE_PREFIX);
- if (!(isPDXRegion && isWbcl)) {
-putAndGetKey(event);
-return true;
- }
- return false;
-} finally {
- lock.writeLock().unlock();
+ public synchronized boolean put(Object event) throws CacheException {
+GatewaySenderEventImpl eventImpl = (GatewaySenderEventImpl) event;
+final Region r = eventImpl.getRegion();
+final boolean isPDXRegion =
+(r instanceof DistributedRegion &&
r.getName().equals(PeerTypeRegistration.REGION_NAME));
+final boolean isWbcl =
+
this.regionName.startsWith(AsyncEventQueueImpl.ASYNC_EVENT_QUEUE_PREFIX);
+if (!(isPDXRegion && isWbcl)) {
+ putAndGetKey(event);
+ return true;
}
+return false;
}
private long putAndGetKey(Object object) throws CacheException {
@@ -259,6 +246,7 @@ public class SerialGatewaySenderQueue implements
RegionQueue {
return key.longValue();
}
+
@Override
public AsyncEvent take() throws CacheException {
// Unsupported since we have no callers.
@@ -280,49 +268,44 @@ public class SerialGatewaySenderQueue implements
RegionQueue {
* have peeked. If the entry was not peeked, this method will silently
return.
*/
@Override
- public void remove() throws CacheException {
-lock.writeLock().lock();
+ public synchronized void remove() throws CacheException {
+if (this.peekedIds.isEmpty()) {
+ return;
+}
+Long key = this.peekedIds.remove();
try {
- if (this.peekedIds.isEmpty()) {
-return;
- }
- Long key = this.peekedIds.remove();
- try {
-// Increment the head key
-updateHeadKey(key.longValue());
-removeIndex(key);
-// Remove the entry at that key with a callback arg signifying it is
-// a WAN queue so that AbstractRegionEntry.destroy can get the value
-// even if it has been evicted to disk. In the normal case, the
-// AbstractRegionEntry.destroy only gets the value in the VM.
-this.region.localDestroy(key, WAN_QUEUE_TOKEN);
-this.stats.decQueueSize();
-
- } catch (EntryNotFoundException ok) {
-// this is acceptable because the conflation can remove entries
[geode] branch develop updated (1636a42 -> ab16f68)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 1636a42 GEODE-8804: Partitioned region description doc improvement (#5877) add ab16f68 GEODE-5922: concurrency problems in SerialGatewaySenderQueue (#5870) No new revisions were added by this update. Summary of changes: .../cache/wan/serial/SerialGatewaySenderQueue.java | 162 + 1 file changed, 68 insertions(+), 94 deletions(-)
[geode] branch support/1.12 updated: GEODE-8697: Propagate ForcedDisconnectException to the user application (#5739)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.12 by this push:
new 929852d GEODE-8697: Propagate ForcedDisconnectException to the user
application (#5739)
929852d is described below
commit 929852d16ac1b44c1c51d6b2c61d643bf4bdac64
Author: Bruce Schuchardt
AuthorDate: Mon Nov 16 07:51:14 2020 -0800
GEODE-8697: Propagate ForcedDisconnectException to the user application
(#5739)
* GEODE-8697: Propagate ForcedDisconnectException to the user application
avoid setting MemberDisconnectedException as a rootCause in
ClusterDistributionManager, even temporarily, as it's an internal
exception that should not be exposed to applications.
* addressing Ernie's comments
(cherry picked from commit 403e19c0a2b85369274e8254c16e0ae508b82e94)
---
.../internal/ClusterDistributionManager.java | 50 --
.../internal/ClusterDistributionManagerTest.java | 40 +
2 files changed, 67 insertions(+), 23 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
index eda8aef..612ed9c 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
@@ -2206,6 +2206,11 @@ public class ClusterDistributionManager implements
DistributionManager {
}
}
+ private List getMembershipTestHooks() {
+return membershipTestHooks;
+ }
+
+
@Override
public Set getAdminMemberSet() {
return distribution.getView().getMembers().stream()
@@ -2280,7 +2285,7 @@ public class ClusterDistributionManager implements
DistributionManager {
* This is the listener implementation for responding from events from the
Membership Manager.
*
*/
- private class DMListener implements
+ static class DMListener implements
org.apache.geode.distributed.internal.membership.api.MembershipListener
{
ClusterDistributionManager dm;
@@ -2290,26 +2295,25 @@ public class ClusterDistributionManager implements
DistributionManager {
@Override
public void membershipFailure(String reason, Throwable t) {
- exceptionInThreads = true;
- rootCause = t;
- if (rootCause != null && !(rootCause instanceof
ForcedDisconnectException)) {
-logger.info("cluster membership failed due to ", rootCause);
-rootCause = new ForcedDisconnectException(rootCause.getMessage());
+ dm.exceptionInThreads = true;
+ Throwable cause = t;
+ if (cause != null && !(cause instanceof ForcedDisconnectException)) {
+logger.info("cluster membership failed due to ", cause);
+cause = new ForcedDisconnectException(cause.getMessage());
}
+ dm.setRootCause(cause);
try {
-if (membershipTestHooks != null) {
- List l = membershipTestHooks;
- for (final MembershipTestHook aL : l) {
-MembershipTestHook dml = aL;
-dml.beforeMembershipFailure(reason, rootCause);
+List testHooks = dm.getMembershipTestHooks();
+if (testHooks != null) {
+ for (final MembershipTestHook testHook : testHooks) {
+testHook.beforeMembershipFailure(reason, cause);
}
}
-getSystem().disconnect(reason, true);
-if (membershipTestHooks != null) {
- List l = membershipTestHooks;
- for (final MembershipTestHook aL : l) {
-MembershipTestHook dml = aL;
-dml.afterMembershipFailure(reason, rootCause);
+dm.getSystem().disconnect(reason, true);
+testHooks = dm.getMembershipTestHooks();
+if (testHooks != null) {
+ for (final MembershipTestHook testHook : testHooks) {
+testHook.afterMembershipFailure(reason, cause);
}
}
} catch (RuntimeException re) {
@@ -2341,7 +2345,7 @@ public class ClusterDistributionManager implements
DistributionManager {
@Override
public void memberDeparted(InternalDistributedMember theId, boolean
crashed, String reason) {
try {
-boolean wasAdmin = getAdminMemberSet().contains(theId);
+boolean wasAdmin = dm.getAdminMemberSet().contains(theId);
if (wasAdmin) {
// Pretend we received an AdminConsoleDisconnectMessage from the
console that
// is no longer in the JavaGroup view.
@@ -2354,9 +2358,9 @@ public class ClusterDistributionManager implements
DistributionManager {
message.setIgnoreAlertListenerRemovalFailure(true); //
[geode] branch support/1.12 updated: GEODE-8721: member that should become coordinator never detects loss of current coordinator (#5758)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.12 by this push:
new 0f23c76 GEODE-8721: member that should become coordinator never
detects loss of current coordinator (#5758)
0f23c76 is described below
commit 0f23c7643d8410cba0badbb126946869d36f4523
Author: Bruce Schuchardt
AuthorDate: Mon Nov 30 08:08:58 2020 -0800
GEODE-8721: member that should become coordinator never detects loss of
current coordinator (#5758)
* GEODE-8721: member that should become coordinator never detects loss of
current coordinator
If a server is in the process of performing an availability check on
another server we shouldn't update the contact timestamp for
the suspected server based on gossip from another server. Doing so
will make the availability check pass and send out another gossip
message that would likewise update their timestamps for the suspected
server, perpetuating the notion that the suspect is still around.
* added VisibleForTesting
(cherry picked from commit b7afc604b9c2fafe4388dcdcf05fc7ec49c0ce86)
---
.../gms/fd/GMSHealthMonitorJUnitTest.java | 51 ++
.../membership/gms/fd/GMSHealthMonitor.java| 19 ++--
2 files changed, 66 insertions(+), 4 deletions(-)
diff --git
a/geode-core/src/integrationTest/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitorJUnitTest.java
b/geode-core/src/integrationTest/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitorJUnitTest.java
index a146f1a..e458dcf 100644
---
a/geode-core/src/integrationTest/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitorJUnitTest.java
+++
b/geode-core/src/integrationTest/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitorJUnitTest.java
@@ -671,6 +671,29 @@ public class GMSHealthMonitorJUnitTest {
}
@Test
+ public void testFinalCheckInProgressPreemptsLivenessGossip() throws
Exception {
+// if a member is undergoing a final check we should not accept another
member's
+// gossip about the suspect being alive and should not update the contact
timestamp
+// because that interferes with the final check
+useGMSHealthMonitorTestClass = true;
+simulateHeartbeatInGMSHealthMonitorTestClass = false;
+
+GMSMembershipView v = installAView();
+setFailureDetectionPorts(v);
+
+MemberIdentifier memberToCheck = gmsHealthMonitor.getNextNeighbor();
+GMSHealthMonitorTest testMonitor = (GMSHealthMonitorTest) gmsHealthMonitor;
+
+// set an old contact timestamp for the suspect, tell the monitor that an
availability
+// check succeeded and then make sure it didn't update the timestamp for
the suspect
+final long timestamp = testMonitor.establishCurrentTime() - 2000;
+testMonitor.setContactTimestamp(memberToCheck, timestamp);
+testMonitor.addMemberInFinalCheck(memberToCheck);
+testMonitor.processMessage(new FinalCheckPassedMessage(null,
memberToCheck));
+
assertThat(testMonitor.getContactTimestamp(memberToCheck)).isEqualTo(timestamp);
+ }
+
+ @Test
public void testFailedSelfCheckRemovesMemberAsSuspect() throws Exception {
useGMSHealthMonitorTestClass = true;
simulateHeartbeatInGMSHealthMonitorTestClass = false;
@@ -1057,6 +1080,34 @@ public class GMSHealthMonitorJUnitTest {
return serverSocket;
}
}
+
+/**
+ * when a suspect is undergoing an availability check its identifier will
+ * be in the membersInFinalCheck collection
+ */
+public void addMemberInFinalCheck(MemberIdentifier memberToCheck) {
+ membersInFinalCheck.add(memberToCheck);
+}
+
+public void setContactTimestamp(MemberIdentifier memberToCheck, long
timestamp) {
+ memberTimeStamps.put(memberToCheck, new TimeStamp(timestamp));
+}
+
+public long getContactTimestamp(MemberIdentifier memberIdentifier) {
+ return ((TimeStamp) memberTimeStamps.get(memberIdentifier)).getTime();
+}
+
+/**
+ * Establish the currentTimeStamp for the health monitor. This is the
timestamp
+ * used in contactedBy() updates and is usually established by the Monitor
thread
+ * in GMSHealthMonitor but is initially zero.
+ *
+ * @return the timestamp
+ */
+public long establishCurrentTime() {
+ currentTimeStamp = System.currentTimeMillis();
+ return currentTimeStamp;
+}
}
public class TrickySocket extends ServerSocket {
diff --git
a/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitor.java
b/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitor.java
index 803f4de..73af605 100644
---
a/geode-membership/src/main/java/org
[geode] branch support/1.13 updated: GEODE-8697: Propagate ForcedDisconnectException to the user application (#5739)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new acae4bb GEODE-8697: Propagate ForcedDisconnectException to the user
application (#5739)
acae4bb is described below
commit acae4bb3fa8e4a20f0bad8eb63259eb1b6d79fa3
Author: Bruce Schuchardt
AuthorDate: Mon Nov 16 07:51:14 2020 -0800
GEODE-8697: Propagate ForcedDisconnectException to the user application
(#5739)
* GEODE-8697: Propagate ForcedDisconnectException to the user application
avoid setting MemberDisconnectedException as a rootCause in
ClusterDistributionManager, even temporarily, as it's an internal
exception that should not be exposed to applications.
* addressing Ernie's comments
(cherry picked from commit 403e19c0a2b85369274e8254c16e0ae508b82e94)
---
.../internal/ClusterDistributionManager.java | 50 --
.../internal/ClusterDistributionManagerTest.java | 40 +
2 files changed, 67 insertions(+), 23 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
index d17ae80..4196965 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
@@ -2206,6 +2206,11 @@ public class ClusterDistributionManager implements
DistributionManager {
}
}
+ private List getMembershipTestHooks() {
+return membershipTestHooks;
+ }
+
+
@Override
public Set getAdminMemberSet() {
return distribution.getView().getMembers().stream()
@@ -2280,7 +2285,7 @@ public class ClusterDistributionManager implements
DistributionManager {
* This is the listener implementation for responding from events from the
Membership Manager.
*
*/
- private class DMListener implements
+ static class DMListener implements
org.apache.geode.distributed.internal.membership.api.MembershipListener
{
ClusterDistributionManager dm;
@@ -2290,26 +2295,25 @@ public class ClusterDistributionManager implements
DistributionManager {
@Override
public void membershipFailure(String reason, Throwable t) {
- exceptionInThreads = true;
- rootCause = t;
- if (rootCause != null && !(rootCause instanceof
ForcedDisconnectException)) {
-logger.info("cluster membership failed due to ", rootCause);
-rootCause = new ForcedDisconnectException(rootCause.getMessage());
+ dm.exceptionInThreads = true;
+ Throwable cause = t;
+ if (cause != null && !(cause instanceof ForcedDisconnectException)) {
+logger.info("cluster membership failed due to ", cause);
+cause = new ForcedDisconnectException(cause.getMessage());
}
+ dm.setRootCause(cause);
try {
-if (membershipTestHooks != null) {
- List l = membershipTestHooks;
- for (final MembershipTestHook aL : l) {
-MembershipTestHook dml = aL;
-dml.beforeMembershipFailure(reason, rootCause);
+List testHooks = dm.getMembershipTestHooks();
+if (testHooks != null) {
+ for (final MembershipTestHook testHook : testHooks) {
+testHook.beforeMembershipFailure(reason, cause);
}
}
-getSystem().disconnect(reason, true);
-if (membershipTestHooks != null) {
- List l = membershipTestHooks;
- for (final MembershipTestHook aL : l) {
-MembershipTestHook dml = aL;
-dml.afterMembershipFailure(reason, rootCause);
+dm.getSystem().disconnect(reason, true);
+testHooks = dm.getMembershipTestHooks();
+if (testHooks != null) {
+ for (final MembershipTestHook testHook : testHooks) {
+testHook.afterMembershipFailure(reason, cause);
}
}
} catch (RuntimeException re) {
@@ -2341,7 +2345,7 @@ public class ClusterDistributionManager implements
DistributionManager {
@Override
public void memberDeparted(InternalDistributedMember theId, boolean
crashed, String reason) {
try {
-boolean wasAdmin = getAdminMemberSet().contains(theId);
+boolean wasAdmin = dm.getAdminMemberSet().contains(theId);
if (wasAdmin) {
// Pretend we received an AdminConsoleDisconnectMessage from the
console that
// is no longer in the JavaGroup view.
@@ -2354,9 +2358,9 @@ public class ClusterDistributionManager implements
DistributionManager {
message.setIgnoreAlertListenerRemovalFailure(true); //
[geode] branch support/1.13 updated: GEODE-8721: member that should become coordinator never detects loss of current coordinator (#5758)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 956c9aa GEODE-8721: member that should become coordinator never
detects loss of current coordinator (#5758)
956c9aa is described below
commit 956c9aa66328a8bed14892f16d230f8d4f6c8105
Author: Bruce Schuchardt
AuthorDate: Mon Nov 30 08:08:58 2020 -0800
GEODE-8721: member that should become coordinator never detects loss of
current coordinator (#5758)
* GEODE-8721: member that should become coordinator never detects loss of
current coordinator
If a server is in the process of performing an availability check on
another server we shouldn't update the contact timestamp for
the suspected server based on gossip from another server. Doing so
will make the availability check pass and send out another gossip
message that would likewise update their timestamps for the suspected
server, perpetuating the notion that the suspect is still around.
* added VisibleForTesting
(cherry picked from commit b7afc604b9c2fafe4388dcdcf05fc7ec49c0ce86)
---
.../gms/fd/GMSHealthMonitorJUnitTest.java | 51 ++
.../membership/gms/fd/GMSHealthMonitor.java| 19 ++--
2 files changed, 66 insertions(+), 4 deletions(-)
diff --git
a/geode-membership/src/integrationTest/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitorJUnitTest.java
b/geode-membership/src/integrationTest/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitorJUnitTest.java
index f734e81..2aaf2f5 100644
---
a/geode-membership/src/integrationTest/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitorJUnitTest.java
+++
b/geode-membership/src/integrationTest/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitorJUnitTest.java
@@ -634,6 +634,29 @@ public class GMSHealthMonitorJUnitTest {
}
@Test
+ public void testFinalCheckInProgressPreemptsLivenessGossip() throws
Exception {
+// if a member is undergoing a final check we should not accept another
member's
+// gossip about the suspect being alive and should not update the contact
timestamp
+// because that interferes with the final check
+useGMSHealthMonitorTestClass = true;
+simulateHeartbeatInGMSHealthMonitorTestClass = false;
+
+GMSMembershipView v = installAView();
+setFailureDetectionPorts(v);
+
+MemberIdentifier memberToCheck = gmsHealthMonitor.getNextNeighbor();
+GMSHealthMonitorTest testMonitor = (GMSHealthMonitorTest) gmsHealthMonitor;
+
+// set an old contact timestamp for the suspect, tell the monitor that an
availability
+// check succeeded and then make sure it didn't update the timestamp for
the suspect
+final long timestamp = testMonitor.establishCurrentTime() - 2000;
+testMonitor.setContactTimestamp(memberToCheck, timestamp);
+testMonitor.addMemberInFinalCheck(memberToCheck);
+testMonitor.processMessage(new FinalCheckPassedMessage(null,
memberToCheck));
+
assertThat(testMonitor.getContactTimestamp(memberToCheck)).isEqualTo(timestamp);
+ }
+
+ @Test
public void testFailedSelfCheckRemovesMemberAsSuspect() throws Exception {
useGMSHealthMonitorTestClass = true;
simulateHeartbeatInGMSHealthMonitorTestClass = false;
@@ -1019,6 +1042,34 @@ public class GMSHealthMonitorJUnitTest {
return serverSocket;
}
}
+
+/**
+ * when a suspect is undergoing an availability check its identifier will
+ * be in the membersInFinalCheck collection
+ */
+public void addMemberInFinalCheck(MemberIdentifier memberToCheck) {
+ membersInFinalCheck.add(memberToCheck);
+}
+
+public void setContactTimestamp(MemberIdentifier memberToCheck, long
timestamp) {
+ memberTimeStamps.put(memberToCheck, new TimeStamp(timestamp));
+}
+
+public long getContactTimestamp(MemberIdentifier memberIdentifier) {
+ return ((TimeStamp) memberTimeStamps.get(memberIdentifier)).getTime();
+}
+
+/**
+ * Establish the currentTimeStamp for the health monitor. This is the
timestamp
+ * used in contactedBy() updates and is usually established by the Monitor
thread
+ * in GMSHealthMonitor but is initially zero.
+ *
+ * @return the timestamp
+ */
+public long establishCurrentTime() {
+ currentTimeStamp = System.currentTimeMillis();
+ return currentTimeStamp;
+}
}
public class TrickySocket extends ServerSocket {
diff --git
a/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitor.java
b/geode-membership/src/main/java/org/apache/geode/distributed/internal/membership/gms/fd/GMSHealthMonitor.java
index bd95e57..2590e23 100644
---
a/geode
[geode] branch develop updated: GEODE-7489: DistributionArchUnitTest is running out of memory for some users (#5783)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new 1bca728 GEODE-7489: DistributionArchUnitTest is running out of memory
for some users (#5783)
1bca728 is described below
commit 1bca72803df43184457c03fdde012139f99ea285
Author: Bruce Schuchardt
AuthorDate: Tue Dec 1 08:05:07 2020 -0800
GEODE-7489: DistributionArchUnitTest is running out of memory for some
users (#5783)
CoreOnlyUsesMembershipAPIArchUnitTest already restricts use of the
membership API to certain classes and does so in a way that avoids using
too much memory. Consequently we're removing this test that needs
upwards of 1.5gb of heap since it performs nearly the same test but
for a specific class in the API package instead of the whole package.
---
.../internal/DistributionArchUnitTest.java | 54 --
1 file changed, 54 deletions(-)
diff --git
a/geode-core/src/integrationTest/java/org/apache/geode/distributed/internal/DistributionArchUnitTest.java
b/geode-core/src/integrationTest/java/org/apache/geode/distributed/internal/DistributionArchUnitTest.java
deleted file mode 100644
index 4d40157..000
---
a/geode-core/src/integrationTest/java/org/apache/geode/distributed/internal/DistributionArchUnitTest.java
+++ /dev/null
@@ -1,54 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
contributor license
- * agreements. See the NOTICE file distributed with this work for additional
information regarding
- * copyright ownership. The ASF licenses this file to You under the Apache
License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the
License. You may obtain a
- * copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express
- * or implied. See the License for the specific language governing permissions
and limitations under
- * the License.
- */
-package org.apache.geode.distributed.internal;
-
-import static
com.tngtech.archunit.core.domain.JavaClass.Predicates.resideInAPackage;
-import static com.tngtech.archunit.core.domain.JavaClass.Predicates.type;
-import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.classes;
-
-import com.tngtech.archunit.core.importer.ImportOption;
-import com.tngtech.archunit.junit.AnalyzeClasses;
-import com.tngtech.archunit.junit.ArchTest;
-import com.tngtech.archunit.junit.ArchUnitRunner;
-import com.tngtech.archunit.junit.CacheMode;
-import com.tngtech.archunit.lang.ArchRule;
-import org.junit.Ignore;
-import org.junit.runner.RunWith;
-
-import org.apache.geode.distributed.internal.membership.MembershipJUnitTest;
-import org.apache.geode.distributed.internal.membership.api.Membership;
-
-@RunWith(ArchUnitRunner.class)
-@AnalyzeClasses(packages = "org.apache.geode", cacheMode = CacheMode.PER_CLASS,
-importOptions = ImportOption.DoNotIncludeArchives.class)
-@Ignore("Disabling until we can fix the memory usage of this test")
-public class DistributionArchUnitTest {
-
- @ArchTest
- public static final ArchRule
membershipShouldOnlyBeAccessedThroughDistributionClass = classes()
- .that(type(Membership.class))
- .should()
- .onlyBeAccessed()
- .byClassesThat(type(Distribution.class)
- .or(type(MembershipJUnitTest.class)) // another integrationTest
- .or(type(DistributionImpl.MyDCReceiver.class))
-
.or(resideInAPackage("org.apache.geode.distributed.internal.membership.api.."))
-
.or(resideInAPackage("org.apache.geode.distributed.internal.membership.gms.."))
- .or(resideInAPackage("org.apache.geode.internal.tcp.."))
-
.or(resideInAPackage("org.apache.geode.distributed.internal.direct.."))
- .or(type(DistributionImpl.class))
-
- );
-}
[geode] branch develop updated (b7afc60 -> a6942e7)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from b7afc60 GEODE-8721: member that should become coordinator never detects loss of current coordinator (#5758) add a6942e7 GEODE-8664: Nest errors in DistributionImpl.start (#5751) No new revisions were added by this update. Summary of changes: .../apache/geode/distributed/LocatorDUnitTest.java | 3 +-- .../distributed/LocatorUDPSecurityDUnitTest.java | 2 +- .../distributed/internal/DistributionImpl.java | 4 ++-- .../distributed/internal/DistributionTest.java | 26 +- 4 files changed, 29 insertions(+), 6 deletions(-)
[geode] branch develop updated (217be41 -> b7afc60)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 217be41 GEODE-8496: top up dependency updates (#5772) add b7afc60 GEODE-8721: member that should become coordinator never detects loss of current coordinator (#5758) No new revisions were added by this update. Summary of changes: .../gms/fd/GMSHealthMonitorJUnitTest.java | 51 ++ .../membership/gms/fd/GMSHealthMonitor.java| 19 ++-- 2 files changed, 66 insertions(+), 4 deletions(-)
[geode] branch develop updated (49e4700 -> 986733e)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 49e4700 GEODE-8732: Update Tomcat9 module to publish to Maven (#5762) add 986733e GEODE-7489: Disable DistributionArchUnitTest (#5768) No new revisions were added by this update. Summary of changes: .../org/apache/geode/distributed/internal/DistributionArchUnitTest.java | 2 ++ 1 file changed, 2 insertions(+)
[geode] branch develop updated: GEODE-8697: Propagate ForcedDisconnectException to the user application (#5739)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new 403e19c GEODE-8697: Propagate ForcedDisconnectException to the user
application (#5739)
403e19c is described below
commit 403e19c0a2b85369274e8254c16e0ae508b82e94
Author: Bruce Schuchardt
AuthorDate: Mon Nov 16 07:51:14 2020 -0800
GEODE-8697: Propagate ForcedDisconnectException to the user application
(#5739)
* GEODE-8697: Propagate ForcedDisconnectException to the user application
avoid setting MemberDisconnectedException as a rootCause in
ClusterDistributionManager, even temporarily, as it's an internal
exception that should not be exposed to applications.
* addressing Ernie's comments
---
.../internal/ClusterDistributionManager.java | 50 --
.../internal/ClusterDistributionManagerTest.java | 40 +
2 files changed, 67 insertions(+), 23 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
index a2af201..bc41c87 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
@@ -2209,6 +2209,11 @@ public class ClusterDistributionManager implements
DistributionManager {
}
}
+ private List getMembershipTestHooks() {
+return membershipTestHooks;
+ }
+
+
@Override
public Set getAdminMemberSet() {
return distribution.getView().getMembers().stream()
@@ -2283,7 +2288,7 @@ public class ClusterDistributionManager implements
DistributionManager {
* This is the listener implementation for responding from events from the
Membership Manager.
*
*/
- private class DMListener implements
+ static class DMListener implements
org.apache.geode.distributed.internal.membership.api.MembershipListener
{
ClusterDistributionManager dm;
@@ -2293,26 +2298,25 @@ public class ClusterDistributionManager implements
DistributionManager {
@Override
public void membershipFailure(String reason, Throwable t) {
- exceptionInThreads = true;
- rootCause = t;
- if (rootCause != null && !(rootCause instanceof
ForcedDisconnectException)) {
-logger.info("cluster membership failed due to ", rootCause);
-rootCause = new ForcedDisconnectException(rootCause.getMessage());
+ dm.exceptionInThreads = true;
+ Throwable cause = t;
+ if (cause != null && !(cause instanceof ForcedDisconnectException)) {
+logger.info("cluster membership failed due to ", cause);
+cause = new ForcedDisconnectException(cause.getMessage());
}
+ dm.setRootCause(cause);
try {
-if (membershipTestHooks != null) {
- List l = membershipTestHooks;
- for (final MembershipTestHook aL : l) {
-MembershipTestHook dml = aL;
-dml.beforeMembershipFailure(reason, rootCause);
+List testHooks = dm.getMembershipTestHooks();
+if (testHooks != null) {
+ for (final MembershipTestHook testHook : testHooks) {
+testHook.beforeMembershipFailure(reason, cause);
}
}
-getSystem().disconnect(reason, true);
-if (membershipTestHooks != null) {
- List l = membershipTestHooks;
- for (final MembershipTestHook aL : l) {
-MembershipTestHook dml = aL;
-dml.afterMembershipFailure(reason, rootCause);
+dm.getSystem().disconnect(reason, true);
+testHooks = dm.getMembershipTestHooks();
+if (testHooks != null) {
+ for (final MembershipTestHook testHook : testHooks) {
+testHook.afterMembershipFailure(reason, cause);
}
}
} catch (RuntimeException re) {
@@ -2344,7 +2348,7 @@ public class ClusterDistributionManager implements
DistributionManager {
@Override
public void memberDeparted(InternalDistributedMember theId, boolean
crashed, String reason) {
try {
-boolean wasAdmin = getAdminMemberSet().contains(theId);
+boolean wasAdmin = dm.getAdminMemberSet().contains(theId);
if (wasAdmin) {
// Pretend we received an AdminConsoleDisconnectMessage from the
console that
// is no longer in the JavaGroup view.
@@ -2357,9 +2361,9 @@ public class ClusterDistributionManager implements
DistributionManager {
message.setIgnoreAlertListenerRemovalFailure(true); // we don't know
if it was a listener
// so
[geode] branch develop updated (7a55cba -> 99e7a13)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 7a55cba GEODE-8672: No need in token mode if concurrencyChecksEnabled (#5746) add 99e7a13 GEODE-8670: ReconnectDUnitTest is hiding a NullPointerException (#5744) No new revisions were added by this update. Summary of changes: .../apache/geode/cache30/ReconnectDUnitTest.java | 14 +++-- .../internal/InternalLocatorIntegrationTest.java | 59 +++--- .../distributed/internal/InternalLocator.java | 15 -- 3 files changed, 62 insertions(+), 26 deletions(-)
[geode] branch support/1.12 updated: GEODE-6008: CI Failure: ClientServerHostNameVerificationDistributedTe… (#4839)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.12 by this push:
new 3a89ae5 GEODE-6008: CI Failure:
ClientServerHostNameVerificationDistributedTe… (#4839)
3a89ae5 is described below
commit 3a89ae5448df7b534c82fa594b195057a974d85c
Author: Bruce Schuchardt
AuthorDate: Tue Mar 24 11:07:27 2020 -0700
GEODE-6008: CI Failure: ClientServerHostNameVerificationDistributedTe…
(#4839)
* GEODE-6008: CI Failure:
ClientServerHostNameVerificationDistributedTest.expectConnectionFailureWhenNoHostNameInServerKey
failed
The "Message distribution has terminated" failure is caused by an
unreported NullPointerException in a residual reader thread introduced
in the fix for GEODE-7727. That fix caused a thread to say alive in a
peer-to-peer tcp/ip Connection in order to clean up the receiving side
of a socket. The Connection shutdown method close() method, however,
releases the Connection's input buffer and nulls out the field. The
reader thread then threw an NPE that was caught and caused the
"Message distribution has terminated" message, which is picked up as a
suspect string by the testing infrastructure.
This problem is also seen in GEODE-7894, GEODE-7871, GEODE-7873 and
GEODE-7806.
The fix is to record the fact that a residual reader thread exists and
avoid releasing the Connection's input buffer when the connection is
closed. This lets the reader thread do the cleanup.
While testing the fix I found that the NioSslEngine was throwing an
IllegalStateException when the reader thread tried to use it in this
same situation. This exception wasn't being caught and caused more
suspect strings to be logged. I've changed this to a checked exception
that is already handled by the reader thread.
ClientServerHostNameVerificationDistributedTest also wasn't working on
my Mac due to its /etc/hosts configuration. I changed the test to allow
the IP address selected by LocalHostUtil to be a valid client/server
address for the SSL certificates it generates.
* fixed failing test due to change in exceptions in NioSslEngine
(cherry picked from commit 1dfc496da34916297258b881c5606944d26bfb8b)
---
...tServerHostNameVerificationDistributedTest.java | 4 +++
.../geode/internal/tcp/CloseConnectionTest.java| 13
.../org/apache/geode/internal/tcp/Connection.java | 38 ++
3 files changed, 48 insertions(+), 7 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/cache/client/internal/ClientServerHostNameVerificationDistributedTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/cache/client/internal/ClientServerHostNameVerificationDistributedTest.java
index 8f1db5f..3ef5ac0 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/cache/client/internal/ClientServerHostNameVerificationDistributedTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/cache/client/internal/ClientServerHostNameVerificationDistributedTest.java
@@ -39,6 +39,7 @@ import
org.apache.geode.cache.client.NoAvailableServersException;
import org.apache.geode.cache.ssl.CertStores;
import org.apache.geode.cache.ssl.CertificateBuilder;
import org.apache.geode.cache.ssl.CertificateMaterial;
+import org.apache.geode.internal.inet.LocalHostUtil;
import org.apache.geode.internal.net.SocketCreatorFactory;
import org.apache.geode.test.dunit.IgnoredException;
import org.apache.geode.test.dunit.rules.ClusterStartupRule;
@@ -82,6 +83,7 @@ public class ClientServerHostNameVerificationDistributedTest {
.sanDnsName(InetAddress.getLoopbackAddress().getHostName())
.sanDnsName(InetAddress.getLocalHost().getHostName())
.sanDnsName(InetAddress.getLocalHost().getCanonicalHostName())
+.sanIpAddress(LocalHostUtil.getLocalHost())
.sanIpAddress(InetAddress.getLocalHost())
.sanIpAddress(InetAddress.getByName("0.0.0.0")) // to pass on windows
.generate();
@@ -91,6 +93,7 @@ public class ClientServerHostNameVerificationDistributedTest {
.issuedBy(ca)
.sanDnsName(InetAddress.getLocalHost().getHostName())
.sanDnsName(InetAddress.getLocalHost().getCanonicalHostName())
+.sanIpAddress(LocalHostUtil.getLocalHost())
.sanIpAddress(InetAddress.getLocalHost())
.generate();
@@ -162,6 +165,7 @@ public class
ClientServerHostNameVerificationDistributedTest {
.sanDnsName(InetAddress.getLoopbackAddress().getHostName())
.sanDnsName(InetAddress.getLocalHost().getHostName())
.sanDnsName(InetAddress.getLocalHost().getCanonicalHostName())
+.sanIpAddress(LocalHostUtil.getLocal
[geode] branch support/1.12 updated: GEODE-6008: CI Failure: ClientServerHostNameVerificationDistributedTe… (#4839)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.12 by this push:
new 3a89ae5 GEODE-6008: CI Failure:
ClientServerHostNameVerificationDistributedTe… (#4839)
3a89ae5 is described below
commit 3a89ae5448df7b534c82fa594b195057a974d85c
Author: Bruce Schuchardt
AuthorDate: Tue Mar 24 11:07:27 2020 -0700
GEODE-6008: CI Failure: ClientServerHostNameVerificationDistributedTe…
(#4839)
* GEODE-6008: CI Failure:
ClientServerHostNameVerificationDistributedTest.expectConnectionFailureWhenNoHostNameInServerKey
failed
The "Message distribution has terminated" failure is caused by an
unreported NullPointerException in a residual reader thread introduced
in the fix for GEODE-7727. That fix caused a thread to say alive in a
peer-to-peer tcp/ip Connection in order to clean up the receiving side
of a socket. The Connection shutdown method close() method, however,
releases the Connection's input buffer and nulls out the field. The
reader thread then threw an NPE that was caught and caused the
"Message distribution has terminated" message, which is picked up as a
suspect string by the testing infrastructure.
This problem is also seen in GEODE-7894, GEODE-7871, GEODE-7873 and
GEODE-7806.
The fix is to record the fact that a residual reader thread exists and
avoid releasing the Connection's input buffer when the connection is
closed. This lets the reader thread do the cleanup.
While testing the fix I found that the NioSslEngine was throwing an
IllegalStateException when the reader thread tried to use it in this
same situation. This exception wasn't being caught and caused more
suspect strings to be logged. I've changed this to a checked exception
that is already handled by the reader thread.
ClientServerHostNameVerificationDistributedTest also wasn't working on
my Mac due to its /etc/hosts configuration. I changed the test to allow
the IP address selected by LocalHostUtil to be a valid client/server
address for the SSL certificates it generates.
* fixed failing test due to change in exceptions in NioSslEngine
(cherry picked from commit 1dfc496da34916297258b881c5606944d26bfb8b)
---
...tServerHostNameVerificationDistributedTest.java | 4 +++
.../geode/internal/tcp/CloseConnectionTest.java| 13
.../org/apache/geode/internal/tcp/Connection.java | 38 ++
3 files changed, 48 insertions(+), 7 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/cache/client/internal/ClientServerHostNameVerificationDistributedTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/cache/client/internal/ClientServerHostNameVerificationDistributedTest.java
index 8f1db5f..3ef5ac0 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/cache/client/internal/ClientServerHostNameVerificationDistributedTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/cache/client/internal/ClientServerHostNameVerificationDistributedTest.java
@@ -39,6 +39,7 @@ import
org.apache.geode.cache.client.NoAvailableServersException;
import org.apache.geode.cache.ssl.CertStores;
import org.apache.geode.cache.ssl.CertificateBuilder;
import org.apache.geode.cache.ssl.CertificateMaterial;
+import org.apache.geode.internal.inet.LocalHostUtil;
import org.apache.geode.internal.net.SocketCreatorFactory;
import org.apache.geode.test.dunit.IgnoredException;
import org.apache.geode.test.dunit.rules.ClusterStartupRule;
@@ -82,6 +83,7 @@ public class ClientServerHostNameVerificationDistributedTest {
.sanDnsName(InetAddress.getLoopbackAddress().getHostName())
.sanDnsName(InetAddress.getLocalHost().getHostName())
.sanDnsName(InetAddress.getLocalHost().getCanonicalHostName())
+.sanIpAddress(LocalHostUtil.getLocalHost())
.sanIpAddress(InetAddress.getLocalHost())
.sanIpAddress(InetAddress.getByName("0.0.0.0")) // to pass on windows
.generate();
@@ -91,6 +93,7 @@ public class ClientServerHostNameVerificationDistributedTest {
.issuedBy(ca)
.sanDnsName(InetAddress.getLocalHost().getHostName())
.sanDnsName(InetAddress.getLocalHost().getCanonicalHostName())
+.sanIpAddress(LocalHostUtil.getLocalHost())
.sanIpAddress(InetAddress.getLocalHost())
.generate();
@@ -162,6 +165,7 @@ public class
ClientServerHostNameVerificationDistributedTest {
.sanDnsName(InetAddress.getLoopbackAddress().getHostName())
.sanDnsName(InetAddress.getLocalHost().getHostName())
.sanDnsName(InetAddress.getLocalHost().getCanonicalHostName())
+.sanIpAddress(LocalHostUtil.getLocal
[geode] branch develop updated (39db792 -> 7da8f9b)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 39db792 GEODE-8682: Bump jetty from 9.4.32.v20200930 to 9.4.33.v20201020 (#5700) add 7da8f9b GEODE-8681: peer-to-peer message loss due to sending connection closing with TLS enabled (#5699) No new revisions were added by this update. Summary of changes: .../geode/ClusterCommunicationsDUnitTest.java | 87 -- .../apache/geode/internal/net/NioSslEngine.java| 11 ++- .../geode/internal/net/NioSslEngineTest.java | 28 ++- 3 files changed, 115 insertions(+), 11 deletions(-)
[geode] branch develop updated (39db792 -> 7da8f9b)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 39db792 GEODE-8682: Bump jetty from 9.4.32.v20200930 to 9.4.33.v20201020 (#5700) add 7da8f9b GEODE-8681: peer-to-peer message loss due to sending connection closing with TLS enabled (#5699) No new revisions were added by this update. Summary of changes: .../geode/ClusterCommunicationsDUnitTest.java | 87 -- .../apache/geode/internal/net/NioSslEngine.java| 11 ++- .../geode/internal/net/NioSslEngineTest.java | 28 ++- 3 files changed, 115 insertions(+), 11 deletions(-)
[geode] branch develop updated (39db792 -> 7da8f9b)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 39db792 GEODE-8682: Bump jetty from 9.4.32.v20200930 to 9.4.33.v20201020 (#5700) add 7da8f9b GEODE-8681: peer-to-peer message loss due to sending connection closing with TLS enabled (#5699) No new revisions were added by this update. Summary of changes: .../geode/ClusterCommunicationsDUnitTest.java | 87 -- .../apache/geode/internal/net/NioSslEngine.java| 11 ++- .../geode/internal/net/NioSslEngineTest.java | 28 ++- 3 files changed, 115 insertions(+), 11 deletions(-)
[geode] branch develop updated (39db792 -> 7da8f9b)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 39db792 GEODE-8682: Bump jetty from 9.4.32.v20200930 to 9.4.33.v20201020 (#5700) add 7da8f9b GEODE-8681: peer-to-peer message loss due to sending connection closing with TLS enabled (#5699) No new revisions were added by this update. Summary of changes: .../geode/ClusterCommunicationsDUnitTest.java | 87 -- .../apache/geode/internal/net/NioSslEngine.java| 11 ++- .../geode/internal/net/NioSslEngineTest.java | 28 ++- 3 files changed, 115 insertions(+), 11 deletions(-)
[geode] branch develop updated (39db792 -> 7da8f9b)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 39db792 GEODE-8682: Bump jetty from 9.4.32.v20200930 to 9.4.33.v20201020 (#5700) add 7da8f9b GEODE-8681: peer-to-peer message loss due to sending connection closing with TLS enabled (#5699) No new revisions were added by this update. Summary of changes: .../geode/ClusterCommunicationsDUnitTest.java | 87 -- .../apache/geode/internal/net/NioSslEngine.java| 11 ++- .../geode/internal/net/NioSslEngineTest.java | 28 ++- 3 files changed, 115 insertions(+), 11 deletions(-)
[geode] branch support/1.12 updated: GEODE-8584: Message transmission fails with IllegalStateException in socket i/o code (#5605)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.12 by this push:
new 2ccaf7b GEODE-8584: Message transmission fails with
IllegalStateException in socket i/o code (#5605)
2ccaf7b is described below
commit 2ccaf7b8fe1cb490a3d3001b61b12ea52c5ed1d0
Author: Bruce Schuchardt
AuthorDate: Thu Oct 8 14:09:42 2020 -0700
GEODE-8584: Message transmission fails with IllegalStateException in socket
i/o code (#5605)
* GEODE-8584: Message transmission fails with IllegalStateException in
socket i/o code
Add appropriate synchronization when using ioFilter's buffers.
to do: add testing
to do: document the need for synchronization in the NioFilter interface.
* revised synchronization across all uses of NioFilter, added test
* remove dangling debug logging
* fix pmd problem
* fix pmd problem
* remove unnecessary volatile modifier
(cherry picked from commit f4d44d658a6273d7de27479a67c05117310205a7)
---
.../org/apache/geode/internal/net/BufferPool.java | 7 +-
.../org/apache/geode/internal/net/NioFilter.java | 22 +-
.../apache/geode/internal/net/NioSslEngine.java| 18 +-
.../org/apache/geode/internal/tcp/Connection.java | 333 +++--
.../org/apache/geode/internal/tcp/MsgReader.java | 82 ++---
.../geode/internal/net/NioSslEngineTest.java | 26 +-
6 files changed, 266 insertions(+), 222 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
index e119250..3938422 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
@@ -255,10 +255,15 @@ public class BufferPool {
* "slice" of the buffer having the requested capacity and hand that out
instead.
* When we put the buffer back in the pool we need to find the original,
non-sliced,
* buffer. This is held in DirectBuffer in its "attachment" field, which is
a public
- * method, though DirectBuffer is package-private.
+ * method, though DirectBuffer is package-private. This method is visible
for use
+ * in debugging and testing. For debugging, invoke this method if you need
to see
+ * the non-sliced buffer for some reason, such as logging its hashcode.
*/
@VisibleForTesting
public ByteBuffer getPoolableBuffer(ByteBuffer buffer) {
+if (!buffer.isDirect()) {
+ return buffer;
+}
ByteBuffer result = buffer;
if (parentOfSliceMethod == null) {
Class clazz = buffer.getClass();
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/NioFilter.java
b/geode-core/src/main/java/org/apache/geode/internal/net/NioFilter.java
index 01556dc..9c437ad 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/NioFilter.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/NioFilter.java
@@ -21,7 +21,13 @@ import java.nio.channels.SocketChannel;
/**
* Prior to transmitting a buffer or processing a received buffer
* a NioFilter should be called to wrap (transmit) or unwrap (received)
- * the buffer in case SSL is being used.
+ * the buffer in case SSL is being used.
+ * Implementations of this class may not be thread-safe in regard to
+ * the buffers their methods return. These may be internal state that,
+ * if used concurrently by multiple threads could cause corruption.
+ * Appropriate external synchronization must be used in order to provide
+ * thread-safety. Do this by invoking getSynchObject() and synchronizing on
+ * the returned object while using the buffer.
*/
public interface NioFilter {
@@ -75,6 +81,10 @@ public interface NioFilter {
}
}
+ default boolean isClosed() {
+return false;
+ }
+
/**
* invoke this method when you are done using the NioFilter
*
@@ -84,9 +94,15 @@ public interface NioFilter {
}
/**
- * returns the unwrapped byte buffer associated with the given wrapped buffer
+ * returns the unwrapped byte buffer associated with the given wrapped
buffer.
*/
ByteBuffer getUnwrappedBuffer(ByteBuffer wrappedBuffer);
-
+ /**
+ * returns an object to be used in synchronizing on the use of buffers
returned by
+ * a NioFilter.
+ */
+ default Object getSynchObject() {
+return this;
+ }
}
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/NioSslEngine.java
b/geode-core/src/main/java/org/apache/geode/internal/net/NioSslEngine.java
index 2d55fa3..2398b35 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/NioSslEngine.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/NioSslEngine.java
@@ -40,6 +40,7 @@ import java
[geode] branch support/1.13 updated: GEODE-8584: Message transmission fails with IllegalStateException in socket i/o code (#5605)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new fb901bc GEODE-8584: Message transmission fails with
IllegalStateException in socket i/o code (#5605)
fb901bc is described below
commit fb901bc6e3b4502b9ddf82849b82217adcef9629
Author: Bruce Schuchardt
AuthorDate: Thu Oct 8 14:09:42 2020 -0700
GEODE-8584: Message transmission fails with IllegalStateException in socket
i/o code (#5605)
* GEODE-8584: Message transmission fails with IllegalStateException in
socket i/o code
Add appropriate synchronization when using ioFilter's buffers.
to do: add testing
to do: document the need for synchronization in the NioFilter interface.
* revised synchronization across all uses of NioFilter, added test
* remove dangling debug logging
* fix pmd problem
* fix pmd problem
* remove unnecessary volatile modifier
(cherry picked from commit f4d44d658a6273d7de27479a67c05117310205a7)
---
.../org/apache/geode/internal/net/BufferPool.java | 7 +-
.../org/apache/geode/internal/net/NioFilter.java | 22 +-
.../apache/geode/internal/net/NioSslEngine.java| 17 +-
.../org/apache/geode/internal/tcp/Connection.java | 333 +++--
.../org/apache/geode/internal/tcp/MsgReader.java | 82 ++---
.../geode/internal/net/NioSslEngineTest.java | 26 +-
6 files changed, 265 insertions(+), 222 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
index 20b138e..74a2e4d 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
@@ -326,10 +326,15 @@ public class BufferPool {
* "slice" of the buffer having the requested capacity and hand that out
instead.
* When we put the buffer back in the pool we need to find the original,
non-sliced,
* buffer. This is held in DirectBuffer in its "attachment" field, which is
a public
- * method, though DirectBuffer is package-private.
+ * method, though DirectBuffer is package-private. This method is visible
for use
+ * in debugging and testing. For debugging, invoke this method if you need
to see
+ * the non-sliced buffer for some reason, such as logging its hashcode.
*/
@VisibleForTesting
public ByteBuffer getPoolableBuffer(ByteBuffer buffer) {
+if (!buffer.isDirect()) {
+ return buffer;
+}
ByteBuffer result = buffer;
if (parentOfSliceMethod == null) {
Class clazz = buffer.getClass();
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/NioFilter.java
b/geode-core/src/main/java/org/apache/geode/internal/net/NioFilter.java
index 01556dc..9c437ad 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/NioFilter.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/NioFilter.java
@@ -21,7 +21,13 @@ import java.nio.channels.SocketChannel;
/**
* Prior to transmitting a buffer or processing a received buffer
* a NioFilter should be called to wrap (transmit) or unwrap (received)
- * the buffer in case SSL is being used.
+ * the buffer in case SSL is being used.
+ * Implementations of this class may not be thread-safe in regard to
+ * the buffers their methods return. These may be internal state that,
+ * if used concurrently by multiple threads could cause corruption.
+ * Appropriate external synchronization must be used in order to provide
+ * thread-safety. Do this by invoking getSynchObject() and synchronizing on
+ * the returned object while using the buffer.
*/
public interface NioFilter {
@@ -75,6 +81,10 @@ public interface NioFilter {
}
}
+ default boolean isClosed() {
+return false;
+ }
+
/**
* invoke this method when you are done using the NioFilter
*
@@ -84,9 +94,15 @@ public interface NioFilter {
}
/**
- * returns the unwrapped byte buffer associated with the given wrapped buffer
+ * returns the unwrapped byte buffer associated with the given wrapped
buffer.
*/
ByteBuffer getUnwrappedBuffer(ByteBuffer wrappedBuffer);
-
+ /**
+ * returns an object to be used in synchronizing on the use of buffers
returned by
+ * a NioFilter.
+ */
+ default Object getSynchObject() {
+return this;
+ }
}
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/NioSslEngine.java
b/geode-core/src/main/java/org/apache/geode/internal/net/NioSslEngine.java
index d948ae7..6f32501 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/NioSslEngine.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/NioSslEngine.java
@@ -40,6 +40,7 @@ import java
[geode] branch develop updated (bcdf3ca -> f4d44d6)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from bcdf3ca GEODE-8582: Redis SCAN returns internal server error (#5603) add f4d44d6 GEODE-8584: Message transmission fails with IllegalStateException in socket i/o code (#5605) No new revisions were added by this update. Summary of changes: .../org/apache/geode/internal/net/BufferPool.java | 7 +- .../org/apache/geode/internal/net/NioFilter.java | 22 +- .../apache/geode/internal/net/NioSslEngine.java| 17 +- .../org/apache/geode/internal/tcp/Connection.java | 333 +++-- .../org/apache/geode/internal/tcp/MsgReader.java | 82 ++--- .../geode/internal/net/NioSslEngineTest.java | 26 +- 6 files changed, 265 insertions(+), 222 deletions(-)
[geode] branch support/1.12 updated: GEODE-8238: message loss during shutdown in Shutdown Hook when JVM exits (#5232)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.12 by this push:
new dff5b2b GEODE-8238: message loss during shutdown in Shutdown Hook
when JVM exits (#5232)
dff5b2b is described below
commit dff5b2b938b73455426db8ad895a84d2a8599538
Author: Bruce Schuchardt
AuthorDate: Wed Jun 10 11:34:25 2020 -0700
GEODE-8238: message loss during shutdown in Shutdown Hook when JVM exits
(#5232)
Remove invocation of removeEndpoint when a shared/unordered connection
shuts down. Endpoint cleanup is already initiated by DistributionImpl
during membership view installation, so it isn't needed here.
(cherry picked from commit ece3a5a6c045075509a5097054e33a64d4194fae)
Also resolved compilation errors after the cherry-pick.
---
.../geode/internal/tcp/TCPConduitDUnitTest.java| 22 ++
.../distributed/internal/DistributionImpl.java | 4
.../distributed/internal/direct/DirectChannel.java | 6 +-
.../org/apache/geode/internal/tcp/Connection.java | 7 ---
.../apache/geode/internal/tcp/ConnectionTable.java | 8
.../org/apache/geode/internal/tcp/TCPConduit.java | 2 +-
6 files changed, 40 insertions(+), 9 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
index 9462c72..698e7fe 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
@@ -16,6 +16,7 @@ package org.apache.geode.internal.tcp;
import static org.apache.geode.test.awaitility.GeodeAwaitility.await;
import static
org.apache.geode.test.util.ResourceUtils.createTempFileFromResource;
+import static org.assertj.core.api.Assertions.assertThat;
import java.io.File;
import java.io.IOException;
@@ -30,8 +31,10 @@ import org.junit.runners.Parameterized;
import org.apache.geode.distributed.ConfigurationProperties;
import org.apache.geode.distributed.DistributedSystem;
import org.apache.geode.distributed.Locator;
+import org.apache.geode.distributed.internal.DistributionImpl;
import org.apache.geode.distributed.internal.InternalDistributedSystem;
import org.apache.geode.distributed.internal.SerialAckedMessage;
+import
org.apache.geode.distributed.internal.membership.InternalDistributedMember;
import org.apache.geode.test.dunit.DistributedTestCase;
import org.apache.geode.test.dunit.DistributedTestUtils;
import org.apache.geode.test.dunit.IgnoredException;
@@ -93,6 +96,25 @@ public class TCPConduitDUnitTest extends DistributedTestCase
{
Thread.sleep(5000);
+// ensure that the closing of a shared/unordered connection to another
node does not
+// remove all connections for that node
+InternalDistributedMember otherMember =
+(InternalDistributedMember)
system.getAllOtherMembers().iterator().next();
+DistributionImpl distribution =
+(DistributionImpl) system.getDistributionManager().getDistribution();
+final ConnectionTable connectionTable =
+distribution.getDirectChannel().getConduit().getConTable();
+
+assertThat(connectionTable.hasReceiversFor(otherMember)).isTrue();
+
+Connection sharedUnordered = connectionTable.get(otherMember, false,
+System.currentTimeMillis(), 15000, 0);
+sharedUnordered.requestClose("for testing");
+// the sender connection has been closed so we should only have 2 senders
now
+assertThat(ConnectionTable.getNumSenderSharedConnections()).isEqualTo(2);
+// there should still be receivers for the other member - endpoint not
removed!
+assertThat(connectionTable.hasReceiversFor(otherMember)).isTrue();
+
try {
await("for message to be sent").until(() -> {
final SerialAckedMessage serialAckedMessage = new SerialAckedMessage();
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionImpl.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionImpl.java
index a89d23f..2fad2e6 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionImpl.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionImpl.java
@@ -794,6 +794,10 @@ public class DistributionImpl implements Distribution {
return result;
}
+ public DirectChannel getDirectChannel() {
+return directChannel;
+ }
+
/**
* Insert our own MessageReceiver between us and the direct channel, in
order to correctly filter
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/direct/DirectChannel.java
b/geode
[geode] branch support/1.12 updated: GEODE-8238: message loss during shutdown in Shutdown Hook when JVM exits (#5232)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.12 by this push:
new dff5b2b GEODE-8238: message loss during shutdown in Shutdown Hook
when JVM exits (#5232)
dff5b2b is described below
commit dff5b2b938b73455426db8ad895a84d2a8599538
Author: Bruce Schuchardt
AuthorDate: Wed Jun 10 11:34:25 2020 -0700
GEODE-8238: message loss during shutdown in Shutdown Hook when JVM exits
(#5232)
Remove invocation of removeEndpoint when a shared/unordered connection
shuts down. Endpoint cleanup is already initiated by DistributionImpl
during membership view installation, so it isn't needed here.
(cherry picked from commit ece3a5a6c045075509a5097054e33a64d4194fae)
Also resolved compilation errors after the cherry-pick.
---
.../geode/internal/tcp/TCPConduitDUnitTest.java| 22 ++
.../distributed/internal/DistributionImpl.java | 4
.../distributed/internal/direct/DirectChannel.java | 6 +-
.../org/apache/geode/internal/tcp/Connection.java | 7 ---
.../apache/geode/internal/tcp/ConnectionTable.java | 8
.../org/apache/geode/internal/tcp/TCPConduit.java | 2 +-
6 files changed, 40 insertions(+), 9 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
index 9462c72..698e7fe 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
@@ -16,6 +16,7 @@ package org.apache.geode.internal.tcp;
import static org.apache.geode.test.awaitility.GeodeAwaitility.await;
import static
org.apache.geode.test.util.ResourceUtils.createTempFileFromResource;
+import static org.assertj.core.api.Assertions.assertThat;
import java.io.File;
import java.io.IOException;
@@ -30,8 +31,10 @@ import org.junit.runners.Parameterized;
import org.apache.geode.distributed.ConfigurationProperties;
import org.apache.geode.distributed.DistributedSystem;
import org.apache.geode.distributed.Locator;
+import org.apache.geode.distributed.internal.DistributionImpl;
import org.apache.geode.distributed.internal.InternalDistributedSystem;
import org.apache.geode.distributed.internal.SerialAckedMessage;
+import
org.apache.geode.distributed.internal.membership.InternalDistributedMember;
import org.apache.geode.test.dunit.DistributedTestCase;
import org.apache.geode.test.dunit.DistributedTestUtils;
import org.apache.geode.test.dunit.IgnoredException;
@@ -93,6 +96,25 @@ public class TCPConduitDUnitTest extends DistributedTestCase
{
Thread.sleep(5000);
+// ensure that the closing of a shared/unordered connection to another
node does not
+// remove all connections for that node
+InternalDistributedMember otherMember =
+(InternalDistributedMember)
system.getAllOtherMembers().iterator().next();
+DistributionImpl distribution =
+(DistributionImpl) system.getDistributionManager().getDistribution();
+final ConnectionTable connectionTable =
+distribution.getDirectChannel().getConduit().getConTable();
+
+assertThat(connectionTable.hasReceiversFor(otherMember)).isTrue();
+
+Connection sharedUnordered = connectionTable.get(otherMember, false,
+System.currentTimeMillis(), 15000, 0);
+sharedUnordered.requestClose("for testing");
+// the sender connection has been closed so we should only have 2 senders
now
+assertThat(ConnectionTable.getNumSenderSharedConnections()).isEqualTo(2);
+// there should still be receivers for the other member - endpoint not
removed!
+assertThat(connectionTable.hasReceiversFor(otherMember)).isTrue();
+
try {
await("for message to be sent").until(() -> {
final SerialAckedMessage serialAckedMessage = new SerialAckedMessage();
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionImpl.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionImpl.java
index a89d23f..2fad2e6 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionImpl.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionImpl.java
@@ -794,6 +794,10 @@ public class DistributionImpl implements Distribution {
return result;
}
+ public DirectChannel getDirectChannel() {
+return directChannel;
+ }
+
/**
* Insert our own MessageReceiver between us and the direct channel, in
order to correctly filter
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/direct/DirectChannel.java
b/geode
[geode] branch develop updated: GEODE-8542: java.lang.IllegalStateException: tcp message exceeded max… (#5562)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new b439d33 GEODE-8542: java.lang.IllegalStateException: tcp message
exceeded max… (#5562)
b439d33 is described below
commit b439d3301dc15a81a9917b05ca4bd0717d1718bc
Author: Bruce Schuchardt
AuthorDate: Tue Sep 29 10:17:56 2020 -0700
GEODE-8542: java.lang.IllegalStateException: tcp message exceeded max…
(#5562)
* GEODE-8542: java.lang.IllegalStateException: tcp message exceeded max
size of 16777215
Limit the size of message chunks to the maximum message size allowed
by org.apache.geode.internal.tcp.Connection.
* implementing the change that Bill requested
---
.../org/apache/geode/internal/tcp/MsgStreamer.java | 9 ++-
.../apache/geode/internal/tcp/MsgStreamerTest.java | 29 --
2 files changed, 35 insertions(+), 3 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/tcp/MsgStreamer.java
b/geode-core/src/main/java/org/apache/geode/internal/tcp/MsgStreamer.java
index 74bc829..9bf8e4f 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/tcp/MsgStreamer.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/tcp/MsgStreamer.java
@@ -27,6 +27,7 @@ import it.unimi.dsi.fastutil.objects.Object2ObjectOpenHashMap;
import it.unimi.dsi.fastutil.objects.ObjectIterator;
import org.apache.geode.DataSerializer;
+import org.apache.geode.annotations.VisibleForTesting;
import org.apache.geode.distributed.internal.DMStats;
import org.apache.geode.distributed.internal.DistributionMessage;
import org.apache.geode.internal.Assert;
@@ -129,7 +130,8 @@ public class MsgStreamer extends OutputStream
this.stats = stats;
this.msg = msg;
this.cons = cons;
-this.buffer = bufferPool.acquireDirectSenderBuffer(sendBufferSize);
+int bufferSize = Math.min(sendBufferSize, Connection.MAX_MSG_SIZE);
+this.buffer = bufferPool.acquireDirectSenderBuffer(bufferSize);
this.buffer.clear();
this.buffer.position(Connection.MSG_HEADER_BYTES);
this.msgId = MsgIdGenerator.NO_MSG_ID;
@@ -349,6 +351,11 @@ public class MsgStreamer extends OutputStream
this.buffer.position(Connection.MSG_HEADER_BYTES);
}
+ @VisibleForTesting
+ protected ByteBuffer getBuffer() {
+return buffer;
+ }
+
@Override
public void close() throws IOException {
try {
diff --git
a/geode-core/src/test/java/org/apache/geode/internal/tcp/MsgStreamerTest.java
b/geode-core/src/test/java/org/apache/geode/internal/tcp/MsgStreamerTest.java
index 60d4578..0c41a0a 100644
---
a/geode-core/src/test/java/org/apache/geode/internal/tcp/MsgStreamerTest.java
+++
b/geode-core/src/test/java/org/apache/geode/internal/tcp/MsgStreamerTest.java
@@ -77,6 +77,31 @@ public class MsgStreamerTest {
verify(pool, times(2)).releaseSenderBuffer(isA(ByteBuffer.class));
}
+ @Test
+ public void streamerRespectsMaxMessageSize() {
+InternalDistributedMember member1;
+member1 = new InternalDistributedMember("localhost", 1234);
+
+DistributionMessage message = new SerialAckedMessage();
+message.setRecipients(Arrays.asList(member1));
+
+when(connection1.getRemoteAddress()).thenReturn(member1);
+when(connection1.getRemoteVersion()).thenReturn(KnownVersion.CURRENT);
+// create a streamer for a Connection that has a buffer size that's larger
than the
+// biggest message we can actually send. This is picked up by the
MsgStreamer to allocate
+// a buffer
+when(connection1.getSendBufferSize()).thenReturn(Connection.MAX_MSG_SIZE +
1);
+List connections = Arrays.asList(connection1);
+
+final BaseMsgStreamer msgStreamer =
+MsgStreamer.create(connections, message, false, stats, pool);
+// the streamer ought to have limited the message buffer to MAX_MSG_SIZE
+assertThat(((MsgStreamer) msgStreamer).getBuffer().capacity())
+.isEqualTo(Connection.MAX_MSG_SIZE);
+ }
+
+
+
protected BaseMsgStreamer createMsgStreamer(boolean
mixedDestinationVersions) {
InternalDistributedMember member1, member2;
@@ -92,9 +117,9 @@ public class MsgStreamerTest {
when(connection2.getRemoteAddress()).thenReturn(member2);
when(connection2.getSendBufferSize()).thenReturn(Connection.SMALL_BUFFER_SIZE);
if (mixedDestinationVersions) {
-
when(connection1.getRemoteVersion()).thenReturn(KnownVersion.GEODE_1_12_0);
+
when(connection2.getRemoteVersion()).thenReturn(KnownVersion.GEODE_1_12_0);
} else {
- when(connection1.getRemoteVersion()).thenReturn(KnownVersion.CURRENT);
+ when(connection2.getRemoteVersion()).thenReturn(KnownVersion.CURRENT);
}
List connections = Arrays.asList(connection1, connection2);
[geode] branch develop updated (40a2764 -> 76edadc)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 40a2764 GEODE-8516: Add Redis tests for multiple subscriptions for the same client (#5529) add 76edadc Revert "GEODE-8463: server's log filled with SSLException: Tag mismatch! (#5482)" (#5531) No new revisions were added by this update. Summary of changes: .../org/apache/geode/internal/net/SSLUtil.java | 39 +++- .../org/apache/geode/internal/net/SSLUtilTest.java | 41 ++ 2 files changed, 6 insertions(+), 74 deletions(-)
[geode] branch support/1.12 updated: GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.12 by this push:
new caf1c47 GEODE-8506: BufferPool returns byte buffers that may be much
larger t… (#5525)
caf1c47 is described below
commit caf1c4783478c2b5c5e5c239874092346316f4fe
Author: Bruce Schuchardt
AuthorDate: Fri Sep 18 14:21:55 2020 -0700
GEODE-8506: BufferPool returns byte buffers that may be much larger t…
(#5525)
* GEODE-8506: BufferPool returns byte buffers that may be much larger than
requested
Create a "slice" of the acquired buffer to return to the caller instead
of returning a buffer larger than what was requested. On return we fish
out the parent buffer and put it back in the pool.
* cache reflection method, remove magic numbers in test and BufferPool
(cherry picked from commit a11b9c076a72609ff00802c010b6e32262228776)
---
.../distributed/internal/DistributionConfig.java | 2 +-
.../org/apache/geode/internal/net/BufferPool.java | 75 ++
.../apache/geode/internal/net/BufferPoolTest.java | 53 +++
.../geode/internal/net/NioPlainEngineTest.java | 1 -
4 files changed, 115 insertions(+), 16 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
index 89a19a6..0074881 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
@@ -1386,7 +1386,7 @@ public interface DistributionConfig extends Config,
LogConfig, StatisticsConfig
/**
* The default value of the {@link
ConfigurationProperties#SOCKET_BUFFER_SIZE} property
*/
- int DEFAULT_SOCKET_BUFFER_SIZE = 32768;
+ static int DEFAULT_SOCKET_BUFFER_SIZE = 32768;
/**
* The minimum {@link ConfigurationProperties#SOCKET_BUFFER_SIZE}.
*
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
index 0997c6e..e119250 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
@@ -15,16 +15,21 @@
package org.apache.geode.internal.net;
import java.lang.ref.SoftReference;
+import java.lang.reflect.Method;
import java.nio.ByteBuffer;
import java.util.IdentityHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
+import org.apache.geode.InternalGemFireException;
+import org.apache.geode.annotations.VisibleForTesting;
import org.apache.geode.distributed.internal.DMStats;
import org.apache.geode.internal.Assert;
public class BufferPool {
private final DMStats stats;
+ private Method parentOfSliceMethod;
+
/**
* Buffers may be acquired from the Buffers pool
* or they may be allocated using Buffer.allocate(). This enum is used
@@ -85,21 +90,27 @@ public class BufferPool {
stats.incReceiverBufferSize(-refSize, true);
}
}
-} else if (bb.capacity() >= size) {
- bb.rewind();
- bb.limit(size);
- return bb;
} else {
- // wasn't big enough so put it back in the queue
- Assert.assertTrue(bufferQueue.offer(ref));
- if (alreadySeen == null) {
-alreadySeen = new IdentityHashMap<>();
- }
- if (alreadySeen.put(ref, ref) != null) {
-// if it returns non-null then we have already seen this item
-// so we have worked all the way through the queue once.
-// So it is time to give up and allocate a new buffer.
-break;
+ int capacity = bb.capacity();
+ if (capacity > size) {
+bb.position(0).limit(size);
+return bb.slice();
+ } else if (capacity == size) {
+bb.rewind();
+bb.limit(size);
+return bb;
+ } else {
+// wasn't big enough so put it back in the queue
+Assert.assertTrue(bufferQueue.offer(ref));
+if (alreadySeen == null) {
+ alreadySeen = new IdentityHashMap<>();
+}
+if (alreadySeen.put(ref, ref) != null) {
+ // if it returns non-null then we have already seen this item
+ // so we have worked all the way through the queue once.
+ // So it is time to give up and allocate a new buffer.
+ break;
+}
}
}
ref = bufferQueue.poll();
@@ -227,6 +238,7 @@ public class BufferPool {
*/
private void releaseBuffer
[geode] branch support/1.12 updated: GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.12 by this push:
new caf1c47 GEODE-8506: BufferPool returns byte buffers that may be much
larger t… (#5525)
caf1c47 is described below
commit caf1c4783478c2b5c5e5c239874092346316f4fe
Author: Bruce Schuchardt
AuthorDate: Fri Sep 18 14:21:55 2020 -0700
GEODE-8506: BufferPool returns byte buffers that may be much larger t…
(#5525)
* GEODE-8506: BufferPool returns byte buffers that may be much larger than
requested
Create a "slice" of the acquired buffer to return to the caller instead
of returning a buffer larger than what was requested. On return we fish
out the parent buffer and put it back in the pool.
* cache reflection method, remove magic numbers in test and BufferPool
(cherry picked from commit a11b9c076a72609ff00802c010b6e32262228776)
---
.../distributed/internal/DistributionConfig.java | 2 +-
.../org/apache/geode/internal/net/BufferPool.java | 75 ++
.../apache/geode/internal/net/BufferPoolTest.java | 53 +++
.../geode/internal/net/NioPlainEngineTest.java | 1 -
4 files changed, 115 insertions(+), 16 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
index 89a19a6..0074881 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
@@ -1386,7 +1386,7 @@ public interface DistributionConfig extends Config,
LogConfig, StatisticsConfig
/**
* The default value of the {@link
ConfigurationProperties#SOCKET_BUFFER_SIZE} property
*/
- int DEFAULT_SOCKET_BUFFER_SIZE = 32768;
+ static int DEFAULT_SOCKET_BUFFER_SIZE = 32768;
/**
* The minimum {@link ConfigurationProperties#SOCKET_BUFFER_SIZE}.
*
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
index 0997c6e..e119250 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
@@ -15,16 +15,21 @@
package org.apache.geode.internal.net;
import java.lang.ref.SoftReference;
+import java.lang.reflect.Method;
import java.nio.ByteBuffer;
import java.util.IdentityHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
+import org.apache.geode.InternalGemFireException;
+import org.apache.geode.annotations.VisibleForTesting;
import org.apache.geode.distributed.internal.DMStats;
import org.apache.geode.internal.Assert;
public class BufferPool {
private final DMStats stats;
+ private Method parentOfSliceMethod;
+
/**
* Buffers may be acquired from the Buffers pool
* or they may be allocated using Buffer.allocate(). This enum is used
@@ -85,21 +90,27 @@ public class BufferPool {
stats.incReceiverBufferSize(-refSize, true);
}
}
-} else if (bb.capacity() >= size) {
- bb.rewind();
- bb.limit(size);
- return bb;
} else {
- // wasn't big enough so put it back in the queue
- Assert.assertTrue(bufferQueue.offer(ref));
- if (alreadySeen == null) {
-alreadySeen = new IdentityHashMap<>();
- }
- if (alreadySeen.put(ref, ref) != null) {
-// if it returns non-null then we have already seen this item
-// so we have worked all the way through the queue once.
-// So it is time to give up and allocate a new buffer.
-break;
+ int capacity = bb.capacity();
+ if (capacity > size) {
+bb.position(0).limit(size);
+return bb.slice();
+ } else if (capacity == size) {
+bb.rewind();
+bb.limit(size);
+return bb;
+ } else {
+// wasn't big enough so put it back in the queue
+Assert.assertTrue(bufferQueue.offer(ref));
+if (alreadySeen == null) {
+ alreadySeen = new IdentityHashMap<>();
+}
+if (alreadySeen.put(ref, ref) != null) {
+ // if it returns non-null then we have already seen this item
+ // so we have worked all the way through the queue once.
+ // So it is time to give up and allocate a new buffer.
+ break;
+}
}
}
ref = bufferQueue.poll();
@@ -227,6 +238,7 @@ public class BufferPool {
*/
private void releaseBuffer
[geode] branch support/1.13 updated: GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 709fe7b GEODE-8506: BufferPool returns byte buffers that may be much
larger t… (#5525)
709fe7b is described below
commit 709fe7b8c4c61eb610a73fcf71005fe93a4d6e1c
Author: Bruce Schuchardt
AuthorDate: Fri Sep 18 14:21:55 2020 -0700
GEODE-8506: BufferPool returns byte buffers that may be much larger t…
(#5525)
* GEODE-8506: BufferPool returns byte buffers that may be much larger than
requested
Create a "slice" of the acquired buffer to return to the caller instead
of returning a buffer larger than what was requested. On return we fish
out the parent buffer and put it back in the pool.
* cache reflection method, remove magic numbers in test and BufferPool
(cherry picked from commit a11b9c076a72609ff00802c010b6e32262228776)
---
.../distributed/internal/DistributionConfig.java | 2 +-
.../org/apache/geode/internal/net/BufferPool.java | 74 ++
.../apache/geode/internal/net/BufferPoolTest.java | 28 ++--
.../geode/internal/net/NioPlainEngineTest.java | 3 +-
.../apache/geode/internal/tcp/MsgStreamerTest.java | 2 +
5 files changed, 85 insertions(+), 24 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
index 3c66211..bcb73ed 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
@@ -1386,7 +1386,7 @@ public interface DistributionConfig extends Config,
LogConfig, StatisticsConfig
/**
* The default value of the {@link
ConfigurationProperties#SOCKET_BUFFER_SIZE} property
*/
- int DEFAULT_SOCKET_BUFFER_SIZE = 32768;
+ static int DEFAULT_SOCKET_BUFFER_SIZE = 32768;
/**
* The minimum {@link ConfigurationProperties#SOCKET_BUFFER_SIZE}.
*
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
index 26d069b..20b138e 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
@@ -15,12 +15,15 @@
package org.apache.geode.internal.net;
import java.lang.ref.SoftReference;
+import java.lang.reflect.Method;
import java.nio.ByteBuffer;
import java.util.IdentityHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import org.apache.logging.log4j.Logger;
+import org.apache.geode.InternalGemFireException;
+import org.apache.geode.annotations.VisibleForTesting;
import org.apache.geode.distributed.internal.DMStats;
import org.apache.geode.distributed.internal.DistributionConfig;
import org.apache.geode.internal.Assert;
@@ -32,6 +35,8 @@ public class BufferPool {
private final DMStats stats;
private static final Logger logger = LogService.getLogger();
+ private Method parentOfSliceMethod;
+
/**
* Buffers may be acquired from the Buffers pool
* or they may be allocated using Buffer.allocate(). This enum is used
@@ -65,10 +70,10 @@ public class BufferPool {
private final ConcurrentLinkedQueue bufferLargeQueue =
new ConcurrentLinkedQueue<>();
- private final int SMALL_BUFFER_SIZE = Connection.SMALL_BUFFER_SIZE;
+ static final int SMALL_BUFFER_SIZE = Connection.SMALL_BUFFER_SIZE;
- private final int MEDIUM_BUFFER_SIZE =
DistributionConfig.DEFAULT_SOCKET_BUFFER_SIZE;
+ static final int MEDIUM_BUFFER_SIZE =
DistributionConfig.DEFAULT_SOCKET_BUFFER_SIZE;
/**
@@ -98,14 +103,18 @@ public class BufferPool {
if (useDirectBuffers) {
if (size <= MEDIUM_BUFFER_SIZE) {
-return acquirePredefinedFixedBuffer(send, size);
+result = acquirePredefinedFixedBuffer(send, size);
} else {
-return acquireLargeBuffer(send, size);
+result = acquireLargeBuffer(send, size);
}
-} else {
- // if we are using heap buffers then don't bother with keeping them
around
- result = ByteBuffer.allocate(size);
+ if (result.capacity() > size) {
+result.position(0).limit(size);
+result = result.slice();
+ }
+ return result;
}
+// if we are using heap buffers then don't bother with keeping them around
+result = ByteBuffer.allocate(size);
updateBufferStats(size, send, false);
return result;
}
@@ -123,7 +132,8 @@ public class BufferPool {
}
/**
- * Acquire direct buffer with predefined default capacity (4096 or 32768)
+ * Acquire direct buffer with predefined default capac
[geode] branch develop updated: GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new a11b9c0 GEODE-8506: BufferPool returns byte buffers that may be much
larger t… (#5525)
a11b9c0 is described below
commit a11b9c076a72609ff00802c010b6e32262228776
Author: Bruce Schuchardt
AuthorDate: Fri Sep 18 14:21:55 2020 -0700
GEODE-8506: BufferPool returns byte buffers that may be much larger t…
(#5525)
* GEODE-8506: BufferPool returns byte buffers that may be much larger than
requested
Create a "slice" of the acquired buffer to return to the caller instead
of returning a buffer larger than what was requested. On return we fish
out the parent buffer and put it back in the pool.
* cache reflection method, remove magic numbers in test and BufferPool
The one dunit failure seems to be GEODE-7710
---
.../distributed/internal/DistributionConfig.java | 2 +-
.../org/apache/geode/internal/net/BufferPool.java | 74 ++
.../apache/geode/internal/net/BufferPoolTest.java | 28 ++--
.../geode/internal/net/NioPlainEngineTest.java | 3 +-
.../apache/geode/internal/tcp/MsgStreamerTest.java | 2 +
5 files changed, 85 insertions(+), 24 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
index 1d69fcb..9aad592 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
@@ -1396,7 +1396,7 @@ public interface DistributionConfig extends Config,
LogConfig, StatisticsConfig
/**
* The default value of the {@link
ConfigurationProperties#SOCKET_BUFFER_SIZE} property
*/
- int DEFAULT_SOCKET_BUFFER_SIZE = 32768;
+ static int DEFAULT_SOCKET_BUFFER_SIZE = 32768;
/**
* The minimum {@link ConfigurationProperties#SOCKET_BUFFER_SIZE}.
*
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
index 26d069b..20b138e 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
@@ -15,12 +15,15 @@
package org.apache.geode.internal.net;
import java.lang.ref.SoftReference;
+import java.lang.reflect.Method;
import java.nio.ByteBuffer;
import java.util.IdentityHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import org.apache.logging.log4j.Logger;
+import org.apache.geode.InternalGemFireException;
+import org.apache.geode.annotations.VisibleForTesting;
import org.apache.geode.distributed.internal.DMStats;
import org.apache.geode.distributed.internal.DistributionConfig;
import org.apache.geode.internal.Assert;
@@ -32,6 +35,8 @@ public class BufferPool {
private final DMStats stats;
private static final Logger logger = LogService.getLogger();
+ private Method parentOfSliceMethod;
+
/**
* Buffers may be acquired from the Buffers pool
* or they may be allocated using Buffer.allocate(). This enum is used
@@ -65,10 +70,10 @@ public class BufferPool {
private final ConcurrentLinkedQueue bufferLargeQueue =
new ConcurrentLinkedQueue<>();
- private final int SMALL_BUFFER_SIZE = Connection.SMALL_BUFFER_SIZE;
+ static final int SMALL_BUFFER_SIZE = Connection.SMALL_BUFFER_SIZE;
- private final int MEDIUM_BUFFER_SIZE =
DistributionConfig.DEFAULT_SOCKET_BUFFER_SIZE;
+ static final int MEDIUM_BUFFER_SIZE =
DistributionConfig.DEFAULT_SOCKET_BUFFER_SIZE;
/**
@@ -98,14 +103,18 @@ public class BufferPool {
if (useDirectBuffers) {
if (size <= MEDIUM_BUFFER_SIZE) {
-return acquirePredefinedFixedBuffer(send, size);
+result = acquirePredefinedFixedBuffer(send, size);
} else {
-return acquireLargeBuffer(send, size);
+result = acquireLargeBuffer(send, size);
}
-} else {
- // if we are using heap buffers then don't bother with keeping them
around
- result = ByteBuffer.allocate(size);
+ if (result.capacity() > size) {
+result.position(0).limit(size);
+result = result.slice();
+ }
+ return result;
}
+// if we are using heap buffers then don't bother with keeping them around
+result = ByteBuffer.allocate(size);
updateBufferStats(size, send, false);
return result;
}
@@ -123,7 +132,8 @@ public class BufferPool {
}
/**
- * Acquire direct buffer with predefined default capacity (4096 or 32768)
+ * Acquire direct buffer with predefined default capacity (SMALL_BUFFER_SIZE
or
+
[geode] branch develop updated: GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new a11b9c0 GEODE-8506: BufferPool returns byte buffers that may be much
larger t… (#5525)
a11b9c0 is described below
commit a11b9c076a72609ff00802c010b6e32262228776
Author: Bruce Schuchardt
AuthorDate: Fri Sep 18 14:21:55 2020 -0700
GEODE-8506: BufferPool returns byte buffers that may be much larger t…
(#5525)
* GEODE-8506: BufferPool returns byte buffers that may be much larger than
requested
Create a "slice" of the acquired buffer to return to the caller instead
of returning a buffer larger than what was requested. On return we fish
out the parent buffer and put it back in the pool.
* cache reflection method, remove magic numbers in test and BufferPool
The one dunit failure seems to be GEODE-7710
---
.../distributed/internal/DistributionConfig.java | 2 +-
.../org/apache/geode/internal/net/BufferPool.java | 74 ++
.../apache/geode/internal/net/BufferPoolTest.java | 28 ++--
.../geode/internal/net/NioPlainEngineTest.java | 3 +-
.../apache/geode/internal/tcp/MsgStreamerTest.java | 2 +
5 files changed, 85 insertions(+), 24 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
index 1d69fcb..9aad592 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/DistributionConfig.java
@@ -1396,7 +1396,7 @@ public interface DistributionConfig extends Config,
LogConfig, StatisticsConfig
/**
* The default value of the {@link
ConfigurationProperties#SOCKET_BUFFER_SIZE} property
*/
- int DEFAULT_SOCKET_BUFFER_SIZE = 32768;
+ static int DEFAULT_SOCKET_BUFFER_SIZE = 32768;
/**
* The minimum {@link ConfigurationProperties#SOCKET_BUFFER_SIZE}.
*
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
index 26d069b..20b138e 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
@@ -15,12 +15,15 @@
package org.apache.geode.internal.net;
import java.lang.ref.SoftReference;
+import java.lang.reflect.Method;
import java.nio.ByteBuffer;
import java.util.IdentityHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import org.apache.logging.log4j.Logger;
+import org.apache.geode.InternalGemFireException;
+import org.apache.geode.annotations.VisibleForTesting;
import org.apache.geode.distributed.internal.DMStats;
import org.apache.geode.distributed.internal.DistributionConfig;
import org.apache.geode.internal.Assert;
@@ -32,6 +35,8 @@ public class BufferPool {
private final DMStats stats;
private static final Logger logger = LogService.getLogger();
+ private Method parentOfSliceMethod;
+
/**
* Buffers may be acquired from the Buffers pool
* or they may be allocated using Buffer.allocate(). This enum is used
@@ -65,10 +70,10 @@ public class BufferPool {
private final ConcurrentLinkedQueue bufferLargeQueue =
new ConcurrentLinkedQueue<>();
- private final int SMALL_BUFFER_SIZE = Connection.SMALL_BUFFER_SIZE;
+ static final int SMALL_BUFFER_SIZE = Connection.SMALL_BUFFER_SIZE;
- private final int MEDIUM_BUFFER_SIZE =
DistributionConfig.DEFAULT_SOCKET_BUFFER_SIZE;
+ static final int MEDIUM_BUFFER_SIZE =
DistributionConfig.DEFAULT_SOCKET_BUFFER_SIZE;
/**
@@ -98,14 +103,18 @@ public class BufferPool {
if (useDirectBuffers) {
if (size <= MEDIUM_BUFFER_SIZE) {
-return acquirePredefinedFixedBuffer(send, size);
+result = acquirePredefinedFixedBuffer(send, size);
} else {
-return acquireLargeBuffer(send, size);
+result = acquireLargeBuffer(send, size);
}
-} else {
- // if we are using heap buffers then don't bother with keeping them
around
- result = ByteBuffer.allocate(size);
+ if (result.capacity() > size) {
+result.position(0).limit(size);
+result = result.slice();
+ }
+ return result;
}
+// if we are using heap buffers then don't bother with keeping them around
+result = ByteBuffer.allocate(size);
updateBufferStats(size, send, false);
return result;
}
@@ -123,7 +132,8 @@ public class BufferPool {
}
/**
- * Acquire direct buffer with predefined default capacity (4096 or 32768)
+ * Acquire direct buffer with predefined default capacity (SMALL_BUFFER_SIZE
or
+
[geode] 01/01: GEODE-8385: hang recovering from disk with cyclic dependencies (#5403)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/9.10
in repository https://gitbox.apache.org/repos/asf/geode.git
commit d49fe266cdbcf01b083944141fe3e667824d27dc
Author: Bruce Schuchardt
AuthorDate: Wed Jul 29 07:58:39 2020 -0700
GEODE-8385: hang recovering from disk with cyclic dependencies (#5403)
* GEODE-8385: hang recovering from disk with cyclic dependencies
This restores the point at which we notify membership listeners of
departures. We used to do this (in 1.12 and earlier) when a ShutdownMessage
is received instead of waiting for a new membership view announcing the
departure.
Membership views can take some time to form and install, which can cause
persistent (disk store) views to be updated later than they used to be.
In the case of this ticket the disk store of one member was being
closed while another was shutting down. The member closing its disk
store did not see the view announcing that shutdown until most of its
disk store regions had closed their persistence advisors. This left the
disk store thinking that the other member was still up at the time it
was closed.
(cherry picked from commit 08316aa05198704d96aefc5497e483052c27a378)
---
.../ClusterDistributionManagerDUnitTest.java | 28
.../internal/ClusterDistributionManager.java | 76 ++
.../distributed/internal/StartupOperation.java | 1 -
.../internal/membership/gms/GMSMemberData.java | 4 +-
.../internal/membership/gms/GMSMembership.java | 6 +-
5 files changed, 69 insertions(+), 46 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
index 5cef2bb..a85d82f 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
@@ -204,6 +204,34 @@ public class ClusterDistributionManagerDUnitTest extends
CacheTestCase {
.until(() -> !membershipManager.isSurpriseMember(member));
}
+ @Test
+ public void shutdownMessageCausesListenerInvocation() {
+final AtomicBoolean listenerInvoked = new AtomicBoolean();
+vm1.invoke("join the cluster", () -> getSystem().getDistributedMember());
// lead member
+system = getSystem(); // non-lead member
+// this membership listener will be invoked when the shutdown message is
received
+system.getDistributionManager().addMembershipListener(new
MembershipListener() {
+ @Override
+ public void memberDeparted(DistributionManager distributionManager,
+ InternalDistributedMember id, boolean crashed) {
+assertThat(crashed).isFalse();
+listenerInvoked.set(Boolean.TRUE);
+ }
+});
+final InternalDistributedMember memberID = system.getDistributedMember();
+locatorvm.invoke("send a shutdown message", () -> {
+ final DistributionManager distributionManager =
+ ((InternalDistributedSystem)
Locator.getLocator().getDistributedSystem())
+ .getDistributionManager();
+ final ShutdownMessage shutdownMessage = new ShutdownMessage();
+ shutdownMessage.setRecipient(memberID);
+
shutdownMessage.setDistributionManagerId(distributionManager.getDistributionManagerId());
+ distributionManager.putOutgoing(shutdownMessage);
+});
+await().until(() -> listenerInvoked.get());
+ }
+
+
/**
* Tests that a severe-level alert is generated if a member does not respond
with an ack quickly
* enough. vm0 and vm1 create a region and set ack-severe-alert-threshold.
vm1 has a cache
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
index 4147948..eda8aef 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
@@ -1782,24 +1782,6 @@ public class ClusterDistributionManager implements
DistributionManager {
}
/**
- * Returns true if id was removed. Returns false if it was not in the list
of managers.
- */
- private boolean removeManager(InternalDistributedMember theId, boolean
crashed, String p_reason) {
-String reason = p_reason;
-
-reason = prettifyReason(reason);
-if (logger.isDebugEnabled()) {
- logger.debug("DistributionManager: removing member <{}>; crashed {};
reason = {}", theId,
- crashed,
[geode] branch support/9.10 created (now d49fe26)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch support/9.10 in repository https://gitbox.apache.org/repos/asf/geode.git. at d49fe26 GEODE-8385: hang recovering from disk with cyclic dependencies (#5403) This branch includes the following new commits: new d49fe26 GEODE-8385: hang recovering from disk with cyclic dependencies (#5403) The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[geode] branch develop updated (1d629e1 -> c48c0c3)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 1d629e1 GEODE-8066: Pull up transitive dependency to clean up javax.activation and mail licenses. (#5291) add c48c0c3 GEODE-8473: Hang in ReplyProcessor21 when forced-disconnect does not establish a cancellation cause (#5491) No new revisions were added by this update. Summary of changes: .../membership/gms/GMSMembershipJUnitTest.java | 15 ++ .../internal/membership/gms/GMSMembership.java | 34 -- 2 files changed, 33 insertions(+), 16 deletions(-)
[geode] branch develop updated (1d629e1 -> c48c0c3)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 1d629e1 GEODE-8066: Pull up transitive dependency to clean up javax.activation and mail licenses. (#5291) add c48c0c3 GEODE-8473: Hang in ReplyProcessor21 when forced-disconnect does not establish a cancellation cause (#5491) No new revisions were added by this update. Summary of changes: .../membership/gms/GMSMembershipJUnitTest.java | 15 ++ .../internal/membership/gms/GMSMembership.java | 34 -- 2 files changed, 33 insertions(+), 16 deletions(-)
[geode] branch support/1.13 updated: GEODE-8467: server fails to notify of a ForcedDisconnect and fails to tear down the cache (#5490)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 8a66bc3 GEODE-8467: server fails to notify of a ForcedDisconnect and
fails to tear down the cache (#5490)
8a66bc3 is described below
commit 8a66bc3b376f391b7f387c74b9b5246d72882178
Author: Bruce Schuchardt
AuthorDate: Tue Sep 1 07:32:47 2020 -0700
GEODE-8467: server fails to notify of a ForcedDisconnect and fails to tear
down the cache (#5490)
Catch exceptions that occur during XML generation and disable auto
reconnect.
Ensure that the DisconnectThread is launched by placing it in a
"finally" block.
(cherry picked from commit e402ed35102a4a885ad24a1052216b0542672bc7)
---
.../geode/internal/cache/GemFireCacheImpl.java | 70 ++
.../geode/internal/cache/GemFireCacheImplTest.java | 12
.../internal/membership/gms/GMSMembership.java | 15 ++---
3 files changed, 65 insertions(+), 32 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/cache/GemFireCacheImpl.java
b/geode-core/src/main/java/org/apache/geode/internal/cache/GemFireCacheImpl.java
index c4c8db5..2f6b0b4 100755
---
a/geode-core/src/main/java/org/apache/geode/internal/cache/GemFireCacheImpl.java
+++
b/geode-core/src/main/java/org/apache/geode/internal/cache/GemFireCacheImpl.java
@@ -59,6 +59,7 @@ import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
+import java.io.PrintWriter;
import java.io.Reader;
import java.io.StringBufferInputStream;
import java.io.StringWriter;
@@ -1126,34 +1127,53 @@ public class GemFireCacheImpl implements InternalCache,
InternalClientCache, Has
@Override
public void saveCacheXmlForReconnect() {
-// there are two versions of this method so it can be unit-tested
-boolean sharedConfigEnabled =
getDistributionManager().getConfig().getUseSharedConfiguration();
+prepareForReconnect((pw) -> CacheXmlGenerator.generate((Cache) this, pw,
false));
+ }
-if (!Boolean.getBoolean(GEMFIRE_PREFIX + "autoReconnect-useCacheXMLFile")
-&& !sharedConfigEnabled) {
- try {
-logger.info("generating XML to rebuild the cache after reconnect
completes");
-StringPrintWriter pw = new StringPrintWriter();
-CacheXmlGenerator.generate((Cache) this, pw, false);
-String cacheXML = pw.toString();
-getCacheConfig().setCacheXMLDescription(cacheXML);
-logger.info("XML generation completed: {}", cacheXML);
- } catch (CancelException e) {
-logger.info("Unable to generate XML description for reconnect of cache
due to exception",
-e);
- }
-} else if (sharedConfigEnabled && !getCacheServers().isEmpty()) {
- // we need to retain a cache-server description if this JVM was started
by gfsh
- List list = new
ArrayList<>(getCacheServers().size());
- for (Object o : getCacheServers()) {
-CacheServerImpl cs = (CacheServerImpl) o;
-if (cs.isDefaultServer()) {
- CacheServerCreation bsc = new CacheServerCreation(this, cs);
- list.add(bsc);
+
+ /**
+ * Testable version of saveCacheXmlForReconnect() that allows us to inject
an XML generator
+ *
+ * @param xmlGenerator a consumer of a PrintWriter that generates a
description of the Cache
+ */
+ protected void prepareForReconnect(Consumer xmlGenerator) {
+boolean sharedConfigEnabled =
+getInternalDistributedSystem().getConfig().getUseSharedConfiguration();
+
+try {
+ if (!Boolean.getBoolean(GEMFIRE_PREFIX + "autoReconnect-useCacheXMLFile")
+ && !sharedConfigEnabled) {
+try {
+ logger.info("generating XML to rebuild the cache after reconnect
completes");
+ StringPrintWriter pw = new StringPrintWriter();
+ xmlGenerator.accept(pw);
+ String cacheXML = pw.toString();
+ getCacheConfig().setCacheXMLDescription(cacheXML);
+ logger.info("XML generation completed: {}", cacheXML);
+} catch (CancelException e) {
+ logger.info("Unable to generate XML description for reconnect of
cache due to exception",
+ e);
}
+ } else if (sharedConfigEnabled && !getCacheServers().isEmpty()) {
+// we need to retain a cache-server description if this JVM was
started by gfsh
+logger.info("saving cache server configuration for use with the
cluster-configuration "
++ "service on reconnect");
+List list = new
ArrayList<>(getCacheServers().size());
+for (Object o : getCacheServers()) {
+
[geode] branch develop updated (304e4d58 -> e402ed3)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 304e4d58 GEODE-8455: Fix difference between create region with gw sender and a… (#5476) add e402ed3 GEODE-8467: server fails to notify of a ForcedDisconnect and fails to tear down the cache (#5490) No new revisions were added by this update. Summary of changes: .../geode/internal/cache/GemFireCacheImpl.java | 70 ++ .../geode/internal/cache/GemFireCacheImplTest.java | 12 .../internal/membership/gms/GMSMembership.java | 15 ++--- 3 files changed, 65 insertions(+), 32 deletions(-)
[geode] branch develop updated: GEODE-8463: server's log filled with SSLException: Tag mismatch! (#5482)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new 20a35ec GEODE-8463: server's log filled with SSLException: Tag
mismatch! (#5482)
20a35ec is described below
commit 20a35ece18054e96eccda70c65a015f4af26b4c7
Author: Bruce Schuchardt
AuthorDate: Fri Aug 28 08:56:53 2020 -0700
GEODE-8463: server's log filled with SSLException: Tag mismatch! (#5482)
This disables the use of TLSv1.3 selection if "any" is specified as the
protocol and throws an exception if TLSv1.3 is requested in a JVM older
than Java 11. Most Java 8 implementations do not support TLSv1.3 - this
is currently only an issue with Oracle's 1.8.0_261 and above.
---
.../org/apache/geode/internal/net/SSLUtil.java | 39 +---
.../org/apache/geode/internal/net/SSLUtilTest.java | 41 --
2 files changed, 74 insertions(+), 6 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/SSLUtil.java
b/geode-core/src/main/java/org/apache/geode/internal/net/SSLUtil.java
index 5093d86..72234f1 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/SSLUtil.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/SSLUtil.java
@@ -19,6 +19,7 @@ import java.security.KeyStore;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.cert.X509Certificate;
+import java.util.Arrays;
import java.util.Objects;
import javax.net.ssl.KeyManager;
@@ -28,7 +29,9 @@ import javax.net.ssl.TrustManager;
import javax.net.ssl.TrustManagerFactory;
import javax.net.ssl.X509TrustManager;
+import org.apache.commons.lang3.JavaVersion;
import org.apache.commons.lang3.StringUtils;
+import org.apache.commons.lang3.SystemUtils;
import org.apache.geode.annotations.VisibleForTesting;
@@ -38,6 +41,8 @@ public class SSLUtil {
* this list as new algorithms become available and are supported by Geode.
Remove old,
* no-longer trusted algorithms.
*/
+ protected static final String[] DEFAULT_ALGORITMS_PRE_JAVA11 = {
+ "TLSv1.2"};
protected static final String[] DEFAULT_ALGORITMS = {
"TLSv1.3",
"TLSv1.2"}; // TLSv1.3 is not available in JDK 8 at this time
@@ -47,7 +52,21 @@ public class SSLUtil {
public static SSLContext getSSLContextInstance(SSLConfig sslConfig)
throws NoSuchAlgorithmException {
String[] protocols = sslConfig.getProtocolsAsStringArray();
-return findSSLContextForProtocols(protocols, DEFAULT_ALGORITMS);
+String[] protocolsForAny = getDefaultAlgorithms();
+return findSSLContextForProtocols(protocols, protocolsForAny);
+ }
+
+ /**
+ * Returns the default algorithms that are used to search for an SSLContext
+ * when "any" is given as the protocol by the user.
+ */
+ public static String[] getDefaultAlgorithms() {
+if (SystemUtils.isJavaVersionAtLeast(JavaVersion.JAVA_11)) {
+ return DEFAULT_ALGORITMS;
+} else {
+ // tlsv1.3 is not supported by Geode before JAVA 11
+ return DEFAULT_ALGORITMS_PRE_JAVA11;
+}
}
/**
@@ -64,22 +83,34 @@ public class SSLUtil {
protected static SSLContext findSSLContextForProtocols(final String[]
protocols,
final String[] protocolsForAny)
throws NoSuchAlgorithmException {
+SSLContext result = null;
for (String protocol : protocols) {
if (protocol.equalsIgnoreCase("any")) {
try {
- return findSSLContextForProtocols(protocolsForAny, new String[0]);
+ result = findSSLContextForProtocols(protocolsForAny, new String[0]);
+ break;
} catch (NoSuchAlgorithmException e) {
// none of the default algorithms is available - continue to see if
there
// are any others in the requested list
}
}
try {
-return SSLContext.getInstance(protocol);
+result = SSLContext.getInstance(protocol);
+break;
} catch (NoSuchAlgorithmException e) {
// continue
}
}
-throw new NoSuchAlgorithmException();
+if (result != null) {
+ if (result.getProtocol().equalsIgnoreCase("tlsv1.3") &&
+ SystemUtils.isJavaVersionAtMost(JavaVersion.JAVA_10)) {
+throw new IllegalStateException("TLSv1.3 is not supported for this JRE
- please use TLSv1.2"
++ " or upgrade to Java 11");
+ }
+ return result;
+}
+throw new NoSuchAlgorithmException("unable to find support for configured
TLS protocols: " +
+Arrays.toString(protocols));
}
/** Read an array of values from a string, whitespace or comma separated. */
diff --git
a/geode-core/src/test/java/org/apache/geode/internal/n
[geode] branch develop updated (6f12a36 -> 537721f)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a change to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git.
from 6f12a36 GEODE-8432: use regionPath directly instead of getRegion when
put eve… (#5464)
add 537721f GEODE-8419: SSL/TLS protocol and cipher suite configuration
is ignored (#5465)
No new revisions were added by this update.
Summary of changes:
.../apache/geode/internal/SSLConfigJUnitTest.java | 2 +-
...LSocketHostNameVerificationIntegrationTest.java | 4 +-
.../internal/net/SSLSocketIntegrationTest.java | 4 +-
.../internal/net/SocketCreatorFailHandshake.java | 2 -
.../admin/internal/AdminDistributedSystemImpl.java | 2 +-
.../apache/geode/distributed/LocatorLauncher.java | 2 +-
.../admin/remote/DistributionLocatorId.java| 2 +-
.../admin/remote/RemoteTransportConfig.java| 2 +-
.../geode/internal/net/SCClusterSocketCreator.java | 1 -
.../geode/internal/{admin => net}/SSLConfig.java | 3 +-
.../internal/net/SSLConfigurationFactory.java | 1 -
.../org/apache/geode/internal/net/SSLUtil.java | 1 -
.../apache/geode/internal/net/SocketCreator.java | 96 +++---
.../geode/internal/net/SocketCreatorFactory.java | 1 -
.../org/apache/geode/internal/tcp/Connection.java | 2 +-
.../ContextAwareSSLRMIClientSocketFactory.java | 2 +-
.../management/internal/JmxManagerAdvisee.java | 2 +-
.../internal/JmxManagerLocatorRequest.java | 2 +-
.../geode/management/internal/ManagementAgent.java | 2 +-
.../internal/api/GeodeConnectionConfig.java| 2 +-
...ClusterManagementServiceInfoRequestHandler.java | 2 +-
.../functions/GetMemberInformationFunction.java| 2 +-
.../net/SSLConfigurationFactoryJUnitTest.java | 1 -
.../org/apache/geode/internal/net/SSLUtilTest.java | 2 -
.../geode/internal/net/SocketCreatorJUnitTest.java | 55 -
.../apache/geode/internal/tcp/TCPConduitTest.java | 2 +-
.../internal/cli/commands/ConnectCommand.java | 2 +-
.../internal/cli/shell/JmxOperationInvoker.java| 2 +-
.../geode/internal/cache/InternalHttpService.java | 2 +-
.../acceptance/CacheConnectionIntegrationTest.java | 2 +-
.../v1/acceptance/CacheOperationsJUnitTest.java| 2 +-
.../geode/tools/pulse/tests/rules/ServerRule.java | 2 +-
.../java/org/apache/geode/redis/SSLTest.java | 2 +-
.../redis/internal/netty/NettyRedisServer.java | 2 +-
34 files changed, 148 insertions(+), 67 deletions(-)
rename geode-core/src/main/java/org/apache/geode/internal/{admin =>
net}/SSLConfig.java (99%)
[geode] branch develop updated (6f12a36 -> 537721f)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a change to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git.
from 6f12a36 GEODE-8432: use regionPath directly instead of getRegion when
put eve… (#5464)
add 537721f GEODE-8419: SSL/TLS protocol and cipher suite configuration
is ignored (#5465)
No new revisions were added by this update.
Summary of changes:
.../apache/geode/internal/SSLConfigJUnitTest.java | 2 +-
...LSocketHostNameVerificationIntegrationTest.java | 4 +-
.../internal/net/SSLSocketIntegrationTest.java | 4 +-
.../internal/net/SocketCreatorFailHandshake.java | 2 -
.../admin/internal/AdminDistributedSystemImpl.java | 2 +-
.../apache/geode/distributed/LocatorLauncher.java | 2 +-
.../admin/remote/DistributionLocatorId.java| 2 +-
.../admin/remote/RemoteTransportConfig.java| 2 +-
.../geode/internal/net/SCClusterSocketCreator.java | 1 -
.../geode/internal/{admin => net}/SSLConfig.java | 3 +-
.../internal/net/SSLConfigurationFactory.java | 1 -
.../org/apache/geode/internal/net/SSLUtil.java | 1 -
.../apache/geode/internal/net/SocketCreator.java | 96 +++---
.../geode/internal/net/SocketCreatorFactory.java | 1 -
.../org/apache/geode/internal/tcp/Connection.java | 2 +-
.../ContextAwareSSLRMIClientSocketFactory.java | 2 +-
.../management/internal/JmxManagerAdvisee.java | 2 +-
.../internal/JmxManagerLocatorRequest.java | 2 +-
.../geode/management/internal/ManagementAgent.java | 2 +-
.../internal/api/GeodeConnectionConfig.java| 2 +-
...ClusterManagementServiceInfoRequestHandler.java | 2 +-
.../functions/GetMemberInformationFunction.java| 2 +-
.../net/SSLConfigurationFactoryJUnitTest.java | 1 -
.../org/apache/geode/internal/net/SSLUtilTest.java | 2 -
.../geode/internal/net/SocketCreatorJUnitTest.java | 55 -
.../apache/geode/internal/tcp/TCPConduitTest.java | 2 +-
.../internal/cli/commands/ConnectCommand.java | 2 +-
.../internal/cli/shell/JmxOperationInvoker.java| 2 +-
.../geode/internal/cache/InternalHttpService.java | 2 +-
.../acceptance/CacheConnectionIntegrationTest.java | 2 +-
.../v1/acceptance/CacheOperationsJUnitTest.java| 2 +-
.../geode/tools/pulse/tests/rules/ServerRule.java | 2 +-
.../java/org/apache/geode/redis/SSLTest.java | 2 +-
.../redis/internal/netty/NettyRedisServer.java | 2 +-
34 files changed, 148 insertions(+), 67 deletions(-)
rename geode-core/src/main/java/org/apache/geode/internal/{admin =>
net}/SSLConfig.java (99%)
[geode] branch develop updated: GEODE-8419: SSL/TLS protocol and cipher suite configuration is ignored (#5465)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new 537721f GEODE-8419: SSL/TLS protocol and cipher suite configuration
is ignored (#5465)
537721f is described below
commit 537721ff815cf40eff85fde65db9b5e787471c89
Author: Bruce Schuchardt
AuthorDate: Wed Aug 19 15:27:07 2020 -0700
GEODE-8419: SSL/TLS protocol and cipher suite configuration is ignored
(#5465)
* GEODE-8419: SSL/TLS protocol and cipher suite configuration is ignored
Configure cipher suites when creating an SSLEngine
* addressing test issues
* fixing error in SSLSocket endpoint validation
* addressing Jake's comments
* change test to use ArgumentCaptor - thanks Jake\!
* check captured argument content
---
.../apache/geode/internal/SSLConfigJUnitTest.java | 2 +-
...LSocketHostNameVerificationIntegrationTest.java | 4 +-
.../internal/net/SSLSocketIntegrationTest.java | 4 +-
.../internal/net/SocketCreatorFailHandshake.java | 2 -
.../admin/internal/AdminDistributedSystemImpl.java | 2 +-
.../apache/geode/distributed/LocatorLauncher.java | 2 +-
.../admin/remote/DistributionLocatorId.java| 2 +-
.../admin/remote/RemoteTransportConfig.java| 2 +-
.../geode/internal/net/SCClusterSocketCreator.java | 1 -
.../geode/internal/{admin => net}/SSLConfig.java | 3 +-
.../internal/net/SSLConfigurationFactory.java | 1 -
.../org/apache/geode/internal/net/SSLUtil.java | 1 -
.../apache/geode/internal/net/SocketCreator.java | 96 +++---
.../geode/internal/net/SocketCreatorFactory.java | 1 -
.../org/apache/geode/internal/tcp/Connection.java | 2 +-
.../ContextAwareSSLRMIClientSocketFactory.java | 2 +-
.../management/internal/JmxManagerAdvisee.java | 2 +-
.../internal/JmxManagerLocatorRequest.java | 2 +-
.../geode/management/internal/ManagementAgent.java | 2 +-
.../internal/api/GeodeConnectionConfig.java| 2 +-
...ClusterManagementServiceInfoRequestHandler.java | 2 +-
.../functions/GetMemberInformationFunction.java| 2 +-
.../net/SSLConfigurationFactoryJUnitTest.java | 1 -
.../org/apache/geode/internal/net/SSLUtilTest.java | 2 -
.../geode/internal/net/SocketCreatorJUnitTest.java | 55 -
.../apache/geode/internal/tcp/TCPConduitTest.java | 2 +-
.../internal/cli/commands/ConnectCommand.java | 2 +-
.../internal/cli/shell/JmxOperationInvoker.java| 2 +-
.../geode/internal/cache/InternalHttpService.java | 2 +-
.../acceptance/CacheConnectionIntegrationTest.java | 2 +-
.../v1/acceptance/CacheOperationsJUnitTest.java| 2 +-
.../geode/tools/pulse/tests/rules/ServerRule.java | 2 +-
.../java/org/apache/geode/redis/SSLTest.java | 2 +-
.../redis/internal/netty/NettyRedisServer.java | 2 +-
34 files changed, 148 insertions(+), 67 deletions(-)
diff --git
a/geode-core/src/integrationTest/java/org/apache/geode/internal/SSLConfigJUnitTest.java
b/geode-core/src/integrationTest/java/org/apache/geode/internal/SSLConfigJUnitTest.java
index 99ec074..2a3ded9 100755
---
a/geode-core/src/integrationTest/java/org/apache/geode/internal/SSLConfigJUnitTest.java
+++
b/geode-core/src/integrationTest/java/org/apache/geode/internal/SSLConfigJUnitTest.java
@@ -67,7 +67,7 @@ import org.junit.Test;
import org.junit.experimental.categories.Category;
import org.apache.geode.distributed.internal.DistributionConfigImpl;
-import org.apache.geode.internal.admin.SSLConfig;
+import org.apache.geode.internal.net.SSLConfig;
import org.apache.geode.internal.net.SSLConfigurationFactory;
import org.apache.geode.test.junit.categories.SecurityTest;
diff --git
a/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketHostNameVerificationIntegrationTest.java
b/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketHostNameVerificationIntegrationTest.java
index 5483457..dc7df44 100755
---
a/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketHostNameVerificationIntegrationTest.java
+++
b/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketHostNameVerificationIntegrationTest.java
@@ -168,7 +168,7 @@ public class SSLSocketHostNameVerificationIntegrationTest {
this.clientSocket = clientChannel.socket();
SSLEngine sslEngine =
-this.socketCreator.createSSLEngine(this.localHost.getHostName(), 1234);
+this.socketCreator.createSSLEngine(this.localHost.getHostName(), 1234,
true);
try {
this.socketCreator.handshakeSSLSocketChannel(clientSocket.getChannel(),
@@ -200,7 +200,7 @@ public class SSLSocketHostNameVerificationIntegrationTest {
try {
socket = serverSocket.accept();
SocketCreator
[geode] branch develop updated (4759840 -> 6a08bba)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 4759840 GEODE-8396: Fixing NullPointerException in create jdbc-mapping command (#5418) add 6a08bba GEODE-8413: CI failure: MergeLogFilesIntegrationTest.testDircountZero failes on Windows (#5435) No new revisions were added by this update. Summary of changes: .../internal/logging/MergeLogFilesIntegrationTest.java | 13 ++--- .../geode/internal/logging/MergeLogFilesIntegrationTest.txt | 1 + 2 files changed, 7 insertions(+), 7 deletions(-) create mode 100644 geode-core/src/integrationTest/resources/org/apache/geode/internal/logging/MergeLogFilesIntegrationTest.txt
[geode] branch develop updated (d6c3b1f -> 1a6e192)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from d6c3b1f GEODE-8407: MergeLogFiles fails to include files with the same name b… (#5428) add 1a6e192 GEODE-6950: hot loop in PrimaryHandler.processRequest() (#5422) No new revisions were added by this update. Summary of changes: .../gms/locator/MembershipLocatorImpl.java | 4 +- .../membership/gms/locator/PrimaryHandler.java | 36 ++--- .../membership/gms/locator/PrimaryHandlerTest.java | 87 ++ 3 files changed, 117 insertions(+), 10 deletions(-) create mode 100644 geode-membership/src/test/java/org/apache/geode/distributed/internal/membership/gms/locator/PrimaryHandlerTest.java
[geode] branch develop updated (d6c3b1f -> 1a6e192)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from d6c3b1f GEODE-8407: MergeLogFiles fails to include files with the same name b… (#5428) add 1a6e192 GEODE-6950: hot loop in PrimaryHandler.processRequest() (#5422) No new revisions were added by this update. Summary of changes: .../gms/locator/MembershipLocatorImpl.java | 4 +- .../membership/gms/locator/PrimaryHandler.java | 36 ++--- .../membership/gms/locator/PrimaryHandlerTest.java | 87 ++ 3 files changed, 117 insertions(+), 10 deletions(-) create mode 100644 geode-membership/src/test/java/org/apache/geode/distributed/internal/membership/gms/locator/PrimaryHandlerTest.java
[geode] branch develop updated (eb8668a -> d6c3b1f)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from eb8668a GEODE-8382: Run Redis tests against Redis API for Geode (#5416) add d6c3b1f GEODE-8407: MergeLogFiles fails to include files with the same name b… (#5428) No new revisions were added by this update. Summary of changes: .../logging/MergeLogFilesIntegrationTest.java | 33 + .../geode/internal/logging/dir1/systemlog.txt | 992 .../geode/internal/logging/dir2/systemlog.txt | 1571 .../geode/internal/logging/MergeLogFiles.java | 58 +- 4 files changed, 2641 insertions(+), 13 deletions(-) create mode 100644 geode-core/src/integrationTest/resources/org/apache/geode/internal/logging/dir1/systemlog.txt create mode 100644 geode-core/src/integrationTest/resources/org/apache/geode/internal/logging/dir2/systemlog.txt
[geode] branch develop updated (eb8668a -> d6c3b1f)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from eb8668a GEODE-8382: Run Redis tests against Redis API for Geode (#5416) add d6c3b1f GEODE-8407: MergeLogFiles fails to include files with the same name b… (#5428) No new revisions were added by this update. Summary of changes: .../logging/MergeLogFilesIntegrationTest.java | 33 + .../geode/internal/logging/dir1/systemlog.txt | 992 .../geode/internal/logging/dir2/systemlog.txt | 1571 .../geode/internal/logging/MergeLogFiles.java | 58 +- 4 files changed, 2641 insertions(+), 13 deletions(-) create mode 100644 geode-core/src/integrationTest/resources/org/apache/geode/internal/logging/dir1/systemlog.txt create mode 100644 geode-core/src/integrationTest/resources/org/apache/geode/internal/logging/dir2/systemlog.txt
[geode] branch develop updated (54421c4 -> 6e07aac)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a change to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git.
from 54421c4 GEODE-8395: fix product name in gfsh help banner (#5417)
add 6e07aac GEODE-8389: reconnect attempt fails due to distribution
configuration… (#5413)
No new revisions were added by this update.
Summary of changes:
.../distributed/internal/DistributionConfig.java | 21 +++-
.../internal/DistributionConfigImpl.java | 40 +-
...ibuteDesc.java => InternalConfigAttribute.java} | 6 ++--
.../internal/DistributionConfigJUnitTest.java | 26 ++
4 files changed, 82 insertions(+), 11 deletions(-)
copy
geode-core/src/main/java/org/apache/geode/distributed/internal/{ConfigAttributeDesc.java
=> InternalConfigAttribute.java} (89%)
[geode] branch support/1.13 updated: GEODE-8385: hang recovering from disk with cyclic dependencies (#5403)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 2eb66cc GEODE-8385: hang recovering from disk with cyclic
dependencies (#5403)
2eb66cc is described below
commit 2eb66cc795e3d08218a4cd32f96a4b47d64d116e
Author: Bruce Schuchardt
AuthorDate: Wed Jul 29 07:58:39 2020 -0700
GEODE-8385: hang recovering from disk with cyclic dependencies (#5403)
* GEODE-8385: hang recovering from disk with cyclic dependencies
This restores the point at which we notify membership listeners of
departures. We used to do this (in 1.12 and earlier) when a ShutdownMessage
is received instead of waiting for a new membership view announcing the
departure.
Membership views can take some time to form and install, which can cause
persistent (disk store) views to be updated later than they used to be.
In the case of this ticket the disk store of one member was being
closed while another was shutting down. The member closing its disk
store did not see the view announcing that shutdown until most of its
disk store regions had closed their persistence advisors. This left the
disk store thinking that the other member was still up at the time it
was closed.
(cherry picked from commit 08316aa05198704d96aefc5497e483052c27a378)
---
.../ClusterDistributionManagerDUnitTest.java | 28
.../internal/ClusterDistributionManager.java | 76 ++
.../distributed/internal/StartupOperation.java | 1 -
.../internal/membership/gms/GMSMemberData.java | 4 +-
.../internal/membership/gms/GMSMembership.java | 6 +-
5 files changed, 69 insertions(+), 46 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
index 5cef2bb..a85d82f 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
@@ -204,6 +204,34 @@ public class ClusterDistributionManagerDUnitTest extends
CacheTestCase {
.until(() -> !membershipManager.isSurpriseMember(member));
}
+ @Test
+ public void shutdownMessageCausesListenerInvocation() {
+final AtomicBoolean listenerInvoked = new AtomicBoolean();
+vm1.invoke("join the cluster", () -> getSystem().getDistributedMember());
// lead member
+system = getSystem(); // non-lead member
+// this membership listener will be invoked when the shutdown message is
received
+system.getDistributionManager().addMembershipListener(new
MembershipListener() {
+ @Override
+ public void memberDeparted(DistributionManager distributionManager,
+ InternalDistributedMember id, boolean crashed) {
+assertThat(crashed).isFalse();
+listenerInvoked.set(Boolean.TRUE);
+ }
+});
+final InternalDistributedMember memberID = system.getDistributedMember();
+locatorvm.invoke("send a shutdown message", () -> {
+ final DistributionManager distributionManager =
+ ((InternalDistributedSystem)
Locator.getLocator().getDistributedSystem())
+ .getDistributionManager();
+ final ShutdownMessage shutdownMessage = new ShutdownMessage();
+ shutdownMessage.setRecipient(memberID);
+
shutdownMessage.setDistributionManagerId(distributionManager.getDistributionManagerId());
+ distributionManager.putOutgoing(shutdownMessage);
+});
+await().until(() -> listenerInvoked.get());
+ }
+
+
/**
* Tests that a severe-level alert is generated if a member does not respond
with an ack quickly
* enough. vm0 and vm1 create a region and set ack-severe-alert-threshold.
vm1 has a cache
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
index a5e363c..d17ae80 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
@@ -1782,24 +1782,6 @@ public class ClusterDistributionManager implements
DistributionManager {
}
/**
- * Returns true if id was removed. Returns false if it was not in the list
of managers.
- */
- private boolean removeManager(InternalDistributedMember theId, boolean
crashed, String p_reason) {
-String reason = p_reason;
-
-
[geode] branch develop updated: GEODE-8385: hang recovering from disk with cyclic dependencies (#5403)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new 08316aa GEODE-8385: hang recovering from disk with cyclic
dependencies (#5403)
08316aa is described below
commit 08316aa05198704d96aefc5497e483052c27a378
Author: Bruce Schuchardt
AuthorDate: Wed Jul 29 07:58:39 2020 -0700
GEODE-8385: hang recovering from disk with cyclic dependencies (#5403)
* GEODE-8385: hang recovering from disk with cyclic dependencies
This restores the point at which we notify membership listeners of
departures. We used to do this (in 1.12 and earlier) when a ShutdownMessage
is received instead of waiting for a new membership view announcing the
departure.
Membership views can take some time to form and install, which can cause
persistent (disk store) views to be updated later than they used to be.
In the case of this ticket the disk store of one member was being
closed while another was shutting down. The member closing its disk
store did not see the view announcing that shutdown until most of its
disk store regions had closed their persistence advisors. This left the
disk store thinking that the other member was still up at the time it
was closed.
---
.../ClusterDistributionManagerDUnitTest.java | 28
.../internal/ClusterDistributionManager.java | 76 ++
.../distributed/internal/StartupOperation.java | 1 -
.../internal/membership/gms/GMSMemberData.java | 4 +-
.../internal/membership/gms/GMSMembership.java | 6 +-
5 files changed, 69 insertions(+), 46 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
index 9d3bbd9..e28f15ab 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
@@ -211,6 +211,34 @@ public class ClusterDistributionManagerDUnitTest extends
CacheTestCase {
.until(() -> !membershipManager.isSurpriseMember(member));
}
+ @Test
+ public void shutdownMessageCausesListenerInvocation() {
+final AtomicBoolean listenerInvoked = new AtomicBoolean();
+vm1.invoke("join the cluster", () -> getSystem().getDistributedMember());
// lead member
+system = getSystem(); // non-lead member
+// this membership listener will be invoked when the shutdown message is
received
+system.getDistributionManager().addMembershipListener(new
MembershipListener() {
+ @Override
+ public void memberDeparted(DistributionManager distributionManager,
+ InternalDistributedMember id, boolean crashed) {
+assertThat(crashed).isFalse();
+listenerInvoked.set(Boolean.TRUE);
+ }
+});
+final InternalDistributedMember memberID = system.getDistributedMember();
+locatorvm.invoke("send a shutdown message", () -> {
+ final DistributionManager distributionManager =
+ ((InternalDistributedSystem)
Locator.getLocator().getDistributedSystem())
+ .getDistributionManager();
+ final ShutdownMessage shutdownMessage = new ShutdownMessage();
+ shutdownMessage.setRecipient(memberID);
+
shutdownMessage.setDistributionManagerId(distributionManager.getDistributionManagerId());
+ distributionManager.putOutgoing(shutdownMessage);
+});
+await().until(() -> listenerInvoked.get());
+ }
+
+
/**
* Tests that a severe-level alert is generated if a member does not respond
with an ack quickly
* enough. vm0 and vm1 create a region and set ack-severe-alert-threshold.
vm1 has a cache
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
index 9a52f60..1f58344 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
@@ -1782,24 +1782,6 @@ public class ClusterDistributionManager implements
DistributionManager {
}
/**
- * Returns true if id was removed. Returns false if it was not in the list
of managers.
- */
- private boolean removeManager(InternalDistributedMember theId, boolean
crashed, String p_reason) {
-String reason = p_reason;
-
-reason = prettifyReason(reason);
-if (logger.isDebugEnabled()) {
- logger.debug("DistributionM
[geode] branch develop updated: GEODE-8385: hang recovering from disk with cyclic dependencies (#5403)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new 08316aa GEODE-8385: hang recovering from disk with cyclic
dependencies (#5403)
08316aa is described below
commit 08316aa05198704d96aefc5497e483052c27a378
Author: Bruce Schuchardt
AuthorDate: Wed Jul 29 07:58:39 2020 -0700
GEODE-8385: hang recovering from disk with cyclic dependencies (#5403)
* GEODE-8385: hang recovering from disk with cyclic dependencies
This restores the point at which we notify membership listeners of
departures. We used to do this (in 1.12 and earlier) when a ShutdownMessage
is received instead of waiting for a new membership view announcing the
departure.
Membership views can take some time to form and install, which can cause
persistent (disk store) views to be updated later than they used to be.
In the case of this ticket the disk store of one member was being
closed while another was shutting down. The member closing its disk
store did not see the view announcing that shutdown until most of its
disk store regions had closed their persistence advisors. This left the
disk store thinking that the other member was still up at the time it
was closed.
---
.../ClusterDistributionManagerDUnitTest.java | 28
.../internal/ClusterDistributionManager.java | 76 ++
.../distributed/internal/StartupOperation.java | 1 -
.../internal/membership/gms/GMSMemberData.java | 4 +-
.../internal/membership/gms/GMSMembership.java | 6 +-
5 files changed, 69 insertions(+), 46 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
index 9d3bbd9..e28f15ab 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/distributed/internal/ClusterDistributionManagerDUnitTest.java
@@ -211,6 +211,34 @@ public class ClusterDistributionManagerDUnitTest extends
CacheTestCase {
.until(() -> !membershipManager.isSurpriseMember(member));
}
+ @Test
+ public void shutdownMessageCausesListenerInvocation() {
+final AtomicBoolean listenerInvoked = new AtomicBoolean();
+vm1.invoke("join the cluster", () -> getSystem().getDistributedMember());
// lead member
+system = getSystem(); // non-lead member
+// this membership listener will be invoked when the shutdown message is
received
+system.getDistributionManager().addMembershipListener(new
MembershipListener() {
+ @Override
+ public void memberDeparted(DistributionManager distributionManager,
+ InternalDistributedMember id, boolean crashed) {
+assertThat(crashed).isFalse();
+listenerInvoked.set(Boolean.TRUE);
+ }
+});
+final InternalDistributedMember memberID = system.getDistributedMember();
+locatorvm.invoke("send a shutdown message", () -> {
+ final DistributionManager distributionManager =
+ ((InternalDistributedSystem)
Locator.getLocator().getDistributedSystem())
+ .getDistributionManager();
+ final ShutdownMessage shutdownMessage = new ShutdownMessage();
+ shutdownMessage.setRecipient(memberID);
+
shutdownMessage.setDistributionManagerId(distributionManager.getDistributionManagerId());
+ distributionManager.putOutgoing(shutdownMessage);
+});
+await().until(() -> listenerInvoked.get());
+ }
+
+
/**
* Tests that a severe-level alert is generated if a member does not respond
with an ack quickly
* enough. vm0 and vm1 create a region and set ack-severe-alert-threshold.
vm1 has a cache
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
index 9a52f60..1f58344 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
@@ -1782,24 +1782,6 @@ public class ClusterDistributionManager implements
DistributionManager {
}
/**
- * Returns true if id was removed. Returns false if it was not in the list
of managers.
- */
- private boolean removeManager(InternalDistributedMember theId, boolean
crashed, String p_reason) {
-String reason = p_reason;
-
-reason = prettifyReason(reason);
-if (logger.isDebugEnabled()) {
- logger.debug("DistributionM
[geode] branch support/1.13 updated: GEODE-8020: buffer management problems (#5048)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 120f94a GEODE-8020: buffer management problems (#5048)
120f94a is described below
commit 120f94a3ee1b7934673978ae9c82f1d3e30cb9c8
Author: Bruce Schuchardt
AuthorDate: Thu May 7 11:33:02 2020 -0700
GEODE-8020: buffer management problems (#5048)
* GEODE-8020: buffer management problems
This fixes some buffer handling in MsgStreamerList and alters
MstStreamer to avoid creating MsgStreamerList and VersionedMsgStreamers
during normal, non-upgrade, operations.
It also changes NioSslEngine to use synchronization in more places,
notably the close() method, which was possibly allowing multiple threads to
change the state of the engine.
* revert unnecessary change to ClusterCommunicationsDUnitTest
* fixing another null version check
* renamed new BufferPool property
* restore logging of ssl exceptions
(cherry picked from commit 7375c591f25bbba413237aed1f56f8a9f70075df)
---
.../internal/ClusterDistributionManager.java | 5 +-
.../apache/geode/internal/cache/properties.html| 9 +-
.../org/apache/geode/internal/net/BufferPool.java | 11 ++-
.../apache/geode/internal/net/NioSslEngine.java| 7 +-
.../org/apache/geode/internal/tcp/Connection.java | 20 ++--
.../org/apache/geode/internal/tcp/MsgStreamer.java | 16 ++--
.../apache/geode/internal/tcp/MsgStreamerList.java | 29 ++
.../geode/internal/net/NioSslEngineTest.java | 5 +
.../apache/geode/internal/tcp/MsgStreamerTest.java | 101 +
9 files changed, 157 insertions(+), 46 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
index 338a8b7..a5e363c 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/ClusterDistributionManager.java
@@ -608,9 +608,8 @@ public class ClusterDistributionManager implements
DistributionManager {
}
if (member != getDistributionManagerId()) {
String relationship = areInSameZone(getDistributionManagerId(), member)
? "" : "not ";
- Object[] logArgs = new Object[] {member, relationship};
- logger.info("Member {} is {} equivalent or in the same redundancy zone.",
- logArgs);
+ logger.info("Member {} is {}equivalent or in the same redundancy zone.",
+ member, relationship);
}
}
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/cache/properties.html
b/geode-core/src/main/java/org/apache/geode/internal/cache/properties.html
index 1f5b88d..0419553 100755
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/properties.html
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/properties.html
@@ -2787,16 +2787,19 @@ TBA
-p2p.nodirectBuffers
+gemfire.BufferPool.useHeapBuffers
Public: false
Boolean (default is false)
-See org.apache.geode.internal.tcp.TCPConduit#useDirectBuffers.
+See org.apache.geode.internal.net.BufferPool#useDirectBuffers.
- use direct ByteBuffers instead of heap ByteBuffers for NIO operations
+ use java "heap" ByteBuffers instead of direct ByteBuffers for NIO
operations. Recommended if TLSv1
+is being used or if you find you are running out of direct-memory and do
not want to
+increase the amount of direct-memory available to the JVM. Use of heap
buffers can
+reduce performance in some cases.
TBA
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
index c156c2c..26d069b 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
@@ -19,13 +19,18 @@ import java.nio.ByteBuffer;
import java.util.IdentityHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
+import org.apache.logging.log4j.Logger;
+
import org.apache.geode.distributed.internal.DMStats;
import org.apache.geode.distributed.internal.DistributionConfig;
import org.apache.geode.internal.Assert;
import org.apache.geode.internal.tcp.Connection;
+import org.apache.geode.logging.internal.log4j.api.LogService;
+import org.apache.geode.util.internal.GeodeGlossary;
public class BufferPool {
private final DMStats stats;
+ private static final Logger logger = LogService.getLogger();
/**
* Buffers may be acquired from the Buffers pool
@@ -69,7 +74,8 @@ public clas
[geode] 02/02: fixed compilation error in cherry-pick
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
commit 0f95d0b4fac88816c80204f7539784d9db88e56a
Author: Bruce Schuchardt
AuthorDate: Fri Jun 26 14:30:26 2020 -0700
fixed compilation error in cherry-pick
---
.../client/internal/locator/wan/LocatorMembershipListenerTest.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
b/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
index b2b3c14..fabb128 100644
---
a/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
+++
b/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
@@ -356,7 +356,7 @@ public class LocatorMembershipListenerTest {
// The sendMessage loop in the listener will try to send 4 messages. Two
to the remoteLocators
// and two to the joiningLocator. The retry loop will try to send the
messages again and
// fail (4 more messages) and then it will succeed (4 more messages, for a
total of 12).
-verify(tcpClient, times(12)).requestToServer(isA(HostAndPort.class),
+verify(tcpClient, times(12)).requestToServer(isA(InetSocketAddress.class),
isA(LocatorJoinMessage.class), isA(Integer.class), isA(Boolean.class));
}
[geode] branch support/1.12 updated (17c26fe -> 0f95d0b)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch support/1.12 in repository https://gitbox.apache.org/repos/asf/geode.git. from 17c26fe First pass at switching to liberica jdk. (#5312) new 5d2181d GEODE-8195: ConcurrentModificationException from LocatorMembershipListenerImpl (#5306) new 0f95d0b fixed compilation error in cherry-pick The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../locator/wan/LocatorMembershipListenerImpl.java | 14 ++-- .../locator/wan/LocatorMembershipListenerTest.java | 88 -- 2 files changed, 72 insertions(+), 30 deletions(-)
[geode] 01/02: GEODE-8195: ConcurrentModificationException from LocatorMembershipListenerImpl (#5306)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.12
in repository https://gitbox.apache.org/repos/asf/geode.git
commit 5d2181d6a259035aaa77a73daa0dcbacefb56038
Author: Bruce Schuchardt
AuthorDate: Fri Jun 26 07:47:23 2020 -0700
GEODE-8195: ConcurrentModificationException from
LocatorMembershipListenerImpl (#5306)
I've replaced the "for" loop using an implicit Iterator with one using an
explicit Iterator so that its safe "remove()" method can be used. The
Iterator method is stated as being the only safe way to modify the
collection while iterating over its contents.
I've also modified a test to validate the fix. The test forces a
failure to send two messages to an address. The failures are then
handled in the code that was throwing the
ConcurrentModificationException and, since there are two failures,
it causes two removals to be performedon the failedMessages collection.
(cherry picked from commit 3cda1b1a213f2195ff0b97361883f6a6c3972b14)
---
.../locator/wan/LocatorMembershipListenerImpl.java | 14 ++--
.../locator/wan/LocatorMembershipListenerTest.java | 88 --
2 files changed, 72 insertions(+), 30 deletions(-)
diff --git
a/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
b/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
index 31315ed..434231d 100644
---
a/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
+++
b/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
@@ -18,6 +18,7 @@ package org.apache.geode.cache.client.internal.locator.wan;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -291,15 +292,15 @@ public class LocatorMembershipListenerImpl implements
LocatorMembershipListener
public void run() {
Map> failedMessages = new
HashMap<>();
for (Map.Entry> entry :
remoteLocators.entrySet()) {
-for (DistributionLocatorId value : entry.getValue()) {
+for (DistributionLocatorId remoteLocator : entry.getValue()) {
// Notify known remote locator about the advertised locator.
LocatorJoinMessage advertiseNewLocatorMessage = new
LocatorJoinMessage(
joiningLocatorDistributedSystemId, joiningLocator,
localLocatorId, "");
- sendMessage(value, advertiseNewLocatorMessage, failedMessages);
+ sendMessage(remoteLocator, advertiseNewLocatorMessage,
failedMessages);
// Notify the advertised locator about remote known locator.
LocatorJoinMessage advertiseKnownLocatorMessage =
- new LocatorJoinMessage(entry.getKey(), value, localLocatorId,
"");
+ new LocatorJoinMessage(entry.getKey(), remoteLocator,
localLocatorId, "");
sendMessage(joiningLocator, advertiseKnownLocatorMessage,
failedMessages);
}
}
@@ -313,9 +314,11 @@ public class LocatorMembershipListenerImpl implements
LocatorMembershipListener
DistributionLocatorId targetLocator = entry.getKey();
Set joinMessages = entry.getValue();
-for (LocatorJoinMessage locatorJoinMessage : joinMessages) {
+for (Iterator iterator =
joinMessages.iterator(); iterator
+.hasNext();) {
+ LocatorJoinMessage locatorJoinMessage = iterator.next();
if (retryMessage(targetLocator, locatorJoinMessage, attempt)) {
-joinMessages.remove(locatorJoinMessage);
+iterator.remove();
} else {
// Sleep between retries.
try {
@@ -324,6 +327,7 @@ public class LocatorMembershipListenerImpl implements
LocatorMembershipListener
Thread.currentThread().interrupt();
logger.warn(
"Locator Membership listener permanently failed to
exchange locator information due to interruption.");
+ return;
}
}
}
diff --git
a/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
b/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
index f3770a6..b2b3c14 100644
---
a/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
+++
b/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
@@ -21,6 +21,7 @@ import static org.mockito.ArgumentMatch
[geode] branch support/1.13 updated: GEODE-8195: ConcurrentModificationException from LocatorMembershipListenerImpl (#5306)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 64b727e GEODE-8195: ConcurrentModificationException from
LocatorMembershipListenerImpl (#5306)
64b727e is described below
commit 64b727e105117d806b6e680ec5f2b0f9bbce0afd
Author: Bruce Schuchardt
AuthorDate: Fri Jun 26 07:47:23 2020 -0700
GEODE-8195: ConcurrentModificationException from
LocatorMembershipListenerImpl (#5306)
I've replaced the "for" loop using an implicit Iterator with one using an
explicit Iterator so that its safe "remove()" method can be used. The
Iterator method is stated as being the only safe way to modify the
collection while iterating over its contents.
I've also modified a test to validate the fix. The test forces a
failure to send two messages to an address. The failures are then
handled in the code that was throwing the
ConcurrentModificationException and, since there are two failures,
it causes two removals to be performedon the failedMessages collection.
(cherry picked from commit 3cda1b1a213f2195ff0b97361883f6a6c3972b14)
---
.../locator/wan/LocatorMembershipListenerImpl.java | 14 ++--
.../locator/wan/LocatorMembershipListenerTest.java | 88 --
2 files changed, 72 insertions(+), 30 deletions(-)
diff --git
a/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
b/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
index acb4f0a..253f6dc 100644
---
a/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
+++
b/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
@@ -18,6 +18,7 @@ package org.apache.geode.cache.client.internal.locator.wan;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -295,15 +296,15 @@ public class LocatorMembershipListenerImpl implements
LocatorMembershipListener
public void run() {
Map> failedMessages = new
HashMap<>();
for (Map.Entry> entry :
remoteLocators.entrySet()) {
-for (DistributionLocatorId value : entry.getValue()) {
+for (DistributionLocatorId remoteLocator : entry.getValue()) {
// Notify known remote locator about the advertised locator.
LocatorJoinMessage advertiseNewLocatorMessage = new
LocatorJoinMessage(
joiningLocatorDistributedSystemId, joiningLocator,
localLocatorId, "");
- sendMessage(value, advertiseNewLocatorMessage, failedMessages);
+ sendMessage(remoteLocator, advertiseNewLocatorMessage,
failedMessages);
// Notify the advertised locator about remote known locator.
LocatorJoinMessage advertiseKnownLocatorMessage =
- new LocatorJoinMessage(entry.getKey(), value, localLocatorId,
"");
+ new LocatorJoinMessage(entry.getKey(), remoteLocator,
localLocatorId, "");
sendMessage(joiningLocator, advertiseKnownLocatorMessage,
failedMessages);
}
}
@@ -317,9 +318,11 @@ public class LocatorMembershipListenerImpl implements
LocatorMembershipListener
DistributionLocatorId targetLocator = entry.getKey();
Set joinMessages = entry.getValue();
-for (LocatorJoinMessage locatorJoinMessage : joinMessages) {
+for (Iterator iterator =
joinMessages.iterator(); iterator
+.hasNext();) {
+ LocatorJoinMessage locatorJoinMessage = iterator.next();
if (retryMessage(targetLocator, locatorJoinMessage, attempt)) {
-joinMessages.remove(locatorJoinMessage);
+iterator.remove();
} else {
// Sleep between retries.
try {
@@ -328,6 +331,7 @@ public class LocatorMembershipListenerImpl implements
LocatorMembershipListener
Thread.currentThread().interrupt();
logger.warn(
"Locator Membership listener permanently failed to
exchange locator information due to interruption.");
+ return;
}
}
}
diff --git
a/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
b/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
index d282128..2869d4d 100644
---
a/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/w
[geode] branch support/1.13 updated: GEODE-8195: ConcurrentModificationException from LocatorMembershipListenerImpl (#5306)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 64b727e GEODE-8195: ConcurrentModificationException from
LocatorMembershipListenerImpl (#5306)
64b727e is described below
commit 64b727e105117d806b6e680ec5f2b0f9bbce0afd
Author: Bruce Schuchardt
AuthorDate: Fri Jun 26 07:47:23 2020 -0700
GEODE-8195: ConcurrentModificationException from
LocatorMembershipListenerImpl (#5306)
I've replaced the "for" loop using an implicit Iterator with one using an
explicit Iterator so that its safe "remove()" method can be used. The
Iterator method is stated as being the only safe way to modify the
collection while iterating over its contents.
I've also modified a test to validate the fix. The test forces a
failure to send two messages to an address. The failures are then
handled in the code that was throwing the
ConcurrentModificationException and, since there are two failures,
it causes two removals to be performedon the failedMessages collection.
(cherry picked from commit 3cda1b1a213f2195ff0b97361883f6a6c3972b14)
---
.../locator/wan/LocatorMembershipListenerImpl.java | 14 ++--
.../locator/wan/LocatorMembershipListenerTest.java | 88 --
2 files changed, 72 insertions(+), 30 deletions(-)
diff --git
a/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
b/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
index acb4f0a..253f6dc 100644
---
a/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
+++
b/geode-wan/src/main/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerImpl.java
@@ -18,6 +18,7 @@ package org.apache.geode.cache.client.internal.locator.wan;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -295,15 +296,15 @@ public class LocatorMembershipListenerImpl implements
LocatorMembershipListener
public void run() {
Map> failedMessages = new
HashMap<>();
for (Map.Entry> entry :
remoteLocators.entrySet()) {
-for (DistributionLocatorId value : entry.getValue()) {
+for (DistributionLocatorId remoteLocator : entry.getValue()) {
// Notify known remote locator about the advertised locator.
LocatorJoinMessage advertiseNewLocatorMessage = new
LocatorJoinMessage(
joiningLocatorDistributedSystemId, joiningLocator,
localLocatorId, "");
- sendMessage(value, advertiseNewLocatorMessage, failedMessages);
+ sendMessage(remoteLocator, advertiseNewLocatorMessage,
failedMessages);
// Notify the advertised locator about remote known locator.
LocatorJoinMessage advertiseKnownLocatorMessage =
- new LocatorJoinMessage(entry.getKey(), value, localLocatorId,
"");
+ new LocatorJoinMessage(entry.getKey(), remoteLocator,
localLocatorId, "");
sendMessage(joiningLocator, advertiseKnownLocatorMessage,
failedMessages);
}
}
@@ -317,9 +318,11 @@ public class LocatorMembershipListenerImpl implements
LocatorMembershipListener
DistributionLocatorId targetLocator = entry.getKey();
Set joinMessages = entry.getValue();
-for (LocatorJoinMessage locatorJoinMessage : joinMessages) {
+for (Iterator iterator =
joinMessages.iterator(); iterator
+.hasNext();) {
+ LocatorJoinMessage locatorJoinMessage = iterator.next();
if (retryMessage(targetLocator, locatorJoinMessage, attempt)) {
-joinMessages.remove(locatorJoinMessage);
+iterator.remove();
} else {
// Sleep between retries.
try {
@@ -328,6 +331,7 @@ public class LocatorMembershipListenerImpl implements
LocatorMembershipListener
Thread.currentThread().interrupt();
logger.warn(
"Locator Membership listener permanently failed to
exchange locator information due to interruption.");
+ return;
}
}
}
diff --git
a/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
b/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/wan/LocatorMembershipListenerTest.java
index d282128..2869d4d 100644
---
a/geode-wan/src/test/java/org/apache/geode/cache/client/internal/locator/w
[geode] branch develop updated (c9ed7d7 -> 3cda1b1)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from c9ed7d7 GEODE-8283: Provide REST interface for disk-store creation (#5288) add 3cda1b1 GEODE-8195: ConcurrentModificationException from LocatorMembershipListenerImpl (#5306) No new revisions were added by this update. Summary of changes: .../locator/wan/LocatorMembershipListenerImpl.java | 14 ++-- .../locator/wan/LocatorMembershipListenerTest.java | 88 -- 2 files changed, 72 insertions(+), 30 deletions(-)
[geode] branch develop updated (eed47f4 -> ab83b61)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from eed47f4 Fix metric-tools image python requirements. (#5298) add ab83b61 GEODE-8285: Change location of generated test file to build dir and fix error message path for sanctioned text file (#5280) No new revisions were added by this update. Summary of changes: .../AnalyzeDataSerializablesJUnitTestBase.java | 31 -- 1 file changed, 23 insertions(+), 8 deletions(-)
[geode] branch support/1.13 updated: GEODE-8144: setting SNI server name is not needed if endpoint verification is disabled (#5250)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 85a42d5 GEODE-8144: setting SNI server name is not needed if endpoint
verification is disabled (#5250)
85a42d5 is described below
commit 85a42d5213d784373772e6e39a1f273838b049a4
Author: Bruce Schuchardt
AuthorDate: Tue Jun 16 10:27:59 2020 -0700
GEODE-8144: setting SNI server name is not needed if endpoint verification
is disabled (#5250)
* GEODE-8144: endpoint identification in servers is not working
modified the fix for this issue to not set the SNI server name parameter
if endpoint verification is disabled. We're doing this because setting
this parameter appears to decrease performance in large performance
tests.
* changed test to throw exceptions instead of asserting they don't exist
* replaced check for SNI server name in SSL parameters with a more in-depth
check
* SSLParameters.getServerNames() may return a null value
(cherry picked from commit b1107d2e403404337c22830a4964eefc2490ef50)
---
.../internal/net/SSLSocketIntegrationTest.java | 29 +-
.../apache/geode/internal/net/SocketCreator.java | 10 +---
2 files changed, 29 insertions(+), 10 deletions(-)
diff --git
a/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketIntegrationTest.java
b/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketIntegrationTest.java
index f1d7397..4800940 100755
---
a/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketIntegrationTest.java
+++
b/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketIntegrationTest.java
@@ -17,6 +17,7 @@ package org.apache.geode.internal.net;
import static org.apache.geode.distributed.ConfigurationProperties.MCAST_PORT;
import static org.apache.geode.distributed.ConfigurationProperties.SSL_CIPHERS;
import static
org.apache.geode.distributed.ConfigurationProperties.SSL_ENABLED_COMPONENTS;
+import static
org.apache.geode.distributed.ConfigurationProperties.SSL_ENDPOINT_IDENTIFICATION_ENABLED;
import static
org.apache.geode.distributed.ConfigurationProperties.SSL_KEYSTORE;
import static
org.apache.geode.distributed.ConfigurationProperties.SSL_KEYSTORE_PASSWORD;
import static
org.apache.geode.distributed.ConfigurationProperties.SSL_PROTOCOLS;
@@ -32,7 +33,6 @@ import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotEquals;
import static org.junit.Assert.assertNotNull;
-import static org.junit.Assert.assertNull;
import static org.mockito.Mockito.mock;
import java.io.DataInputStream;
@@ -52,13 +52,17 @@ import java.net.URL;
import java.nio.ByteBuffer;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
+import java.util.List;
import java.util.Properties;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicReference;
+import javax.net.ssl.SNIServerName;
import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLEngine;
import javax.net.ssl.SSLException;
+import javax.net.ssl.StandardConstants;
import org.apache.commons.io.FileUtils;
import org.junit.After;
@@ -130,6 +134,7 @@ public class SSLSocketIntegrationTest {
properties.setProperty(SSL_TRUSTSTORE, keystore.getCanonicalPath());
properties.setProperty(SSL_TRUSTSTORE_PASSWORD, "password");
properties.setProperty(SSL_REQUIRE_AUTHENTICATION, "true");
+properties.setProperty(SSL_ENDPOINT_IDENTIFICATION_ENABLED, "false");
properties.setProperty(SSL_CIPHERS, "any");
properties.setProperty(SSL_PROTOCOLS, "TLSv1.2");
@@ -176,7 +181,7 @@ public class SSLSocketIntegrationTest {
}
@Test
- public void securedSocketTransmissionShouldWork() throws Exception {
+ public void securedSocketTransmissionShouldWork() throws Throwable {
this.serverSocket = this.socketCreator.forCluster().createServerSocket(0,
0, this.localHost);
this.serverThread = startServer(this.serverSocket, 15000);
@@ -194,12 +199,14 @@ public class SSLSocketIntegrationTest {
await().until(() -> {
return !serverThread.isAlive();
});
-assertNull(serverException);
+if (serverException != null) {
+ throw serverException;
+}
assertThat(this.messageFromClient.get()).isEqualTo(MESSAGE);
}
@Test
- public void testSecuredSocketTransmissionShouldWorkUsingNIO() throws
Exception {
+ public void testSecuredSocketTransmissionShouldWorkUsingNIO() throws
Throwable {
ServerSocketChannel serverChannel = ServerSocketChannel.open();
serverSocket = serverChannel.socket();
@@ -
[geode] branch support/1.13 updated: GEODE-8277: acceptance test certificates expired in Dockerized SNI acceptance tests (#5274)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 3dc3237 GEODE-8277: acceptance test certificates expired in
Dockerized SNI acceptance tests (#5274)
3dc3237 is described below
commit 3dc32370b8d0f734eb0889719bad8ce3f9ca420b
Author: Bruce Schuchardt
AuthorDate: Thu Jun 18 10:42:41 2020 -0700
GEODE-8277: acceptance test certificates expired in Dockerized SNI
acceptance tests (#5274)
The old keystores have expired. I've generated 100 year keystores and
added a
program to recreate them if necessary.
(cherry picked from commit a6640d71e1b2aca2f7f2f861508162f08691891d)
---
.../client/sni/GenerateSNIKeyAndTrustStores.java | 79 +
.../sni/geode-config/locator-maeve-keystore.jks| Bin 3525 -> 3529 bytes
.../geode-config/server-clementine-keystore.jks| Bin 3537 -> 3537 bytes
.../sni/geode-config/server-dolores-keystore.jks | Bin 3528 -> 3533 bytes
.../geode/client/sni/geode-config/truststore.jks | Bin 1126 -> 1129 bytes
.../org/apache/geode/cache/ssl/CertStores.java | 4 +-
6 files changed, 81 insertions(+), 2 deletions(-)
diff --git
a/geode-assembly/src/acceptanceTest/java/org/apache/geode/client/sni/GenerateSNIKeyAndTrustStores.java
b/geode-assembly/src/acceptanceTest/java/org/apache/geode/client/sni/GenerateSNIKeyAndTrustStores.java
new file mode 100644
index 000..1e5168c
--- /dev/null
+++
b/geode-assembly/src/acceptanceTest/java/org/apache/geode/client/sni/GenerateSNIKeyAndTrustStores.java
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
contributor license
+ * agreements. See the NOTICE file distributed with this work for additional
information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache
License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express
+ * or implied. See the License for the specific language governing permissions
and limitations under
+ * the License.
+ */
+package org.apache.geode.client.sni;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.URL;
+
+import org.apache.geode.cache.ssl.CertStores;
+import org.apache.geode.cache.ssl.CertificateBuilder;
+import org.apache.geode.cache.ssl.CertificateMaterial;
+
+/**
+ * This program generates the trust and key stores used by SNI acceptance
tests.
+ * The stores have a 100 year expiration date, but if you need to generate new
ones
+ * use this program, modified as necessary to correct problems, to generate new
+ * stores.
+ */
+public class GenerateSNIKeyAndTrustStores {
+
+ public static void main(String... args) throws Exception {
+new GenerateSNIKeyAndTrustStores().generateStores();
+ }
+
+ public void generateStores() throws Exception {
+CertificateMaterial ca = new CertificateBuilder(365 * 100, "SHA256withRSA")
+.commonName("Test CA")
+.isCA()
+.generate();
+
+final String resourceFilename = "geode-config/gemfire.properties";
+final URL resource =
SingleServerSNIAcceptanceTest.class.getResource(resourceFilename);
+String path = resource.getPath();
+path = path.substring(0, path.length() - "gemfire.properties".length());
+
+boolean trustStoreCreated = false;
+
+for (String certName : new String[] {"locator-maeve", "server-clementine",
"server-dolores"}) {
+ CertificateMaterial certificate = new CertificateBuilder(365 * 100,
"SHA256withRSA")
+ .commonName(certName)
+ .issuedBy(ca)
+ .sanDnsName("geode") // for inside the docker container
+ .sanDnsName("localhost") // for inside the docker container
+ .sanIpAddress(InetAddress.getByName("0.0.0.0")) // for inside the
docker container
+ .sanDnsName(certName) // for client endpoint validation
+ .generate();
+
+ CertStores store = new CertStores(certName);
+ store.withCertificate("locator-maeve", certificate);
+ store.trust("ca", ca);
+
+ File keyStoreFile = new File(path + certName + "-keystore.jks");
+ keyStoreFile.createNewFile();
+ store.createKeyStore(keyStoreFile.getAbsolutePath(), "geode");
+ System.out.println("created " + keyStoreFile.getAbsolutePath());
+
+ if (!
[geode] branch develop updated (87a2f10 -> a6640d7)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 87a2f10 GEODE-8237: Add note about 'alter region' & cluster conf service (#5231) add a6640d7 GEODE-8277: acceptance test certificates expired in Dockerized SNI acceptance tests (#5274) No new revisions were added by this update. Summary of changes: .../client/sni/GenerateSNIKeyAndTrustStores.java | 79 + .../sni/geode-config/locator-maeve-keystore.jks| Bin 3525 -> 3529 bytes .../geode-config/server-clementine-keystore.jks| Bin 3537 -> 3537 bytes .../sni/geode-config/server-dolores-keystore.jks | Bin 3528 -> 3533 bytes .../geode/client/sni/geode-config/truststore.jks | Bin 1126 -> 1129 bytes .../org/apache/geode/cache/ssl/CertStores.java | 4 +- 6 files changed, 81 insertions(+), 2 deletions(-) create mode 100644 geode-assembly/src/acceptanceTest/java/org/apache/geode/client/sni/GenerateSNIKeyAndTrustStores.java
[geode] branch develop updated (a5c8164 -> d405e4b)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from a5c8164 GEODE-8260: add toString to RedisData classes (#5255) add d405e4b GEODE-8176: Move test to ClientServerMiscDUnitTest (#5238) No new revisions were added by this update. Summary of changes: .../tier/sockets/ClientServerMiscDUnitTest.java| 34 ++ .../sockets/ClientServerMiscDUnitTestBase.java | 30 --- 2 files changed, 34 insertions(+), 30 deletions(-)
[geode] branch develop updated: GEODE-8144: setting SNI server name is not needed if endpoint verification is disabled (#5250)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new b1107d2 GEODE-8144: setting SNI server name is not needed if endpoint
verification is disabled (#5250)
b1107d2 is described below
commit b1107d2e403404337c22830a4964eefc2490ef50
Author: Bruce Schuchardt
AuthorDate: Tue Jun 16 10:27:59 2020 -0700
GEODE-8144: setting SNI server name is not needed if endpoint verification
is disabled (#5250)
* GEODE-8144: endpoint identification in servers is not working
modified the fix for this issue to not set the SNI server name parameter
if endpoint verification is disabled. We're doing this because setting
this parameter appears to decrease performance in large performance
tests.
* changed test to throw exceptions instead of asserting they don't exist
* replaced check for SNI server name in SSL parameters with a more in-depth
check
* SSLParameters.getServerNames() may return a null value
---
.../internal/net/SSLSocketIntegrationTest.java | 29 +-
.../apache/geode/internal/net/SocketCreator.java | 10 +---
2 files changed, 29 insertions(+), 10 deletions(-)
diff --git
a/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketIntegrationTest.java
b/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketIntegrationTest.java
index f1d7397..4800940 100755
---
a/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketIntegrationTest.java
+++
b/geode-core/src/integrationTest/java/org/apache/geode/internal/net/SSLSocketIntegrationTest.java
@@ -17,6 +17,7 @@ package org.apache.geode.internal.net;
import static org.apache.geode.distributed.ConfigurationProperties.MCAST_PORT;
import static org.apache.geode.distributed.ConfigurationProperties.SSL_CIPHERS;
import static
org.apache.geode.distributed.ConfigurationProperties.SSL_ENABLED_COMPONENTS;
+import static
org.apache.geode.distributed.ConfigurationProperties.SSL_ENDPOINT_IDENTIFICATION_ENABLED;
import static
org.apache.geode.distributed.ConfigurationProperties.SSL_KEYSTORE;
import static
org.apache.geode.distributed.ConfigurationProperties.SSL_KEYSTORE_PASSWORD;
import static
org.apache.geode.distributed.ConfigurationProperties.SSL_PROTOCOLS;
@@ -32,7 +33,6 @@ import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotEquals;
import static org.junit.Assert.assertNotNull;
-import static org.junit.Assert.assertNull;
import static org.mockito.Mockito.mock;
import java.io.DataInputStream;
@@ -52,13 +52,17 @@ import java.net.URL;
import java.nio.ByteBuffer;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
+import java.util.List;
import java.util.Properties;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicReference;
+import javax.net.ssl.SNIServerName;
import javax.net.ssl.SSLContext;
+import javax.net.ssl.SSLEngine;
import javax.net.ssl.SSLException;
+import javax.net.ssl.StandardConstants;
import org.apache.commons.io.FileUtils;
import org.junit.After;
@@ -130,6 +134,7 @@ public class SSLSocketIntegrationTest {
properties.setProperty(SSL_TRUSTSTORE, keystore.getCanonicalPath());
properties.setProperty(SSL_TRUSTSTORE_PASSWORD, "password");
properties.setProperty(SSL_REQUIRE_AUTHENTICATION, "true");
+properties.setProperty(SSL_ENDPOINT_IDENTIFICATION_ENABLED, "false");
properties.setProperty(SSL_CIPHERS, "any");
properties.setProperty(SSL_PROTOCOLS, "TLSv1.2");
@@ -176,7 +181,7 @@ public class SSLSocketIntegrationTest {
}
@Test
- public void securedSocketTransmissionShouldWork() throws Exception {
+ public void securedSocketTransmissionShouldWork() throws Throwable {
this.serverSocket = this.socketCreator.forCluster().createServerSocket(0,
0, this.localHost);
this.serverThread = startServer(this.serverSocket, 15000);
@@ -194,12 +199,14 @@ public class SSLSocketIntegrationTest {
await().until(() -> {
return !serverThread.isAlive();
});
-assertNull(serverException);
+if (serverException != null) {
+ throw serverException;
+}
assertThat(this.messageFromClient.get()).isEqualTo(MESSAGE);
}
@Test
- public void testSecuredSocketTransmissionShouldWorkUsingNIO() throws
Exception {
+ public void testSecuredSocketTransmissionShouldWorkUsingNIO() throws
Throwable {
ServerSocketChannel serverChannel = ServerSocketChannel.open();
serverSocket = serverChannel.socket();
@@ -232,7 +239,9 @@ public class SSLSocketIntegrationTest {
awai
[geode] branch support/1.13 updated: GEODE-8238: message loss during shutdown in Shutdown Hook when JVM exits (#5232)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 850a040 GEODE-8238: message loss during shutdown in Shutdown Hook
when JVM exits (#5232)
850a040 is described below
commit 850a04088b73fa23ee2816f00efbf5bc418520ba
Author: Bruce Schuchardt
AuthorDate: Wed Jun 10 11:34:25 2020 -0700
GEODE-8238: message loss during shutdown in Shutdown Hook when JVM exits
(#5232)
Remove invocation of removeEndpoint when a shared/unordered connection
shuts down. Endpoint cleanup is already initiated by DistributionImpl
during membership view installation, so it isn't needed here.
(cherry picked from commit ece3a5a6c045075509a5097054e33a64d4194fae)
---
.../geode/internal/tcp/TCPConduitDUnitTest.java | 21 +
.../distributed/internal/direct/DirectChannel.java | 6 +-
.../org/apache/geode/internal/tcp/Connection.java | 10 --
3 files changed, 26 insertions(+), 11 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
index 9ffb76c..41d64c6 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/internal/tcp/TCPConduitDUnitTest.java
@@ -31,8 +31,10 @@ import org.junit.runners.Parameterized;
import org.apache.geode.distributed.ConfigurationProperties;
import org.apache.geode.distributed.DistributedSystem;
import org.apache.geode.distributed.Locator;
+import org.apache.geode.distributed.internal.DistributionImpl;
import org.apache.geode.distributed.internal.InternalDistributedSystem;
import org.apache.geode.distributed.internal.SerialAckedMessage;
+import
org.apache.geode.distributed.internal.membership.InternalDistributedMember;
import org.apache.geode.test.dunit.DistributedTestCase;
import org.apache.geode.test.dunit.DistributedTestUtils;
import org.apache.geode.test.dunit.IgnoredException;
@@ -96,6 +98,25 @@ public class TCPConduitDUnitTest extends DistributedTestCase
{
assertThat(ConnectionTable.getNumSenderSharedConnections()).isEqualTo(3);
});
+// ensure that the closing of a shared/unordered connection to another
node does not
+// remove all connections for that node
+InternalDistributedMember otherMember =
+(InternalDistributedMember)
system.getAllOtherMembers().iterator().next();
+DistributionImpl distribution =
+(DistributionImpl) system.getDistributionManager().getDistribution();
+final ConnectionTable connectionTable =
+distribution.getDirectChannel().getConduit().getConTable();
+
+assertThat(connectionTable.hasReceiversFor(otherMember)).isTrue();
+
+Connection sharedUnordered = connectionTable.get(otherMember, false,
+System.currentTimeMillis(), 15000, 0);
+sharedUnordered.requestClose("for testing");
+// the sender connection has been closed so we should only have 2 senders
now
+assertThat(ConnectionTable.getNumSenderSharedConnections()).isEqualTo(2);
+// there should still be receivers for the other member - endpoint not
removed!
+assertThat(connectionTable.hasReceiversFor(otherMember)).isTrue();
+
try {
await("for message to be sent").until(() -> {
final SerialAckedMessage serialAckedMessage = new SerialAckedMessage();
diff --git
a/geode-core/src/main/java/org/apache/geode/distributed/internal/direct/DirectChannel.java
b/geode-core/src/main/java/org/apache/geode/distributed/internal/direct/DirectChannel.java
index 1ab4bd5..c195655 100644
---
a/geode-core/src/main/java/org/apache/geode/distributed/internal/direct/DirectChannel.java
+++
b/geode-core/src/main/java/org/apache/geode/distributed/internal/direct/DirectChannel.java
@@ -267,6 +267,7 @@ public class DirectChannel {
final List cons = new ArrayList(destinations.length);
ConnectExceptions ce = getConnections(mgr, msg, destinations,
orderedMsg, retry, ackTimeout,
ackSDTimeout, cons);
+
if (directReply && msg.getProcessorId() > 0) { // no longer a
direct-reply message?
directReply = false;
}
@@ -690,11 +691,6 @@ public class DirectChannel {
}
}
- public void closeEndpoint(InternalDistributedMember member, String reason) {
-closeEndpoint(member, reason, true);
- }
-
-
/**
* Closes any connections used to communicate with the given jgroupsAddress.
*/
diff --git
a/geode-core/src/main/java/org/apache/geode/internal/tcp/Connection.java
b/geode-core/src/main/java/org/apache/geode/internal/tcp/Connection.java
index 92da89d..841b86c 100644
--- a/geode-core/src/main/
[geode] branch develop updated: GEODE-8206: CI Failure: ReconnectWithClusterConfigurationDUnitTest.testReconnectAfterMeltdown hang (#5192)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/develop by this push:
new 426d7fd GEODE-8206: CI Failure:
ReconnectWithClusterConfigurationDUnitTest.testReconnectAfterMeltdown hang
(#5192)
426d7fd is described below
commit 426d7fd41d0d66a7557f86236d6773f582e7ef0a
Author: Bruce Schuchardt
AuthorDate: Tue Jun 2 07:51:39 2020 -0700
GEODE-8206: CI Failure:
ReconnectWithClusterConfigurationDUnitTest.testReconnectAfterMeltdown hang
(#5192)
Modified the test to set the correct locator ports.
Modified the teardown code to tell the cache to stop
reconnecting.
Modified each run to use a temporary working directory so that runs
don't leave behind artifacts on disk that can taint subsequent runs.
---
...ReconnectWithClusterConfigurationDUnitTest.java | 33 ++
1 file changed, 21 insertions(+), 12 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/cache30/ReconnectWithClusterConfigurationDUnitTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/cache30/ReconnectWithClusterConfigurationDUnitTest.java
index d2c6d95..1c8f92c 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/cache30/ReconnectWithClusterConfigurationDUnitTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/cache30/ReconnectWithClusterConfigurationDUnitTest.java
@@ -31,6 +31,8 @@ import static org.assertj.core.api.Assertions.assertThat;
import java.io.File;
import java.io.IOException;
import java.io.Serializable;
+import java.net.UnknownHostException;
+import java.nio.file.Paths;
import java.util.List;
import java.util.Properties;
import java.util.concurrent.TimeUnit;
@@ -39,16 +41,17 @@ import org.junit.After;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
import org.apache.geode.cache.Cache;
import org.apache.geode.cache.CacheFactory;
import org.apache.geode.distributed.DistributedSystem;
import org.apache.geode.distributed.Locator;
-import
org.apache.geode.distributed.internal.InternalConfigurationPersistenceService;
import org.apache.geode.distributed.internal.InternalLocator;
import
org.apache.geode.distributed.internal.membership.api.MembershipManagerHelper;
import org.apache.geode.internal.AvailablePort;
import org.apache.geode.internal.AvailablePortHelper;
+import org.apache.geode.internal.inet.LocalHostUtil;
import org.apache.geode.test.awaitility.GeodeAwaitility;
import org.apache.geode.test.dunit.Assert;
import org.apache.geode.test.dunit.AsyncInvocation;
@@ -68,10 +71,14 @@ public class ReconnectWithClusterConfigurationDUnitTest
implements Serializable
static Properties dsProperties;
@Rule
- public DistributedRule distributedRule =
DistributedRule.builder().withVMCount(NUM_VMS).build();
+ public transient DistributedRule distributedRule =
+ DistributedRule.builder().withVMCount(NUM_VMS).build();
+
+ @Rule
+ public transient TemporaryFolder temporaryFolder = new TemporaryFolder();
@Before
- public void setup() {
+ public void setup() throws IOException {
List randomAvailableTCPPortKeepers =
AvailablePortHelper.getRandomAvailableTCPPortKeepers(NUM_LOCATORS);
for (int i = 0; i < NUM_LOCATORS; i++) {
@@ -81,6 +88,7 @@ public class ReconnectWithClusterConfigurationDUnitTest
implements Serializable
final int[] locPorts = locatorPorts;
Invoke.invokeInEveryVM("set locator ports", () -> locatorPorts = locPorts);
for (int i = 0; i < NUM_LOCATORS; i++) {
+ final String workingDir = temporaryFolder.newFolder().getAbsolutePath();
final int locatorNumber = i;
randomAvailableTCPPortKeepers.get(locatorNumber).release();
VM.getVM(i).invoke("start locator", () -> {
@@ -88,7 +96,9 @@ public class ReconnectWithClusterConfigurationDUnitTest
implements Serializable
Disconnect.disconnectFromDS();
dsProperties = null;
Properties props = getDistributedSystemProperties();
- locator = Locator.startLocatorAndDS(locatorPorts[locatorNumber], new
File(""), props);
+ locator = InternalLocator.startLocator(locatorPorts[locatorNumber],
new File(""),
+ null, null, LocalHostUtil.getLocalHost(), true,
+ props, null, Paths.get(workingDir));
system = locator.getDistributedSystem();
cache = ((InternalLocator) locator).getCache();
IgnoredException.addIgnoredException(
@@ -106,10 +116,8 @@ public class ReconnectWithClusterConfigurationDUnitTest
implements Serializable
VM.getVM(i).invoke(() -> {
InternalLocator locator = InternalLocator.getLocator();
if (locator != null) {
-
[geode] branch feature/GEODE-8206 created (now bc6bc73)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch feature/GEODE-8206 in repository https://gitbox.apache.org/repos/asf/geode.git. at bc6bc73 GEODE-8206: CI Failure: ReconnectWithClusterConfigurationDUnitTest.testReconnectAfterMeltdown hang This branch includes the following new commits: new bc6bc73 GEODE-8206: CI Failure: ReconnectWithClusterConfigurationDUnitTest.testReconnectAfterMeltdown hang The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[geode] 01/01: GEODE-8206: CI Failure: ReconnectWithClusterConfigurationDUnitTest.testReconnectAfterMeltdown hang
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch feature/GEODE-8206
in repository https://gitbox.apache.org/repos/asf/geode.git
commit bc6bc73c28d15d538c7ac1e23a0050b67c406e4c
Author: Bruce Schuchardt
AuthorDate: Mon Jun 1 15:16:53 2020 -0700
GEODE-8206: CI Failure:
ReconnectWithClusterConfigurationDUnitTest.testReconnectAfterMeltdown hang
Modified the test to set the correct locator ports.
Modified the teardown code to tell the cache to stop
reconnecting.
Modified each run to use a temporary working directory so that runs
don't leave behind artifacts on disk that can taint subsequent runs.
---
...ReconnectWithClusterConfigurationDUnitTest.java | 33 ++
1 file changed, 21 insertions(+), 12 deletions(-)
diff --git
a/geode-core/src/distributedTest/java/org/apache/geode/cache30/ReconnectWithClusterConfigurationDUnitTest.java
b/geode-core/src/distributedTest/java/org/apache/geode/cache30/ReconnectWithClusterConfigurationDUnitTest.java
index d2c6d95..1c8f92c 100644
---
a/geode-core/src/distributedTest/java/org/apache/geode/cache30/ReconnectWithClusterConfigurationDUnitTest.java
+++
b/geode-core/src/distributedTest/java/org/apache/geode/cache30/ReconnectWithClusterConfigurationDUnitTest.java
@@ -31,6 +31,8 @@ import static org.assertj.core.api.Assertions.assertThat;
import java.io.File;
import java.io.IOException;
import java.io.Serializable;
+import java.net.UnknownHostException;
+import java.nio.file.Paths;
import java.util.List;
import java.util.Properties;
import java.util.concurrent.TimeUnit;
@@ -39,16 +41,17 @@ import org.junit.After;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
import org.apache.geode.cache.Cache;
import org.apache.geode.cache.CacheFactory;
import org.apache.geode.distributed.DistributedSystem;
import org.apache.geode.distributed.Locator;
-import
org.apache.geode.distributed.internal.InternalConfigurationPersistenceService;
import org.apache.geode.distributed.internal.InternalLocator;
import
org.apache.geode.distributed.internal.membership.api.MembershipManagerHelper;
import org.apache.geode.internal.AvailablePort;
import org.apache.geode.internal.AvailablePortHelper;
+import org.apache.geode.internal.inet.LocalHostUtil;
import org.apache.geode.test.awaitility.GeodeAwaitility;
import org.apache.geode.test.dunit.Assert;
import org.apache.geode.test.dunit.AsyncInvocation;
@@ -68,10 +71,14 @@ public class ReconnectWithClusterConfigurationDUnitTest
implements Serializable
static Properties dsProperties;
@Rule
- public DistributedRule distributedRule =
DistributedRule.builder().withVMCount(NUM_VMS).build();
+ public transient DistributedRule distributedRule =
+ DistributedRule.builder().withVMCount(NUM_VMS).build();
+
+ @Rule
+ public transient TemporaryFolder temporaryFolder = new TemporaryFolder();
@Before
- public void setup() {
+ public void setup() throws IOException {
List randomAvailableTCPPortKeepers =
AvailablePortHelper.getRandomAvailableTCPPortKeepers(NUM_LOCATORS);
for (int i = 0; i < NUM_LOCATORS; i++) {
@@ -81,6 +88,7 @@ public class ReconnectWithClusterConfigurationDUnitTest
implements Serializable
final int[] locPorts = locatorPorts;
Invoke.invokeInEveryVM("set locator ports", () -> locatorPorts = locPorts);
for (int i = 0; i < NUM_LOCATORS; i++) {
+ final String workingDir = temporaryFolder.newFolder().getAbsolutePath();
final int locatorNumber = i;
randomAvailableTCPPortKeepers.get(locatorNumber).release();
VM.getVM(i).invoke("start locator", () -> {
@@ -88,7 +96,9 @@ public class ReconnectWithClusterConfigurationDUnitTest
implements Serializable
Disconnect.disconnectFromDS();
dsProperties = null;
Properties props = getDistributedSystemProperties();
- locator = Locator.startLocatorAndDS(locatorPorts[locatorNumber], new
File(""), props);
+ locator = InternalLocator.startLocator(locatorPorts[locatorNumber],
new File(""),
+ null, null, LocalHostUtil.getLocalHost(), true,
+ props, null, Paths.get(workingDir));
system = locator.getDistributedSystem();
cache = ((InternalLocator) locator).getCache();
IgnoredException.addIgnoredException(
@@ -106,10 +116,8 @@ public class ReconnectWithClusterConfigurationDUnitTest
implements Serializable
VM.getVM(i).invoke(() -> {
InternalLocator locator = InternalLocator.getLocator();
if (locator != null) {
- InternalConfigurationPersistenceService sharedConfig =
- locator.getConfigurationPersistenceService();
- if (sharedConfig != null) {
-sharedConfig.destroySharedConfiguration(
[geode] branch support/1.13 updated: GEODE-8144 another attempt to fix a failing test (#5172)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/support/1.13 by this push:
new 367e0a1 GEODE-8144 another attempt to fix a failing test (#5172)
367e0a1 is described below
commit 367e0a1557307ab6b8ba30c5a8b8c15324f6b552
Author: Bruce Schuchardt
AuthorDate: Thu May 28 14:44:58 2020 -0700
GEODE-8144 another attempt to fix a failing test (#5172)
one more test change
(cherry picked from commit 42c11640c75ae87038e07cbde3c98a3ca6b36987)
---
.../internal/cli/commands/GfshHostNameVerificationDistributedTest.java | 2 ++
1 file changed, 2 insertions(+)
diff --git
a/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
b/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
index d8014f4..3f125a3 100644
---
a/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
+++
b/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
@@ -65,8 +65,10 @@ public class GfshHostNameVerificationDistributedTest {
final CertificateBuilder builder = new CertificateBuilder()
.commonName("locator")
.issuedBy(ca)
+.sanDnsName(InetAddress.getLocalHost().getHostName())
.sanDnsName(InetAddress.getLoopbackAddress().getHostName())
.sanDnsName(hostname)
+.sanIpAddress(InetAddress.getByName("127.0.0.1"))
.sanIpAddress(InetAddress.getByName("0.0.0.0"));
// the rules used by this test use "localhost" as a hostname, which
// causes it to use a non-loopback IP literal address instead of the
[geode] branch support/1.13 updated (b3fd74a -> bcc097a)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch support/1.13 in repository https://gitbox.apache.org/repos/asf/geode.git. from b3fd74a GEODE-8174: Fix ConcurrentModificationException when using JTA transaction. (#5170) new 207e3ee GEODE-8144: endpoint identification in servers is not working (#5131) new 214fb5d Feature/geode 8144 (#5164) new bcc097a GEODE-8144 another attempt to fix a failing test (#5172) The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../client/sni/DualServerSNIAcceptanceTest.java| 3 +- .../org/apache/geode/client/sni/docker-compose.yml | 1 + .../client/sni/geode-config/gemfire.properties | 1 + .../client/sni/geode-config/gfsecurity.properties | 2 +- .../sni/geode-config/locator-maeve-keystore.jks| Bin 2048 -> 3525 bytes .../geode-config/server-clementine-keystore.jks| Bin 2059 -> 3537 bytes .../sni/geode-config/server-dolores-keystore.jks | Bin 2050 -> 3528 bytes .../geode/client/sni/geode-config/truststore.jks | Bin 8095 -> 1126 bytes .../geode/client/sni/scripts/geode-starter-2.gfsh | 8 ++--- .../geode/client/sni/scripts/geode-starter.gfsh| 6 ++-- .../RestrictUseOfInetAddressJUnitTest.java | 1 + .../geode/admin/internal/InetAddressUtils.java | 9 ++ .../distributed/internal/InternalLocator.java | 2 +- .../apache/geode/internal/net/SocketCreator.java | 33 + .../GfshHostNameVerificationDistributedTest.java | 15 ++ 15 files changed, 60 insertions(+), 21 deletions(-)
[geode] 01/03: GEODE-8144: endpoint identification in servers is not working (#5131)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
commit 207e3ee0c3aae704216bd0e9ec1275606b9116a2
Author: Bruce Schuchardt
AuthorDate: Tue May 26 07:51:18 2020 -0700
GEODE-8144: endpoint identification in servers is not working (#5131)
* GEODE-8144: endpoint identification in servers is not working
Set the SNI server-name field in SSL parameters for p2p communications,
allowing endpoint identification to work properly.
I modified one of the SNI haproxy tests to have keystores with the
proper subject-alternative-names for p2p communications in the docker
containers and for client/server off-platform communications. I used
Sai's keystore/truststore construction CertificateMaterial/CertStores
classes to generate the stores...
.sanDnsName("geode") // for inside the docker container
.sanDnsName("localhost") // for inside the docker container
.sanIpAddress(InetAddress.getByName("0.0.0.0")) // for inside the
docker container
.sanDnsName(certName) // for client endpoint validation
(locator-maeve for instance)
* modified SocketCreator to look for a hostname if one is not present and
endpoint verification is enabled
This fixes some problems when running in docker containers
* removed test shell script
* sanction used of getCanonicalHostName() in SocketCreator
(cherry picked from commit b22df8cf0f11b73237dd88dc1de9217f5f7cc8c3)
---
.../client/sni/DualServerSNIAcceptanceTest.java| 3 +-
.../org/apache/geode/client/sni/docker-compose.yml | 1 +
.../client/sni/geode-config/gemfire.properties | 1 +
.../client/sni/geode-config/gfsecurity.properties | 2 +-
.../sni/geode-config/locator-maeve-keystore.jks| Bin 2048 -> 3525 bytes
.../geode-config/server-clementine-keystore.jks| Bin 2059 -> 3537 bytes
.../sni/geode-config/server-dolores-keystore.jks | Bin 2050 -> 3528 bytes
.../geode/client/sni/geode-config/truststore.jks | Bin 8095 -> 1126 bytes
.../geode/client/sni/scripts/geode-starter-2.gfsh | 8 ++---
.../geode/client/sni/scripts/geode-starter.gfsh| 6 ++--
.../RestrictUseOfInetAddressJUnitTest.java | 1 +
.../distributed/internal/InternalLocator.java | 2 +-
.../apache/geode/internal/net/SocketCreator.java | 33 +
13 files changed, 41 insertions(+), 16 deletions(-)
diff --git
a/geode-assembly/src/acceptanceTest/java/org/apache/geode/client/sni/DualServerSNIAcceptanceTest.java
b/geode-assembly/src/acceptanceTest/java/org/apache/geode/client/sni/DualServerSNIAcceptanceTest.java
index 640d92a..7b08be2 100644
---
a/geode-assembly/src/acceptanceTest/java/org/apache/geode/client/sni/DualServerSNIAcceptanceTest.java
+++
b/geode-assembly/src/acceptanceTest/java/org/apache/geode/client/sni/DualServerSNIAcceptanceTest.java
@@ -26,7 +26,6 @@ import static
org.apache.geode.test.util.ResourceUtils.createTempFileFromResourc
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
-import java.io.IOException;
import java.net.URL;
import java.util.Properties;
@@ -74,7 +73,7 @@ public class DualServerSNIAcceptanceTest {
private ClientCache cache;
@BeforeClass
- public static void beforeClass() throws IOException, InterruptedException {
+ public static void beforeClass() throws Exception {
docker.get().exec(options("-T"), "geode",
arguments("gfsh", "run",
"--file=/geode/scripts/geode-starter-2.gfsh"));
diff --git
a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/docker-compose.yml
b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/docker-compose.yml
index 8caa12f..dd52102 100644
---
a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/docker-compose.yml
+++
b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/docker-compose.yml
@@ -19,6 +19,7 @@ services:
geode:
container_name: 'geode'
image: 'geode:develop'
+hostname: geode
expose:
- '10334'
- '40404'
diff --git
a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/geode-config/gemfire.properties
b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/geode-config/gemfire.properties
index 1f13fb0..9ae3497 100644
---
a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/geode-config/gemfire.properties
+++
b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/geode-config/gemfire.properties
@@ -17,3 +17,4 @@
statistic-sampling-enabled=true
statistic-archive-file=statArchive.gfs
+
diff --git
a/geode-assembly/src/accep
[geode] 03/03: GEODE-8144 another attempt to fix a failing test (#5172)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
commit bcc097a6dda38ab276720521ab0308cef7a4c1be
Author: Bruce Schuchardt
AuthorDate: Wed May 27 16:55:15 2020 -0700
GEODE-8144 another attempt to fix a failing test (#5172)
* modified SocketCreator to look for a hostname if one is not present and
endpoint verification is enabled
This fixes some problems when running in docker containers
* removed test shell script
* GEODE-8144 another attempt to fix a failing test
The JMX tests in this test class use "localhost" to connect to a
Manager. This is being transformed somehow into an IP numeric address
and is failing endpoint verification. The test passes on my Windows machine
and also passes on Mac and Ubuntu when I run it there. I'm
adding the "localhost" IP address to the certificate to get past this.
Another fix would be to change the Rule that's using "localhost" to
use the real host name but that would affect a lot of other tests.
Before I started messing with this test it was adding
InetAddress.getLocalHost() as an IP address to the certificate so I
don't think this is a big change to the original test. The test now
uses LocalHostUtils.getLocalHost() to get an IP to add to the
certificate, which is the correct "localhost" to use with Geode.
(cherry picked from commit 42c11640c75ae87038e07cbde3c98a3ca6b36987)
---
.../cli/commands/GfshHostNameVerificationDistributedTest.java | 10 --
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git
a/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
b/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
index 4d5f40d..d8014f4 100644
---
a/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
+++
b/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
@@ -29,7 +29,6 @@ import org.junit.Rule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
-import org.apache.geode.admin.internal.InetAddressUtils;
import org.apache.geode.cache.ssl.CertStores;
import org.apache.geode.cache.ssl.CertificateBuilder;
import org.apache.geode.cache.ssl.CertificateMaterial;
@@ -69,11 +68,10 @@ public class GfshHostNameVerificationDistributedTest {
.sanDnsName(InetAddress.getLoopbackAddress().getHostName())
.sanDnsName(hostname)
.sanIpAddress(InetAddress.getByName("0.0.0.0"));
-if (InetAddressUtils.isIPLiteral(hostname)) {
- // no valid hostname for this machine's IP address, so sanDnsName won't
work
- // and we need to use a sanIpAddress
- builder.sanIpAddress(LocalHostUtil.getLocalHost());
-}
+// the rules used by this test use "localhost" as a hostname, which
+// causes it to use a non-loopback IP literal address instead of the
+// host's name on CI Windows runs
+builder.sanIpAddress(LocalHostUtil.getLocalHost());
CertificateMaterial locatorCertificate = builder.generate();
CertificateMaterial gfshCertificate = new CertificateBuilder()
[geode] 02/03: Feature/geode 8144 (#5164)
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch support/1.13
in repository https://gitbox.apache.org/repos/asf/geode.git
commit 214fb5d6d19d7624e55811dcd16b16923a0027bd
Author: Bruce Schuchardt
AuthorDate: Wed May 27 11:12:43 2020 -0700
Feature/geode 8144 (#5164)
* modified SocketCreator to look for a hostname if one is not present and
endpoint verification is enabled
This fixes some problems when running in docker containers
* removed test shell script
* modified a test to use LocalHostUtil to get localhost
* remove use of InetAddress.getLocalHost & add a sanIpAddress if there is
no name for the localhost address
(cherry picked from commit a185267dca016954f857f6848863bc4f74e6de81)
---
.../apache/geode/admin/internal/InetAddressUtils.java | 9 +
.../GfshHostNameVerificationDistributedTest.java| 17 -
2 files changed, 21 insertions(+), 5 deletions(-)
diff --git
a/geode-core/src/main/java/org/apache/geode/admin/internal/InetAddressUtils.java
b/geode-core/src/main/java/org/apache/geode/admin/internal/InetAddressUtils.java
index ef24055..4609c1f 100644
---
a/geode-core/src/main/java/org/apache/geode/admin/internal/InetAddressUtils.java
+++
b/geode-core/src/main/java/org/apache/geode/admin/internal/InetAddressUtils.java
@@ -17,6 +17,8 @@ package org.apache.geode.admin.internal;
import java.net.InetAddress;
import java.net.UnknownHostException;
+import org.apache.commons.validator.routines.InetAddressValidator;
+
import org.apache.geode.internal.inet.LocalHostUtil;
/**
@@ -31,6 +33,13 @@ public class InetAddressUtils {
}
/**
+ * Returns true if the given hostname is an IPv4 or IPv6 numeric address
+ */
+ public static boolean isIPLiteral(String hostName) {
+return InetAddressValidator.getInstance().isValid(hostName);
+ }
+
+ /**
* Returns a string version of {@code InetAddress} which can be converted
back later. Essentially
* any leading slash is trimmed.
*
diff --git
a/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
b/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
index f232748..4d5f40d 100644
---
a/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
+++
b/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
@@ -29,9 +29,11 @@ import org.junit.Rule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
+import org.apache.geode.admin.internal.InetAddressUtils;
import org.apache.geode.cache.ssl.CertStores;
import org.apache.geode.cache.ssl.CertificateBuilder;
import org.apache.geode.cache.ssl.CertificateMaterial;
+import org.apache.geode.internal.inet.LocalHostUtil;
import org.apache.geode.test.dunit.IgnoredException;
import org.apache.geode.test.dunit.VM;
import org.apache.geode.test.dunit.rules.ClusterStartupRule;
@@ -60,14 +62,19 @@ public class GfshHostNameVerificationDistributedTest {
.isCA()
.generate();
-CertificateMaterial locatorCertificate = new CertificateBuilder()
+String hostname = LocalHostUtil.getCanonicalLocalHostName();
+final CertificateBuilder builder = new CertificateBuilder()
.commonName("locator")
.issuedBy(ca)
.sanDnsName(InetAddress.getLoopbackAddress().getHostName())
-.sanDnsName(InetAddress.getLocalHost().getHostName())
-.sanIpAddress(InetAddress.getLocalHost())
-.sanIpAddress(InetAddress.getByName("0.0.0.0")) // to pass on windows
-.generate();
+.sanDnsName(hostname)
+.sanIpAddress(InetAddress.getByName("0.0.0.0"));
+if (InetAddressUtils.isIPLiteral(hostname)) {
+ // no valid hostname for this machine's IP address, so sanDnsName won't
work
+ // and we need to use a sanIpAddress
+ builder.sanIpAddress(LocalHostUtil.getLocalHost());
+}
+CertificateMaterial locatorCertificate = builder.generate();
CertificateMaterial gfshCertificate = new CertificateBuilder()
.commonName("gfsh")
[geode] branch develop updated (53d1631 -> 42c1164)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 53d1631 GEODE-8171: javadoc for putAll need to have accurate exception (#5147) add 42c1164 GEODE-8144 another attempt to fix a failing test (#5172) No new revisions were added by this update. Summary of changes: .../cli/commands/GfshHostNameVerificationDistributedTest.java | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-)
[geode] 03/03: GEODE-8144 another attempt to fix a failing test
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch feature/GEODE-8144
in repository https://gitbox.apache.org/repos/asf/geode.git
commit d5cdaf09f560f84ab2adb0abde6ddb10ab3e5a7e
Author: Bruce Schuchardt
AuthorDate: Wed May 27 14:53:43 2020 -0700
GEODE-8144 another attempt to fix a failing test
The JMX tests in this test class use "localhost" to connect to a
Manager. This is being transformed somehow into an IP numeric address
and is failing endpoint verification. The test passes on my Windows machine
and also passes on Mac and Ubuntu when I run it there. I'm
adding the "localhost" IP address to the certificate to get past this.
Another fix would be to change the Rule that's using "localhost" to
use the real host name but that would affect a lot of other tests.
Before I started messing with this test it was adding
InetAddress.getLocalHost() as an IP address to the certificate so I
don't think this is a big change to the original test. The test now
uses LocalHostUtils.getLocalHost() to get an IP to add to the
certificate, which is the correct "localhost" to use with Geode.
---
.../cli/commands/GfshHostNameVerificationDistributedTest.java | 10 --
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git
a/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
b/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
index 4d5f40d..d8014f4 100644
---
a/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
+++
b/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
@@ -29,7 +29,6 @@ import org.junit.Rule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
-import org.apache.geode.admin.internal.InetAddressUtils;
import org.apache.geode.cache.ssl.CertStores;
import org.apache.geode.cache.ssl.CertificateBuilder;
import org.apache.geode.cache.ssl.CertificateMaterial;
@@ -69,11 +68,10 @@ public class GfshHostNameVerificationDistributedTest {
.sanDnsName(InetAddress.getLoopbackAddress().getHostName())
.sanDnsName(hostname)
.sanIpAddress(InetAddress.getByName("0.0.0.0"));
-if (InetAddressUtils.isIPLiteral(hostname)) {
- // no valid hostname for this machine's IP address, so sanDnsName won't
work
- // and we need to use a sanIpAddress
- builder.sanIpAddress(LocalHostUtil.getLocalHost());
-}
+// the rules used by this test use "localhost" as a hostname, which
+// causes it to use a non-loopback IP literal address instead of the
+// host's name on CI Windows runs
+builder.sanIpAddress(LocalHostUtil.getLocalHost());
CertificateMaterial locatorCertificate = builder.generate();
CertificateMaterial gfshCertificate = new CertificateBuilder()
[geode] 02/03: removed test shell script
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a commit to branch feature/GEODE-8144 in repository https://gitbox.apache.org/repos/asf/geode.git commit ca66e3ae98d827211ff05c929f18c96b831d3220 Author: Bruce Schuchardt AuthorDate: Thu May 21 15:05:42 2020 -0700 removed test shell script --- .../org/apache/geode/client/sni/scripts/cleandocker.sh| 11 --- 1 file changed, 11 deletions(-) diff --git a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh deleted file mode 100755 index 2d84de5..000 --- a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh +++ /dev/null @@ -1,11 +0,0 @@ -#!/bin/sh -# use this script to get rid of containers when test execution in gradle leaves them -# up, or when you've created the containers for command-line debugging, as in... - -# in the sni directory, use -# docker-compose up -# then use this to rsh to the container -# docker-compose exec geode sh - -docker kill haproxy geode -docker rm haproxy geode
[geode] branch feature/GEODE-8144 updated (37a2989 -> d5cdaf0)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch feature/GEODE-8144 in repository https://gitbox.apache.org/repos/asf/geode.git. omit 37a2989 remove use of InetAddress.getLocalHost & add a sanIpAddress if there is no name for the localhost address omit 342ab9a modified a test to use LocalHostUtil to get localhost omit 0b53d73 removed test shell script omit 989 modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled add 17dd67a GEODE-8182: change String commands to use one region (#5165) add a185267 Feature/geode 8144 (#5164) new 0323599 modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled new ca66e3a removed test shell script new d5cdaf0 GEODE-8144 another attempt to fix a failing test This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (37a2989) \ N -- N -- N refs/heads/feature/GEODE-8144 (d5cdaf0) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../GfshHostNameVerificationDistributedTest.java | 10 +- .../apache/geode/redis/GeodeRedisServerRule.java | 10 - .../redis/RedisLockServiceIntegrationTest.java | 63 -- .../apache/geode/redis/StringsIntegrationTest.java | 7 +- .../geode/redis/sets/SRemIntegrationTest.java | 63 -- .../codeAnalysis/sanctionedDataSerializables.txt | 4 - .../geode/redis/internal/AutoCloseableLock.java| 36 .../geode/redis/internal/ByteArrayWrapper.java | 13 ++ .../redis/internal/ExecutionHandlerContext.java| 19 +- .../geode/redis/internal/GeodeRedisServer.java | 72 +-- .../geode/redis/internal/KeyHashIdentifier.java| 70 --- .../apache/geode/redis/internal/KeyRegistrar.java | 173 - .../geode/redis/internal/RedisLockService.java | 110 --- .../redis/internal/RedisLockServiceMBean.java | 33 .../geode/redis/internal/RegionProvider.java | 86 + .../redis/internal/executor/AbstractExecutor.java | 33 +--- .../redis/internal/executor/CommandFunction.java | 5 - .../redis/internal/executor/DBSizeExecutor.java| 2 +- .../geode/redis/internal/executor/DelExecutor.java | 3 +- .../redis/internal/executor/ExistsExecutor.java| 3 +- .../internal/executor/ExpirationExecutor.java | 2 +- .../redis/internal/executor/ExpireAtExecutor.java | 7 +- .../redis/internal/executor/FlushAllExecutor.java | 10 +- .../redis/internal/executor/KeysExecutor.java | 2 +- .../redis/internal/executor/RedisHashInRegion.java | 5 - .../redis/internal/executor/RedisKeyInRegion.java | 10 +- .../redis/internal/executor/RenameExecutor.java| 94 - .../redis/internal/executor/ScanExecutor.java | 2 +- .../geode/redis/internal/executor/TTLExecutor.java | 9 +- .../redis/internal/executor/TypeExecutor.java | 7 +- .../internal/executor/hash/RedisHashCommands.java | 3 - .../hash/RedisHashCommandsFunctionExecutor.java| 7 - .../internal/executor/set/RedisSetCommands.java| 2 - .../set/RedisSetCommandsFunctionExecutor.java | 7 - .../internal/executor/set/RedisSetInRegion.java| 5 - .../internal/executor/set/SIsMemberExecutor.java | 9 +- .../redis/internal/executor/set/SMoveExecutor.java | 13 +- .../internal/executor/string/AppendExecutor.java | 13 +- .../internal/executor/string/BitCountExecutor.java | 9 +- .../internal/executor/string/BitOpExecutor.java| 38 ++-- .../internal/executor/string/BitPosExecutor.java | 9 +- .../internal/executor/string/DecrByExecutor.java | 18 +- .../internal/executor/string/DecrExecutor.java | 92 - .../internal/executor/string/GetBitExecutor.java | 11 +- .../internal/executor/string/GetExecutor.java | 24 +-- .../internal/executor/string/GetRangeExecutor.java | 11 +- .../internal/executor/string/GetSetExecutor.java | 41 +--- .../internal/executor/string/IncrByExecutor.java | 15 +- .../executor/string/IncrByFloatExecutor.java | 15 +- .../internal
[geode] 01/03: modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a commit to branch feature/GEODE-8144 in repository https://gitbox.apache.org/repos/asf/geode.git commit 03235994d7a4e0e277e38be767fe492f1c049f53 Author: Bruce Schuchardt AuthorDate: Thu May 21 14:16:11 2020 -0700 modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled This fixes some problems when running in docker containers --- .../org/apache/geode/client/sni/scripts/cleandocker.sh| 11 +++ 1 file changed, 11 insertions(+) diff --git a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh new file mode 100755 index 000..2d84de5 --- /dev/null +++ b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh @@ -0,0 +1,11 @@ +#!/bin/sh +# use this script to get rid of containers when test execution in gradle leaves them +# up, or when you've created the containers for command-line debugging, as in... + +# in the sni directory, use +# docker-compose up +# then use this to rsh to the container +# docker-compose exec geode sh + +docker kill haproxy geode +docker rm haproxy geode
[geode] branch develop updated (17dd67a -> a185267)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/geode.git. from 17dd67a GEODE-8182: change String commands to use one region (#5165) add a185267 Feature/geode 8144 (#5164) No new revisions were added by this update. Summary of changes: .../apache/geode/admin/internal/InetAddressUtils.java | 9 + .../GfshHostNameVerificationDistributedTest.java| 17 - 2 files changed, 21 insertions(+), 5 deletions(-)
[geode] branch feature/GEODE-8144 updated (342ab9a -> 37a2989)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch feature/GEODE-8144 in repository https://gitbox.apache.org/repos/asf/geode.git. from 342ab9a modified a test to use LocalHostUtil to get localhost add 37a2989 remove use of InetAddress.getLocalHost & add a sanIpAddress if there is no name for the localhost address No new revisions were added by this update. Summary of changes: .../apache/geode/admin/internal/InetAddressUtils.java | 9 + .../GfshHostNameVerificationDistributedTest.java | 18 +++--- 2 files changed, 20 insertions(+), 7 deletions(-)
[geode] branch feature/GEODE-8144 updated: modified a test to use LocalHostUtil to get localhost
This is an automated email from the ASF dual-hosted git repository.
bschuchardt pushed a commit to branch feature/GEODE-8144
in repository https://gitbox.apache.org/repos/asf/geode.git
The following commit(s) were added to refs/heads/feature/GEODE-8144 by this
push:
new 342ab9a modified a test to use LocalHostUtil to get localhost
342ab9a is described below
commit 342ab9a06be9bac3da5bf71d05a2b9750d7ac3d8
Author: Bruce Schuchardt
AuthorDate: Tue May 26 14:40:55 2020 -0700
modified a test to use LocalHostUtil to get localhost
---
.../internal/cli/commands/GfshHostNameVerificationDistributedTest.java | 3 +++
1 file changed, 3 insertions(+)
diff --git
a/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
b/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
index f232748..5e509b9 100644
---
a/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
+++
b/geode-gfsh/src/distributedTest/java/org/apache/geode/management/internal/cli/commands/GfshHostNameVerificationDistributedTest.java
@@ -32,6 +32,7 @@ import org.junit.experimental.categories.Category;
import org.apache.geode.cache.ssl.CertStores;
import org.apache.geode.cache.ssl.CertificateBuilder;
import org.apache.geode.cache.ssl.CertificateMaterial;
+import org.apache.geode.internal.inet.LocalHostUtil;
import org.apache.geode.test.dunit.IgnoredException;
import org.apache.geode.test.dunit.VM;
import org.apache.geode.test.dunit.rules.ClusterStartupRule;
@@ -65,6 +66,8 @@ public class GfshHostNameVerificationDistributedTest {
.issuedBy(ca)
.sanDnsName(InetAddress.getLoopbackAddress().getHostName())
.sanDnsName(InetAddress.getLocalHost().getHostName())
+.sanDnsName(LocalHostUtil.getCanonicalLocalHostName())
+.sanIpAddress(LocalHostUtil.getLocalHost())
.sanIpAddress(InetAddress.getLocalHost())
.sanIpAddress(InetAddress.getByName("0.0.0.0")) // to pass on windows
.generate();
[geode] branch feature/GEODE-8144 updated (6dc29fd -> 0b53d73)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch feature/GEODE-8144 in repository https://gitbox.apache.org/repos/asf/geode.git. omit 6dc29fd sanction used of getCanonicalHostName() in SocketCreator omit 3b4c6a7 removed test shell script omit 2682745 modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled omit 88e7fd9 GEODE-8144: endpoint identification in servers is not working add b38ae30 GEODE-7792: configure logging for geode-membership integration tests (#5116) add 1af3be3 GEODE-8100: update cluster-management-service documentation (#5121) add 6aa2420 Introduce RedisResponse to SET executors (#5114) add 7e222df GEODE-8147: change redis DELETE and EXISTS to use Function+Delta (#5128) add 47a5709 GEODE-8095: refactor operation BiFunction into a specific OperationPerformer interface (#5135) add aca029e GEODE-8100: Update cluster management service docs - style edit (#5141) add 7538de5 GEODE-8167: bump spring security to recommended version (#5144) add 19d5f78 GEODE-7458: Adding option in gfsh command "start gateway sender" to control clearing of existing queues (#4387) add c3c226a GEODE-8119:closing threads when offline disk store is executed (#5106) add 07bf3dd GEODE-8150: Downgrade classgraph to 4.8.52 (#5138) add d08847b GEODE-8112: Add --member option in query command. (#5102) add 9a0563e GEODE-8170: change all hash and set commands to use function (#5125) add 0f8da9e GEODE-8145: Add Redis configuration properties to gemfire_properties.html (#5130) add f2392d0 GEODE-8146: use latest winrm in tools image (#5134) add 358fd70 GEODE-8131: reader thread blocked attempting to issue an alert (#5132) add e0cbd78 GEODE-8127: ensure that redis function executes on primary (#5133) add d47e073 GEODE-8138: Improve semantics of the redis-port option (#5142) add 0a7f8ae GEODE-8168: Redis pipelined command responses can be corrupted (#5145) add af1ea6d GEODE-8151: Convert hash commands to return RedisResponse (#5140) add f243c4d Revert GEODE-8127: the test is flakey (#5153) add a05b86d GEODE-8175: remove unsupported redis commands (#5146) add 03355b9 GEODE-8175: fix compile error (#5156) add e5e803e GEODE-8180: add 1.14 to management wiki (#5154) add 99b12ed GEODE-8108: Remove System.out.println calls from geode-redis (#5149) add c79905f Revert "GEODE-8119:closing threads when offline disk store is executed (#5106)" (#5158) add e432855 GEODE-7971: Gw sender deliver TX events atomically to Gw receivers (#4928) add b22df8c GEODE-8144: endpoint identification in servers is not working (#5131) add 253d667 GEODE-8136: Move UncheckedUtils to geode-common (#5123) add 5e00b10 GEODE-8134: convert String commands to use Function+Delta (#5160) add bef07b3 GEODE-8174: Fix ConcurrentModificationException when using JTA transaction. (#5161) new 989 modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled new 0b53d73 removed test shell script This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (6dc29fd) \ N -- N -- N refs/heads/feature/GEODE-8144 (0b53d73) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../src/test/resources/expected-pom.xml| 18 +- .../gradle/plugins/DependencyConstraints.groovy|5 +- ci/images/alpine-tools/Dockerfile | 22 +- ci/pipelines/meta/jinja.template.yml |1 + .../integrationTest/resources/assembly_content.txt |2 +- .../resources/dependency_classpath.txt |2 +- geode-assembly/src/main/dist/LICENSE |2 +- .../apache/geode/util/internal/UncheckedUtils.java | 25 +- .../geode/util/internal/UncheckedUtilsTest.java| 60 + .../cache/PartitionedRegionSingleHopDUnitTest.java | 14 +- ...istributedRegionFunctionExecutionDUnitTest.java | 59 +
[geode] 02/02: removed test shell script
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a commit to branch feature/GEODE-8144 in repository https://gitbox.apache.org/repos/asf/geode.git commit 0b53d7378efa4c907c5e0c8497a3c190239766fe Author: Bruce Schuchardt AuthorDate: Thu May 21 15:05:42 2020 -0700 removed test shell script --- .../org/apache/geode/client/sni/scripts/cleandocker.sh| 11 --- 1 file changed, 11 deletions(-) diff --git a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh deleted file mode 100755 index 2d84de5..000 --- a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh +++ /dev/null @@ -1,11 +0,0 @@ -#!/bin/sh -# use this script to get rid of containers when test execution in gradle leaves them -# up, or when you've created the containers for command-line debugging, as in... - -# in the sni directory, use -# docker-compose up -# then use this to rsh to the container -# docker-compose exec geode sh - -docker kill haproxy geode -docker rm haproxy geode
[geode] 01/02: modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a commit to branch feature/GEODE-8144 in repository https://gitbox.apache.org/repos/asf/geode.git commit 9890203df763cac3651eaccab9938708073b Author: Bruce Schuchardt AuthorDate: Thu May 21 14:16:11 2020 -0700 modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled This fixes some problems when running in docker containers --- .../org/apache/geode/client/sni/scripts/cleandocker.sh| 11 +++ 1 file changed, 11 insertions(+) diff --git a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh new file mode 100755 index 000..2d84de5 --- /dev/null +++ b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh @@ -0,0 +1,11 @@ +#!/bin/sh +# use this script to get rid of containers when test execution in gradle leaves them +# up, or when you've created the containers for command-line debugging, as in... + +# in the sni directory, use +# docker-compose up +# then use this to rsh to the container +# docker-compose exec geode sh + +docker kill haproxy geode +docker rm haproxy geode
[geode] 01/02: modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a commit to branch feature/GEODE-8144 in repository https://gitbox.apache.org/repos/asf/geode.git commit 9890203df763cac3651eaccab9938708073b Author: Bruce Schuchardt AuthorDate: Thu May 21 14:16:11 2020 -0700 modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled This fixes some problems when running in docker containers --- .../org/apache/geode/client/sni/scripts/cleandocker.sh| 11 +++ 1 file changed, 11 insertions(+) diff --git a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh new file mode 100755 index 000..2d84de5 --- /dev/null +++ b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh @@ -0,0 +1,11 @@ +#!/bin/sh +# use this script to get rid of containers when test execution in gradle leaves them +# up, or when you've created the containers for command-line debugging, as in... + +# in the sni directory, use +# docker-compose up +# then use this to rsh to the container +# docker-compose exec geode sh + +docker kill haproxy geode +docker rm haproxy geode
[geode] 02/02: removed test shell script
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a commit to branch feature/GEODE-8144 in repository https://gitbox.apache.org/repos/asf/geode.git commit 0b53d7378efa4c907c5e0c8497a3c190239766fe Author: Bruce Schuchardt AuthorDate: Thu May 21 15:05:42 2020 -0700 removed test shell script --- .../org/apache/geode/client/sni/scripts/cleandocker.sh| 11 --- 1 file changed, 11 deletions(-) diff --git a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh b/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh deleted file mode 100755 index 2d84de5..000 --- a/geode-assembly/src/acceptanceTest/resources/org/apache/geode/client/sni/scripts/cleandocker.sh +++ /dev/null @@ -1,11 +0,0 @@ -#!/bin/sh -# use this script to get rid of containers when test execution in gradle leaves them -# up, or when you've created the containers for command-line debugging, as in... - -# in the sni directory, use -# docker-compose up -# then use this to rsh to the container -# docker-compose exec geode sh - -docker kill haproxy geode -docker rm haproxy geode
[geode] branch feature/GEODE-8144 updated (6dc29fd -> 0b53d73)
This is an automated email from the ASF dual-hosted git repository. bschuchardt pushed a change to branch feature/GEODE-8144 in repository https://gitbox.apache.org/repos/asf/geode.git. omit 6dc29fd sanction used of getCanonicalHostName() in SocketCreator omit 3b4c6a7 removed test shell script omit 2682745 modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled omit 88e7fd9 GEODE-8144: endpoint identification in servers is not working add b38ae30 GEODE-7792: configure logging for geode-membership integration tests (#5116) add 1af3be3 GEODE-8100: update cluster-management-service documentation (#5121) add 6aa2420 Introduce RedisResponse to SET executors (#5114) add 7e222df GEODE-8147: change redis DELETE and EXISTS to use Function+Delta (#5128) add 47a5709 GEODE-8095: refactor operation BiFunction into a specific OperationPerformer interface (#5135) add aca029e GEODE-8100: Update cluster management service docs - style edit (#5141) add 7538de5 GEODE-8167: bump spring security to recommended version (#5144) add 19d5f78 GEODE-7458: Adding option in gfsh command "start gateway sender" to control clearing of existing queues (#4387) add c3c226a GEODE-8119:closing threads when offline disk store is executed (#5106) add 07bf3dd GEODE-8150: Downgrade classgraph to 4.8.52 (#5138) add d08847b GEODE-8112: Add --member option in query command. (#5102) add 9a0563e GEODE-8170: change all hash and set commands to use function (#5125) add 0f8da9e GEODE-8145: Add Redis configuration properties to gemfire_properties.html (#5130) add f2392d0 GEODE-8146: use latest winrm in tools image (#5134) add 358fd70 GEODE-8131: reader thread blocked attempting to issue an alert (#5132) add e0cbd78 GEODE-8127: ensure that redis function executes on primary (#5133) add d47e073 GEODE-8138: Improve semantics of the redis-port option (#5142) add 0a7f8ae GEODE-8168: Redis pipelined command responses can be corrupted (#5145) add af1ea6d GEODE-8151: Convert hash commands to return RedisResponse (#5140) add f243c4d Revert GEODE-8127: the test is flakey (#5153) add a05b86d GEODE-8175: remove unsupported redis commands (#5146) add 03355b9 GEODE-8175: fix compile error (#5156) add e5e803e GEODE-8180: add 1.14 to management wiki (#5154) add 99b12ed GEODE-8108: Remove System.out.println calls from geode-redis (#5149) add c79905f Revert "GEODE-8119:closing threads when offline disk store is executed (#5106)" (#5158) add e432855 GEODE-7971: Gw sender deliver TX events atomically to Gw receivers (#4928) add b22df8c GEODE-8144: endpoint identification in servers is not working (#5131) add 253d667 GEODE-8136: Move UncheckedUtils to geode-common (#5123) add 5e00b10 GEODE-8134: convert String commands to use Function+Delta (#5160) add bef07b3 GEODE-8174: Fix ConcurrentModificationException when using JTA transaction. (#5161) new 989 modified SocketCreator to look for a hostname if one is not present and endpoint verification is enabled new 0b53d73 removed test shell script This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (6dc29fd) \ N -- N -- N refs/heads/feature/GEODE-8144 (0b53d73) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../src/test/resources/expected-pom.xml| 18 +- .../gradle/plugins/DependencyConstraints.groovy|5 +- ci/images/alpine-tools/Dockerfile | 22 +- ci/pipelines/meta/jinja.template.yml |1 + .../integrationTest/resources/assembly_content.txt |2 +- .../resources/dependency_classpath.txt |2 +- geode-assembly/src/main/dist/LICENSE |2 +- .../apache/geode/util/internal/UncheckedUtils.java | 25 +- .../geode/util/internal/UncheckedUtilsTest.java| 60 + .../cache/PartitionedRegionSingleHopDUnitTest.java | 14 +- ...istributedRegionFunctionExecutionDUnitTest.java | 59 +
