Build failed in Jenkins: Geode-nightly #924

[Apache Jenkins Server]

See 


Changes:

[sjewll] GEODE-3423: Provide support for running parallel docker builds in

[jdeppe] GEODE-3423: Use openjdk:8 as the base

[dbarnes] GEODE-3395 Variable-ize product version and name in user guide - 
reflect

[dbarnes] GEODE-3395 Variable-ize product version and name in user guide -

[ukohlmeyer] GEODE-3393: One-way SSL commit failing with userHome/.keystore not

[ukohlmeyer] GEODE-3402: Mark ProtoBuf interface as experimental. This now 
closes

--
[...truncated 128.45 KB...]
at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44)

Caused by:
java.net.BindException: Failed to create server socket on  null[40,404]
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:786)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:748)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:715)
at 
org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.(AcceptorImpl.java:469)
at 
org.apache.geode.internal.cache.CacheServerImpl.start(CacheServerImpl.java:344)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65)

Caused by:
java.net.BindException: Address already in use (Bind failed)
at java.net.PlainSocketImpl.socketBind(Native Method)
at 
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
at java.net.ServerSocket.bind(ServerSocket.java:375)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:782)
... 5 more

org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > 
testMultipleAttributeUpdates FAILED
org.apache.geode.test.dunit.RMIException: While invoking 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$55/2095854288.call
 in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs
at org.apache.geode.test.dunit.VM.invoke(VM.java:387)
at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44)

Caused by:
java.net.BindException: Failed to create server socket on  null[40,404]
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:786)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:748)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:715)
at 
org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.(AcceptorImpl.java:469)
at 
org.apache.geode.internal.cache.CacheServerImpl.start(CacheServerImpl.java:344)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65)

Caused by:
java.net.BindException: Address already in use (Bind failed)
at java.net.PlainSocketImpl.socketBind(Native Method)
at 
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
at java.net.ServerSocket.bind(ServerSocket.java:375)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:782)
... 5 more

org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > 
testSanity FAILED
org.apache.geode.test.dunit.RMIException: While invoking 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$55/2095854288.call
 in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs
at org.apache.geode.test.dunit.VM.invoke(VM.java:387)
at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60)
at 

[GitHub] geode pull request #702: GEODE-3416: Reduce synchronization blockages in Soc...

[kohlmu-pivotal]

Github user kohlmu-pivotal commented on a diff in the pull request:

https://github.com/apache/geode/pull/702#discussion_r133353995
  
--- Diff: 
geode-core/src/main/java/org/apache/geode/internal/tcp/ConnectionTable.java ---
@@ -929,10 +929,6 @@ protected void removeEndpoint(DistributedMember 
memberID, String reason,
   owner.getDM().getMembershipManager().getShutdownCause());
 }
   }
-
-  if (remoteAddress != null) {
-
this.socketCloser.releaseResourcesForAddress(remoteAddress.toString());
-  }
--- End diff --

it does a missed revert...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Permission to assign and close JIRA tickets

[Anthony Baker]

done

> On Aug 15, 2017, at 9:39 AM, Alexander Murmann  wrote:
> 
> amurmann
> 
> Thanks!
> 
> On Tue, Aug 15, 2017 at 9:05 AM, Anthony Baker  wrote:
> 
>> What’s your JIRA username?
>> 
>>> On Aug 15, 2017, at 8:40 AM, Alexander Murmann 
>> wrote:
>>> 
>>> Good morning,
>>> 
>>> Can I please get permission to assign JIRA tickets to myself as well as
>>> close them?
>>> 
>>> Thank you!
>> 
>> 

[GitHub] geode pull request #702: GEODE-3416: Reduce synchronization blockages in Soc...

[bschuchardt]

Github user bschuchardt commented on a diff in the pull request:

https://github.com/apache/geode/pull/702#discussion_r133325744
  
--- Diff: 
geode-core/src/main/java/org/apache/geode/internal/net/SocketCloser.java ---
@@ -96,46 +99,55 @@ public int getMaxThreads() {
 return this.asyncClosePoolMaxThreads;
   }
 
-  private ThreadPoolExecutor getAsyncThreadExecutor(String address) {
-synchronized (asyncCloseExecutors) {
-  ThreadPoolExecutor pool = asyncCloseExecutors.get(address);
-  if (pool == null) {
-final ThreadGroup tg = 
LoggingThreadGroup.createThreadGroup("Socket asyncClose", logger);
-ThreadFactory tf = new ThreadFactory() {
-  public Thread newThread(final Runnable command) {
-Thread thread = new Thread(tg, command);
-thread.setDaemon(true);
-return thread;
-  }
-};
-BlockingQueue workQueue = new 
LinkedBlockingQueue();
-pool = new ThreadPoolExecutor(this.asyncClosePoolMaxThreads, 
this.asyncClosePoolMaxThreads,
-this.asyncClosePoolKeepAliveSeconds, TimeUnit.SECONDS, 
workQueue, tf);
-pool.allowCoreThreadTimeOut(true);
-asyncCloseExecutors.put(address, pool);
+  private ExecutorService getAsyncThreadExecutor(String address) {
+ExecutorService executorService = asyncCloseExecutors.get(address);
+if (executorService == null) {
+  // To be used for pre-1.8 jdk releases.
+  // createThreadPool();
+
+  executorService = 
Executors.newWorkStealingPool(asyncClosePoolMaxThreads);
+
+  ExecutorService previousThreadPoolExecutor =
+  asyncCloseExecutors.putIfAbsent(address, executorService);
+
+  if (previousThreadPoolExecutor != null) {
+executorService.shutdownNow();
+return previousThreadPoolExecutor;
   }
-  return pool;
 }
+return executorService;
+  }
+
+  /**
+   * @deprecated this method is to be used for pre 1.8 jdk.
+   */
+  @Deprecated
+  private void createThreadPool() {
+ExecutorService executorService;
+final ThreadGroup threadGroup =
+LoggingThreadGroup.createThreadGroup("Socket asyncClose", logger);
+ThreadFactory threadFactory = new ThreadFactory() {
+  public Thread newThread(final Runnable command) {
+Thread thread = new Thread(threadGroup, command);
+thread.setDaemon(true);
+return thread;
+  }
+};
+
+executorService = new ThreadPoolExecutor(asyncClosePoolMaxThreads, 
asyncClosePoolMaxThreads,
+asyncCloseWaitTime, asyncCloseWaitUnits, new 
LinkedBlockingQueue<>(), threadFactory);
   }
 
   /**
* Call this method if you know all the resources in the closer for the 
given address are no
* longer needed. Currently a thread pool is kept for each address and 
if you know that an address
* no longer needs its pool then you should call this method.
*/
-  public void releaseResourcesForAddress(String address) {
-synchronized (asyncCloseExecutors) {
-  ThreadPoolExecutor pool = asyncCloseExecutors.get(address);
-  if (pool != null) {
-pool.shutdown();
-asyncCloseExecutors.remove(address);
-  }
-}
-  }
 
-  private boolean isClosed() {
-synchronized (asyncCloseExecutors) {
-  return this.closed;
+  public void releaseResourcesForAddress(String address) {
+ExecutorService executorService = asyncCloseExecutors.remove(address);
+if (executorService != null) {
+  executorService.shutdown();
--- End diff --

When the cache closes it invokes ConnectionTable.close() which closes each 
connection and _afterwards_ closes its SocketCloser.  All socket closing in 
this case will be performed asynchronously.

The same is done in CacheClientNotifier.  All dispatching sockets are 
closed and then the SocketCloser is closed.

Client proxy and connection table receiver threads are closed by shutting 
down their executors.

Both of these behaviors are essential if the auto-reconnect mechanism is 
going to work properly.  Otherwise shutdown of the old cache may hang waiting 
on the keepalive timeout.

I think extensive testing of this change is needed.  Network partition 
detection and auto-reconnect testing needs to be done to ensure that the cache 
can be properly closed under harsh conditions.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA 

Re: Review Request 61676: Release the lock held when beforeCompletion failed with CommitConflictException

[Nick Reich]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61676/#review183009
---


Ship it!




Ship It!

- Nick Reich


On Aug. 15, 2017, 10:47 p.m., Eric Shu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61676/
> ---
> 
> (Updated Aug. 15, 2017, 10:47 p.m.)
> 
> 
> Review request for geode, anilkumar gingade, Darrel Schneider, Lynn Gallinat, 
> and Nick Reich.
> 
> 
> Bugs: GEODE-3444
> https://issues.apache.org/jira/browse/GEODE-3444
> 
> 
> Repository: geode
> 
> 
> Description
> ---
> 
> When JTA transaction failed with exception, it is better to release the locks 
> it held right away.
> 
> 
> Diffs
> -
> 
>   geode-core/src/main/java/org/apache/geode/internal/cache/TXState.java 
> 55415e3 
> 
> 
> Diff: https://reviews.apache.org/r/61676/diff/1/
> 
> 
> Testing
> ---
> 
> Precheckin.
> 
> 
> Thanks,
> 
> Eric Shu
> 
>

Review Request 61676: Release the lock held when beforeCompletion failed with CommitConflictException

[Eric Shu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61676/
---

Review request for geode, anilkumar gingade, Darrel Schneider, Lynn Gallinat, 
and Nick Reich.


Bugs: GEODE-3444
https://issues.apache.org/jira/browse/GEODE-3444


Repository: geode


Description
---

When JTA transaction failed with exception, it is better to release the locks 
it held right away.


Diffs
-

  geode-core/src/main/java/org/apache/geode/internal/cache/TXState.java 55415e3 


Diff: https://reviews.apache.org/r/61676/diff/1/


Testing
---

Precheckin.


Thanks,

Eric Shu

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #648 was SUCCESSFUL (with 2027 tests)

[Spring CI]

---
Spring Data GemFire > Nightly-ApacheGeode > #648 was successful.
---
Scheduled
2029 tests in total.

https://build.spring.io/browse/SGF-NAG-648/





--
This message is automatically generated by Atlassian Bamboo

Re: continuous query internal mechanism questions

[Anilkumar Gingade]

In Geode, high availability for subscription events are achieved by having
redundant event-queues (HAQueues) on multiple severs; this is configured
using redundancy-level with client connection. Based on the redundancy
level, the client register CQs on multiple servers. During the subscription
(CQ) registration, it elects/assigns one of the server to host primary
HAQueue.

The client keeps monitoring the redundancy level during node join or
failure; to satisfy the redundancy level.

You can find more about HAQueues at
https://cwiki.apache.org/confluence/display/GEODE/HA+Client+Event+Queues

I assume, you have 2 node cluster. What is your subscription redundancy
level?

>> For some reason, sometimes there is a failure to complete the first
registration
Is there any log message, stack trace, reporting reason for failure? If its
dev environment, you can run client/server with debug/fine level log to see
additional info.

Are you trying to stop your server, while registering the CQs? Can you give
more detail about your test scenario...

-Anil.


On Tue, Aug 15, 2017 at 11:25 AM, Jason Huynh  wrote:

> I am not quite sure how native client registers cqs. From my understanding:
> with the java api, I believe there is only one message (ExecuteCQ message)
> that is executed on the server side and then replicated to the other nodes
> through the profile (OperationMessage).
>
> It seems the extra ExecuteCQ message failing and then closing the cq might
> be putting the system in a weird state...
>
> On Tue, Aug 15, 2017 at 7:56 AM Roi Apelker 
> wrote:
>
> > Hi,
> >
> > I have been examining the continuous query registration mechanism for
> > quite some time
> > This is related to an issue that I have, where sometimes a node crashes
> (1
> > node out of 2), and the other one does not send CQ events. The CQ is
> > registered on a partitioned region which resides on these 2 nodes.
> >
> > I noticed the following behavior, and I wonder if anyone can comment
> > regarding it, if it is justified or not and what is the reason:
> >
> > 1. When the software using the client (native client) registers for the
> > CQ, a CQ command (ExecuteCQ61) is received on both servers.
> >  -- is this normal behaviour? Does the client actually send this command
> > to both servers?
> >
> > 2. When this command is received by a server, and the CQ is registered,
> > another registration message is sent to the other node via an
> > OperationMessage (REGISTER_CQ)
> >  -- it seems that regularly, the server can handle this situation as the
> > second registration identifies the previous one and does not affect it.
> but
> > the question, why do we need this 2nd registration, if there is a command
> > sent to each server?
> >
> > 3. For some reason, sometimes there is a failure to complete the first
> > registration (executed by ExecuteCQ61) and then this failure causes a
> > closure to the CQ, which is accompanied with a close request to the other
> > node.
> >  -- I assume by now, since 2 registrations and one closure have occurred
> > on node 2, the CQ is still active and the client receives notifications.
> >
> > 4. Sometimes, 1 out of 5, once node 1 crashes, I get a cleanup operation,
> > caused by the crash (via MemberCrashedEvent), and this also closes the
> > existing CQ, and in this case the CQ in node 2 does not operate anymore
> and
> > the client receives no notifications.
> >  -- fact is, that 4 out of 4 times, I do not get this cleanup by
> > MemberCrashedEvent (maybe due to some other error), and that the CQ
> > notifications are received normally.
> >
> > Can anyone clear things up for me? Any comment on any of the statements
> > above will be greatly appreciated.
> >
> > Thanks,
> >
> > Roi
> >
> >
> > -Original Message-
> > From: Roi Apelker
> > Sent: Wednesday, August 09, 2017 3:21 PM
> > To: dev@geode.apache.org
> > Subject: RE: continuous query internal mechanism
> >
> > Dhanyavad
> >
> > -Original Message-
> > From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> > Sent: Tuesday, August 08, 2017 9:55 PM
> > To: dev@geode.apache.org
> > Subject: Re: continuous query internal mechanism
> >
> > Registered events, i meant, are events generated for interest
> registration
> > "region.registerInterest(*)". And CqEvents are for CQs registered.
> >
> > -Anil.
> >
> >
> > On Tue, Aug 8, 2017 at 12:27 AM, Roi Apelker 
> > wrote:
> >
> > > Shukriya
> > >
> > > What is the difference between registered events and CQ events?
> > >
> > > -Original Message-
> > > From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> > > Sent: Monday, August 07, 2017 10:12 PM
> > > To: dev@geode.apache.org
> > > Subject: Re: continuous query internal mechanism
> > >
> > > CQ Processing on server side is same for all clients (Java, C++)...
> > >
> > > The subscription events are sent to client as ClientUpdateMessage,
> > > which holds information about 

Re: Review Request 61563: GEODE-3383: Refactor deploy tests

[Kirk Lund]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61563/#review183006
---


Ship it!




Ship It!

- Kirk Lund


On Aug. 11, 2017, 9:52 p.m., Jared Stewart wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61563/
> ---
> 
> (Updated Aug. 11, 2017, 9:52 p.m.)
> 
> 
> Review request for geode, Emily Yeh, Jared Stewart, Ken Howe, Kirk Lund, and 
> Patrick Rhomberg.
> 
> 
> Repository: geode
> 
> 
> Description
> ---
> 
> GEODE-3383: Refactor deploy tests
> 
> - Refactor DeployedJarJUnitTest
> - Move several tests into more appropriate locations
> 
> 
> Diffs
> -
> 
>   
> geode-core/src/test/java/org/apache/geode/internal/ClassPathLoaderIntegrationTest.java
>  34d8a2318422edbb3bbdc04920a41693f8d3fe69 
>   
> geode-core/src/test/java/org/apache/geode/internal/DeployedJarJUnitTest.java 
> 178dbae2eaada0cc054502fdf4b6d2ff102b4ed6 
>   
> geode-core/src/test/java/org/apache/geode/internal/JarDeployerDeadlockTest.java
>  PRE-CREATION 
>   
> geode-core/src/test/java/org/apache/geode/management/DeployJarTestSuite.java 
> 6dfab66926c7b246bf839632b293330959f1d728 
>   
> geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/DeployCommandsDUnitTest.java
>  89148d7752ae1f69e25671ffc43101a63cf7dc64 
>   
> geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/CommandOverHttpDUnitTest.java
>  7753aafbd7dc5ea4288e27f088400163f6739347 
> 
> 
> Diff: https://reviews.apache.org/r/61563/diff/1/
> 
> 
> Testing
> ---
> 
> Precheckin passed
> 
> 
> Thanks,
> 
> Jared Stewart
> 
>

Re: Review Request 61627: GEODE-3437: Fix list and describe region tests

[Kirk Lund]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61627/#review183007
---


Ship it!




Ship It!

- Kirk Lund


On Aug. 14, 2017, 10:40 p.m., Jared Stewart wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61627/
> ---
> 
> (Updated Aug. 14, 2017, 10:40 p.m.)
> 
> 
> Review request for geode, Emily Yeh, Jared Stewart, Ken Howe, Kirk Lund, and 
> Patrick Rhomberg.
> 
> 
> Repository: geode
> 
> 
> Description
> ---
> 
> GEODE-3437: Fix list and describe region tests
> 
> 
> Diffs
> -
> 
>   
> geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ListAndDescribeRegionDUnitTest.java
>  ab8c69b7cc99c88dd4e928efeb441d7d1a1d9b1b 
>   
> geode-core/src/test/java/org/apache/geode/test/dunit/rules/LocatorServerStartupRule.java
>  fc7966f5eb2a9ca4c30369a20ce664d3929ecc22 
> 
> 
> Diff: https://reviews.apache.org/r/61627/diff/1/
> 
> 
> Testing
> ---
> 
> Precheckin running
> 
> 
> Thanks,
> 
> Jared Stewart
> 
>

Re: Review Request 61671: GEODE-3328: fix testAddGemFirePropertyFileToCommandLineNew on Windows

[Kirk Lund]

> On Aug. 15, 2017, 9:38 p.m., Ken Howe wrote:
> > geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/GfshCommandJUnitTest.java
> > Line 411 (original), 411 (patched)
> > 
> >
> > Why was this test renamed? Not really a problem but on the surface 
> > looks unneeded.

Oops that was caused by me have both versions of the test in the class briefly. 
I'll remove "New"


- Kirk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61671/#review182999
---


On Aug. 15, 2017, 7:29 p.m., Kirk Lund wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61671/
> ---
> 
> (Updated Aug. 15, 2017, 7:29 p.m.)
> 
> 
> Review request for geode, Jinmei Liao, Jared Stewart, and Ken Howe.
> 
> 
> Bugs: GEODE-3328
> https://issues.apache.org/jira/browse/GEODE-3328
> 
> 
> Repository: geode
> 
> 
> Description
> ---
> 
> I had to temporarily revert Jinmei's fix for GEODE-3328 in order to fully 
> revert Emily's refactorings of GFSH commands. This commit reintroduces her 
> fix with minor tweak to get it to work without Emily's changes.
> 
> 
> Diffs
> -
> 
>   
> geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/GfshCommandJUnitTest.java
>  da60c7aa481954be0acc8c7e2b88717cf8bc9c37 
> 
> 
> Diff: https://reviews.apache.org/r/61671/diff/1/
> 
> 
> Testing
> ---
> 
> precheckin in progress
> 
> 
> Thanks,
> 
> Kirk Lund
> 
>

Re: Review Request 61671: GEODE-3328: fix testAddGemFirePropertyFileToCommandLineNew on Windows

[Ken Howe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61671/#review182999
---


Ship it!





geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/GfshCommandJUnitTest.java
Line 411 (original), 411 (patched)


Why was this test renamed? Not really a problem but on the surface looks 
unneeded.


- Ken Howe


On Aug. 15, 2017, 7:29 p.m., Kirk Lund wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61671/
> ---
> 
> (Updated Aug. 15, 2017, 7:29 p.m.)
> 
> 
> Review request for geode, Jinmei Liao, Jared Stewart, and Ken Howe.
> 
> 
> Bugs: GEODE-3328
> https://issues.apache.org/jira/browse/GEODE-3328
> 
> 
> Repository: geode
> 
> 
> Description
> ---
> 
> I had to temporarily revert Jinmei's fix for GEODE-3328 in order to fully 
> revert Emily's refactorings of GFSH commands. This commit reintroduces her 
> fix with minor tweak to get it to work without Emily's changes.
> 
> 
> Diffs
> -
> 
>   
> geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/GfshCommandJUnitTest.java
>  da60c7aa481954be0acc8c7e2b88717cf8bc9c37 
> 
> 
> Diff: https://reviews.apache.org/r/61671/diff/1/
> 
> 
> Testing
> ---
> 
> precheckin in progress
> 
> 
> Thanks,
> 
> Kirk Lund
> 
>

Re: Review Request 61411: GEODE-3286 Failing to cleanup connections from ConnectionTable receiver table (corrected "stopped" check in previous fix)

[Udo Kohlmeyer]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61411/#review182998
---


Ship it!




Ship It!

- Udo Kohlmeyer


On Aug. 3, 2017, 11:01 p.m., Hitesh Khamesra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61411/
> ---
> 
> (Updated Aug. 3, 2017, 11:01 p.m.)
> 
> 
> Review request for geode, Alexander Murmann, Bruce Schuchardt, Galen 
> O'Sullivan, Udo Kohlmeyer, and Brian Rowe.
> 
> 
> Repository: geode
> 
> 
> Description
> ---
> 
> In previous fix, we were checking "if thread is stopped then add connection 
> to receiver list". Instead, it should be if thread is stopped then we should 
> not add connection to receiver list. As a fix, we add connection to reciver 
> list if connection is not closed or thread is not stopped.
> 
> Added unit for it.
> 
> 
> Diffs
> -
> 
>   geode-core/src/main/java/org/apache/geode/internal/tcp/Connection.java 
> c3ad596 
>   geode-core/src/main/java/org/apache/geode/internal/tcp/ConnectionTable.java 
> 69fb7a2 
>   
> geode-core/src/test/java/org/apache/geode/internal/tcp/ConnectionTableTest.java
>  312c64d 
> 
> 
> Diff: https://reviews.apache.org/r/61411/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hitesh Khamesra
> 
>

Re: Build failed in Jenkins: Geode-nightly #923

[Mark Bretl]

Hi Jared,

Short answer: No, Longer answer: Ask Infra ;)

We are generally running on VMs with multiple Jenkins executors, which
means we could be sharing the box with other jobs. We have been
piggybacking on other project's systems so far, trying to find 'available'
systems. I have a feeling this is going to be an ongoing issue of getting a
stable environment until we have another solution, such as requesting
dedicated VMs or containers or etc.

Open to ideas here...

--Mark


On Tue, Aug 15, 2017 at 10:59 AM, Jared Stewart  wrote:

> It looks like all of our tests using default ports are failing in the
> nightly build.  Here is my suspicion of what happened: We have some
> Acceptance Tests that spin up separate JVMs for Locators and Servers.
> Those tests make sure to kill their child processes as part of their
> cleanup, but I suspect that when the build times out our test process gets
> hard-killed, which would prevent the proper cleanup from taking place.
> Does anyone have ssh access into this box to look for orphaned Java
> processes?
>
> Thanks,
> Jared
>
> > Begin forwarded message:
> >
> > From: Apache Jenkins Server 
> > Subject: Build failed in Jenkins: Geode-nightly #923
> > Date: August 14, 2017 at 8:53:03 PM PDT
> > To: dev@geode.apache.org, kmil...@pivotal.io, kh...@apache.org,
> bschucha...@pivotal.io, jstew...@pivotal.io, kl...@pivotal.io,
> dschnei...@pivotal.io, dbar...@pivotal.io, n...@pivotal.io
> > Reply-To: dev@geode.apache.org
> >
> > See  irect?page=changes>
> >
> > Changes:
> >
> > [jiliao] GEODE-3328: fix a test failure on windows.
> >
> > [klund] Revert "GEODE-3328: fix a test failure on windows."
> >
> > [klund] GEODE-3436: revert recent refactoring of GFSH commands
> >
> > --
> > [...truncated 129.88 KB...]
> >org.apache.geode.test.dunit.RMIException: While invoking
> org.apache.geode.modules.session.Tomcat8SessionsClientServer
> DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host
> asf905.gq1.ygridcore.net with 4 VMs
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:387)
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
> >at org.apache.geode.modules.session.Tomcat8SessionsClientServer
> DUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60)
> >at org.apache.geode.modules.session.Tomcat8SessionsClientServer
> DUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44)
> >
> >Caused by:
> >java.net.BindException: Failed to create server socket on
> null[40,404]
> >at org.apache.geode.internal.net.
> SocketCreator.createServerSocket(SocketCreator.java:783)
> >at org.apache.geode.internal.net.
> SocketCreator.createServerSocket(SocketCreator.java:745)
> >at org.apache.geode.internal.net.
> SocketCreator.createServerSocket(SocketCreator.java:712)
> >at org.apache.geode.internal.cach
> e.tier.sockets.AcceptorImpl.(AcceptorImpl.java:469)
> >at org.apache.geode.internal.cach
> e.CacheServerImpl.start(CacheServerImpl.java:344)
> >at org.apache.geode.modules.sessi
> on.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f
> 0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65)
> >
> >Caused by:
> >java.net.BindException: Address already in use (Bind failed)
> >at java.net.PlainSocketImpl.socketBind(Native Method)
> >at java.net.AbstractPlainSocketIm
> pl.bind(AbstractPlainSocketImpl.java:387)
> >at java.net.ServerSocket.bind(ServerSocket.java:375)
> >at org.apache.geode.internal.net.
> SocketCreator.createServerSocket(SocketCreator.java:779)
> >... 5 more
> >
> > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest >
> testSanity FAILED
> >org.apache.geode.test.dunit.RMIException: While invoking
> org.apache.geode.modules.session.Tomcat8SessionsClientServer
> DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host
> asf905.gq1.ygridcore.net with 4 VMs
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:387)
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
> >at org.apache.geode.modules.session.Tomcat8SessionsClientServer
> DUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60)
> >at org.apache.geode.modules.session.Tomcat8SessionsClientServer
> DUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44)
> >
> >Caused by:
> >java.net.BindException: Failed to create server socket on
> null[40,404]
> >at org.apache.geode.internal.net.
> SocketCreator.createServerSocket(SocketCreator.java:783)
> >at 

Re: Review Request 61672: GEODE-3249: internal messages should require credentials

[Udo Kohlmeyer]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61672/#review182997
---


Ship it!




Ship It!

- Udo Kohlmeyer


On Aug. 15, 2017, 8:46 p.m., Bruce Schuchardt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61672/
> ---
> 
> (Updated Aug. 15, 2017, 8:46 p.m.)
> 
> 
> Review request for geode, Alexander Murmann, Galen O'Sullivan, Hitesh 
> Khamesra, Udo Kohlmeyer, and Brian Rowe.
> 
> 
> Bugs: GEODE-3249
> https://issues.apache.org/jira/browse/GEODE-3249
> 
> 
> Repository: geode
> 
> 
> Description
> ---
> 
> Internal messages that could alter server state now require security 
> credentials.
> 
> This was merely a matter of changing the server to require the credentials 
> and changing the client to send credentials.  I removed the general 
> overriding of AbstractOp.processSecureBytes() because it made no sense.  If 
> the server sends a secure byte "part" in a message the client is obligated to 
> process it or the next message it sends will cause a security violation.
> 
> I've added a server-side property that folks can set to allow old clients to 
> continue to work.  This must be used to roll the servers forward to the new 
> version that contains this change.  Clients must then be rolled forward & the 
> servers can then be rolled once again without the property set.
> 
> 
> Diffs
> -
> 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/AbstractOp.java
>  c4035f9cf5db1c031e35eef4be0908afbddefffb 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/AddPDXEnumOp.java
>  ca7790aca5cab703c2180f85f01e37c91fa3c956 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/AddPDXTypeOp.java
>  88c85514c891d19399257bb2d85cb463b92ed6bb 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/CloseConnectionOp.java
>  ffcdc39c3ba05e90bf7b9c49509b72de70451f85 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/CommitOp.java 
> edffb2b18bde31435c9555b13c3e630aee1e4027 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/GetClientPRMetaDataOp.java
>  2ba3e3a9a8044fcd7d991fd444fcaf75b2a5c2f4 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/GetClientPartitionAttributesOp.java
>  49567dd31d9f617162768b5066bbb5307785a85f 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/GetEventValueOp.java
>  3fb5fcfa497264d5e0a14d95ed0935f392216680 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/GetFunctionAttributeOp.java
>  c7edbfea719e75291287824c3654c0e7fac3e7bb 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXEnumByIdOp.java
>  7bbf74056f6ecfb7efe27c575029281b98d01b47 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXEnumsOp.java
>  be4c092298df497f6c145b26d8b87234d59c6be8 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXIdForEnumOp.java
>  d87371c6778e9a9ea44c956dbef9e169338c7930 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXIdForTypeOp.java
>  27f600e3e5e2803cfd2f1c312036b57f61a12751 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXTypeByIdOp.java
>  bee50b5f02c2d891f8c450ce1dc799757a39453f 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXTypesOp.java
>  5256924e94fd533dc27c8eb28073a4e68bd68174 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/MakePrimaryOp.java
>  e1d3d5030bb2b31f6471cfc14f147d7780357dc1 
>   geode-core/src/main/java/org/apache/geode/cache/client/internal/PingOp.java 
> 2e5254226c3ef461e93033bb623dfca31cdce1c5 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/PrimaryAckOp.java
>  e380e99e00815d3d56763d429dfc8ad51c3f4113 
>   geode-core/src/main/java/org/apache/geode/cache/client/internal/PutOp.java 
> 447ed382cda810c99f3400ba862db9537794a01b 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/ReadyForEventsOp.java
>  f6d0ccb5a9892e38d83b7fafc831fef6f1f14bb7 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/RegisterDataSerializersOp.java
>  5b259615a1482a6c4835fec12096012001f616d4 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/RegisterInstantiatorsOp.java
>  114bebee931ad4b890adf54d3fdadf1d0d7bbc23 
>   
> geode-core/src/main/java/org/apache/geode/cache/client/internal/RollbackOp.java
>  4704f3a3f4651f9d719e5f3226c9c372307804f8 
>   geode-core/src/main/java/org/apache/geode/cache/client/internal/SizeOp.java 
> ac8c95e9145d601a23d7fe4e6e67039cefa1d1be 
>   
> 

Review Request 61672: GEODE-3249: internal messages should require credentials

[Bruce Schuchardt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61672/
---

Review request for geode, Alexander Murmann, Galen O'Sullivan, Hitesh Khamesra, 
Udo Kohlmeyer, and Brian Rowe.


Bugs: GEODE-3249
https://issues.apache.org/jira/browse/GEODE-3249


Repository: geode


Description
---

Internal messages that could alter server state now require security 
credentials.

This was merely a matter of changing the server to require the credentials and 
changing the client to send credentials.  I removed the general overriding of 
AbstractOp.processSecureBytes() because it made no sense.  If the server sends 
a secure byte "part" in a message the client is obligated to process it or the 
next message it sends will cause a security violation.

I've added a server-side property that folks can set to allow old clients to 
continue to work.  This must be used to roll the servers forward to the new 
version that contains this change.  Clients must then be rolled forward & the 
servers can then be rolled once again without the property set.


Diffs
-

  
geode-core/src/main/java/org/apache/geode/cache/client/internal/AbstractOp.java 
c4035f9cf5db1c031e35eef4be0908afbddefffb 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/AddPDXEnumOp.java
 ca7790aca5cab703c2180f85f01e37c91fa3c956 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/AddPDXTypeOp.java
 88c85514c891d19399257bb2d85cb463b92ed6bb 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/CloseConnectionOp.java
 ffcdc39c3ba05e90bf7b9c49509b72de70451f85 
  geode-core/src/main/java/org/apache/geode/cache/client/internal/CommitOp.java 
edffb2b18bde31435c9555b13c3e630aee1e4027 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/GetClientPRMetaDataOp.java
 2ba3e3a9a8044fcd7d991fd444fcaf75b2a5c2f4 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/GetClientPartitionAttributesOp.java
 49567dd31d9f617162768b5066bbb5307785a85f 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/GetEventValueOp.java
 3fb5fcfa497264d5e0a14d95ed0935f392216680 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/GetFunctionAttributeOp.java
 c7edbfea719e75291287824c3654c0e7fac3e7bb 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXEnumByIdOp.java
 7bbf74056f6ecfb7efe27c575029281b98d01b47 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXEnumsOp.java
 be4c092298df497f6c145b26d8b87234d59c6be8 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXIdForEnumOp.java
 d87371c6778e9a9ea44c956dbef9e169338c7930 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXIdForTypeOp.java
 27f600e3e5e2803cfd2f1c312036b57f61a12751 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXTypeByIdOp.java
 bee50b5f02c2d891f8c450ce1dc799757a39453f 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/GetPDXTypesOp.java
 5256924e94fd533dc27c8eb28073a4e68bd68174 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/MakePrimaryOp.java
 e1d3d5030bb2b31f6471cfc14f147d7780357dc1 
  geode-core/src/main/java/org/apache/geode/cache/client/internal/PingOp.java 
2e5254226c3ef461e93033bb623dfca31cdce1c5 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/PrimaryAckOp.java
 e380e99e00815d3d56763d429dfc8ad51c3f4113 
  geode-core/src/main/java/org/apache/geode/cache/client/internal/PutOp.java 
447ed382cda810c99f3400ba862db9537794a01b 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/ReadyForEventsOp.java
 f6d0ccb5a9892e38d83b7fafc831fef6f1f14bb7 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/RegisterDataSerializersOp.java
 5b259615a1482a6c4835fec12096012001f616d4 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/RegisterInstantiatorsOp.java
 114bebee931ad4b890adf54d3fdadf1d0d7bbc23 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/RollbackOp.java 
4704f3a3f4651f9d719e5f3226c9c372307804f8 
  geode-core/src/main/java/org/apache/geode/cache/client/internal/SizeOp.java 
ac8c95e9145d601a23d7fe4e6e67039cefa1d1be 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/TXFailoverOp.java
 17fc701f6da43ae56748af57590ac4f0c13f77aa 
  
geode-core/src/main/java/org/apache/geode/cache/client/internal/TXSynchronizationOp.java
 0c4086cf23bfcfd8d2d24f4e2b3390fccc79a0a0 
  
geode-core/src/main/java/org/apache/geode/internal/cache/tier/sockets/Message.java
 1f9ef91b22b382e94b6c98158a04dd1f992772bc 
  
geode-core/src/main/java/org/apache/geode/internal/cache/tier/sockets/ServerConnection.java
 870d0ff5cc624271992649acc049ea3a727332d8 
  
geode-core/src/main/java/org/apache/geode/internal/cache/tier/sockets/command/AddPdxType.java
 

[GitHub] geode pull request #714: GEODE-3412: adding files missing from last commit

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/geode/pull/714


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Passed: apache/geode#3570 (GEODE-3328 - 27475ea)

[Travis CI]

Build Update for apache/geode
-

Build: #3570
Status: Passed

Duration: 20 minutes and 45 seconds
Commit: 27475ea (GEODE-3328)
Author: Kirk Lund
Message: GEODE-3328: fix testAddGemFirePropertyFileToCommandLineNew on Windows

Modification of ca4b81 committed by Jinmei Liao

View the changeset: https://github.com/apache/geode/commit/27475ea80374

View the full build log and details: 
https://travis-ci.org/apache/geode/builds/264876311?utm_source=email_medium=notification

--

You can configure recipients for build notifications in your .travis.yml file. 
See https://docs.travis-ci.com/user/notifications

Review Request 61671: GEODE-3328: fix testAddGemFirePropertyFileToCommandLineNew on Windows

[Kirk Lund]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61671/
---

Review request for geode, Jinmei Liao, Jared Stewart, and Ken Howe.


Bugs: GEODE-3328
https://issues.apache.org/jira/browse/GEODE-3328


Repository: geode


Description
---

I had to temporarily revert Jinmei's fix for GEODE-3328 in order to fully 
revert Emily's refactorings of GFSH commands. This commit reintroduces her fix 
with minor tweak to get it to work without Emily's changes.


Diffs
-

  
geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/GfshCommandJUnitTest.java
 da60c7aa481954be0acc8c7e2b88717cf8bc9c37 


Diff: https://reviews.apache.org/r/61671/diff/1/


Testing
---

precheckin in progress


Thanks,

Kirk Lund

[GitHub] geode pull request #714: GEODE-3412: adding files missing from last commit

[WireBaron]

GitHub user WireBaron opened a pull request:

https://github.com/apache/geode/pull/714

GEODE-3412: adding files missing from last commit

This is adding some files which should have been in the last commit for 
this feature.

Thank you for submitting a contribution to Apache Geode.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced in 
the commit message?

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically `develop`)?

- [x] Is your initial contribution a single, squashed commit?

- [ ] Does `gradlew build` run cleanly?

- [ ] Have you written or updated unit tests to verify your changes?

- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and
submit an update to your PR as soon as possible. If you need help, please 
send an
email to dev@geode.apache.org.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/WireBaron/geode feature/GEODE3412

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/geode/pull/714.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #714


commit 1022f323a0061f43155bd5ea74879981b8bc8ffc
Author: Brian Rowe 
Date:   2017-08-15T18:21:51Z

GEODE-3412: adding files missing from last commit




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode pull request #713: Feature/geode 3412

[WireBaron]

Github user WireBaron closed the pull request at:

https://github.com/apache/geode/pull/713


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode pull request #713: Feature/geode 3412

[WireBaron]

GitHub user WireBaron opened a pull request:

https://github.com/apache/geode/pull/713

Feature/geode 3412

This is adding some files which should have been in the last commit for 
this feature.

Thank you for submitting a contribution to Apache Geode.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced in 
the commit message?

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically `develop`)?

- [ ] Is your initial contribution a single, squashed commit?

- [ ] Does `gradlew build` run cleanly?

- [ ] Have you written or updated unit tests to verify your changes?

- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and
submit an update to your PR as soon as possible. If you need help, please 
send an
email to dev@geode.apache.org.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/WireBaron/geode feature/GEODE-3412

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/geode/pull/713.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #713


commit daa140875c4f65838dde4ae0f627a9551bea7516
Author: Brian Rowe 
Date:   2017-08-10T18:16:25Z

GEODE-3412: Add simple authentication flow to protobuf protocol.

This change adds a simple username/password validation to the protobuf 
protocol.
It also adds a new configuration parameter to specify the type of 
authentication required.

Signed-off-by: Galen O'Sullivan 

commit 6753916687b9d916e18c3be5959f4c6aff490800
Author: Brian Rowe 
Date:   2017-08-14T17:29:21Z

GEODE-3412: implementing review feedback

commit b11c02a59cb2d71d4ac7d0b821ca668c825e2f6c
Author: Brian Rowe 
Date:   2017-08-15T17:09:39Z

GEODE-3412: moving authenticator interface to geode.security package

commit 704b6953e01a152e6859d4b254f4d8d39bd40011
Author: Brian Rowe 
Date:   2017-08-15T18:21:51Z

GEODE-3412: adding files missing from last commit




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode pull request #712: GEODE-3434: Allow the modules to be interoperable w...

[jhuynh1]

Github user jhuynh1 commented on a diff in the pull request:

https://github.com/apache/geode/pull/712#discussion_r133270585
  
--- Diff: 
geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventListener.java
 ---
@@ -34,7 +34,6 @@
 import org.apache.geode.cache.query.internal.DefaultQuery;
 import org.apache.geode.internal.cache.BucketNotFoundException;
 import org.apache.geode.internal.cache.PrimaryBucketException;
-import org.apache.geode.internal.cache.tier.sockets.command.Default;
--- End diff --

I reverted the first set of changes to this file as it didn't have anything 
to do with this change set.  It shouldn't be showing up in this diff anymore 
but if it is I'll make sure to revert it...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode pull request #712: GEODE-3434: Allow the modules to be interoperable w...

[DivineEnder]

Github user DivineEnder commented on a diff in the pull request:

https://github.com/apache/geode/pull/712#discussion_r133268501
  
--- Diff: 
geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventListener.java
 ---
@@ -34,7 +34,6 @@
 import org.apache.geode.cache.query.internal.DefaultQuery;
 import org.apache.geode.internal.cache.BucketNotFoundException;
 import org.apache.geode.internal.cache.PrimaryBucketException;
-import org.apache.geode.internal.cache.tier.sockets.command.Default;
--- End diff --

Can you organize imports on this file since this import seems to have 
changed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: continuous query internal mechanism questions

[Jason Huynh]

I am not quite sure how native client registers cqs. From my understanding:
with the java api, I believe there is only one message (ExecuteCQ message)
that is executed on the server side and then replicated to the other nodes
through the profile (OperationMessage).

It seems the extra ExecuteCQ message failing and then closing the cq might
be putting the system in a weird state...

On Tue, Aug 15, 2017 at 7:56 AM Roi Apelker  wrote:

> Hi,
>
> I have been examining the continuous query registration mechanism for
> quite some time
> This is related to an issue that I have, where sometimes a node crashes (1
> node out of 2), and the other one does not send CQ events. The CQ is
> registered on a partitioned region which resides on these 2 nodes.
>
> I noticed the following behavior, and I wonder if anyone can comment
> regarding it, if it is justified or not and what is the reason:
>
> 1. When the software using the client (native client) registers for the
> CQ, a CQ command (ExecuteCQ61) is received on both servers.
>  -- is this normal behaviour? Does the client actually send this command
> to both servers?
>
> 2. When this command is received by a server, and the CQ is registered,
> another registration message is sent to the other node via an
> OperationMessage (REGISTER_CQ)
>  -- it seems that regularly, the server can handle this situation as the
> second registration identifies the previous one and does not affect it. but
> the question, why do we need this 2nd registration, if there is a command
> sent to each server?
>
> 3. For some reason, sometimes there is a failure to complete the first
> registration (executed by ExecuteCQ61) and then this failure causes a
> closure to the CQ, which is accompanied with a close request to the other
> node.
>  -- I assume by now, since 2 registrations and one closure have occurred
> on node 2, the CQ is still active and the client receives notifications.
>
> 4. Sometimes, 1 out of 5, once node 1 crashes, I get a cleanup operation,
> caused by the crash (via MemberCrashedEvent), and this also closes the
> existing CQ, and in this case the CQ in node 2 does not operate anymore and
> the client receives no notifications.
>  -- fact is, that 4 out of 4 times, I do not get this cleanup by
> MemberCrashedEvent (maybe due to some other error), and that the CQ
> notifications are received normally.
>
> Can anyone clear things up for me? Any comment on any of the statements
> above will be greatly appreciated.
>
> Thanks,
>
> Roi
>
>
> -Original Message-
> From: Roi Apelker
> Sent: Wednesday, August 09, 2017 3:21 PM
> To: dev@geode.apache.org
> Subject: RE: continuous query internal mechanism
>
> Dhanyavad
>
> -Original Message-
> From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> Sent: Tuesday, August 08, 2017 9:55 PM
> To: dev@geode.apache.org
> Subject: Re: continuous query internal mechanism
>
> Registered events, i meant, are events generated for interest registration
> "region.registerInterest(*)". And CqEvents are for CQs registered.
>
> -Anil.
>
>
> On Tue, Aug 8, 2017 at 12:27 AM, Roi Apelker 
> wrote:
>
> > Shukriya
> >
> > What is the difference between registered events and CQ events?
> >
> > -Original Message-
> > From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> > Sent: Monday, August 07, 2017 10:12 PM
> > To: dev@geode.apache.org
> > Subject: Re: continuous query internal mechanism
> >
> > CQ Processing on server side is same for all clients (Java, C++)...
> >
> > The subscription events are sent to client as ClientUpdateMessage,
> > which holds information about registered events and CQ events. The
> > client process this and updates/invokes the client side
> > cache/listeners with respective event. Look into
> > ClientUpdateMessageImpl and CacheClientUpdater (for client side
> processing).
> >
> > -Anil.
> >
> >
> >
> >
> > On Mon, Aug 7, 2017 at 11:01 AM, Roi Apelker 
> > wrote:
> >
> > > Thanks,
> > >
> > > By the way, is there any difference in the behaviour of the server,
> > > if the client that registered the CQ is a native (C++) client?
> > >
> > > I have been going over the classes and code for some time and can't
> > > seem to find the actual location where a CQ update/notification is
> > sent...
> > >
> > > It's like CqEventImpl class is never even generated in this scenario.
> > >
> > > If anyone can help here I would be most grateful :-)
> > >
> > > Thanks
> > >
> > > Roi
> > >
> > >
> > >
> > > -Original Message-
> > > From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> > > Sent: Monday, August 07, 2017 8:23 PM
> > > To: dev@geode.apache.org
> > > Subject: Re: continuous query internal mechanism
> > >
> > > You can find those in CqServiceImpl.process*()...
> > >
> > > -Anil.
> > >
> > >
> > > On Mon, Aug 7, 2017 at 9:14 AM, Roi Apelker 
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I am trying to look 

[GitHub] geode pull request #712: GEODE-3434: Allow the modules to be interoperable w...

[DivineEnder]

Github user DivineEnder commented on a diff in the pull request:

https://github.com/apache/geode/pull/712#discussion_r133256792
  
--- Diff: 
geode-assembly/src/test/java/org/apache/geode/session/tests/ContainerInstall.java
 ---
@@ -18,6 +18,7 @@
 import java.io.FileInputStream;
 import java.io.FileOutputStream;
 import java.io.IOException;
+import java.net.URI;
--- End diff --

I think this is extraneous. It doesn't seem to be used anywhere in this 
file.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode pull request #707: GEODE-3412: Add simple authentication flow to proto...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/geode/pull/707


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode pull request #702: GEODE-3416: Reduce synchronization blockages in Soc...

[kohlmu-pivotal]

Github user kohlmu-pivotal commented on a diff in the pull request:

https://github.com/apache/geode/pull/702#discussion_r133255222
  
--- Diff: 
geode-core/src/main/java/org/apache/geode/internal/net/SocketCloser.java ---
@@ -96,46 +99,55 @@ public int getMaxThreads() {
 return this.asyncClosePoolMaxThreads;
   }
 
-  private ThreadPoolExecutor getAsyncThreadExecutor(String address) {
-synchronized (asyncCloseExecutors) {
-  ThreadPoolExecutor pool = asyncCloseExecutors.get(address);
-  if (pool == null) {
-final ThreadGroup tg = 
LoggingThreadGroup.createThreadGroup("Socket asyncClose", logger);
-ThreadFactory tf = new ThreadFactory() {
-  public Thread newThread(final Runnable command) {
-Thread thread = new Thread(tg, command);
-thread.setDaemon(true);
-return thread;
-  }
-};
-BlockingQueue workQueue = new 
LinkedBlockingQueue();
-pool = new ThreadPoolExecutor(this.asyncClosePoolMaxThreads, 
this.asyncClosePoolMaxThreads,
-this.asyncClosePoolKeepAliveSeconds, TimeUnit.SECONDS, 
workQueue, tf);
-pool.allowCoreThreadTimeOut(true);
-asyncCloseExecutors.put(address, pool);
+  private ExecutorService getAsyncThreadExecutor(String address) {
+ExecutorService executorService = asyncCloseExecutors.get(address);
+if (executorService == null) {
+  // To be used for pre-1.8 jdk releases.
+  // createThreadPool();
+
+  executorService = 
Executors.newWorkStealingPool(asyncClosePoolMaxThreads);
+
+  ExecutorService previousThreadPoolExecutor =
+  asyncCloseExecutors.putIfAbsent(address, executorService);
+
+  if (previousThreadPoolExecutor != null) {
+executorService.shutdownNow();
+return previousThreadPoolExecutor;
   }
-  return pool;
 }
+return executorService;
+  }
+
+  /**
+   * @deprecated this method is to be used for pre 1.8 jdk.
+   */
+  @Deprecated
+  private void createThreadPool() {
+ExecutorService executorService;
+final ThreadGroup threadGroup =
+LoggingThreadGroup.createThreadGroup("Socket asyncClose", logger);
+ThreadFactory threadFactory = new ThreadFactory() {
+  public Thread newThread(final Runnable command) {
+Thread thread = new Thread(threadGroup, command);
+thread.setDaemon(true);
+return thread;
+  }
+};
+
+executorService = new ThreadPoolExecutor(asyncClosePoolMaxThreads, 
asyncClosePoolMaxThreads,
+asyncCloseWaitTime, asyncCloseWaitUnits, new 
LinkedBlockingQueue<>(), threadFactory);
   }
 
   /**
* Call this method if you know all the resources in the closer for the 
given address are no
* longer needed. Currently a thread pool is kept for each address and 
if you know that an address
* no longer needs its pool then you should call this method.
*/
-  public void releaseResourcesForAddress(String address) {
-synchronized (asyncCloseExecutors) {
-  ThreadPoolExecutor pool = asyncCloseExecutors.get(address);
-  if (pool != null) {
-pool.shutdown();
-asyncCloseExecutors.remove(address);
-  }
-}
-  }
 
-  private boolean isClosed() {
-synchronized (asyncCloseExecutors) {
-  return this.closed;
+  public void releaseResourcesForAddress(String address) {
+ExecutorService executorService = asyncCloseExecutors.remove(address);
+if (executorService != null) {
+  executorService.shutdown();
--- End diff --

Correct and agreed. When the cache closes, I imagine SocketCloser.close() 
is called. Which means the closing of any socket post that would be done 
in-line. 
I don't know what the expected behavior is supposed to be, for the closing 
of the sockets when that cache is closing. But you are correct that it could 
mean that the closing of the socket could be holding up the shutting down of 
the cache.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: continuous query - high availability and assurance

[Udo Kohlmeyer]

Hi there Roi,

If you have turned on Durable Client-Server messaging 
 
then the cluster should deal with the failure of a client. All client 
bound messages would have a "at-least-once" delivery policy. All 
client-bound messages are acknowledged asynchronously after delivery to 
the client.


If you have enabled "subscription-redundancy" on the client pool and set 
it to either -1 or any number > 0, then the server nodes will have 
redundant client queues, which will enable to server cluster to handle 
the case where the client-queue is lost due to the server having failed. 
see documentation 



As stated above, client messages are delivered to the client from the 
server. The acknowledgement upon receiving these messages are done in an 
asynchronous manner back to the server. There is no manual intervention 
required to acknowledge the receiving of the messages. In addition to 
this you cannot apply the messaging paradigm where you want to only 
acknowledge a message AFTER the processing of that message by the 
application.


A client message (CQ or subscription) will be available for 
acknowledgement upon successful delivery to the client and not the 
custom processing of the message by the application.


--Udo



On 8/15/17 10:25, Mark Bretl wrote:

+ user

On Tue, Aug 15, 2017 at 10:17 AM, Roi Apelker 
wrote:


Hi,

I have been working on an issue related to continuous query mechanism, and
I would like to know if this mechanism, with High availability, assures
100% data integrity?

For example, if I have 2 nodes where a client registered to CQ, and one
node fails, is there any scenario, in which a notification might be lost,
either not sent at all, or hasn’t reached its destination and will not be
sent, or any other?

Is there a guarantee that all information pertinent for the CQ will reach
the subscriber?

Thanks,

Roi
This message and the information contained herein is proprietary and
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer <
https://www.amdocs.com/about/email-disclaimer>

[GitHub] geode pull request #702: GEODE-3416: Reduce synchronization blockages in Soc...

[hiteshk25]

Github user hiteshk25 commented on a diff in the pull request:

https://github.com/apache/geode/pull/702#discussion_r133251379
  
--- Diff: 
geode-core/src/main/java/org/apache/geode/internal/net/SocketCloser.java ---
@@ -96,46 +99,55 @@ public int getMaxThreads() {
 return this.asyncClosePoolMaxThreads;
   }
 
-  private ThreadPoolExecutor getAsyncThreadExecutor(String address) {
-synchronized (asyncCloseExecutors) {
-  ThreadPoolExecutor pool = asyncCloseExecutors.get(address);
-  if (pool == null) {
-final ThreadGroup tg = 
LoggingThreadGroup.createThreadGroup("Socket asyncClose", logger);
-ThreadFactory tf = new ThreadFactory() {
-  public Thread newThread(final Runnable command) {
-Thread thread = new Thread(tg, command);
-thread.setDaemon(true);
-return thread;
-  }
-};
-BlockingQueue workQueue = new 
LinkedBlockingQueue();
-pool = new ThreadPoolExecutor(this.asyncClosePoolMaxThreads, 
this.asyncClosePoolMaxThreads,
-this.asyncClosePoolKeepAliveSeconds, TimeUnit.SECONDS, 
workQueue, tf);
-pool.allowCoreThreadTimeOut(true);
-asyncCloseExecutors.put(address, pool);
+  private ExecutorService getAsyncThreadExecutor(String address) {
+ExecutorService executorService = asyncCloseExecutors.get(address);
+if (executorService == null) {
+  // To be used for pre-1.8 jdk releases.
+  // createThreadPool();
+
+  executorService = 
Executors.newWorkStealingPool(asyncClosePoolMaxThreads);
+
+  ExecutorService previousThreadPoolExecutor =
+  asyncCloseExecutors.putIfAbsent(address, executorService);
+
+  if (previousThreadPoolExecutor != null) {
+executorService.shutdownNow();
+return previousThreadPoolExecutor;
   }
-  return pool;
 }
+return executorService;
+  }
+
+  /**
+   * @deprecated this method is to be used for pre 1.8 jdk.
+   */
+  @Deprecated
+  private void createThreadPool() {
+ExecutorService executorService;
+final ThreadGroup threadGroup =
+LoggingThreadGroup.createThreadGroup("Socket asyncClose", logger);
+ThreadFactory threadFactory = new ThreadFactory() {
+  public Thread newThread(final Runnable command) {
+Thread thread = new Thread(threadGroup, command);
+thread.setDaemon(true);
+return thread;
+  }
+};
+
+executorService = new ThreadPoolExecutor(asyncClosePoolMaxThreads, 
asyncClosePoolMaxThreads,
+asyncCloseWaitTime, asyncCloseWaitUnits, new 
LinkedBlockingQueue<>(), threadFactory);
   }
 
   /**
* Call this method if you know all the resources in the closer for the 
given address are no
* longer needed. Currently a thread pool is kept for each address and 
if you know that an address
* no longer needs its pool then you should call this method.
*/
-  public void releaseResourcesForAddress(String address) {
-synchronized (asyncCloseExecutors) {
-  ThreadPoolExecutor pool = asyncCloseExecutors.get(address);
-  if (pool != null) {
-pool.shutdown();
-asyncCloseExecutors.remove(address);
-  }
-}
-  }
 
-  private boolean isClosed() {
-synchronized (asyncCloseExecutors) {
-  return this.closed;
+  public void releaseResourcesForAddress(String address) {
+ExecutorService executorService = asyncCloseExecutors.remove(address);
+if (executorService != null) {
+  executorService.shutdown();
--- End diff --

We close the socket in a new thread when we are not closing the cache. 
Otherwise, we close all sockets inline, correct? Given that, how often multiple 
clients are closing the connection same time?? I feel cache.close() may take 
some time if we close all things inline but there may be some good reason for 
doing that. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode pull request #702: GEODE-3416: Reduce synchronization blockages in Soc...

[hiteshk25]

Github user hiteshk25 commented on a diff in the pull request:

https://github.com/apache/geode/pull/702#discussion_r133249141
  
--- Diff: 
geode-core/src/main/java/org/apache/geode/internal/tcp/ConnectionTable.java ---
@@ -929,10 +929,6 @@ protected void removeEndpoint(DistributedMember 
memberID, String reason,
   owner.getDM().getMembershipManager().getShutdownCause());
 }
   }
-
-  if (remoteAddress != null) {
-
this.socketCloser.releaseResourcesForAddress(remoteAddress.toString());
-  }
--- End diff --

This doesn't require??


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: continuous query - high availability and assurance

[Mark Bretl]

+ user

On Tue, Aug 15, 2017 at 10:17 AM, Roi Apelker 
wrote:

> Hi,
>
> I have been working on an issue related to continuous query mechanism, and
> I would like to know if this mechanism, with High availability, assures
> 100% data integrity?
>
> For example, if I have 2 nodes where a client registered to CQ, and one
> node fails, is there any scenario, in which a notification might be lost,
> either not sent at all, or hasn’t reached its destination and will not be
> sent, or any other?
>
> Is there a guarantee that all information pertinent for the CQ will reach
> the subscriber?
>
> Thanks,
>
> Roi
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
>

[GitHub] geode pull request #707: GEODE-3412: Add simple authentication flow to proto...

[pivotal-amurmann]

Github user pivotal-amurmann commented on a diff in the pull request:

https://github.com/apache/geode/pull/707#discussion_r133246611
  
--- Diff: 
geode-core/src/main/java/org/apache/geode/internal/cache/tier/sockets/ServerConnectionFactory.java
 ---
@@ -22,59 +22,89 @@
 
 import java.io.IOException;
 import java.net.Socket;
+import java.util.HashMap;
 import java.util.Iterator;
+import java.util.Map;
 import java.util.ServiceLoader;
-import javax.management.ServiceNotFoundException;
 
 /**
  * Creates instances of ServerConnection based on the connection mode 
provided.
  */
 public class ServerConnectionFactory {
-  private static ClientProtocolMessageHandler protobufProtocolHandler;
-  private static final Object protocolLoadLock = new Object();
+  private ClientProtocolMessageHandler protobufProtocolHandler;
+  private Map authenticators 
= null;
 
-  private static ClientProtocolMessageHandler 
findClientProtocolMessageHandler() {
+  public ServerConnectionFactory() {}
+
+  private synchronized void initializeAuthenticatorsMap() {
+if (authenticators != null) {
+  return;
+}
+authenticators = new HashMap<>();
+ServiceLoader loader = 
ServiceLoader.load(StreamAuthenticator.class);
+for (StreamAuthenticator streamAuthenticator : loader) {
+  authenticators.put(streamAuthenticator.implementationID(), 
streamAuthenticator.getClass());
+}
+  }
+
+  private synchronized ClientProtocolMessageHandler 
initializeMessageHandler() {
 if (protobufProtocolHandler != null) {
   return protobufProtocolHandler;
 }
+ServiceLoader loader =
+ServiceLoader.load(ClientProtocolMessageHandler.class);
+Iterator iterator = loader.iterator();
 
-synchronized (protocolLoadLock) {
-  if (protobufProtocolHandler != null) {
-return protobufProtocolHandler;
-  }
-
-  ServiceLoader loader =
-  ServiceLoader.load(ClientProtocolMessageHandler.class);
-  Iterator iterator = loader.iterator();
-
-  if (!iterator.hasNext()) {
-throw new ServiceLoadingFailureException(
-"ClientProtocolMessageHandler implementation not found in 
JVM");
-  }
+if (!iterator.hasNext()) {
+  throw new ServiceLoadingFailureException(
+  "There is no ClientProtocolMessageHandler implementation found 
in JVM");
+}
 
-  ClientProtocolMessageHandler returnValue = iterator.next();
+protobufProtocolHandler = iterator.next();
+return protobufProtocolHandler;
+  }
 
-  if (iterator.hasNext()) {
+  private StreamAuthenticator findStreamAuthenticator(String 
implementationID) {
+if (authenticators == null) {
+  initializeAuthenticatorsMap();
+}
+Class streamAuthenticatorClass =
+authenticators.get(implementationID);
+if (streamAuthenticatorClass == null) {
+  throw new ServiceLoadingFailureException(
+  "Could not find implementation for StreamAuthenticator with 
implementation ID "
+  + implementationID);
+} else {
+  try {
+return streamAuthenticatorClass.newInstance();
+  } catch (InstantiationException | IllegalAccessException e) {
 throw new ServiceLoadingFailureException(
-"Multiple service implementations found for 
ClientProtocolMessageHandler");
--- End diff --

Where are we now handling the case of having multiple implementations?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

continuous query - high availability and assurance

[Roi Apelker]

Hi,

I have been working on an issue related to continuous query mechanism, and I 
would like to know if this mechanism, with High availability, assures 100% data 
integrity?

For example, if I have 2 nodes where a client registered to CQ, and one node 
fails, is there any scenario, in which a notification might be lost, either not 
sent at all, or hasn’t reached its destination and will not be sent, or any 
other? 

Is there a guarantee that all information pertinent for the CQ will reach the 
subscriber?

Thanks,

Roi
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

[GitHub] geode pull request #683: GEODE-3314 - additional refactoring for developer Q...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/geode/pull/683


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode pull request #702: GEODE-3416: Reduce synchronization blockages in Soc...

[kohlmu-pivotal]

Github user kohlmu-pivotal commented on a diff in the pull request:

https://github.com/apache/geode/pull/702#discussion_r133238599
  
--- Diff: 
geode-core/src/main/java/org/apache/geode/internal/net/SocketCloser.java ---
@@ -96,46 +99,55 @@ public int getMaxThreads() {
 return this.asyncClosePoolMaxThreads;
   }
 
-  private ThreadPoolExecutor getAsyncThreadExecutor(String address) {
-synchronized (asyncCloseExecutors) {
-  ThreadPoolExecutor pool = asyncCloseExecutors.get(address);
-  if (pool == null) {
-final ThreadGroup tg = 
LoggingThreadGroup.createThreadGroup("Socket asyncClose", logger);
-ThreadFactory tf = new ThreadFactory() {
-  public Thread newThread(final Runnable command) {
-Thread thread = new Thread(tg, command);
-thread.setDaemon(true);
-return thread;
-  }
-};
-BlockingQueue workQueue = new 
LinkedBlockingQueue();
-pool = new ThreadPoolExecutor(this.asyncClosePoolMaxThreads, 
this.asyncClosePoolMaxThreads,
-this.asyncClosePoolKeepAliveSeconds, TimeUnit.SECONDS, 
workQueue, tf);
-pool.allowCoreThreadTimeOut(true);
-asyncCloseExecutors.put(address, pool);
+  private ExecutorService getAsyncThreadExecutor(String address) {
+ExecutorService executorService = asyncCloseExecutors.get(address);
+if (executorService == null) {
+  // To be used for pre-1.8 jdk releases.
+  // createThreadPool();
+
+  executorService = 
Executors.newWorkStealingPool(asyncClosePoolMaxThreads);
+
+  ExecutorService previousThreadPoolExecutor =
+  asyncCloseExecutors.putIfAbsent(address, executorService);
+
+  if (previousThreadPoolExecutor != null) {
+executorService.shutdownNow();
+return previousThreadPoolExecutor;
   }
-  return pool;
 }
+return executorService;
+  }
+
+  /**
+   * @deprecated this method is to be used for pre 1.8 jdk.
+   */
+  @Deprecated
+  private void createThreadPool() {
+ExecutorService executorService;
+final ThreadGroup threadGroup =
+LoggingThreadGroup.createThreadGroup("Socket asyncClose", logger);
+ThreadFactory threadFactory = new ThreadFactory() {
+  public Thread newThread(final Runnable command) {
+Thread thread = new Thread(threadGroup, command);
+thread.setDaemon(true);
+return thread;
+  }
+};
+
+executorService = new ThreadPoolExecutor(asyncClosePoolMaxThreads, 
asyncClosePoolMaxThreads,
+asyncCloseWaitTime, asyncCloseWaitUnits, new 
LinkedBlockingQueue<>(), threadFactory);
   }
 
   /**
* Call this method if you know all the resources in the closer for the 
given address are no
* longer needed. Currently a thread pool is kept for each address and 
if you know that an address
* no longer needs its pool then you should call this method.
*/
-  public void releaseResourcesForAddress(String address) {
-synchronized (asyncCloseExecutors) {
-  ThreadPoolExecutor pool = asyncCloseExecutors.get(address);
-  if (pool != null) {
-pool.shutdown();
-asyncCloseExecutors.remove(address);
-  }
-}
-  }
 
-  private boolean isClosed() {
-synchronized (asyncCloseExecutors) {
-  return this.closed;
+  public void releaseResourcesForAddress(String address) {
+ExecutorService executorService = asyncCloseExecutors.remove(address);
+if (executorService != null) {
+  executorService.shutdown();
--- End diff --

ExecutorService.shutdown does protect itself internally with locks, etc.. 
so we don't have to worry about multiple threads calling shutdown on the same 
executor service


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Permission to assign and close JIRA tickets

[Alexander Murmann]

amurmann

Thanks!

On Tue, Aug 15, 2017 at 9:05 AM, Anthony Baker  wrote:

> What’s your JIRA username?
>
> > On Aug 15, 2017, at 8:40 AM, Alexander Murmann 
> wrote:
> >
> > Good morning,
> >
> > Can I please get permission to assign JIRA tickets to myself as well as
> > close them?
> >
> > Thank you!
>
>

[GitHub] geode pull request #702: GEODE-3416: Reduce synchronization blockages in Soc...

[kohlmu-pivotal]

Github user kohlmu-pivotal commented on a diff in the pull request:

https://github.com/apache/geode/pull/702#discussion_r133238417
  
--- Diff: 
geode-core/src/main/java/org/apache/geode/internal/net/SocketCloser.java ---
@@ -144,35 +156,22 @@ private boolean isClosed() {
* called then the asyncClose will be done synchronously.
*/
   public void close() {
-synchronized (asyncCloseExecutors) {
+synchronized (closed) {
   if (!this.closed) {
 this.closed = true;
-for (ThreadPoolExecutor pool : asyncCloseExecutors.values()) {
-  pool.shutdown();
-}
-asyncCloseExecutors.clear();
+  } else {
+return;
   }
 }
+for (ExecutorService executorService : asyncCloseExecutors.values()) {
+  executorService.shutdown();
+  asyncCloseExecutors.clear();
--- End diff --

Good catch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode pull request #702: GEODE-3416: Reduce synchronization blockages in Soc...

[WireBaron]

Github user WireBaron commented on a diff in the pull request:

https://github.com/apache/geode/pull/702#discussion_r133230687
  
--- Diff: 
geode-core/src/main/java/org/apache/geode/internal/net/SocketCloser.java ---
@@ -144,35 +156,22 @@ private boolean isClosed() {
* called then the asyncClose will be done synchronously.
*/
   public void close() {
-synchronized (asyncCloseExecutors) {
+synchronized (closed) {
   if (!this.closed) {
 this.closed = true;
-for (ThreadPoolExecutor pool : asyncCloseExecutors.values()) {
-  pool.shutdown();
-}
-asyncCloseExecutors.clear();
+  } else {
+return;
   }
 }
+for (ExecutorService executorService : asyncCloseExecutors.values()) {
+  executorService.shutdown();
+  asyncCloseExecutors.clear();
--- End diff --

Shouldn't this be outside the for loop?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode pull request #702: GEODE-3416: Reduce synchronization blockages in Soc...

[WireBaron]

Github user WireBaron commented on a diff in the pull request:

https://github.com/apache/geode/pull/702#discussion_r133230527
  
--- Diff: 
geode-core/src/main/java/org/apache/geode/internal/net/SocketCloser.java ---
@@ -96,46 +99,55 @@ public int getMaxThreads() {
 return this.asyncClosePoolMaxThreads;
   }
 
-  private ThreadPoolExecutor getAsyncThreadExecutor(String address) {
-synchronized (asyncCloseExecutors) {
-  ThreadPoolExecutor pool = asyncCloseExecutors.get(address);
-  if (pool == null) {
-final ThreadGroup tg = 
LoggingThreadGroup.createThreadGroup("Socket asyncClose", logger);
-ThreadFactory tf = new ThreadFactory() {
-  public Thread newThread(final Runnable command) {
-Thread thread = new Thread(tg, command);
-thread.setDaemon(true);
-return thread;
-  }
-};
-BlockingQueue workQueue = new 
LinkedBlockingQueue();
-pool = new ThreadPoolExecutor(this.asyncClosePoolMaxThreads, 
this.asyncClosePoolMaxThreads,
-this.asyncClosePoolKeepAliveSeconds, TimeUnit.SECONDS, 
workQueue, tf);
-pool.allowCoreThreadTimeOut(true);
-asyncCloseExecutors.put(address, pool);
+  private ExecutorService getAsyncThreadExecutor(String address) {
+ExecutorService executorService = asyncCloseExecutors.get(address);
+if (executorService == null) {
+  // To be used for pre-1.8 jdk releases.
+  // createThreadPool();
+
+  executorService = 
Executors.newWorkStealingPool(asyncClosePoolMaxThreads);
+
+  ExecutorService previousThreadPoolExecutor =
+  asyncCloseExecutors.putIfAbsent(address, executorService);
+
+  if (previousThreadPoolExecutor != null) {
+executorService.shutdownNow();
+return previousThreadPoolExecutor;
   }
-  return pool;
 }
+return executorService;
+  }
+
+  /**
+   * @deprecated this method is to be used for pre 1.8 jdk.
+   */
+  @Deprecated
+  private void createThreadPool() {
+ExecutorService executorService;
+final ThreadGroup threadGroup =
+LoggingThreadGroup.createThreadGroup("Socket asyncClose", logger);
+ThreadFactory threadFactory = new ThreadFactory() {
+  public Thread newThread(final Runnable command) {
+Thread thread = new Thread(threadGroup, command);
+thread.setDaemon(true);
+return thread;
+  }
+};
+
+executorService = new ThreadPoolExecutor(asyncClosePoolMaxThreads, 
asyncClosePoolMaxThreads,
+asyncCloseWaitTime, asyncCloseWaitUnits, new 
LinkedBlockingQueue<>(), threadFactory);
   }
 
   /**
* Call this method if you know all the resources in the closer for the 
given address are no
* longer needed. Currently a thread pool is kept for each address and 
if you know that an address
* no longer needs its pool then you should call this method.
*/
-  public void releaseResourcesForAddress(String address) {
-synchronized (asyncCloseExecutors) {
-  ThreadPoolExecutor pool = asyncCloseExecutors.get(address);
-  if (pool != null) {
-pool.shutdown();
-asyncCloseExecutors.remove(address);
-  }
-}
-  }
 
-  private boolean isClosed() {
-synchronized (asyncCloseExecutors) {
-  return this.closed;
+  public void releaseResourcesForAddress(String address) {
+ExecutorService executorService = asyncCloseExecutors.remove(address);
+if (executorService != null) {
+  executorService.shutdown();
--- End diff --

This could still race with a thread running the close() method below and 
both could end up calling shutdown on the same service.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] geode issue #649: GEODE-2997: Change new protocol GetAllResponse.

[kohlmu-pivotal]

Github user kohlmu-pivotal commented on the issue:

https://github.com/apache/geode/pull/649
  
@galen-pivotal is this PR active anymore?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Permission to assign and close JIRA tickets

[Anthony Baker]

What’s your JIRA username?

> On Aug 15, 2017, at 8:40 AM, Alexander Murmann  wrote:
> 
> Good morning,
> 
> Can I please get permission to assign JIRA tickets to myself as well as
> close them?
> 
> Thank you!

[GitHub] geode pull request #700: GEODE-3386 - Make KeyedErrorResponse & ErrorRespons...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/geode/pull/700


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Permission to assign and close JIRA tickets

[Alexander Murmann]

Good morning,

Can I please get permission to assign JIRA tickets to myself as well as
close them?

Thank you!

continuous query internal mechanism questions

[Roi Apelker]

Hi,

I have been examining the continuous query registration mechanism for quite 
some time 
This is related to an issue that I have, where sometimes a node crashes (1 node 
out of 2), and the other one does not send CQ events. The CQ is registered on a 
partitioned region which resides on these 2 nodes.

I noticed the following behavior, and I wonder if anyone can comment regarding 
it, if it is justified or not and what is the reason:

1. When the software using the client (native client) registers for the CQ, a 
CQ command (ExecuteCQ61) is received on both servers.
 -- is this normal behaviour? Does the client actually send this command to 
both servers?

2. When this command is received by a server, and the CQ is registered, another 
registration message is sent to the other node via an OperationMessage 
(REGISTER_CQ)
 -- it seems that regularly, the server can handle this situation as the second 
registration identifies the previous one and does not affect it. but the 
question, why do we need this 2nd registration, if there is a command sent to 
each server?

3. For some reason, sometimes there is a failure to complete the first 
registration (executed by ExecuteCQ61) and then this failure causes a closure 
to the CQ, which is accompanied with a close request to the other node.
 -- I assume by now, since 2 registrations and one closure have occurred on 
node 2, the CQ is still active and the client receives notifications.

4. Sometimes, 1 out of 5, once node 1 crashes, I get a cleanup operation, 
caused by the crash (via MemberCrashedEvent), and this also closes the existing 
CQ, and in this case the CQ in node 2 does not operate anymore and the client 
receives no notifications.
 -- fact is, that 4 out of 4 times, I do not get this cleanup by 
MemberCrashedEvent (maybe due to some other error), and that the CQ 
notifications are received normally.

Can anyone clear things up for me? Any comment on any of the statements above 
will be greatly appreciated. 
 
Thanks,

Roi


-Original Message-
From: Roi Apelker 
Sent: Wednesday, August 09, 2017 3:21 PM
To: dev@geode.apache.org
Subject: RE: continuous query internal mechanism

Dhanyavad

-Original Message-
From: Anilkumar Gingade [mailto:aging...@pivotal.io]
Sent: Tuesday, August 08, 2017 9:55 PM
To: dev@geode.apache.org
Subject: Re: continuous query internal mechanism

Registered events, i meant, are events generated for interest registration 
"region.registerInterest(*)". And CqEvents are for CQs registered.

-Anil.


On Tue, Aug 8, 2017 at 12:27 AM, Roi Apelker  wrote:

> Shukriya
>
> What is the difference between registered events and CQ events?
>
> -Original Message-
> From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> Sent: Monday, August 07, 2017 10:12 PM
> To: dev@geode.apache.org
> Subject: Re: continuous query internal mechanism
>
> CQ Processing on server side is same for all clients (Java, C++)...
>
> The subscription events are sent to client as ClientUpdateMessage, 
> which holds information about registered events and CQ events. The 
> client process this and updates/invokes the client side 
> cache/listeners with respective event. Look into 
> ClientUpdateMessageImpl and CacheClientUpdater (for client side processing).
>
> -Anil.
>
>
>
>
> On Mon, Aug 7, 2017 at 11:01 AM, Roi Apelker 
> wrote:
>
> > Thanks,
> >
> > By the way, is there any difference in the behaviour of the server, 
> > if the client that registered the CQ is a native (C++) client?
> >
> > I have been going over the classes and code for some time and can't 
> > seem to find the actual location where a CQ update/notification is
> sent...
> >
> > It's like CqEventImpl class is never even generated in this scenario.
> >
> > If anyone can help here I would be most grateful :-)
> >
> > Thanks
> >
> > Roi
> >
> >
> >
> > -Original Message-
> > From: Anilkumar Gingade [mailto:aging...@pivotal.io]
> > Sent: Monday, August 07, 2017 8:23 PM
> > To: dev@geode.apache.org
> > Subject: Re: continuous query internal mechanism
> >
> > You can find those in CqServiceImpl.process*()...
> >
> > -Anil.
> >
> >
> > On Mon, Aug 7, 2017 at 9:14 AM, Roi Apelker 
> > wrote:
> >
> > > Hello,
> > >
> > > I am trying to look into the code of the continuous query 
> > > mechanism
> > > - where the GEODE server sends the notification back to the client.
> > >
> > > Can anyone point me to the central classes of continuous query, 
> > > especially to the one that is responsible for the calculation of 
> > > the new data and packing it as a message back to the client?
> > >
> > > Thanks,
> > >
> > > Roi
> > >
> > > This message and the information contained herein is proprietary 
> > > and confidential and subject to the Amdocs policy statement,
> > >
> > > you may review at https://www.amdocs.com/about/email-disclaimer < 
> > > https://www.amdocs.com/about/email-disclaimer>
> > >
> > This