Re: [INFO] Latest test run of 200 DistributedTestOpenJDK8 passes

[Alexander Murmann]

Looking more into this, it looks like this was introduced by the changes
for GEODE-7458 - "Adding additional option in gfsh command "start gateway
sender" to control clearing of existing queues".

That happened about a month ago, but it's inherent to those flaky tests
that we discover them only after a while. Nonetheless, they become paper
cuts that ultimately slow us down substantially if they persist.

@Mario Ivanac If I am correct and GEODE-7458 introduced this you were the
one making that change. Might you be able to take a look at making that
test more reliable or reverting the change?

Thank you!

On Fri, Jun 19, 2020 at 7:57 AM Alexander Murmann 
wrote:

> Thank you so much for sharing this, Mark!
>
> It looks like there is a big cluster around WAN Gateway. Is anyone already
> looking into the WAN issues?
>
> On Thu, Jun 18, 2020 at 10:06 PM Mark Hanson  wrote:
>
>> FYI, the build success rate was around 90% or so about two months ago.
>>
>> Here are the DUnit tests that are currently failing in our tests, most
>> likely in CI, and PR pipelines.
>>
>> Please let me know if you have any questions.
>>
>> Thanks,
>> Mark
>>
>>
>>
>> ***
>>
>>  Overall build success rate: 78.0% (156 of 200)
>>
>>
>> ***
>>
>>
>>
>> The following test methods see failures in more than one class.  There
>> may be a failing *TestBase class
>>
>>
>>
>> *.testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived:
>> 12 failures :
>>
>>   SerialWANPersistenceEnabledGatewaySenderDUnitTest:  8 failures (96.000%
>> success rate)
>>
>>   SerialWANPersistenceEnabledGatewaySenderOffHeapDUnitTest:  4 failures
>> (98.000% success rate)
>>
>>
>>
>> *.testpersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived:
>> 12 failures :
>>
>>   ParallelWANPersistenceEnabledGatewaySenderOffHeapDUnitTest:  5 failures
>> (97.500% success rate)
>>
>>   ParallelWANPersistenceEnabledGatewaySenderDUnitTest:  7 failures
>> (96.500% success rate)
>>
>>
>>
>> *.testPingWrongServer:  4 failures :
>>
>>   ClientServerMiscSelectorDUnitTest:  3 failures (98.500% success rate)
>>
>>   ClientServerMiscDUnitTest:  1 failures (99.500% success rate)
>>
>>
>>
>>
>> ***
>>
>>
>>
>>
>>
>> org.apache.geode.internal.cache.wan.serial.SerialWANPersistenceEnabledGatewaySenderDUnitTest:
>> 8 failures (96.000% success rate)
>>
>>
>>
>>
>>  
>> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>>
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3539
>>
>>
>>  
>> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>>
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3526
>>
>>
>>  
>> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>>
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3505
>>
>>
>>  
>> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>>
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3435
>>
>>
>>  
>> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>>
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3414
>>
>>
>>  
>> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>>
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3391
>>
>>
>>  
>> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>>
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3363
>>
>>
>>  
>> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>>
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3359
>>
>>
>>
>> org.apache.geode.management.MemberMXBeanDistributedTest:  2 failures
>> (99.000% success rate)
>>
>>
>>
>>  testBucketCount
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3463
>>
>>  testBucketCount
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3411
>>
>>
>>
>> 

Re: Odg: Certificate Based Authorization

[Jinmei Liao]

In the old management team, we have been considering the idea of getting rid of 
jmx connection in gfsh and only using http connection mechanism.

On Jun 19, 2020 2:53 PM, Jacob Barrett  wrote:
So I can see why this research paper was so bleak about the options in trying 
to get the SSL certificate for the current connection being serviced. As they 
discovered the accept loop in OpenJDK’s (and older Oracle implementations) 
immediately fires the RMI operation to a thread pool after connected. This is 
after SSLSocket would have would’ve done the handshake and been passed to any 
of our validation callbacks so stashing anything in a thread local storage is 
dead.

Good news is deep in the sun.rmi.transport.tcp.TCPTransport there is a 
ThreadLocal that has the socket used to establish the 
connection and this thread local is set before each invocation of an RMI 
operation. The bad news is that it's private on an internal class. I think this 
is where the age of the research is in our favor. Back when I think it was 
writing we didn’t have OpenJDK. We had Oracle, IBM, and a few others. Now with 
everything pretty much converging on OpenJDK I don’t believe it as as nasty to 
go poke at this internal using reflection. I think it is less dirty then their 
nasty trick of utilizing the IPv6 address as a unique identifier in a custom 
Socket.

Once we have the SSLSocket for this connection then we are golden. From there 
you have public API access to the SSLSession.

Looking at the OpenJDK source this class has largely been unchanged since its 
initial import into the repo in 2007. Most importantly the private member in 
question has been and its sill available in all versions of OpenJDK. Sure this 
limits us to OpenJDK support for certificate based authentication by SSL 
handshake via RMI but in Geode that’s really only gfsh. This is a really small 
surface area. With the focus being on converting gfsh activities into REST APIs 
this surface area is shrinking. Personally I would be inclined to leave RMI out 
of the solution initially. Second I would use this private variable to compete 
the support in OpenJDK.

-Jake


> On Jun 19, 2020, at 11:14 AM, Jacob Barrett  wrote:
>
>
>
>>
>> On Jun 18, 2020, at 4:24 AM, Jakov Varenina 
>> mailto:jakov.varen...@est.tech>> wrote:
>>
>> In order to completely remove the need for username/password, it is required 
>> that we implement this new kind of authorization on *all* geode 
>> interfaces/components (cluster, gateway, web, jmx, locator, server). The 
>> reason why we didn't have any progress is because we faced major obstacle 
>> during development when we tried to retrieve clients certificate from RMI 
>> connections (e.g. jmx connections). It seems there are no easy/nice way to 
>> retrieve it, and what we came up so far is following:
>>
>> 1) We have found some possible "hack solution" that could be implemented and 
>> it is described in the following paper 
>> (https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.224.2915%26rep%3Drep1%26type%3Dpdfdata=02%7C01%7Cjiliao%40vmware.com%7C047d1b26112f4948b0ab08d8149b3ad3%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637282004228769823sdata=6fB1iM1Dk%2BJ%2B8oCHankNhd4%2FSd5EwKcxZlex%2BTcUg8U%3Dreserved=0).
>>  We have started to work on the prototype that will implement this solution.
>
> Wow, that is a hack. Have you found any implementation of this solution. 
> There doesn’t appear to be a repository listed. There also doesn’t appear to 
> be a publish date on this document. The most recent references are from 2010. 
> I wonder if things are better now. I am going to poke at the Java source code 
> a bit and report back.
>
> Would your needs be dependent on Java 8? If we found a solution that only 
> worked say with java 12, would that work?
>
> -Jake
>

Re: Odg: Certificate Based Authorization

[Jacob Barrett]

So I can see why this research paper was so bleak about the options in trying 
to get the SSL certificate for the current connection being serviced. As they 
discovered the accept loop in OpenJDK’s (and older Oracle implementations) 
immediately fires the RMI operation to a thread pool after connected. This is 
after SSLSocket would have would’ve done the handshake and been passed to any 
of our validation callbacks so stashing anything in a thread local storage is 
dead.

Good news is deep in the sun.rmi.transport.tcp.TCPTransport there is a 
ThreadLocal that has the socket used to establish the 
connection and this thread local is set before each invocation of an RMI 
operation. The bad news is that it's private on an internal class. I think this 
is where the age of the research is in our favor. Back when I think it was 
writing we didn’t have OpenJDK. We had Oracle, IBM, and a few others. Now with 
everything pretty much converging on OpenJDK I don’t believe it as as nasty to 
go poke at this internal using reflection. I think it is less dirty then their 
nasty trick of utilizing the IPv6 address as a unique identifier in a custom 
Socket. 

Once we have the SSLSocket for this connection then we are golden. From there 
you have public API access to the SSLSession. 

Looking at the OpenJDK source this class has largely been unchanged since its 
initial import into the repo in 2007. Most importantly the private member in 
question has been and its sill available in all versions of OpenJDK. Sure this 
limits us to OpenJDK support for certificate based authentication by SSL 
handshake via RMI but in Geode that’s really only gfsh. This is a really small 
surface area. With the focus being on converting gfsh activities into REST APIs 
this surface area is shrinking. Personally I would be inclined to leave RMI out 
of the solution initially. Second I would use this private variable to compete 
the support in OpenJDK.

-Jake


> On Jun 19, 2020, at 11:14 AM, Jacob Barrett  wrote:
> 
> 
> 
>> 
>> On Jun 18, 2020, at 4:24 AM, Jakov Varenina 
>> mailto:jakov.varen...@est.tech>> wrote:
>> 
>> In order to completely remove the need for username/password, it is required 
>> that we implement this new kind of authorization on *all* geode 
>> interfaces/components (cluster, gateway, web, jmx, locator, server). The 
>> reason why we didn't have any progress is because we faced major obstacle 
>> during development when we tried to retrieve clients certificate from RMI 
>> connections (e.g. jmx connections). It seems there are no easy/nice way to 
>> retrieve it, and what we came up so far is following:
>> 
>> 1) We have found some possible "hack solution" that could be implemented and 
>> it is described in the following paper 
>> (https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.224.2915%26rep%3Drep1%26type%3Dpdfdata=02%7C01%7Cjabarrett%40vmware.com%7C0540782b5fd04f118ec108d8147c9e31%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637281872755348036sdata=AXxPkuFmrfvqQLKWQSTf9w0%2B7tFjsQ8%2FuATCNAx3hT8%3Dreserved=0).
>>  We have started to work on the prototype that will implement this solution.
> 
> Wow, that is a hack. Have you found any implementation of this solution. 
> There doesn’t appear to be a repository listed. There also doesn’t appear to 
> be a publish date on this document. The most recent references are from 2010. 
> I wonder if things are better now. I am going to poke at the Java source code 
> a bit and report back. 
> 
> Would your needs be dependent on Java 8? If we found a solution that only 
> worked say with java 12, would that work?
> 
> -Jake
> 

Re: Odg: Certificate Based Authorization

[Jacob Barrett]

On Jun 19, 2020, at 12:20 PM, Anthony Baker 
mailto:bak...@vmware.com>> wrote:

That’s fine, I just want to understand what happens when I use this API:

createdAuthenticatedView(…)

Does it throw an exception?  Silently work but not switch to the new user?


I would expect that first off we document the usage of certificate based 
authentication is generally mutually exclusive to the use of multiuser 
authentication. Secondly, I would expect the SecurityManager should reject any 
authentication request that it receives if it can’t authenticate the given 
credentials it was sent so the behavior at the client should reflect that those 
credentials were denied.

It would however be possible to mix both application and user authentication in 
a SecurityManger in theory you could have multi user authentication, just not 
with certificates for the user (can be done with client/server protocol 
changes). An operation performed on an authenticated view would have both the 
application’s certificate credentials and user credentials sent to the 
SecurityManager. That implementation can do whatever it wants with them.

-Jake

RE: [VOTE] Backporting of GEODE-8261 to 1.13 release branch.

[Nabarun Nag]

Thank you all for the  votes needed for the backport. It has been backported to 
support/1.13 branch

GEODE-8261: Added a null check for the proxyID. (#5251)

Regards
Naba


-Original Message-
From: Jianxia Chen  
Sent: Friday, June 19, 2020 12:22 PM
To: dev@geode.apache.org
Subject: Re: [VOTE] Backporting of GEODE-8261 to 1.13 release branch.

+1

On Fri, Jun 19, 2020 at 12:16 PM Nabarun Nag  wrote:

> Hi Geode devs,
>
> Requesting vote to backport of GEODE-8261 to 1.13
>
> Why?
> This commit fixes an issue with servers throwing null pointer 
> exceptions while a member is being shutdown and registering interest is in 
> process.
>
> SHA
> 720a4caea2ddb22296aa3225fc5264d2096cdf20
>
>
> Regards
> Nabarun
>

Re: [VOTE] Backporting of GEODE-8261 to 1.13 release branch.

[Jianxia Chen]

+1

On Fri, Jun 19, 2020 at 12:16 PM Nabarun Nag  wrote:

> Hi Geode devs,
>
> Requesting vote to backport of GEODE-8261 to 1.13
>
> Why?
> This commit fixes an issue with servers throwing null pointer exceptions
> while a member is being shutdown and registering interest is in process.
>
> SHA
> 720a4caea2ddb22296aa3225fc5264d2096cdf20
>
>
> Regards
> Nabarun
>

Re: [VOTE] Backporting of GEODE-8261 to 1.13 release branch.

[Ivan Godwin]

+1

On 6/19/20, 12:16 PM, "Nabarun Nag"  wrote:

Hi Geode devs,

Requesting vote to backport of GEODE-8261 to 1.13

Why?
This commit fixes an issue with servers throwing null pointer exceptions 
while a member is being shutdown and registering interest is in process.

SHA
720a4caea2ddb22296aa3225fc5264d2096cdf20


Regards
Nabarun

Re: Odg: Certificate Based Authorization

[Anthony Baker]

That’s fine, I just want to understand what happens when I use this API:

createdAuthenticatedView(…)

Does it throw an exception?  Silently work but not switch to the new user?


Thanks,
Anthony


On Jun 19, 2020, at 10:14 AM, Jacob Barrett 
mailto:jabarr...@vmware.com>> wrote:

1) Multi-user authentication will not be supported when using this new kind of 
SecurityManager implementation.

I agree here. What we are implementing is application/service level 
authentication. Using certs from the applications connection is not viable for 
multi-user authentication. A method similar to option 2 below would need to be 
implemented for such a thing. (See comments below on option 2 issues)

Re: [VOTE] Backporting of GEODE-8261 to 1.13 release branch.

[Donal Evans]

+1

From: Nabarun Nag 
Sent: Friday, June 19, 2020 12:15 PM
To: dev@geode.apache.org 
Subject: [VOTE] Backporting of GEODE-8261 to 1.13 release branch.

Hi Geode devs,

Requesting vote to backport of GEODE-8261 to 1.13

Why?
This commit fixes an issue with servers throwing null pointer exceptions while 
a member is being shutdown and registering interest is in process.

SHA
720a4caea2ddb22296aa3225fc5264d2096cdf20


Regards
Nabarun

[VOTE] Backporting of GEODE-8261 to 1.13 release branch.

[Nabarun Nag]

Hi Geode devs,

Requesting vote to backport of GEODE-8261 to 1.13

Why?
This commit fixes an issue with servers throwing null pointer exceptions while 
a member is being shutdown and registering interest is in process.

SHA
720a4caea2ddb22296aa3225fc5264d2096cdf20


Regards
Nabarun

Re: Client Usage of RMI

[Kirk Lund]

The only use of RMI in Geode that I'm aware of is:

1) GFSH to Locator communication over jmx/rmi
2) Communication from the main JUnit JVM to DUnit JVM for dunit tests

In theory, a User could connect to the Locator over jmx/rmi from any JVM
using JDK JMX code. This could include deploying a Function (that uses JMX)
to a Server or executing JMX code directly within a Geode Java Client.

On Fri, Jun 19, 2020 at 11:17 AM Jacob Barrett  wrote:

> All,
>
> Related to the conversation about certificate based authentication, how
> much of the client side would invoke any RMI requests? I am trying to gauge
> the impact of not having RMI as an option for client applications wishing
> to use certificate based authentication.
>
> -Jake
>
>

Client Usage of RMI

[Jacob Barrett]

All,

Related to the conversation about certificate based authentication, how much of 
the client side would invoke any RMI requests? I am trying to gauge the impact 
of not having RMI as an option for client applications wishing to use 
certificate based authentication.

-Jake

Re: Odg: Certificate Based Authorization

[Jacob Barrett]

> 
> On Jun 18, 2020, at 4:24 AM, Jakov Varenina 
> mailto:jakov.varen...@est.tech>> wrote:
> 
> In order to completely remove the need for username/password, it is required 
> that we implement this new kind of authorization on *all* geode 
> interfaces/components (cluster, gateway, web, jmx, locator, server). The 
> reason why we didn't have any progress is because we faced major obstacle 
> during development when we tried to retrieve clients certificate from RMI 
> connections (e.g. jmx connections). It seems there are no easy/nice way to 
> retrieve it, and what we came up so far is following:
> 
> 1) We have found some possible "hack solution" that could be implemented and 
> it is described in the following paper 
> (https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.224.2915%26rep%3Drep1%26type%3Dpdfdata=02%7C01%7Cjabarrett%40vmware.com%7Ce9d45eb604334359a29708d814745030%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637281837084459652sdata=vWYeTsTQmP6RyIz5xEIdsVRjnXii%2Buk02TwLXuyGBEo%3Dreserved=0).
>  We have started to work on the prototype that will implement this solution.

Wow, that is a hack. Have you found any implementation of this solution. There 
doesn’t appear to be a repository listed. There also doesn’t appear to be a 
publish date on this document. The most recent references are from 2010. I 
wonder if things are better now. I am going to poke at the Java source code a 
bit and report back. 

Would your needs be dependent on Java 8? If we found a solution that only 
worked say with java 12, would that work?

-Jake

Re: Odg: Certificate Based Authorization

[Jacob Barrett]

On Jun 18, 2020, at 4:24 AM, Jakov Varenina 
mailto:jakov.varen...@est.tech>> wrote:

Hi Anthony and all,

I have been working with Mario on this feature. Let me first answer the 
questions:

1) Multi-user authentication will not be supported when using this new kind of 
SecurityManager implementation.

I agree here. What we are implementing is application/service level 
authentication. Using certs from the applications connection is not viable for 
multi-user authentication. A method similar to option 2 below would need to be 
implemented for such a thing. (See comments below on option 2 issues)

2) The idea was to use only CN for principal, and ignore SAN (this would be 
documented). But we could as you suggested forward both, or even whole 
certificate, and let the user to decide which one to use. According to RFC 6125 
,
 SAN is not replacement for CN but they complement each other.

I have reservations about only forwarding the CN in the credential properties. 
I have been doing some research in this area I strongly feel we need to forward 
the entire certificate. It should also be done in a way the prevents the client 
from being able to forward this same key/value to the server in the 
authentication part of the handshake. What the SecurityMangaer is trusting is 
that the SSL engine did its due diligence and verified and validated the 
certificate. So only the SSL Engine should be able to export this 
“authenticated” certificate. The SecurityManager is simply trusting the third 
party authentication by its mean presence in the credentials, or some other out 
of band means. By passing the entire certificate it allows the SM implantation 
to use whatever portions of the certificate it wants for identity mapping. It 
could use the CN, something the SAN, and some extension.

In order to completely remove the need for username/password, it is required 
that we implement this new kind of authorization on *all* geode 
interfaces/components (cluster, gateway, web, jmx, locator, server). The reason 
why we didn't have any progress is because we faced major obstacle during 
development when we tried to retrieve clients certificate from RMI connections 
(e.g. jmx connections). It seems there are no easy/nice way to retrieve it, and 
what we came up so far is following:

1) We have found some possible "hack solution" that could be implemented and it 
is described in the following paper 
(https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.224.2915%26rep%3Drep1%26type%3Dpdfdata=02%7C01%7Cjabarrett%40vmware.com%7Cca7271e56ce84e558a2a08d8137a41be%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637280763124731300sdata=%2Fdk70AwZSk0YVPDrYESvMq2u6cV%2Bn4ia3i3BQGpgE2c%3Dreserved=0).
 We have started to work on the prototype that will implement this solution.

I want to spend some time digging into this issue with you. I just saw 
yesterday you all were stuck on this and haven’t had a chance to dig in. Can I 
sync up with you later today?

2) Second idea is that client reads CN (principal) from certificate and sends 
it towards server in a same way as Username/Password is sent now over RMI 
connections. The downside of this solution is that server doesn't verify the 
originator identity, that is, will not compare received principal with one 
received within the client certificate. With this solution, client's that uses 
certificates signed by geode trusted Certificate Authority and with lower 
authorization privileges, can "hack" client implementation and send principal 
with higher ( then which was given to them with a certificate) authorization 
privileges over RMI connection, since that principal will not compared with one 
received in the certificate.

As you pointed out, having the client send the certificate in the auth part of 
the handshake doesn’t authenticate the origin. To do this you must have a 
challenge. The current protocol does not have a challenge response handshake so 
the challenge must be pre-arranged. Like simply hashing the a password gives 
you the ability to replay the hash should you intercept it, same would go for 
the cert. Pre-arranged challenges have also their own security issues because 
you know some part of the message, think cracking the Enigma. A common practice 
would be a known challenge with salt. It solves the known message part since 
there is randomness at the start of the message it randomizes the output of the 
encryption. It doesn’t solve the playback issue that pre-arranged challenges 
have though. For this you need a once token of some kind. This token can be 
used once and only once when making 

Re: [PROPOSAL] back port fix for GEODE-8251 to support branches

[Dave Barnes]

OK to backport, Jinmei. Thanks for your contribution.
-Dave

(I saw a +1 vote from Owen in another email client, though it doesn’t seem to 
show up in this rendering of the thread. Thought I had successfully worked 
around the email issue. Hmmm…)

> On Jun 19, 2020, at 8:47 AM, Jianxia Chen  wrote:
> 
> +1
> 
> On Fri, Jun 19, 2020 at 8:46 AM Jinmei Liao  wrote:
> 
>> Need one more vote 
>> 
>> From: Bruce Schuchardt 
>> Sent: Thursday, June 18, 2020 3:43 PM
>> To: dev@geode.apache.org 
>> Subject: Re: [PROPOSAL] back port fix for GEODE-8251 to support branches
>> 
>> +1
>> 
>> On 6/18/20, 3:24 PM, "Jinmei Liao"  wrote:
>> 
>>The fix for this issue
>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FGEODE-8251data=02%7C01%7Cjiliao%40vmware.com%7C76cb38f955c84ade5d9f08d813d91e9c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637281170530289424sdata=a8fdpKs4MUwr90bE0oOgobk5so840TJhK47%2FJFE0JDs%3Dreserved=0
>> is needed on support 1.13 branch in order for rolling upgrade to work from
>> 1.12 to 1.13.
>> 
>>Thanks!
>> 
>>Jinmei
>> 
>> 

Re: Setting your commit email address

[Jacob Barrett]

I think it's useful for contact those that feel compelled to hide their email 
addresses. If there is an email address you can surely use that directly. I 
have used the commenting feature a few times and like it because it puts the 
comment with the code making it dead clear what I am talking about. 

I think the issue with PRs is one of email fatigue. Because every JIRA and 
every PR gets blasted to mailing list we tend to ignore the stream of messages. 
But what if the only JIRAs and PRs you got were just those your were tagged in? 
I sort of have this working for myself for JIRAs but still experimenting. I 
unsubscribed from the issues list. Now the only JIRAs I get notifications on 
are ones I am watching. The downside is I don’t get to see new JIRAs as they 
come in. There is a feature in JIRA to email everyone or a set of people on 
JIRA create events. It wasn’t user settable before but maybe it is now. I plan 
to investigate this in the future. For now I am enjoying less of the JIRA spam. 
I want to find a similar pattern for PRs. Notify me on new PRs and then only 
for PRs I am subscribed to or mentioned.

-Jake


> On Jun 19, 2020, at 9:16 AM, Kirk Lund  wrote:
> 
> Since most folks don't even notice they've been requested to review a PR, I
> don't have high confidence that they'll notice a notification about a
> comment on a past commit. Seems sort of useful though. Thank you!
> 
> On Thu, Jun 18, 2020 at 3:53 PM Owen Nichols  wrote:
> 
>> Perhaps a little-known feature of GitHub: you can comment on commits too,
>> not just on PRs.  For example:
>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode%2Fcommit%2Fd82e30d3dd50e3983c332063c41915e97aca721adata=02%7C01%7Cjabarrett%40vmware.com%7C81f534ce227840414ac508d8146c1ff4%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637281801914055624sdata=fa6TNqaGXy8jkqUokxJSLbDEIKi1QiRaT1zqvyFWCBs%3Dreserved=0
>> 
>> On 6/18/20, 3:48 PM, "Kirk Lund"  wrote:
>> 
>>I'm not sure what you're talking about Jake (PRs?). I'm talking about
>>commits that are already on develop.
>> 
>>For example, if someone adds a new dunit test that turns out to be very
>>flaky, I'd like to find out from the git log who authored it and send
>> them
>>an email directly (privately) and I have no way to do that if they're
>>hiding their email address.
>> 
>>Thanks though,
>>Kirk
>> 
>>On Thu, Jun 18, 2020 at 3:43 PM Jacob Barrett 
>> wrote:
>> 
>>> Regardless of the email address issue, you can go to the commit and
>> make
>>> comments and @johndoe and they will get a notification.
>>> 
>>> -Jake
>>> 
>>> 
 On Jun 18, 2020, at 10:31 AM, Kirk Lund  wrote:
 
 I guess my main point is that I don't like private emails in OSS
>> commit
 messages.
 
 Author: John Doe 
 
 If there's a problem with a commit made by John Doe and he doesn't
>> work
>>> for
 Pivotal, then I have no idea how to contact him directly to
>> discuss the
 commit. If I visit github.com/johndoe, there's no way to send a
>> message
>>> to
 Mr. Doe. If Mr. Doe is a committer, then I know john...@apache.org
>>> exists.
 
 Is there really a privacy problem with having "john...@apache.org"
>> in
>>> the
 commit message? Hiding it makes it very difficult for the rest of
>> us in
>>> the
 same community to contact that person which is why I don't like
>> private
 emails.
 
 Are you getting lots of spam from git commits??
 
 
 On Wed, Jun 17, 2020 at 1:20 PM Nabarun Nag 
>> wrote:
 
> Hi Kirk,
> 
> I think it is also now in the privacy setting in GitHub for
>> anyone who
> wants to keep emails private. [
>>> 
>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsettings%2Femailsdata=02%7C01%7Cjabarrett%40vmware.com%7C81f534ce227840414ac508d8146c1ff4%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637281801914055624sdata=0oZrwvG70fpOqPFl%2F3by%2FPY4nEN299QUb77UHspnbiI%3Dreserved=0
>>> :
> 
>>> 
>> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsettings%2Femailsdata=02%7C01%7Cjabarrett%40vmware.com%7C81f534ce227840414ac508d8146c1ff4%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637281801914055624sdata=0oZrwvG70fpOqPFl%2F3by%2FPY4nEN299QUb77UHspnbiI%3Dreserved=0
>>> ]
> 
> This setting is needed for web based git operations like squash
>> merging
> PRs etc.
> 
> In GitHub:
> "Keep my email addresses private
> We’ll remove your public profile email and use
> nabarun...@users.noreply.github.com when performing web-based Git
> operations (e.g. edits and merges) and sending email on your
>> behalf. If
>>> you
> want command line Git operations to use your private email you
>> must set
> your email in Git."
> 
> Regards
> Nabarun
> 
> -Original Message-
> From: Kirk Lund 
> Sent: Wednesday, June 17, 2020 1:12 PM
> 

Re: Setting your commit email address

[Kirk Lund]

Since most folks don't even notice they've been requested to review a PR, I
don't have high confidence that they'll notice a notification about a
comment on a past commit. Seems sort of useful though. Thank you!

On Thu, Jun 18, 2020 at 3:53 PM Owen Nichols  wrote:

> Perhaps a little-known feature of GitHub: you can comment on commits too,
> not just on PRs.  For example:
> https://github.com/apache/geode/commit/d82e30d3dd50e3983c332063c41915e97aca721a
>
> On 6/18/20, 3:48 PM, "Kirk Lund"  wrote:
>
> I'm not sure what you're talking about Jake (PRs?). I'm talking about
> commits that are already on develop.
>
> For example, if someone adds a new dunit test that turns out to be very
> flaky, I'd like to find out from the git log who authored it and send
> them
> an email directly (privately) and I have no way to do that if they're
> hiding their email address.
>
> Thanks though,
> Kirk
>
> On Thu, Jun 18, 2020 at 3:43 PM Jacob Barrett 
> wrote:
>
> > Regardless of the email address issue, you can go to the commit and
> make
> > comments and @johndoe and they will get a notification.
> >
> > -Jake
> >
> >
> > > On Jun 18, 2020, at 10:31 AM, Kirk Lund  wrote:
> > >
> > > I guess my main point is that I don't like private emails in OSS
> commit
> > > messages.
> > >
> > > Author: John Doe 
> > >
> > > If there's a problem with a commit made by John Doe and he doesn't
> work
> > for
> > > Pivotal, then I have no idea how to contact him directly to
> discuss the
> > > commit. If I visit github.com/johndoe, there's no way to send a
> message
> > to
> > > Mr. Doe. If Mr. Doe is a committer, then I know john...@apache.org
> > exists.
> > >
> > > Is there really a privacy problem with having "john...@apache.org"
> in
> > the
> > > commit message? Hiding it makes it very difficult for the rest of
> us in
> > the
> > > same community to contact that person which is why I don't like
> private
> > > emails.
> > >
> > > Are you getting lots of spam from git commits??
> > >
> > >
> > > On Wed, Jun 17, 2020 at 1:20 PM Nabarun Nag 
> wrote:
> > >
> > >> Hi Kirk,
> > >>
> > >> I think it is also now in the privacy setting in GitHub for
> anyone who
> > >> wants to keep emails private. [
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsettings%2Femailsdata=02%7C01%7Conichols%40vmware.com%7C7ac9167a37b54c070d7e08d813d9aa6e%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637281172880655331sdata=QrtQe4ftMGNQIaf1KOXMuQXDQQAnL4x09Cpa1gW47%2Bs%3Dreserved=0
> > :
> > >>
> >
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsettings%2Femailsdata=02%7C01%7Conichols%40vmware.com%7C7ac9167a37b54c070d7e08d813d9aa6e%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637281172880655331sdata=QrtQe4ftMGNQIaf1KOXMuQXDQQAnL4x09Cpa1gW47%2Bs%3Dreserved=0
> > ]
> > >>
> > >> This setting is needed for web based git operations like squash
> merging
> > >> PRs etc.
> > >>
> > >> In GitHub:
> > >> "Keep my email addresses private
> > >> We’ll remove your public profile email and use
> > >> nabarun...@users.noreply.github.com when performing web-based Git
> > >> operations (e.g. edits and merges) and sending email on your
> behalf. If
> > you
> > >> want command line Git operations to use your private email you
> must set
> > >> your email in Git."
> > >>
> > >> Regards
> > >> Nabarun
> > >>
> > >> -Original Message-
> > >> From: Kirk Lund 
> > >> Sent: Wednesday, June 17, 2020 1:12 PM
> > >> To: dev@geode.apache.org
> > >> Subject: Setting your commit email address
> > >>
> > >> Please make sure you've setup your commit email address. It makes
> it
> > much
> > >> easier to find out who committed something and how to contact
> them if
> > >> there's a problem.
> > >>
> > >> You typically use the following to set your email address
> globally in
> > git:
> > >>
> > >> $ git config --global user.email "em...@example.com"
> > >>
> > >> You can also setup different repos with different email addresses
> by
> > using:
> > >>
> > >> $ git config user.email "em...@example.com"
> > >>
> > >> In the below example, it's much easier to follow up with the
> author of
> > the
> > >> 1st commit than the author of the 2nd commit:
> > >>
> > >> commit b1107d2e403404337c22830a4964eefc2490ef50
> > >> Author: John Doe 
> > >> Date:   Tue Jun 16 12:25:30 2020 -0700
> > >>
> > >>GEODE-: add something new
> > >>
> > >> commit e159238175766b46cbb6fe1e3459aa2da68db756
> > >> Author: John Doe 
> > >> Date:   Tue Jun 16 10:55:16 2020 -0700
> > >>
> > >>GEODE-: fix something bad
> > 

Re: [PROPOSAL] back port fix for GEODE-8251 to support branches

[Jianxia Chen]

+1

On Fri, Jun 19, 2020 at 8:46 AM Jinmei Liao  wrote:

> Need one more vote 
> 
> From: Bruce Schuchardt 
> Sent: Thursday, June 18, 2020 3:43 PM
> To: dev@geode.apache.org 
> Subject: Re: [PROPOSAL] back port fix for GEODE-8251 to support branches
>
> +1
>
> On 6/18/20, 3:24 PM, "Jinmei Liao"  wrote:
>
> The fix for this issue
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FGEODE-8251data=02%7C01%7Cjiliao%40vmware.com%7C76cb38f955c84ade5d9f08d813d91e9c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637281170530289424sdata=a8fdpKs4MUwr90bE0oOgobk5so840TJhK47%2FJFE0JDs%3Dreserved=0
> is needed on support 1.13 branch in order for rolling upgrade to work from
> 1.12 to 1.13.
>
> Thanks!
>
> Jinmei
>
>

Re: [PROPOSAL] back port fix for GEODE-8251 to support branches

[Jinmei Liao]

Need one more vote 

From: Bruce Schuchardt 
Sent: Thursday, June 18, 2020 3:43 PM
To: dev@geode.apache.org 
Subject: Re: [PROPOSAL] back port fix for GEODE-8251 to support branches

+1

On 6/18/20, 3:24 PM, "Jinmei Liao"  wrote:

The fix for this issue 
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FGEODE-8251data=02%7C01%7Cjiliao%40vmware.com%7C76cb38f955c84ade5d9f08d813d91e9c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637281170530289424sdata=a8fdpKs4MUwr90bE0oOgobk5so840TJhK47%2FJFE0JDs%3Dreserved=0
 is needed on support 1.13 branch in order for rolling upgrade to work from 
1.12 to 1.13.

Thanks!

Jinmei

Re: [INFO] Latest test run of 200 DistributedTestOpenJDK8 passes

[Alexander Murmann]

Thank you so much for sharing this, Mark!

It looks like there is a big cluster around WAN Gateway. Is anyone already
looking into the WAN issues?

On Thu, Jun 18, 2020 at 10:06 PM Mark Hanson  wrote:

> FYI, the build success rate was around 90% or so about two months ago.
>
> Here are the DUnit tests that are currently failing in our tests, most
> likely in CI, and PR pipelines.
>
> Please let me know if you have any questions.
>
> Thanks,
> Mark
>
>
>
> ***
>
>  Overall build success rate: 78.0% (156 of 200)
>
>
> ***
>
>
>
> The following test methods see failures in more than one class.  There may
> be a failing *TestBase class
>
>
>
> *.testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived:
> 12 failures :
>
>   SerialWANPersistenceEnabledGatewaySenderDUnitTest:  8 failures (96.000%
> success rate)
>
>   SerialWANPersistenceEnabledGatewaySenderOffHeapDUnitTest:  4 failures
> (98.000% success rate)
>
>
>
> *.testpersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived:
> 12 failures :
>
>   ParallelWANPersistenceEnabledGatewaySenderOffHeapDUnitTest:  5 failures
> (97.500% success rate)
>
>   ParallelWANPersistenceEnabledGatewaySenderDUnitTest:  7 failures
> (96.500% success rate)
>
>
>
> *.testPingWrongServer:  4 failures :
>
>   ClientServerMiscSelectorDUnitTest:  3 failures (98.500% success rate)
>
>   ClientServerMiscDUnitTest:  1 failures (99.500% success rate)
>
>
>
>
> ***
>
>
>
>
>
> org.apache.geode.internal.cache.wan.serial.SerialWANPersistenceEnabledGatewaySenderDUnitTest:
> 8 failures (96.000% success rate)
>
>
>
>
>  
> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3539
>
>
>  
> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3526
>
>
>  
> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3505
>
>
>  
> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3435
>
>
>  
> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3414
>
>
>  
> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3391
>
>
>  
> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3363
>
>
>  
> testReplicatedRegionPersistentWanGateway_restartSenderWithCleanQueues_expectNoEventsReceived
>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3359
>
>
>
> org.apache.geode.management.MemberMXBeanDistributedTest:  2 failures
> (99.000% success rate)
>
>
>
>  testBucketCount
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3463
>
>  testBucketCount
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3411
>
>
>
> org.apache.geode.internal.cache.tier.sockets.RedundancyLevelPart3DUnitTest:
> 1 failures (99.500% success rate)
>
>
>
>  testRegisterInterestAndMakePrimaryWithFullRedundancy
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3381
>
>
>
> org.apache.geode.management.internal.cli.commands.QueryCommandOverHttpDUnitTest:
> 1 failures (99.500% success rate)
>
>
>
>  testSimpleQueryOnLocator
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-mass-test-run-main/jobs/DistributedTestOpenJDK8/builds/3516
>
>
>
> org.apache.geode.internal.cache.tier.sockets.ClientServerMiscDUnitTest:  1
> failures (99.500% success rate)
>
>
>
>  testPingWrongServer
>