Re: [DISCUSS] Geode 1.14

[Owen Nichols]

support/1.13 was cut on May 4, 2020 as per Geode's published time-based 
schedule [1] (Monday on or after Feb 1, May 1, Aug 1, Nov 1)

prior to the expected Aug 3 date to cut support/1.14, the Geode community 
decided in July [2] that the usual quarterly schedule would not apply to 1.14, 
and then clarified in this thread that stabilization would instead happen on 
develop in parallel with drawing down the number of issues on the 1.14 
dashboard [3].

I still see 7 issues on the dashboard.  I had optimistically hoped it would 
work out to cut the branch today Feb 1 (in furtherance of returning to our 
quarterly schedule), but it doesn't seem like we're ready?  Hopefully this 
isn't a case of the "slippery slope" Geode was trying to avoid [4] by adopted a 
time-based cadence in 2018.

[1] 
https://www.cwiki.us/display/GEODE/Releasing+Apache+Geode#ReleasingApacheGeode-Announceintenttobranch
[2] 
https://lists.apache.org/x/thread.html/ra8ef649c2d02a216f42152d8b468f03e14bed0c0dccea6a0a9b511db@%3Cdev.geode.apache.org%3E
[3] https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336052
[4] 
https://lists.apache.org/x/thread.html/d36a63c3794d13506ecad3d52a2aca938dcf0f8509b61860bbbc50cd@%3Cdev.geode.apache.org%3E

On 1/6/21, 10:18 AM, "Nabarun Nag"  wrote:

Added a couple of tickets with the blocker label and are now available on 
the dashboard.

Please feel free to pick up those tasks.

Do reach out if anyone needs more information.

Regards
Nabarun

From: Alexander Murmann 
Sent: Wednesday, January 6, 2021 7:39 AM
To: dev@geode.apache.org 
Subject: Re: [DISCUSS] Geode 1.14

Hi team!

If I am reading the silence correctly as silent approval, can everyone 
please start to add the blocks-1.14.0​ label to all tickets we should consider 
as release blockers?

I also would encourage us to not remove the label when we resolve a ticket. 
This allows us to get a better understanding of our progress. We should remove 
the label if we decide that we can ship without a fix.


On a practical note: It seems like the dashboard doesn't always show all 
ticket changes immediately. I'll let you know when I get a better understanding 
of that behavior.

From: Alexander Murmann 
Sent: Tuesday, January 5, 2021 08:34
To: dev@geode.apache.org 
Subject: Re: [DISCUSS] Geode 1.14

I agree with Jianxia that a dashboard would be helpful.

Not every bug that affects a release should necessarily be a release 
blocker. Factors like low severity, existence in prior versions and time to fix 
might all play a role in deciding that a bug should be fixed in a later 
version. So we need a way to track what issues are considered a release blocker 
separately from what versions are affected. Ideally we'd use something like a 
blocked-releases fields. We don't have that right now, so I propose we use a 
blocks-1.14.0​ label.

I started on a dashboard that tracks these 
here.
 Everyone should have access to view it.

The dashboard shows all issues with the label. There are two lists one with 
open issues and one with resolved issues. I propose that we keep the blocker 
label on an issue if we resolve the issue, but remove it if we decide against 
resolving it in this version. In that case we might want to consider already 
tagging it with a future version (e.g. blocks-1.14.1​), so that it stays top of 
mind. We should only do that though if we indeed want to fix it in that 
version.?

Once the list of open issues marked as blockers is down to a handful, we 
should revisit the discussion about cutting the release branch.

If we agree that this is the right approach, I'll need all of your help in 
going back to open tickets and labeling them as release blockers where 
appropriate.

How does that sound?

From: Jacob Barrett 
Sent: Monday, January 4, 2021 16:40
To: dev@geode.apache.org 
Subject: Re: [DISCUSS] Geode 1.14



> On Jan 4, 2021, at 3:42 PM, Owen Nichols  wrote:
>
> How to tell whether develop is stable?

Same way we tell if a release branch is stable.

Listen, cutting a release branch from the develop branch shouldn’t result 
in weeks worth of stabilization of the release branch. That is a smell that we 
have bad things going into develop. If there are bad things in develop then we 
are introducing new things to develop on top of bad things, which will just end 

Canceled: apache/geode-native#2986 (develop - a939452)

[Travis CI]

Build Update for apache/geode-native
-

Build: #2986
Status: Canceled

Duration: ?
Commit: a939452 (develop)
Author: M. Oleske
Message: update sqlite (#735)

View the changeset: 
https://github.com/apache/geode-native/compare/5600a6b1fc26...a93945217aef

View the full build log and details: 
https://travis-ci.com/github/apache/geode-native/builds/215634180?utm_medium=notification_source=email

  Restart your build: 
https://travis-ci.com/github/apache/geode-native/builds/215634180?utm_medium=notification_source=email

--

You can unsubscribe from build emails from the apache/geode-native repository 
going to 
https://travis-ci.com/account/preferences/unsubscribe?repository=16807653_medium=notification_source=email.
Or unsubscribe from *all* email updating your settings at 
https://travis-ci.com/account/preferences/unsubscribe?utm_medium=notification_source=email.
Or configure specific recipients for build notifications in your .travis.yml 
file. See https://docs.travis-ci.com/user/notifications.

Re: [DISCUSS] RFC - Add option to allow newer Geode clients to connect to older Geode servers

[Jacob Barrett]

Having just spent some time yanking out some of the really really old version 
support I think a naive version knocking approach would work. During the client 
handshake the server will reject and close the connection of any client with a 
newer version number than it supports. The client could use this as signal to 
downgrade its version and try again. This could continue until the server 
accepts the client. We would need to decide if we would expect the entire 
membership to be a the same versions or if the version knocking needs to be on 
a per member basis. Obviously knocking for every connection is not ideal so 
some sort heuristic should be maintained for the life of the client.

Interestingly enough the clients sort of did this up until the merge of this 
version cleanups. All clients first made connections using the very old 
protocol version so that the server would send its version back. Then the 
client would disconnect and reconnect using its current version. The same could 
be done today with the current protocol version, the clients could make first 
connection with v1.0.0, get the server version, close and reconnect identifying 
themselves at the same server version.

-Jake


On Jan 29, 2021, at 3:35 PM, Dan Smith 
mailto:dasm...@vmware.com>> wrote:

Well, I have no objection to adding a system property for this if you want to 
try it. Since those properties aren't technically part of the public API I 
don't think we need to offer full support for what happens when the setting 
breaks. I'm just thinking ahead to what will happen when the protocol does 
change. At that point setting the system property will not work, unless the 
client has the capability to negotiate and discover the server version and use 
the old protocol the way that WAN does.

Do keep in mind that failures may not be obvious if the serialization protocol 
changes and your client is pretending to be a different version. I think it's 
possible that the errors might show up only in log messages or corrupted 
values, and only if you are using whatever features are affected by a protocol 
change.

-Dan

From: Alberto Gomez mailto:alberto.go...@est.tech>>
Sent: Friday, January 29, 2021 11:40 AM
To: dev@geode.apache.org 
mailto:dev@geode.apache.org>>
Subject: Re: [DISCUSS] RFC - Add option to allow newer Geode clients to connect 
to older Geode servers

Hi Dan,

Thanks a lot for your comments.

The scope of the RFC is not very ambitious. As I pointed out in it, the idea is 
not to implement the backward compatibility of clients with older servers. 
Rather, the aim is to allow to take advantage of the fact that serialization or 
other types of changes that may break this compatibility are not very frequent. 
For those cases where there have been no incompatible changes, with one of the 
proposed System Properties, it would be possible for a client to communicate 
with an older compatible server without the need of implementing anything 
extra. And we would have the test cases in place to assure this. For those 
cases where compatibility has been broken, it will not be possible to 
communicate the client with the older server and we would also have the tests 
showing that this communication is not possible even if the proposed System 
Property is used.

I do not know how costly it would be to implement and maintain the alternative 
approach you suggest with the negotiation required to support full backward 
compatibility. I would leave that to a different RFC. The good thing is that 
the current RFC could serve as a first step to implement the second, if it is 
agreed that this second feature is worth of being put in Geode.

Best regards,

Alberto

From: Dan Smith mailto:dasm...@vmware.com>>
Sent: Friday, January 29, 2021 1:56 AM
To: dev@geode.apache.org 
mailto:dev@geode.apache.org>>
Subject: Re: [DISCUSS] RFC - Add option to allow newer Geode clients to connect 
to older Geode servers

I think just sending the old version will only work until we actually make any 
changes to the protocol. Once we do, serialization will break unless we also 
change the client to pretend to be that old version, including the way it 
serializes and deserializes messages. With this proposal there will be no way 
for the client to use new features with a newer server since the version number 
of the client is set with a system property.

An alternative would be to have the client and the server need a way to 
negotiate which protocol they are going to communicate over. We do this already 
for WAN. WAN senders can be a higher version than receivers, otherwise we 
couldn't upgrade an Active/Active WAN. What happens is that the WAN receiver 
will accept a newer versioned client, and it sends back its own version. The 
client reads the receivers version and adjusts accordingly. You can see this in 

Re: [DISCUSS] RFC - Add option to allow newer Geode clients to connect to older Geode servers

[Anthony Baker]

In my ideal world, the version represents the protocol version and not a 
product release number. As Dan points out, we could add a negotiation option to 
allow more flexibility between clients and servers.

To accomplish this we would need a simpler and well-specified protocol.  The 
current protocol is pretty gnarly and difficult to piece together from the 
implementation.  Complicating this is the fact that the protocol includes not 
only the message definitions but also the data serialization mechanisms for any 
data type included in a message.  Given the challenge of answering “has any 
message or serialization changed?” we've taken the route of using release 
versions.  Perhaps we could build an automated analysis tool that could answer 
this question and reduce frequency of version bumps to match when something 
_really_ changes.

While some work has been done in this direction [1], it’s not far enough along 
to be a viable alternative as yet.

Anthony

[1] https://cwiki.apache.org/confluence/display/GEODE/New+Client+Server+Protocol



On Jan 29, 2021, at 3:35 PM, Dan Smith 
mailto:dasm...@vmware.com>> wrote:

Well, I have no objection to adding a system property for this if you want to 
try it. Since those properties aren't technically part of the public API I 
don't think we need to offer full support for what happens when the setting 
breaks. I'm just thinking ahead to what will happen when the protocol does 
change. At that point setting the system property will not work, unless the 
client has the capability to negotiate and discover the server version and use 
the old protocol the way that WAN does.

Do keep in mind that failures may not be obvious if the serialization protocol 
changes and your client is pretending to be a different version. I think it's 
possible that the errors might show up only in log messages or corrupted 
values, and only if you are using whatever features are affected by a protocol 
change.

-Dan

From: Alberto Gomez mailto:alberto.go...@est.tech>>
Sent: Friday, January 29, 2021 11:40 AM
To: dev@geode.apache.org 
mailto:dev@geode.apache.org>>
Subject: Re: [DISCUSS] RFC - Add option to allow newer Geode clients to connect 
to older Geode servers

Hi Dan,

Thanks a lot for your comments.

The scope of the RFC is not very ambitious. As I pointed out in it, the idea is 
not to implement the backward compatibility of clients with older servers. 
Rather, the aim is to allow to take advantage of the fact that serialization or 
other types of changes that may break this compatibility are not very frequent. 
For those cases where there have been no incompatible changes, with one of the 
proposed System Properties, it would be possible for a client to communicate 
with an older compatible server without the need of implementing anything 
extra. And we would have the test cases in place to assure this. For those 
cases where compatibility has been broken, it will not be possible to 
communicate the client with the older server and we would also have the tests 
showing that this communication is not possible even if the proposed System 
Property is used.

I do not know how costly it would be to implement and maintain the alternative 
approach you suggest with the negotiation required to support full backward 
compatibility. I would leave that to a different RFC. The good thing is that 
the current RFC could serve as a first step to implement the second, if it is 
agreed that this second feature is worth of being put in Geode.

Best regards,

Alberto

From: Dan Smith mailto:dasm...@vmware.com>>
Sent: Friday, January 29, 2021 1:56 AM
To: dev@geode.apache.org 
mailto:dev@geode.apache.org>>
Subject: Re: [DISCUSS] RFC - Add option to allow newer Geode clients to connect 
to older Geode servers

I think just sending the old version will only work until we actually make any 
changes to the protocol. Once we do, serialization will break unless we also 
change the client to pretend to be that old version, including the way it 
serializes and deserializes messages. With this proposal there will be no way 
for the client to use new features with a newer server since the version number 
of the client is set with a system property.

An alternative would be to have the client and the server need a way to 
negotiate which protocol they are going to communicate over. We do this already 
for WAN. WAN senders can be a higher version than receivers, otherwise we 
couldn't upgrade an Active/Active WAN. What happens is that the WAN receiver 
will accept a newer versioned client, and it sends back its own version. The 
client reads the receivers version and adjusts accordingly. You can see this in 
ClientSideHandshakeImpl.handshakeWithServer.

This will require a lot of testing to make sure that users won't see strange 
corruption related errors related to serialization changes.

Re: [Proposal] Geode Native Library Versioning

[Jacob Barrett]

On Jan 29, 2021, at 3:47 PM, Dan Smith 
mailto:dasm...@vmware.com>> wrote:
 I do think at least implementing some automated checking for whatever 
compatibility we intend to provide is a good idea.

I have a branch with a test using Abigail [1]. This branch depends on merging 
of a CI branch. It was actually through this branch that it was clear we have 
not been good about preserving any ABI compatibility between releases. I also 
found that we are exporting more symbols than we should in our library. All 
these need to be addressed over time. Baby steps I supposed…

[1] https://sourceware.org/libabigail/

-Jake