Re: [DISCUSS] proposal to pare down old-version testing
Sounds like there is consensus to implement this proposal.
Support branches and develop have been updated. Release scripts will
automatically check whether the protocol has changed and maintain the list
accordingly.
On 1/4/22, 11:21 AM, "Alexander Murmann" wrote:
+1 Great compromise! Let's not forget about this whenever we change the
protocol
--
Please provide anonymous feedback for me
here<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fforms.gle%2FkpUiaty3jt8X9y4H9data=04%7C01%7Conichols%40vmware.com%7C951690bb3afc4c13b6ab08d9cfb75ec2%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637769208757591829%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=%2FSdd%2ByOW%2B0sf1u46uQr27AVPKejKY6M8cexjeC22WVk%3Dreserved=0>.
From: Anilkumar Gingade
Sent: Tuesday, January 4, 2022 07:09
To: dev@geode.apache.org
Subject: Re: [DISCUSS] proposal to pare down old-version testing
+1
Thanks for bringing this and taking care of this.
-Anil.
On 1/3/22, 10:41 AM, "Dan Smith" wrote:
Looking at KnownVersion.java - we did make protocol changes in 1.12.1
and 1.13.2. So, my suggestion would be to keep 1.12.0 and 1.13.1, but dop all
the other patch versions that aren't the latest.
-Dan
From: Dan Smith
Sent: Monday, January 3, 2022 10:37 AM
To: dev@geode.apache.org
Subject: Re: [DISCUSS] proposal to pare down old-version testing
+1 - this seems reasonable to me. If we do make a protocol change in a
patch, we could potentially keep around an older patch version just in that
specific case, but otherwise I think this makes sense.
-Dan
From: Anthony Baker
Sent: Thursday, December 23, 2021 8:53 AM
To: dev@geode.apache.org
Subject: Re: [DISCUSS] proposal to pare down old-version testing
Interesting data point: 40% of maven central downloads last month were
for version 1.4.0. Of course those numbers can be easily skewed by CI bots, but
still!
@Owen, I think your suggestion nicely improves practicality while
continuing to support strong compatibility. In many cases it’s quite a bit
easier to upgrade the Geode server cluster compared to potentially many, many
client applications. Supporting older client versions gives users time to
upgrade, quicker access to bug fixes, and helps avoids downtime.
+1
Anthony
> On Dec 22, 2021, at 7:13 PM, Owen Nichols wrote:
>
> Since adopting our N-2 support policy, the list of released versions
in /settings.gradle has ballooned to over 30 entries [1].
>
> CI tests use this list to confirm that we don’t break rolling upgrade
ability or compatibility with older clients, but some of these tests don’t seem
to scale well: PR#7203 to add the most recent 3 releases (bringing the total to
33) is unable to pass CI after 8 tries.
>
> Possible solutions fall into two categories: keep the full list and
throw developers and/or more hardware at the struggling tests, or concede that
testing every version is not a scalable approach and find ways to shorten the
list, e.g. randomly select a subset of old versions at runtime, or manually
pare down the list.
>
> I propose to shorten the list [2] by keeping only the latest patch
for each minor (unless the client or server protocol version has changed, so
also keep the patch prior to 1.12.1 and prior to 1.13.2). As long as a patch
release doesn’t change the client or server protocol version, I see low value
in testing upgrades from every patch version to every future version forever.
The months between patch releases already provide plenty of upgrade coverage on
that specific patch, then we can move on to the next…even if there could
somehow be a corner-case where transitive property of upgradability doesn’t
hold, most users probably take the latest-to-latest upgrade path anyway, which
will always be tested.
>
> Let’s keep discussion open until 3PM PST Jan 5. In case of no
response, I will assume lazy consensus and update settings.gradle as proposed
[2].
>
>
>
> [1] Current list from
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode%2Fblob%2Fdevelop%2Fsettings.gradle%23L72-L101data=04%7C01%7Conichols%40vmware.com%7C951690bb3afc4c13b6ab08d9cfb75ec2%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637769208757591829%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=eZ3X5mtDxcSRIb%2FJwFmfwnIhK7zO1byNKC133qlZdDY%3Dreserved=0
:
> 1.0.0-incubating
> 1.1.0
Re: [DISCUSS] proposal to pare down old-version testing
+1 Great compromise! Let's not forget about this whenever we change the protocol
--
Please provide anonymous feedback for me
here<https://forms.gle/kpUiaty3jt8X9y4H9>.
From: Anilkumar Gingade
Sent: Tuesday, January 4, 2022 07:09
To: dev@geode.apache.org
Subject: Re: [DISCUSS] proposal to pare down old-version testing
+1
Thanks for bringing this and taking care of this.
-Anil.
On 1/3/22, 10:41 AM, "Dan Smith" wrote:
Looking at KnownVersion.java - we did make protocol changes in 1.12.1 and
1.13.2. So, my suggestion would be to keep 1.12.0 and 1.13.1, but dop all the
other patch versions that aren't the latest.
-Dan
From: Dan Smith
Sent: Monday, January 3, 2022 10:37 AM
To: dev@geode.apache.org
Subject: Re: [DISCUSS] proposal to pare down old-version testing
+1 - this seems reasonable to me. If we do make a protocol change in a
patch, we could potentially keep around an older patch version just in that
specific case, but otherwise I think this makes sense.
-Dan
From: Anthony Baker
Sent: Thursday, December 23, 2021 8:53 AM
To: dev@geode.apache.org
Subject: Re: [DISCUSS] proposal to pare down old-version testing
Interesting data point: 40% of maven central downloads last month were for
version 1.4.0. Of course those numbers can be easily skewed by CI bots, but
still!
@Owen, I think your suggestion nicely improves practicality while
continuing to support strong compatibility. In many cases it’s quite a bit
easier to upgrade the Geode server cluster compared to potentially many, many
client applications. Supporting older client versions gives users time to
upgrade, quicker access to bug fixes, and helps avoids downtime.
+1
Anthony
> On Dec 22, 2021, at 7:13 PM, Owen Nichols wrote:
>
> Since adopting our N-2 support policy, the list of released versions in
/settings.gradle has ballooned to over 30 entries [1].
>
> CI tests use this list to confirm that we don’t break rolling upgrade
ability or compatibility with older clients, but some of these tests don’t seem
to scale well: PR#7203 to add the most recent 3 releases (bringing the total to
33) is unable to pass CI after 8 tries.
>
> Possible solutions fall into two categories: keep the full list and throw
developers and/or more hardware at the struggling tests, or concede that
testing every version is not a scalable approach and find ways to shorten the
list, e.g. randomly select a subset of old versions at runtime, or manually
pare down the list.
>
> I propose to shorten the list [2] by keeping only the latest patch for
each minor (unless the client or server protocol version has changed, so also
keep the patch prior to 1.12.1 and prior to 1.13.2). As long as a patch
release doesn’t change the client or server protocol version, I see low value
in testing upgrades from every patch version to every future version forever.
The months between patch releases already provide plenty of upgrade coverage on
that specific patch, then we can move on to the next…even if there could
somehow be a corner-case where transitive property of upgradability doesn’t
hold, most users probably take the latest-to-latest upgrade path anyway, which
will always be tested.
>
> Let’s keep discussion open until 3PM PST Jan 5. In case of no response,
I will assume lazy consensus and update settings.gradle as proposed [2].
>
>
>
> [1] Current list from
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode%2Fblob%2Fdevelop%2Fsettings.gradle%23L72-L101data=04%7C01%7Camurmann%40vmware.com%7C579b50b0cacc49ed54df08d9cf9436a0%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637769057757912491%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=8%2F%2BuhyHnyI0ZPbY4Y%2F0vRG6O%2Bi3wuSf3%2BeISAvg0DR8%3Dreserved=0
:
> 1.0.0-incubating
> 1.1.0
> 1.1.1
> 1.2.0
> 1.3.0
> 1.4.0
> 1.5.0
> 1.6.0
> 1.7.0
> 1.8.0
> 1.9.0
> 1.9.1
> 1.9.2
> 1.10.0
> 1.11.0
> 1.12.0
> 1.12.1
> 1.12.2
> 1.12.3
> 1.12.4
> 1.12.5
> 1.12.6
> 1.12.7*
> 1.13.0
> 1.13.1
> 1.13.2
> 1.13.3
> 1.13.4
> 1.13.5
> 1.13.6*
> 1.14.0
> 1.14.1
> 1.14.2*
> *=released, but not yet added to settings.gradle due to PR#7203 not able
to pass CI due to size of version list
>
> [2] Proposed shortlist:
> 1.1.1
> 1.2.0
> 1.3.0
> 1.4.0
> 1.5.0
> 1.6.0
> 1.7.0
> 1.8.0
> 1.9.2
> 1.10.0
> 1.11.0
> 1.12.0
> 1.12.7
> 1.13.1
> 1.13.6
> 1.14.2
>
Re: [DISCUSS] proposal to pare down old-version testing
+1 Thanks for bringing this and taking care of this. -Anil. On 1/3/22, 10:41 AM, "Dan Smith" wrote: Looking at KnownVersion.java - we did make protocol changes in 1.12.1 and 1.13.2. So, my suggestion would be to keep 1.12.0 and 1.13.1, but dop all the other patch versions that aren't the latest. -Dan From: Dan Smith Sent: Monday, January 3, 2022 10:37 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] proposal to pare down old-version testing +1 - this seems reasonable to me. If we do make a protocol change in a patch, we could potentially keep around an older patch version just in that specific case, but otherwise I think this makes sense. -Dan From: Anthony Baker Sent: Thursday, December 23, 2021 8:53 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] proposal to pare down old-version testing Interesting data point: 40% of maven central downloads last month were for version 1.4.0. Of course those numbers can be easily skewed by CI bots, but still! @Owen, I think your suggestion nicely improves practicality while continuing to support strong compatibility. In many cases it’s quite a bit easier to upgrade the Geode server cluster compared to potentially many, many client applications. Supporting older client versions gives users time to upgrade, quicker access to bug fixes, and helps avoids downtime. +1 Anthony > On Dec 22, 2021, at 7:13 PM, Owen Nichols wrote: > > Since adopting our N-2 support policy, the list of released versions in /settings.gradle has ballooned to over 30 entries [1]. > > CI tests use this list to confirm that we don’t break rolling upgrade ability or compatibility with older clients, but some of these tests don’t seem to scale well: PR#7203 to add the most recent 3 releases (bringing the total to 33) is unable to pass CI after 8 tries. > > Possible solutions fall into two categories: keep the full list and throw developers and/or more hardware at the struggling tests, or concede that testing every version is not a scalable approach and find ways to shorten the list, e.g. randomly select a subset of old versions at runtime, or manually pare down the list. > > I propose to shorten the list [2] by keeping only the latest patch for each minor (unless the client or server protocol version has changed, so also keep the patch prior to 1.12.1 and prior to 1.13.2). As long as a patch release doesn’t change the client or server protocol version, I see low value in testing upgrades from every patch version to every future version forever. The months between patch releases already provide plenty of upgrade coverage on that specific patch, then we can move on to the next…even if there could somehow be a corner-case where transitive property of upgradability doesn’t hold, most users probably take the latest-to-latest upgrade path anyway, which will always be tested. > > Let’s keep discussion open until 3PM PST Jan 5. In case of no response, I will assume lazy consensus and update settings.gradle as proposed [2]. > > > > [1] Current list from https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode%2Fblob%2Fdevelop%2Fsettings.gradle%23L72-L101data=04%7C01%7Cagingade%40vmware.com%7C8b7e0a65ccc34f37b8a608d9cee8b189%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637768321075908672%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=Kla9OIPhvSC54LCeIQ%2FhrjOcpn7K39cVBHfAWJlbVh8%3Dreserved=0 : > 1.0.0-incubating > 1.1.0 > 1.1.1 > 1.2.0 > 1.3.0 > 1.4.0 > 1.5.0 > 1.6.0 > 1.7.0 > 1.8.0 > 1.9.0 > 1.9.1 > 1.9.2 > 1.10.0 > 1.11.0 > 1.12.0 > 1.12.1 > 1.12.2 > 1.12.3 > 1.12.4 > 1.12.5 > 1.12.6 > 1.12.7* > 1.13.0 > 1.13.1 > 1.13.2 > 1.13.3 > 1.13.4 > 1.13.5 > 1.13.6* > 1.14.0 > 1.14.1 > 1.14.2* > *=released, but not yet added to settings.gradle due to PR#7203 not able to pass CI due to size of version list > > [2] Proposed shortlist: > 1.1.1 > 1.2.0 > 1.3.0 > 1.4.0 > 1.5.0 > 1.6.0 > 1.7.0 > 1.8.0 > 1.9.2 > 1.10.0 > 1.11.0 > 1.12.0 > 1.12.7 > 1.13.1 > 1.13.6 > 1.14.2 >
Re: [DISCUSS] proposal to pare down old-version testing
Looking at KnownVersion.java - we did make protocol changes in 1.12.1 and 1.13.2. So, my suggestion would be to keep 1.12.0 and 1.13.1, but dop all the other patch versions that aren't the latest. -Dan From: Dan Smith Sent: Monday, January 3, 2022 10:37 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] proposal to pare down old-version testing +1 - this seems reasonable to me. If we do make a protocol change in a patch, we could potentially keep around an older patch version just in that specific case, but otherwise I think this makes sense. -Dan From: Anthony Baker Sent: Thursday, December 23, 2021 8:53 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] proposal to pare down old-version testing Interesting data point: 40% of maven central downloads last month were for version 1.4.0. Of course those numbers can be easily skewed by CI bots, but still! @Owen, I think your suggestion nicely improves practicality while continuing to support strong compatibility. In many cases it’s quite a bit easier to upgrade the Geode server cluster compared to potentially many, many client applications. Supporting older client versions gives users time to upgrade, quicker access to bug fixes, and helps avoids downtime. +1 Anthony > On Dec 22, 2021, at 7:13 PM, Owen Nichols wrote: > > Since adopting our N-2 support policy, the list of released versions in > /settings.gradle has ballooned to over 30 entries [1]. > > CI tests use this list to confirm that we don’t break rolling upgrade ability > or compatibility with older clients, but some of these tests don’t seem to > scale well: PR#7203 to add the most recent 3 releases (bringing the total to > 33) is unable to pass CI after 8 tries. > > Possible solutions fall into two categories: keep the full list and throw > developers and/or more hardware at the struggling tests, or concede that > testing every version is not a scalable approach and find ways to shorten the > list, e.g. randomly select a subset of old versions at runtime, or manually > pare down the list. > > I propose to shorten the list [2] by keeping only the latest patch for each > minor (unless the client or server protocol version has changed, so also keep > the patch prior to 1.12.1 and prior to 1.13.2). As long as a patch release > doesn’t change the client or server protocol version, I see low value in > testing upgrades from every patch version to every future version forever. > The months between patch releases already provide plenty of upgrade coverage > on that specific patch, then we can move on to the next…even if there could > somehow be a corner-case where transitive property of upgradability doesn’t > hold, most users probably take the latest-to-latest upgrade path anyway, > which will always be tested. > > Let’s keep discussion open until 3PM PST Jan 5. In case of no response, I > will assume lazy consensus and update settings.gradle as proposed [2]. > > > > [1] Current list from > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode%2Fblob%2Fdevelop%2Fsettings.gradle%23L72-L101data=04%7C01%7Cdasmith%40vmware.com%7Cff2c00ff1af142cd1bbd08d9cee82048%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637768318660654753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=4ykbbBS3aRaZe2HwN%2BZ3kXINab%2F%2FfyLx%2Fld5TFH%2B1zc%3Dreserved=0 > : > 1.0.0-incubating > 1.1.0 > 1.1.1 > 1.2.0 > 1.3.0 > 1.4.0 > 1.5.0 > 1.6.0 > 1.7.0 > 1.8.0 > 1.9.0 > 1.9.1 > 1.9.2 > 1.10.0 > 1.11.0 > 1.12.0 > 1.12.1 > 1.12.2 > 1.12.3 > 1.12.4 > 1.12.5 > 1.12.6 > 1.12.7* > 1.13.0 > 1.13.1 > 1.13.2 > 1.13.3 > 1.13.4 > 1.13.5 > 1.13.6* > 1.14.0 > 1.14.1 > 1.14.2* > *=released, but not yet added to settings.gradle due to PR#7203 not able to > pass CI due to size of version list > > [2] Proposed shortlist: > 1.1.1 > 1.2.0 > 1.3.0 > 1.4.0 > 1.5.0 > 1.6.0 > 1.7.0 > 1.8.0 > 1.9.2 > 1.10.0 > 1.11.0 > 1.12.0 > 1.12.7 > 1.13.1 > 1.13.6 > 1.14.2 >
Re: [DISCUSS] proposal to pare down old-version testing
+1 - this seems reasonable to me. If we do make a protocol change in a patch, we could potentially keep around an older patch version just in that specific case, but otherwise I think this makes sense. -Dan From: Anthony Baker Sent: Thursday, December 23, 2021 8:53 AM To: dev@geode.apache.org Subject: Re: [DISCUSS] proposal to pare down old-version testing Interesting data point: 40% of maven central downloads last month were for version 1.4.0. Of course those numbers can be easily skewed by CI bots, but still! @Owen, I think your suggestion nicely improves practicality while continuing to support strong compatibility. In many cases it’s quite a bit easier to upgrade the Geode server cluster compared to potentially many, many client applications. Supporting older client versions gives users time to upgrade, quicker access to bug fixes, and helps avoids downtime. +1 Anthony > On Dec 22, 2021, at 7:13 PM, Owen Nichols wrote: > > Since adopting our N-2 support policy, the list of released versions in > /settings.gradle has ballooned to over 30 entries [1]. > > CI tests use this list to confirm that we don’t break rolling upgrade ability > or compatibility with older clients, but some of these tests don’t seem to > scale well: PR#7203 to add the most recent 3 releases (bringing the total to > 33) is unable to pass CI after 8 tries. > > Possible solutions fall into two categories: keep the full list and throw > developers and/or more hardware at the struggling tests, or concede that > testing every version is not a scalable approach and find ways to shorten the > list, e.g. randomly select a subset of old versions at runtime, or manually > pare down the list. > > I propose to shorten the list [2] by keeping only the latest patch for each > minor (unless the client or server protocol version has changed, so also keep > the patch prior to 1.12.1 and prior to 1.13.2). As long as a patch release > doesn’t change the client or server protocol version, I see low value in > testing upgrades from every patch version to every future version forever. > The months between patch releases already provide plenty of upgrade coverage > on that specific patch, then we can move on to the next…even if there could > somehow be a corner-case where transitive property of upgradability doesn’t > hold, most users probably take the latest-to-latest upgrade path anyway, > which will always be tested. > > Let’s keep discussion open until 3PM PST Jan 5. In case of no response, I > will assume lazy consensus and update settings.gradle as proposed [2]. > > > > [1] Current list from > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode%2Fblob%2Fdevelop%2Fsettings.gradle%23L72-L101data=04%7C01%7Cdasmith%40vmware.com%7Ca2aa64ae94d145b7f0d008d9c634ca53%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637758752301700154%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=m09xNUntRb4V3YW8zbFTq1XCYMNnqR38%2B2goSoDlxcY%3Dreserved=0 > : > 1.0.0-incubating > 1.1.0 > 1.1.1 > 1.2.0 > 1.3.0 > 1.4.0 > 1.5.0 > 1.6.0 > 1.7.0 > 1.8.0 > 1.9.0 > 1.9.1 > 1.9.2 > 1.10.0 > 1.11.0 > 1.12.0 > 1.12.1 > 1.12.2 > 1.12.3 > 1.12.4 > 1.12.5 > 1.12.6 > 1.12.7* > 1.13.0 > 1.13.1 > 1.13.2 > 1.13.3 > 1.13.4 > 1.13.5 > 1.13.6* > 1.14.0 > 1.14.1 > 1.14.2* > *=released, but not yet added to settings.gradle due to PR#7203 not able to > pass CI due to size of version list > > [2] Proposed shortlist: > 1.1.1 > 1.2.0 > 1.3.0 > 1.4.0 > 1.5.0 > 1.6.0 > 1.7.0 > 1.8.0 > 1.9.2 > 1.10.0 > 1.11.0 > 1.12.0 > 1.12.7 > 1.13.1 > 1.13.6 > 1.14.2 >
Re: [DISCUSS] proposal to pare down old-version testing
Interesting data point: 40% of maven central downloads last month were for version 1.4.0. Of course those numbers can be easily skewed by CI bots, but still! @Owen, I think your suggestion nicely improves practicality while continuing to support strong compatibility. In many cases it’s quite a bit easier to upgrade the Geode server cluster compared to potentially many, many client applications. Supporting older client versions gives users time to upgrade, quicker access to bug fixes, and helps avoids downtime. +1 Anthony > On Dec 22, 2021, at 7:13 PM, Owen Nichols wrote: > > Since adopting our N-2 support policy, the list of released versions in > /settings.gradle has ballooned to over 30 entries [1]. > > CI tests use this list to confirm that we don’t break rolling upgrade ability > or compatibility with older clients, but some of these tests don’t seem to > scale well: PR#7203 to add the most recent 3 releases (bringing the total to > 33) is unable to pass CI after 8 tries. > > Possible solutions fall into two categories: keep the full list and throw > developers and/or more hardware at the struggling tests, or concede that > testing every version is not a scalable approach and find ways to shorten the > list, e.g. randomly select a subset of old versions at runtime, or manually > pare down the list. > > I propose to shorten the list [2] by keeping only the latest patch for each > minor (unless the client or server protocol version has changed, so also keep > the patch prior to 1.12.1 and prior to 1.13.2). As long as a patch release > doesn’t change the client or server protocol version, I see low value in > testing upgrades from every patch version to every future version forever. > The months between patch releases already provide plenty of upgrade coverage > on that specific patch, then we can move on to the next…even if there could > somehow be a corner-case where transitive property of upgradability doesn’t > hold, most users probably take the latest-to-latest upgrade path anyway, > which will always be tested. > > Let’s keep discussion open until 3PM PST Jan 5. In case of no response, I > will assume lazy consensus and update settings.gradle as proposed [2]. > > > > [1] Current list from > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode%2Fblob%2Fdevelop%2Fsettings.gradle%23L72-L101data=04%7C01%7Cbakera%40vmware.com%7C803bd50439204bf790a208d9c5c24361%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637758261648196421%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=X3SpCCjuvQo9JO01Di8HM29eKpXPzH9mr2vvpt2fSww%3Dreserved=0 > : > 1.0.0-incubating > 1.1.0 > 1.1.1 > 1.2.0 > 1.3.0 > 1.4.0 > 1.5.0 > 1.6.0 > 1.7.0 > 1.8.0 > 1.9.0 > 1.9.1 > 1.9.2 > 1.10.0 > 1.11.0 > 1.12.0 > 1.12.1 > 1.12.2 > 1.12.3 > 1.12.4 > 1.12.5 > 1.12.6 > 1.12.7* > 1.13.0 > 1.13.1 > 1.13.2 > 1.13.3 > 1.13.4 > 1.13.5 > 1.13.6* > 1.14.0 > 1.14.1 > 1.14.2* > *=released, but not yet added to settings.gradle due to PR#7203 not able to > pass CI due to size of version list > > [2] Proposed shortlist: > 1.1.1 > 1.2.0 > 1.3.0 > 1.4.0 > 1.5.0 > 1.6.0 > 1.7.0 > 1.8.0 > 1.9.2 > 1.10.0 > 1.11.0 > 1.12.0 > 1.12.7 > 1.13.1 > 1.13.6 > 1.14.2 >
Re: [DISCUSS] proposal to pare down old-version testing
Yes! This! 100%! > On Dec 22, 2021, at 7:14 PM, Owen Nichols wrote: > > Since adopting our N-2 support policy, the list of released versions in > /settings.gradle has ballooned to over 30 entries [1]. > > CI tests use this list to confirm that we don’t break rolling upgrade ability > or compatibility with older clients, but some of these tests don’t seem to > scale well: PR#7203 to add the most recent 3 releases (bringing the total to > 33) is unable to pass CI after 8 tries. > > Possible solutions fall into two categories: keep the full list and throw > developers and/or more hardware at the struggling tests, or concede that > testing every version is not a scalable approach and find ways to shorten the > list, e.g. randomly select a subset of old versions at runtime, or manually > pare down the list. > > I propose to shorten the list [2] by keeping only the latest patch for each > minor (unless the client or server protocol version has changed, so also keep > the patch prior to 1.12.1 and prior to 1.13.2). As long as a patch release > doesn’t change the client or server protocol version, I see low value in > testing upgrades from every patch version to every future version forever. > The months between patch releases already provide plenty of upgrade coverage > on that specific patch, then we can move on to the next…even if there could > somehow be a corner-case where transitive property of upgradability doesn’t > hold, most users probably take the latest-to-latest upgrade path anyway, > which will always be tested. > > Let’s keep discussion open until 3PM PST Jan 5. In case of no response, I > will assume lazy consensus and update settings.gradle as proposed [2]. > > > > [1] Current list from > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fgeode%2Fblob%2Fdevelop%2Fsettings.gradle%23L72-L101data=04%7C01%7Cjabarrett%40vmware.com%7C1699e45bec75493de2b008d9c5c243c5%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637758260417697506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=qdmkkxmOrSZrjWEzS%2Fmjx6kEqBENrBxKBfBHE7ZiA30%3Dreserved=0 > : > 1.0.0-incubating > 1.1.0 > 1.1.1 > 1.2.0 > 1.3.0 > 1.4.0 > 1.5.0 > 1.6.0 > 1.7.0 > 1.8.0 > 1.9.0 > 1.9.1 > 1.9.2 > 1.10.0 > 1.11.0 > 1.12.0 > 1.12.1 > 1.12.2 > 1.12.3 > 1.12.4 > 1.12.5 > 1.12.6 > 1.12.7* > 1.13.0 > 1.13.1 > 1.13.2 > 1.13.3 > 1.13.4 > 1.13.5 > 1.13.6* > 1.14.0 > 1.14.1 > 1.14.2* > *=released, but not yet added to settings.gradle due to PR#7203 not able to > pass CI due to size of version list > > [2] Proposed shortlist: > 1.1.1 > 1.2.0 > 1.3.0 > 1.4.0 > 1.5.0 > 1.6.0 > 1.7.0 > 1.8.0 > 1.9.2 > 1.10.0 > 1.11.0 > 1.12.0 > 1.12.7 > 1.13.1 > 1.13.6 > 1.14.2 >
[DISCUSS] proposal to pare down old-version testing
Since adopting our N-2 support policy, the list of released versions in /settings.gradle has ballooned to over 30 entries [1]. CI tests use this list to confirm that we don’t break rolling upgrade ability or compatibility with older clients, but some of these tests don’t seem to scale well: PR#7203 to add the most recent 3 releases (bringing the total to 33) is unable to pass CI after 8 tries. Possible solutions fall into two categories: keep the full list and throw developers and/or more hardware at the struggling tests, or concede that testing every version is not a scalable approach and find ways to shorten the list, e.g. randomly select a subset of old versions at runtime, or manually pare down the list. I propose to shorten the list [2] by keeping only the latest patch for each minor (unless the client or server protocol version has changed, so also keep the patch prior to 1.12.1 and prior to 1.13.2). As long as a patch release doesn’t change the client or server protocol version, I see low value in testing upgrades from every patch version to every future version forever. The months between patch releases already provide plenty of upgrade coverage on that specific patch, then we can move on to the next…even if there could somehow be a corner-case where transitive property of upgradability doesn’t hold, most users probably take the latest-to-latest upgrade path anyway, which will always be tested. Let’s keep discussion open until 3PM PST Jan 5. In case of no response, I will assume lazy consensus and update settings.gradle as proposed [2]. [1] Current list from https://github.com/apache/geode/blob/develop/settings.gradle#L72-L101 : 1.0.0-incubating 1.1.0 1.1.1 1.2.0 1.3.0 1.4.0 1.5.0 1.6.0 1.7.0 1.8.0 1.9.0 1.9.1 1.9.2 1.10.0 1.11.0 1.12.0 1.12.1 1.12.2 1.12.3 1.12.4 1.12.5 1.12.6 1.12.7* 1.13.0 1.13.1 1.13.2 1.13.3 1.13.4 1.13.5 1.13.6* 1.14.0 1.14.1 1.14.2* *=released, but not yet added to settings.gradle due to PR#7203 not able to pass CI due to size of version list [2] Proposed shortlist: 1.1.1 1.2.0 1.3.0 1.4.0 1.5.0 1.6.0 1.7.0 1.8.0 1.9.2 1.10.0 1.11.0 1.12.0 1.12.7 1.13.1 1.13.6 1.14.2
