Re: [DISCUSS] Removing erlang 19 support
Hey Russell, have no fear. I'm happy to say your 2014 essay is no longer an issue :) We've been shipping our couchdb binaries with Erlang Solutions' pre-built Erlang for a very long time now, at least since 2.1.0 and possibly since 2.0.0 released. When they don't provide it, we build using kerl. Whatever version of Erlang comes with the OS doesn't matter at all. In fact, we ship the exact same version of Erlang across every OS/arch we support today. The only limitation is whether or not the specific version of Erlang will *compile* on that specific OS/arch combo with the features we need - this usually comes down to the version of openssl shipped by the distro being compatible with Erlang's ssl module. There was a hiccup in the transition fom 0.9.8 to 1.0.x, and again from 1.0.x to 1.1.x, but that's all well in the past now. Cheers, Joan "DRY" Touzet On 2021-01-25 3:40 p.m., Russell Branca wrote: I'm also +1 to removing Erlang 19. I wanted to reiterate what Newson said about Erlang Solutions providing Erlang packaging, and I think we should more strongly lean on options like this rather than being dependent on the OS distros Erlang versions. Many years ago I wrote about the nuances with Erlang versions and CouchDB on the R14/R15/R16 lines [1], and it's worth noting that multiple Debian versions shipped versions of Erlang that were fundamentally broken for use with CouchDB. I think we even blocked a number of those versions in the rebar.config allowed versions. I don't know how much of an issue that is today, but there's certainly precedent for distro shipped Erlang versions not being an option. -Russell [1] https://chewbranca.com/doc/trunk/archive/github/2014-05-07-on-the-viability-of-erlang-releases-and-couchdb.md On Mon, Jan 25, 2021 at 5:07 AM Bessenyei Balázs Donát wrote: Thank you all for the input! I'll remove erlang 19 for `couchdb-config` and we'll be able to refer to this thread when we have to remove it anywhere else. Donat On Sat, Jan 23, 2021 at 10:09 PM Adam Kocoloski wrote: Ah, good research there Joan. +1 to Donat’s suggestion to drop support for 19 from me. Adam On Jan 22, 2021, at 4:49 PM, Joan Touzet wrote: On 2021-01-22 4:37 p.m., Robert Newson wrote: Iteresting. I’m actually surprised at the inversion here (that CouchDB is dependent on IBM to confirm CouchDB’s stability). I’ve always agonised over even the perception that IBM/Cloudant is calling the shots. I appreciate the reassurance that running at scale provides, of course, I just don’t think it can be our official position. It's a tough one. I was pretty aggressive on CouchDB running the very latest until the scheduler collapse problem surfaced. After that, there was another problem (I can't recall) that was pretty serious too. I took a wait-and-see attitude at that point, and after I didn't see IBM move forward to a newer release, didn't move forward ourselves. Looks like we ended up in deadlock! However! See your own comments on this: https://github.com/apache/couchdb/issues/3115#issuecomment-729031967 I knew there was something at the back of my head on this. Guess we're both getting old ;) On the core point of the thread, it seems there’s no barrier to dropping Erlang 19 support, so I think we can go to a VOTE thread, perhaps best to wait till Monday for others to chime in on this discussion though. More important is that we already committed changes on the main repo re: Erlang 19 about 14 months ago by Paul: https://github.com/apache/couchdb/commit/3594f2f1fc16903c1c383ebaf205d31c9c17fb3a I think that makes Donat's request pretty straightforward. I also think that IBM Cloudant’s chosen Erlang release is in part influenced by CouchDB’s lack of support for later versions and requirement of compatible with older releases, which now appears illusory. If we're ready to move to 21 or 22 as a default, we're ready. Let's hope the serious issues in 21/22 are at least mitigated. I'm happy to make the 3.3 release (or whatever is next) use the very latest version of 21 or 22 from GitHub, subject to community recommendations and encouragement. 23 is still a WIP: https://github.com/apache/couchdb/issues/3115 -Jon B. On 22 Jan 2021, at 21:19, Joan Touzet wrote: On 22/01/2021 15:48, Robert Newson wrote: I’m +1 on dropping Erlang 19 support. Erlang is now on major release 23. No problem here. I’d further advocate a general policy of supporting only the most recent 2 or 3 major releases of Erlang/OTP. The main (I think only?) reason to keep compatibility so far back is because of the versions supported by some OS’es. I don’t feel that is a strong reason given the couchdb install, in common with Erlang-based projects, is self-contained. The existence of Erlang Solutions packages for all common platforms is also a factor. That hasn't been the case for at least 2 years, if not longer. As the release engineer, I've been focused on stability for
Re: [PROPOSAL] Archiving git branches
A belated +1 to archiving. Thanks Joan, and also thanks for keeping the remotes/origin/ioq-per-shard-or-user branch. -Russell On Wed, Oct 21, 2020 at 12:27 PM Joan Touzet wrote: > Hah! > > OK, it's all done. At present I cannot remove any of the #.#.# branches > as Infra has protected them. (I may or may not bother to open a ticket > on this.) > > Here's the remaining branch list: > > remotes/origin/1.3.x > remotes/origin/1.4.x > remotes/origin/1.5.x > remotes/origin/1.6.x > remotes/origin/1.x.x > remotes/origin/1278-add-clustered-db-info > remotes/origin/2.0.x > remotes/origin/2.1.x > remotes/origin/2.3.x > remotes/origin/2493-remove-auth-cache > remotes/origin/3.0.x > remotes/origin/3.x > remotes/origin/HEAD -> origin/main > remotes/origin/access > remotes/origin/bump-ibrowse > remotes/origin/feat/access-3.x > remotes/origin/feat/access-master-clean > remotes/origin/feat/add-same-site-secure/master > remotes/origin/ioq-per-shard-or-user > remotes/origin/main > remotes/origin/master > remotes/origin/prototype/fdb-layer-db-version-as-vstamps > remotes/origin/re-enable-most-elixir-tests > remotes/origin/record_last_compaction > remotes/origin/smoosh-update-operator-guide > remotes/origin/update-fauxton-1.2.6 > > That's about 6x shorter. Phew! > > Remember, if you are trying to recover a branch that is now gone, just: > > git checkout -b archive/ > > -Joan > > On 21/10/2020 14:59, Jan Lehnardt wrote: > > *makes chain saw noises* > > > > (trimming branches, get it?) > > > > Thanks Joan! > > > > Best > > Jan > > — > > > >> On 21. Oct 2020, at 20:23, Joan Touzet wrote: > >> > >> I am starting the work now. As there was no response, I'm going to only > >> keep these branches: > >> > >> 2.3.x > >> 3.x > >> main > >> master (with a README saying you're in the wrong place) > >> > >> plus these PR branches: > >> > >> bump-ibrowse (#3208) > >> smoosh-update-operator-guide (#3184) > >> re-enable-most-elixir-tests (#3175) > >> feat/add-same-site-secure/master (#3131) > >> feat/access-master-clean (#3038) > >> prototype/fdb-layer-db-version-as-vstamps (#2952) > >> feat/access-3.x (#2943) > >> ioq-per-shard-or-user (#1998) > >> 1278-add-clustered-db-info (#1443) > >> record_last_compaction (#1272) > >> > >> Any of the above that were targeted to prototype/fdb-layer or master > >> have been re-targeted to main (except a couple clustering-specific ones > >> which were set to 3.x). Please check if this is correct for your PRs. > >> > >> -Joan "clean all the things?" Touzet > >> > >> On 14/10/2020 13:19, Joan Touzet wrote: > >>> A reminder about this: I intend to start this work tomorrow. If you > have > >>> any PRs or branches you want left alone, speak now. > >>> > >>> Based on feedback I've received, it sounds like the prototype/fdb-* > >>> branches are now done? If this is **NOT** the case, speak up. > >>> > >>> -Joan > >>> > >>> On 07/10/2020 18:10, Joan Touzet wrote: > Hi there, > > I'd like to clean up our branches in git on the main couchdb repo. > This > would involve deleting some of our obsolete branches, after tagging > the > final revision on each branch. This way, we retain the history but the > branch no longer appears in the dropdown on GitHub, or in git branch > listings at the cli. > > Example process: > > git tag archive/1.3.x 1.3.x > git branch -d 1.3.x > git push origin :1.3.x > git push --tags > > If we ever needed the branch back, we just: > > git checkout -b 1.3.x archive/1.3.x > > I would propose to do this for all branches except: > > main > master (for now) > 2.3.x > 3.x > prototype/fdb-layer > > ...plus any branches that have been touched in the past 90 days, that > still have open PRs, or that someone specifically asks me to retain in > this thread. > > I'd also like to do this on couchdb-documentation and couchdb-fauxton. > > I would propose to do this about 1 week from now, let's say on October > 15th. > > Thoughts? > > -Joan "fall cleaning" Touzet > > >
Re: [ANNOUNCE] Bessenyei Balázs Donát elected as CouchDB committer
Congrats Donat! Welcome aboard! -Russell On Fri, Jan 15, 2021 at 10:17 AM Alessio 'Blaster' Biancalana < dottorblas...@apache.org> wrote: > Congratulations and welcome! > > Alessio > > On Fri, Jan 15, 2021 at 4:40 PM Dave Cottlehuber > wrote: > > > > > On 14. Jan 2021, at 21:48, Robert Newson wrote: > > > > > > > > Dear community, > > > > > > > > I am pleased to announce that the CouchDB Project Management > Committee > > has elected Bessenyei Balázs Donát as a CouchDB committer. > > > > > > > > Apache ID: bessbd > > > > > > > > Committers are given a binding vote in certain project decisions, as > > well as write access to public project infrastructure. > > > > > > > > This election was made in recognition of Donát's commitment to the > > project. We mean this in the sense of being loyal to the project and its > > interests. > > > > > > > > Please join me in extending a warm welcome to Donát! > > > > > > > > On behalf of the CouchDB PMC, > > > > > > > > Robert Newson > > > > Awesome! welcome Donát! > > > > A+ > > Dave > > >
Re: [DISCUSS] Removing erlang 19 support
I'm also +1 to removing Erlang 19. I wanted to reiterate what Newson said about Erlang Solutions providing Erlang packaging, and I think we should more strongly lean on options like this rather than being dependent on the OS distros Erlang versions. Many years ago I wrote about the nuances with Erlang versions and CouchDB on the R14/R15/R16 lines [1], and it's worth noting that multiple Debian versions shipped versions of Erlang that were fundamentally broken for use with CouchDB. I think we even blocked a number of those versions in the rebar.config allowed versions. I don't know how much of an issue that is today, but there's certainly precedent for distro shipped Erlang versions not being an option. -Russell [1] https://chewbranca.com/doc/trunk/archive/github/2014-05-07-on-the-viability-of-erlang-releases-and-couchdb.md On Mon, Jan 25, 2021 at 5:07 AM Bessenyei Balázs Donát wrote: > Thank you all for the input! > > I'll remove erlang 19 for `couchdb-config` and we'll be able to refer > to this thread when we have to remove it anywhere else. > > > Donat > > On Sat, Jan 23, 2021 at 10:09 PM Adam Kocoloski > wrote: > > > > Ah, good research there Joan. +1 to Donat’s suggestion to drop support > for 19 from me. > > > > Adam > > > > > On Jan 22, 2021, at 4:49 PM, Joan Touzet wrote: > > > > > > On 2021-01-22 4:37 p.m., Robert Newson wrote: > > >> Iteresting. I’m actually surprised at the inversion here (that > CouchDB is dependent on IBM to confirm CouchDB’s stability). I’ve always > agonised over even the perception that IBM/Cloudant is calling the shots. I > appreciate the reassurance that running at scale provides, of course, I > just don’t think it can be our official position. > > > > > > It's a tough one. I was pretty aggressive on CouchDB running the very > latest until the scheduler collapse problem surfaced. After that, there was > another problem (I can't recall) that was pretty serious too. I took a > wait-and-see attitude at that point, and after I didn't see IBM move > forward to a newer release, didn't move forward ourselves. Looks like we > ended up in deadlock! > > > > > > However! See your own comments on this: > > > > > > https://github.com/apache/couchdb/issues/3115#issuecomment-729031967 > > > > > > I knew there was something at the back of my head on this. Guess we're > both getting old ;) > > > > > >> On the core point of the thread, it seems there’s no barrier to > dropping Erlang 19 support, so I think we can go to a VOTE thread, perhaps > best to wait till Monday for others to chime in on this discussion though. > > > > > > More important is that we already committed changes on the main repo > re: Erlang 19 about 14 months ago by Paul: > > > > > > > https://github.com/apache/couchdb/commit/3594f2f1fc16903c1c383ebaf205d31c9c17fb3a > > > > > > I think that makes Donat's request pretty straightforward. > > > > > >> I also think that IBM Cloudant’s chosen Erlang release is in part > influenced by CouchDB’s lack of support for later versions and requirement > of compatible with older releases, which now appears illusory. > > > > > > If we're ready to move to 21 or 22 as a default, we're ready. Let's > hope the serious issues in 21/22 are at least mitigated. I'm happy to make > the 3.3 release (or whatever is next) use the very latest version of 21 or > 22 from GitHub, subject to community recommendations and encouragement. 23 > is still a WIP: https://github.com/apache/couchdb/issues/3115 > > > > > > -Jon > > > > > >> B. > > >>> On 22 Jan 2021, at 21:19, Joan Touzet wrote: > > >>> > > >>> On 22/01/2021 15:48, Robert Newson wrote: > > I’m +1 on dropping Erlang 19 support. Erlang is now on major > release 23. > > >>> > > >>> No problem here. > > >>> > > I’d further advocate a general policy of supporting only the most > recent 2 or 3 major releases of Erlang/OTP. > > > > The main (I think only?) reason to keep compatibility so far back > is because of the versions supported by some OS’es. I don’t feel that is a > strong reason given the couchdb install, in common with Erlang-based > projects, is self-contained. The existence of Erlang Solutions packages for > all common platforms is also a factor. > > >>> > > >>> That hasn't been the case for at least 2 years, if not longer. > > >>> > > >>> As the release engineer, I've been focused on stability for everyone. > > >>> This is largely driven by knowing that IBM/Cloudant largely run > CouchDB > > >>> on 20.x at scale. Standing on the shoulders of giants, our releases > run > > >>> the latest 20.x release at the time of binary generation. > > >>> > > >>> A few times recently issues cropped up in 21 and 22 that we didn't > > >>> encounter in our user base because, at scale, we are deployed on > > >>> 20.3.8.something. Some of these issues were non-trivial. (I'm off > today, > > >>> so I don't have the time to dig into the specifics until Monday.) > > >>> > > >>> So my $0.02 is that: if IBM/Cloudant is
Re: [DISCUSS] Removing erlang 19 support
Thank you all for the input! I'll remove erlang 19 for `couchdb-config` and we'll be able to refer to this thread when we have to remove it anywhere else. Donat On Sat, Jan 23, 2021 at 10:09 PM Adam Kocoloski wrote: > > Ah, good research there Joan. +1 to Donat’s suggestion to drop support for 19 > from me. > > Adam > > > On Jan 22, 2021, at 4:49 PM, Joan Touzet wrote: > > > > On 2021-01-22 4:37 p.m., Robert Newson wrote: > >> Iteresting. I’m actually surprised at the inversion here (that CouchDB > >> is dependent on IBM to confirm CouchDB’s stability). I’ve always agonised > >> over even the perception that IBM/Cloudant is calling the shots. I > >> appreciate the reassurance that running at scale provides, of course, I > >> just don’t think it can be our official position. > > > > It's a tough one. I was pretty aggressive on CouchDB running the very > > latest until the scheduler collapse problem surfaced. After that, there was > > another problem (I can't recall) that was pretty serious too. I took a > > wait-and-see attitude at that point, and after I didn't see IBM move > > forward to a newer release, didn't move forward ourselves. Looks like we > > ended up in deadlock! > > > > However! See your own comments on this: > > > > https://github.com/apache/couchdb/issues/3115#issuecomment-729031967 > > > > I knew there was something at the back of my head on this. Guess we're both > > getting old ;) > > > >> On the core point of the thread, it seems there’s no barrier to dropping > >> Erlang 19 support, so I think we can go to a VOTE thread, perhaps best to > >> wait till Monday for others to chime in on this discussion though. > > > > More important is that we already committed changes on the main repo re: > > Erlang 19 about 14 months ago by Paul: > > > > https://github.com/apache/couchdb/commit/3594f2f1fc16903c1c383ebaf205d31c9c17fb3a > > > > I think that makes Donat's request pretty straightforward. > > > >> I also think that IBM Cloudant’s chosen Erlang release is in part > >> influenced by CouchDB’s lack of support for later versions and requirement > >> of compatible with older releases, which now appears illusory. > > > > If we're ready to move to 21 or 22 as a default, we're ready. Let's hope > > the serious issues in 21/22 are at least mitigated. I'm happy to make the > > 3.3 release (or whatever is next) use the very latest version of 21 or 22 > > from GitHub, subject to community recommendations and encouragement. 23 is > > still a WIP: https://github.com/apache/couchdb/issues/3115 > > > > -Jon > > > >> B. > >>> On 22 Jan 2021, at 21:19, Joan Touzet wrote: > >>> > >>> On 22/01/2021 15:48, Robert Newson wrote: > I’m +1 on dropping Erlang 19 support. Erlang is now on major release 23. > >>> > >>> No problem here. > >>> > I’d further advocate a general policy of supporting only the most recent > 2 or 3 major releases of Erlang/OTP. > > The main (I think only?) reason to keep compatibility so far back is > because of the versions supported by some OS’es. I don’t feel that is a > strong reason given the couchdb install, in common with Erlang-based > projects, is self-contained. The existence of Erlang Solutions packages > for all common platforms is also a factor. > >>> > >>> That hasn't been the case for at least 2 years, if not longer. > >>> > >>> As the release engineer, I've been focused on stability for everyone. > >>> This is largely driven by knowing that IBM/Cloudant largely run CouchDB > >>> on 20.x at scale. Standing on the shoulders of giants, our releases run > >>> the latest 20.x release at the time of binary generation. > >>> > >>> A few times recently issues cropped up in 21 and 22 that we didn't > >>> encounter in our user base because, at scale, we are deployed on > >>> 20.3.8.something. Some of these issues were non-trivial. (I'm off today, > >>> so I don't have the time to dig into the specifics until Monday.) > >>> > >>> So my $0.02 is that: if IBM/Cloudant is ready to move to something newer > >>> at scale, I'm ready to release binaries on a newer Erlang by default. > >>> > >>> The alternative (running newer Erlangs in the binary distributions than > >>> IBM/Cloudant run in production) could possibly be perceived as treating > >>> our open source customers as guinea pigs. I'd rather not risk that > >>> perception, but am willing to be convinced otherwise. > >>> > >>> -Joan > >>> > > B. > > > On 22 Jan 2021, at 19:54, Bessenyei Balázs Donát > > wrote: > > > > Hi All, > > > > CI for https://github.com/apache/couchdb-config appears to be broken. > > I wanted to fix it in > > https://github.com/apache/couchdb-config/pull/34/files , but I'm > > getting issues with erlang 19. Are we okay with dropping 19 support > > there? > > > > On a different note: are we okay with dropping erlang 19 support > > overall in couch project(s)? > > >
