Re: [gentoo-dev] Deprecating repoman
Ühel kenal päeval, N, 10.03.2022 kell 18:18, kirjutas Joshua Kinard: > I stick to the officially-published method of checking and committing > changes: > https://devmanual.gentoo.org/ebuild-maintenance/git/index.html > > The two tools highlighted there for the bulk of the work is repoman > and pkgdev. repoman is cited twelve times, pkgdev is cited six > times. pkgcheck is mentioned once. pkgcommit has no mentions. > > From that, one should not be faulted for assuming that repoman is the > more important tool, if not preferred tool, with pkgdev coming in > second place. > pkgcheck comes across as entirely optional and even seems equivalent > to 'repoman full', and how would one know that pkgcommit even exists? I believe the very purpose of this thread is to have a consensus/pre- announcement before actually editing the official documentation as part of the process of deprecating repoman. > That all said, most of my checks are done pre-commit, not pre-push. > It's annoying trying to unwind a commit with a mistake in git, so I > aim to not have any in the commit itself. By the time I get to the > push stage, everything MUST be perfect before I hit 'git push'. > Cause once something is public, there's no going back. In a handful > of cases, I've gone as far as nuking my local copy of the tree, re- > cloning it, and re-doing everything again just to fix a mistake that > I should have caught pre-commit. git commit --amend, interactive rebase, etc (all before push). I suggest to learn them and embrace them. Re-cloning is not fun, just for something that a quick interactive rebase or even a git reset -- hard destructive operation instead of re-clone would solve. Also the benefit of using pkgcheck is to actually be able to make the same checks that CI would do before you push, so you can amend your commits to fix issues before they hit the server and CI and break the tree. pkgcheck is so fast that it can do full tree checks in a reasonable time (repoman would take days on a radiator mips when you go outside single package), and I believe has features to have it check all your commits that haven't been pushed yet at once, checking only what it can to not be too slow to not use (so you don't need to run the check with each commit but for all of them once you commit - and if issues, again, git interactive rebase). repoman does not catch many many QA issues that pkgcheck (and therefore CI) does. > How about making an edit to repoman to have it throw a deprecation > warning for itself and to install whatever we all agree should > replace it? I would think that's a natural next step after this thread, and why this thread exists. > > Mart
[gentoo-dev] Up for grabs: dev-lang/elixir
dev-lang/elixir is up for grabs due to the proxied-maintainer being inactive. It is currently up-to-date and has no outstanding bugs.
Re: [gentoo-dev] Deprecating repoman
On Thu, 10 Mar 2022 12:07:40 -0600 William Hubbs wrote: > On Thu, Mar 10, 2022 at 09:29:59AM -0800, Matt Turner wrote: > > On Wed, Mar 9, 2022 at 11:09 PM Joonas Niilola > > wrote: > > > > > > On 9.3.2022 23.00, Matt Turner wrote: > > > > I'd like to deprecate and ultimately remove repoman. I believe > > > > that dev-util/pkgcheck and pkgcommit (from > > > > app-portage/mgorny-dev-scripts) are far superior replacements, > > > > and it makes sense to have people using the same tool and > > > > seeing the same warnings as in the CI. > > > > > > > > Are there any useful checks or behaviors of repoman that are > > > > missing from pkgcheck and pkgcommit? > > > > > > > > Thanks, > > > > Matt > > > > > > > > > > I still fail to see the "why" in here. Repoman is better than > > > pure 'git commit' that I know some people still like to use, and > > > as long as it's kept maintained. > > > > repoman is inferior to other tooling mentioned. The other tooling is > > actually run in CI. Developers should get the same warnings locally > > as in CI. Restated another way: I'm tired of telling people to stop > > using repoman or "pkgcheck would have caught that". > > I am going to nit-pick here, but pkgcheck pulls in pkgcore still. As > far asI know, pkgcore was meant to be a portage-like package manager, > but it isn't maintained. So, can we break that dependency before we > make pkgcheck the official tool for qa checks? I would rather not have > pkgcore landing on everyone's systems unless it is usable. If I am > wrong about pkgcore, please correct me and I'll be quiet, but if not > let's make pkgcheck independent from it before we deprecate repoman. > > Thanks, > > William William. pkgcheck uses pkgcore base code as it's library for accessing and evaluating repositories and ebuilds just like repoman uses portage api's to do the same. This is why pkgcheck is primarily faster than repoman. pkgcore was a re-design from the ground up of how to handle repositories and ebuilds within them, with efficiency and speed in mind. Unfortunately, pkgcore never gained enough developer support to to keep up with EAPI changes and the miriad of portage options that kept expanding. So pkgcore never ended up replacing portage as our main package manager. This is one reason why Brian eventually gave up on it and subsequently left gentoo. While I put a lot of effort into the re-write of repoman so that the code is maintainable, more easily extendable. It will never be as fast as pkgcheck due to the legacy portage code it is based on. Yes, repoman could be updated to do more checks and catch everything that pkgcheck does. It can be made to do many of the checks in parrallel, but it will never be as efficient as pkgcheck due to the underlying portage code it uses. I largely stopped working on repoman/gentoo due to the near constant bitching about nearly everything I did from another (ahem) gentoo developer which shall remain un-named. My only concern about the possible deprecation of repoman, is the fact that is is based on the current official and primary package manager code used in gentoo. While pkgcheck is not. For that reason, changes made to the core code is mostly automatically carried over to repoman without need to change repoman as well. Ideally, development should be moved to pkgcore and portage be put into maintenance mode only. With pkgcore taking over as the primary package manager for gentoo. But Zac is just too damn efficient at updating/fixing portage bugs! As well as adding features and options... (too damn many to remember half of them now)
Re: [gentoo-dev] Deprecating repoman
On 3/11/2022 13:25, Alec Warner wrote: [snip] > > The new workflow with pkgcheck was announced at the end of 2019: > https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck > > It's been 2 years, I think we can bring everyone into the fold here. I've searched my -dev archives for part of that URL, and the only hits I am getting is this e-mail thread. Was this URL previously shared on this mailing list or another? I do not track the Gentoo Blogs, so unless something is shared to the mailing lists, I will likely miss it. That said, I will admit I am uncomfortable with post-commit, pre-push validation. I get that git is vastly different, and vastly superior, to CVS. Get it right the first time, and you don't have to worry about fixing it later -- CVS teaches you that very well, and it still works well for git workflows. Going back into git post-commit to fix things is still something I need to learn more about, as my git-fu is still pretty amateurish beyond the common basics. Especially when dealing with kernel patch maintenance and maintaining lots of small, discrete changes that kernel upstream prefers. -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org rsa6144/5C63F4E3F5C6C943 2015-04-27 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic
Re: [gentoo-dev] Deprecating repoman
On Fri, Mar 11, 2022 at 9:14 AM Peter Stuge wrote: > > Matt Turner wrote: > > repoman is inferior to other tooling mentioned. The other tooling is > > actually run in CI. > > The problem seems to be that CI is running something other than > developers run, not the other way around. > > > > Developers should get the same warnings locally as in CI. > > I agree. And developers and their tools should not have to bend to > whatever CI does, I think the other way around makes more sense. > > > CI doesn't use repoman because of performance issues. I think the problem is that making committers use the CI tool is technically a fairly straightforward change; while everything you discuss further in the thread is not; because it implies people will work on improving tools that have shown little or no progress on improving in the 15 years I've been in Gentoo. > > What if pkgcore evolves to provide a portage-compatible API? No API is planned and no one is working on it. > > Then CI could use repoman without performance problems, and > developers would also enjoy improved performance, without spending > time on replacing tooling which already works for them. The benefit of getting people to change their behavior is that the benefits (less breakage, better tooling) are achieved now; as opposed to in some hypothetical future where repoman is improved. Note that we have been waiting for 'improved repoman' for about 8 years; so..I'm not willing to bet on it when we have better tooling that works today and the primary complaint is that...what exactly? The new workflow with pkgcheck was announced at the end of 2019: https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck It's been 2 years, I think we can bring everyone into the fold here. > > Looking into the future then maybe portage could even come to use > pkgcore for the low-level things that pkgcore does, then even users > could enjoy improved performance. Many things could happen but again, if you talk to people who work on pkgcore; there is no concrete plan for this. So I don't see why we are betting on it happening. If you read radhermit's blog: https://pkgcraft.github.io/posts/timesink-chronicles/ you can get one take on the project and why it struggled. -A > > > Right? > > //Peter >
Re: [gentoo-dev] Deprecating repoman
> On Thu, 10 Mar 2022, Alec Warner wrote: > I'm not sure a news item is strictly necessary, we might just p.mask > repoman with a link to the guide that Matt will need to write about > how repoman is being replaced. We should distinguish between deprecating the repoman(-only) workflow and deprecating the repoman tool. Doing the former doesn't imply masking and removing the repoman package, at least not while it's working and being maintained as part of Portage. Also, what would be the harm in having another QA tool available in addition to pkgcheck? (Long time ago, we used to have three of them. Does anyone remember qualudis? :) Ulrich signature.asc Description: PGP signature
Re: [gentoo-dev] Deprecating repoman
On 3/11/2022 03:54, Mart Raudsepp wrote:> Ühel kenal päeval, N, 10.03.2022 kell 18:18, kirjutas Joshua Kinard: >> I stick to the officially-published method of checking and committing >> changes: >> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html >> >> The two tools highlighted there for the bulk of the work is repoman >> and pkgdev. repoman is cited twelve times, pkgdev is cited six times. >> pkgcheck is mentioned once. pkgcommit has no mentions. >> >> From that, one should not be faulted for assuming that repoman is the >> more important tool, if not preferred tool, with pkgdev coming in >> second place. pkgcheck comes across as entirely optional and even >> seems equivalent to 'repoman full', and how would one know that >> pkgcommit even exists? > > I believe the very purpose of this thread is to have a consensus/pre- > announcement before actually editing the official documentation as part > of the process of deprecating repoman. I feel that the documentation should have had more mentions of these newer tools as their adoption by other developers accelerated. Documentation doesn't have to have a fixed point in time when it fully changes over. It can change organically, like almost everything else in the project. > >> That all said, most of my checks are done pre-commit, not pre-push. >> It's annoying trying to unwind a commit with a mistake in git, so I >> aim to not have any in the commit itself. By the time I get to the >> push stage, everything MUST be perfect before I hit 'git push'. Cause >> once something is public, there's no going back. In a handful of >> cases, I've gone as far as nuking my local copy of the tree, re- >> cloning it, and re-doing everything again just to fix a mistake that I >> should have caught pre-commit. > > git commit --amend, interactive rebase, etc (all before push). I suggest > to learn them and embrace them. Re-cloning is not fun, just for > something that a quick interactive rebase or even a git reset -- hard > destructive operation instead of re-clone would solve. The few times I resorted to a re-clone, the damage done was beyond recoverability of --amend or even rebase. I don't really remember the specifics, as the last time I did a full re-clone was maybe 14-17 months ago. My developer tree lives on my NAS now with ZFS snapshotting, so recovery is a lot easier, even if I have a really old snapshot. > Also the benefit of using pkgcheck is to actually be able to make the > same checks that CI would do before you push, so you can amend your > commits to fix issues before they hit the server and CI and break the > tree. pkgcheck is so fast that it can do full tree checks in a > reasonable time (repoman would take days on a radiator mips when you go > outside single package), and I believe has features to have it check all > your commits that haven't been pushed yet at once, checking only what it > can to not be too slow to not use (so you don't need to run the check > with each commit but for all of them once you commit - and if issues, > again, git interactive rebase). Speed is really not a big issue for me. I run repoman from my amd64 dev box, and it's like, maybe 10-13 seconds at most during 'repoman full'? And my MIPS systems, while not the slowest of slow of that arch, they do teach you patience over the years. The other bits you mention about pkgcheck do sound useful, though. But I am a stickler for official documentation, because my risk aversion level when committing to a public repo that can affect hundreds of thousands of users is *extremely* high. When I first signed up as a dev and we had the gentoo-x86 CVS repo, there were no CI checks. It was the responsibility of the dev committing to get it right the first time, or else. The fact I haven't blown the tree up in years, even using archaic workflows, should be proof enough that what I do does work, even if it is sub-optimal in the eyes of others. If we do deprecate repoman, that's fine. I'll learn the new tooling. My initial beef was the use of subjective opinion in proposing the initial change in the first place. Even if it is blatantly obvious to many that A > B, these mailing lists serve as a public archive of our work, so when proposing key changes, regardless of how many people know/don't know about something, full justification needs to be stated clearly and openly. -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org rsa6144/5C63F4E3F5C6C943 2015-04-27 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic
Re: [gentoo-dev] Deprecating repoman
On 11/03/2022 21.51, Joshua Kinard wrote: > On 3/11/2022 13:25, Alec Warner wrote: > > [snip] > >> >> The new workflow with pkgcheck was announced at the end of 2019: >> https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck >> >> It's been 2 years, I think we can bring everyone into the fold here. > > I've searched my -dev archives for part of that URL, and the only hits I am > getting is this e-mail thread. Was this URL previously shared on this > mailing list or another? I do not track the Gentoo Blogs, so unless > something is shared to the mailing lists, I will likely miss it. > > That said, I will admit I am uncomfortable with post-commit, pre-push > validation. I get that git is vastly different, and vastly superior, to > CVS. Get it right the first time, and you don't have to worry about fixing > it later -- CVS teaches you that very well, and it still works well for git > workflows. Going back into git post-commit to fix things is still something > I need to learn more about, as my git-fu is still pretty amateurish beyond > the common basics. Especially when dealing with kernel patch maintenance > and maintaining lots of small, discrete changes that kernel upstream prefers. > Just to note, when using pkgdev, I use mainly the command `pkgdev commit -as`, which stands for auto add files in current working pkg and run scan *before commit* - which is identical to `repoman full -dx`. I think I'm going to add configuration mode to pkgdev, so you can set scan mode to on by default (you can pass `--scan false` to disable it). -- Arthur Zamarin arthur...@gentoo.org Gentoo Linux developer (Python, GURU) OpenPGP_signature Description: OpenPGP digital signature
[gentoo-dev] Up for grabs: net-fs/s3fs
net-fs/s3fs is up grabs due to the proxied-maintainer being inactive. It is currently up-to-date and has no outstanding bugs.
Re: [gentoo-dev] Deprecating repoman
On Fri, 11 Mar 2022 10:25:19 -0800 Alec Warner wrote: > On Fri, Mar 11, 2022 at 9:14 AM Peter Stuge wrote: > > > > Matt Turner wrote: > > > repoman is inferior to other tooling mentioned. The other tooling > > > is actually run in CI. > > > > The problem seems to be that CI is running something other than > > developers run, not the other way around. > > > > > > > Developers should get the same warnings locally as in CI. > > > > I agree. And developers and their tools should not have to bend to > > whatever CI does, I think the other way around makes more sense. > > > > > > CI doesn't use repoman because of performance issues. > > I think the problem is that making committers use the CI tool is > technically a fairly straightforward change; while everything you > discuss further in the thread is not; because it implies people will > work on improving tools that have shown little or no progress on > improving in the 15 years I've been in Gentoo. > Somewhat incorrect here. I did the majority of the re-write to make the code maintainable, extensible not that long ago. It even has the ability to be repository configurable and include the ability for custom repository plugin checks to be added and run. The CI system was being developed at the same time using pkgcheck. See my other reply to WilliamH for more detail about that and fundamental speed differences. > > > > What if pkgcore evolves to provide a portage-compatible API? > > No API is planned and no one is working on it. > I and a few others did some work to ensure pkgcore had some usable api's from early in it's development. But pkgcore is so different from portage code, it was difficult for early to mediocre python devs to wrap their head around. There was even work to make porthole be able to use pkgcore as it's backend package manager, but it was never fully utilized due to more work needed on pkgcore for some functionality. > > > > Then CI could use repoman without performance problems, and > > developers would also enjoy improved performance, without spending > > time on replacing tooling which already works for them. > No, pkgcore and it's QA tool pkgcheck are designed to work together. Repoman is designed to work with portage base code. Checks can be designed to work on either, but not both. The missing checks that the CI does can be added to repoman, but the primary dev(s) creating them on the CI don't recreate them for repoman. They want to work on pkgcheck. Repoman code will NEVER be as fast as pkgcheck due to the legacy portage code structure it relies on. Pkgcore was a re-design from the ground up to to improve upon the shortcomings of the original portage structure. This resulted in vast speed improvements and reduction in lines of code to do the same functionality. Portage (due to Zac) has made vast improvements in structure, but still requires a ton more changes to get to where it should be. > The benefit of getting people to change their behavior is that the > benefits (less breakage, better tooling) are achieved now; as opposed > to in some hypothetical future where repoman is improved. > Note that we have been waiting for 'improved repoman' for about 8 > years; so..I'm not willing to bet on it when we have better tooling > that works today and the primary complaint is that...what exactly? > > The new workflow with pkgcheck was announced at the end of 2019: > https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck > > It's been 2 years, I think we can bring everyone into the fold here. > > > > > Looking into the future then maybe portage could even come to use > > pkgcore for the low-level things that pkgcore does, then even users > > could enjoy improved performance. > > Many things could happen but again, if you talk to people who work on > pkgcore; there is no concrete plan for this. So I don't see why we are > betting on it happening. > > If you read radhermit's blog: > https://pkgcraft.github.io/posts/timesink-chronicles/ you can get one > take on the project and why it struggled. > > -A > > > > > > > Right? > > > > //Peter > > >
[gentoo-dev] Last rites: dev-db/mysql-cluster
# Sam James (2022-03-11) # Significant number of open bugs including unaddressed security issues; # out of date; stuck on deprecated (and soon-to-stop-working) cmake-utils # eclass. See bug #834113. Removal on 2022-04-11. dev-db/mysql-cluster signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] Deprecating repoman
Matt Turner wrote: > repoman is inferior to other tooling mentioned. The other tooling is > actually run in CI. The problem seems to be that CI is running something other than developers run, not the other way around. > Developers should get the same warnings locally as in CI. I agree. And developers and their tools should not have to bend to whatever CI does, I think the other way around makes more sense. CI doesn't use repoman because of performance issues. What if pkgcore evolves to provide a portage-compatible API? Then CI could use repoman without performance problems, and developers would also enjoy improved performance, without spending time on replacing tooling which already works for them. Looking into the future then maybe portage could even come to use pkgcore for the low-level things that pkgcore does, then even users could enjoy improved performance. Right? //Peter
Re: [gentoo-dev] Deprecating repoman
On 3/11/2022 20:57, Sam James wrote: > > >> On 10 Mar 2022, at 23:18, Joshua Kinard wrote: >> >> On 3/10/2022 14:58, Alec Warner wrote: >>> On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard wrote: On 3/9/2022 16:00, Matt Turner wrote: > I'd like to deprecate and ultimately remove repoman. I believe that > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts) > are far superior replacements, and it makes sense to have people using > the same tool and seeing the same warnings as in the CI. > > Are there any useful checks or behaviors of repoman that are missing > from pkgcheck and pkgcommit? > > Thanks, > Matt repoman has been //the// goto tool for checking in a change since before I was a developer in 2003. It does everything we need, in one simple tool. Your proposal looks to replace repoman's functionality (and snark) with two or more packages, including tools from a package named after a fellow developer. Additionally, "I believe that are far superior replacements" is an entirely subjective opinion and, frankly, completely invalid as justification to replace a tool that has worked fine for the last 20+ years. What is so fundamentally broken about repoman that cannot be fixed such that it needs total replacement by multiple independent tools? Please document. the pros and cons here so that we can all make an informed decision. > > Matt could've given more details about why pkgcheck is superior but I think > this is actually showing/exposing the problem again: we've assumed that > everybody should really know repoman lacks a significant number of the checks > pkgcheck has, as well as being much slower. > > Those are the two reasons why it's superior. But again, those are subjective observations. Maybe it's my longer experience with the project, or maybe it's because I usually refrain from poking the more complex bits in the tree, or maybe it's the particular niche I've stuck to, the extra checks that pkgcheck runs haven't really applied to me. If I do too many significant changes to an ebuild, I'll re-read its source a half-dozen times or more to make ***sure*** that I didn't miss something. I will grep the tree for similar bits of code to make sure I'm doing something reasonably sane, I will test that ebuild on at least two different arches (amd64 and mips), etc. Maybe I over think it, or maybe I have some form of hyperpedanticism. Or maybe I've just been really lucky. :: shrugs :: And speed, again, is really quite far down on my list of concerns most of the time. That said, yes, I agree with you wholeheartedly that there was a failure at the Project/Distro level to explain the benefits of using pkgcheck over repoman scan/full. That's a communications failure, and it is really symbolic of a larger issue that projects like ours often struggle with. Each of us tends to operate off in their own little fiefdom, something I am just as guilty of as anyone else, and we don't always know what other devs are doing or how they're doing it. I wish I had suggestions to offer up at the moment on fixing this, but, alas... >>> >>> So here is the more basic argument, you can agree or disagree. >>> >>> *The goal I want is for people to commit to the tree and not break things.* >> >> This goal I agree with, and I am rather pedantic about. If not mildly >> fearful of. We've all been there at least once in our dev-lives. It's >> almost a rite of passage, if you will, to break the tree with a dumb commit >> at least once. Goal is to never repeat that mistake again. >> > > Right. I spend a fair amount of time fixing issues with repoman doesn't > find but pkgcheck / CI does, or filing bugs for them. > >> >>> If we could accomplish this with no tooling at all, that would be >>> great. Sadly humans are fallible and so we have tools to check their >>> work. Currently this tooling has two parts: >>> - pre-push tooling that you run prior to pushing. >>> - post-push CI tooling that informs you when you break the tree. >>> >>> So holding to our goal of not breaking the tree, it's better for >>> developers to run the same QA tool in pre-push that CI is using, >>> because our metric for 'breaking the tree' is the CI tool and if the >>> CI tool passes locally, there is a strong likelihood it will not break >>> in CI either. Note this argument is generic, I'm not even saying which >>> tools are in use, or which tools are better, or anything like that. >>> >>> Here we see Matt discussing a workflow he finds frustrating. >>> - A developer does a push. >>> - Their push breaks CI. >>> - He inquires about their workflow. >>> - He learns they did not run the CI tool in their pre-push workflow. >>> - He tells them they should run the CI tool in their pre-push workflow. >>> - This happens many times, causing this thread. >>> >>> The point of the thread then is to convince people
Re: [gentoo-dev] Deprecating repoman
Il giorno mer 9 mar 2022 alle ore 22:01 Matt Turner ha scritto: > I'd like to deprecate and ultimately remove repoman. I believe that > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts) > Hi using `repoman manifest` in scripts here, what would be the correct replacement for that?
[gentoo-dev] Last rites: dev-libs/processor-trace
# Sam James (2022-03-11) # No maintainer, out of date, stuck on deprecated # (and soon-to-stop-working) cmake-utils. bug #774480. # Removal on 2022-04-11. dev-libs/processor-trace signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] Deprecating repoman
On Fri, Mar 11, 2022 at 4:43 PM Francesco Riosa wrote: > > Il giorno mer 9 mar 2022 alle ore 22:01 Matt Turner ha > scritto: >> >> I'd like to deprecate and ultimately remove repoman. I believe that >> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts) > > > Hi using `repoman manifest` in scripts here, what would be the correct > replacement for that? pkgdev manifest
Re: [gentoo-dev] Deprecating repoman
> On 10 Mar 2022, at 23:18, Joshua Kinard wrote: > > On 3/10/2022 14:58, Alec Warner wrote: >> On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard wrote: >>> >>> On 3/9/2022 16:00, Matt Turner wrote: I'd like to deprecate and ultimately remove repoman. I believe that dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts) are far superior replacements, and it makes sense to have people using the same tool and seeing the same warnings as in the CI. Are there any useful checks or behaviors of repoman that are missing from pkgcheck and pkgcommit? Thanks, Matt >>> >>> repoman has been //the// goto tool for checking in a change since before I >>> was a developer in 2003. It does everything we need, in one simple tool. >>> Your proposal looks to replace repoman's functionality (and snark) with two >>> or more packages, including tools from a package named after a fellow >>> developer. >>> >>> Additionally, "I believe that are far superior replacements" is an >>> entirely subjective opinion and, frankly, completely invalid as >>> justification to replace a tool that has worked fine for the last 20+ years. >>> What is so fundamentally broken about repoman that cannot be fixed such >>> that it needs total replacement by multiple independent tools? Please >>> document. the pros and cons here so that we can all make an informed >>> decision. Matt could've given more details about why pkgcheck is superior but I think this is actually showing/exposing the problem again: we've assumed that everybody should really know repoman lacks a significant number of the checks pkgcheck has, as well as being much slower. Those are the two reasons why it's superior. >> >> So here is the more basic argument, you can agree or disagree. >> >> *The goal I want is for people to commit to the tree and not break things.* > > This goal I agree with, and I am rather pedantic about. If not mildly > fearful of. We've all been there at least once in our dev-lives. It's > almost a rite of passage, if you will, to break the tree with a dumb commit > at least once. Goal is to never repeat that mistake again. > Right. I spend a fair amount of time fixing issues with repoman doesn't find but pkgcheck / CI does, or filing bugs for them. > >> If we could accomplish this with no tooling at all, that would be >> great. Sadly humans are fallible and so we have tools to check their >> work. Currently this tooling has two parts: >> - pre-push tooling that you run prior to pushing. >> - post-push CI tooling that informs you when you break the tree. >> >> So holding to our goal of not breaking the tree, it's better for >> developers to run the same QA tool in pre-push that CI is using, >> because our metric for 'breaking the tree' is the CI tool and if the >> CI tool passes locally, there is a strong likelihood it will not break >> in CI either. Note this argument is generic, I'm not even saying which >> tools are in use, or which tools are better, or anything like that. >> >> Here we see Matt discussing a workflow he finds frustrating. >> - A developer does a push. >> - Their push breaks CI. >> - He inquires about their workflow. >> - He learns they did not run the CI tool in their pre-push workflow. >> - He tells them they should run the CI tool in their pre-push workflow. >> - This happens many times, causing this thread. >> >> The point of the thread then is to convince people to run the CI tool >> in pre-push, as a matter of policy, to reduce tree breakage and reduce >> the occurrence of the above conversation. > > I stick to the officially-published method of checking and committing changes: > https://devmanual.gentoo.org/ebuild-maintenance/git/index.html > > The two tools highlighted there for the bulk of the work is repoman and > pkgdev. repoman is cited twelve times, pkgdev is cited six times. pkgcheck > is mentioned once. pkgcommit has no mentions. pkgcommit is just a wrapper around git and pkgcheck, so it is there. It's not like you aren't allowed to make your own wrappers or aliases or scripts, right? >> > > Back in mid-2002, it was exactly Gentoo's snarkness that tipped the scales > for me. I'd just given up on FreeBSD-4.x on a sparc64 machine (old Sun > Blade 100) and LFS turned out to not be a lot of fun, but Gentoo worked fine > on it. All of the colors on the terminal gave it zing and pop, and made it > rather fun to work with. rpm and apt-get were dull; emerge was cheeky and > playful! Still is to this day. > I have no objection to (and in fact would rather welcome) contributions to make other tools more "Gentoo-like". Could you make a PR or provide some patch to add some more personality to them? signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] Deprecating repoman
On 3/11/2022 20:54, Sam James wrote: > > >> On 11 Mar 2022, at 19:39, Joshua Kinard wrote: >> >> On 3/11/2022 03:54, Mart Raudsepp wrote:> Ühel kenal päeval, N, 10.03.2022 >> kell 18:18, kirjutas Joshua Kinard: I stick to the officially-published method of checking and committing changes: https://devmanual.gentoo.org/ebuild-maintenance/git/index.html The two tools highlighted there for the bulk of the work is repoman and pkgdev. repoman is cited twelve times, pkgdev is cited six times. pkgcheck is mentioned once. pkgcommit has no mentions. From that, one should not be faulted for assuming that repoman is the more important tool, if not preferred tool, with pkgdev coming in second place. pkgcheck comes across as entirely optional and even seems equivalent to 'repoman full', and how would one know that pkgcommit even exists? >>> >>> I believe the very purpose of this thread is to have a consensus/pre- >>> announcement before actually editing the official documentation as part >>> of the process of deprecating repoman. >> >> I feel that the documentation should have had more mentions of these newer >> tools as their adoption by other developers accelerated. Documentation >> doesn't have to have a fixed point in time when it fully changes over. It >> can change organically, like almost everything else in the project. > > Well, I've done that. I've been adding pkgcheck and pkgdev to the devmanual > over time, and to the wiki. Yes, I'd seen that, though at least in the ebuild maintenance section on git, pkgdev's mentions were added in such a way that it was implied to be comparable to repoman. Not better than repoman, as many people here are stating. People like me, when we see that, are going to conduct an internal risk assessment. Use what I know works and get results I know I will get, or use a completely new tool that I know nothing about, and maybe get the same result with no perceived benefit or gain. >> [snip] > >>> Also the benefit of using pkgcheck is to actually be able to make the >>> same checks that CI would do before you push, so you can amend your >>> commits to fix issues before they hit the server and CI and break the >>> tree. pkgcheck is so fast that it can do full tree checks in a >>> reasonable time (repoman would take days on a radiator mips when you go >>> outside single package), and I believe has features to have it check all >>> your commits that haven't been pushed yet at once, checking only what it >>> can to not be too slow to not use (so you don't need to run the check >>> with each commit but for all of them once you commit - and if issues, >>> again, git interactive rebase). >> >> Speed is really not a big issue for me. I run repoman from my amd64 dev >> box, and it's like, maybe 10-13 seconds at most during 'repoman full'? And >> my MIPS systems, while not the slowest of slow of that arch, they do teach >> you patience over the years. >> >> The other bits you mention about pkgcheck do sound useful, though. But I am >> a stickler for official documentation, because my risk aversion level when >> committing to a public repo that can affect hundreds of thousands of users >> is *extremely* high. When I first signed up as a dev and we had the > > It is already mentioned in the devmanual, but we can add it in more places > if you specify which. Matt's posted a pull request in GH that looks to eliminate all remaining references of repoman and replace them with pkgdev. So I think we're done with this point now once that request is merged in. -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org rsa6144/5C63F4E3F5C6C943 2015-04-27 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic
[gentoo-dev] Re: Deprecating repoman
I've filed a PR against devmanual.git to remove references to repoman and replace them with references to pkgdev where appropriate. Most everywhere already had pkgdev/pkgcheck text in place so there wasn't much to do. See: https://github.com/gentoo/devmanual/pull/274
Re: [gentoo-dev] Deprecating repoman
> On 11 Mar 2022, at 19:51, Joshua Kinard wrote: > > On 3/11/2022 13:25, Alec Warner wrote: > > [snip] > >> >> The new workflow with pkgcheck was announced at the end of 2019: >> https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck >> >> It's been 2 years, I think we can bring everyone into the fold here. > > I've searched my -dev archives for part of that URL, and the only hits I am > getting is this e-mail thread. Was this URL previously shared on this > mailing list or another? I do not track the Gentoo Blogs, so unless > something is shared to the mailing lists, I will likely miss it. I think you may be latching on a bit to the pkgcommit thing. Nobody is making you use mgorny's scripts. pkgdev has been advertised on this very ML: https://marc.info/?l=gentoo-dev=161443741531808=2 > > That said, I will admit I am uncomfortable with post-commit, pre-push > validation. I get that git is vastly different, and vastly superior, to > CVS. Get it right the first time, and you don't have to worry about fixing > it later -- CVS teaches you that very well, and it still works well for git > workflows. Going back into git post-commit to fix things is still something > I need to learn more about, as my git-fu is still pretty amateurish beyond > the common basics. Especially when dealing with kernel patch maintenance > and maintaining lots of small, discrete changes that kernel upstream prefers. You can always do 'pkgcheck scan' before committing, or, I think 'pkgdev commit -A' might work. signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] Deprecating repoman
> On 11 Mar 2022, at 17:14, Peter Stuge wrote: > [snip] > Looking into the future then maybe portage could even come to use > pkgcore for the low-level things that pkgcore does, then even users > could enjoy improved performance. > Are you volunteering to work on this substantial task? signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] Deprecating repoman
> On 11 Mar 2022, at 19:39, Joshua Kinard wrote: > > On 3/11/2022 03:54, Mart Raudsepp wrote:> Ühel kenal päeval, N, 10.03.2022 > kell 18:18, kirjutas Joshua Kinard: >>> I stick to the officially-published method of checking and committing >>> changes: >>> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html >>> >>> The two tools highlighted there for the bulk of the work is repoman >>> and pkgdev. repoman is cited twelve times, pkgdev is cited six times. >>> pkgcheck is mentioned once. pkgcommit has no mentions. >>> >>> From that, one should not be faulted for assuming that repoman is the >>> more important tool, if not preferred tool, with pkgdev coming in >>> second place. pkgcheck comes across as entirely optional and even >>> seems equivalent to 'repoman full', and how would one know that >>> pkgcommit even exists? >> >> I believe the very purpose of this thread is to have a consensus/pre- >> announcement before actually editing the official documentation as part >> of the process of deprecating repoman. > > I feel that the documentation should have had more mentions of these newer > tools as their adoption by other developers accelerated. Documentation > doesn't have to have a fixed point in time when it fully changes over. It > can change organically, like almost everything else in the project. Well, I've done that. I've been adding pkgcheck and pkgdev to the devmanual over time, and to the wiki. > [snip] >> Also the benefit of using pkgcheck is to actually be able to make the >> same checks that CI would do before you push, so you can amend your >> commits to fix issues before they hit the server and CI and break the >> tree. pkgcheck is so fast that it can do full tree checks in a >> reasonable time (repoman would take days on a radiator mips when you go >> outside single package), and I believe has features to have it check all >> your commits that haven't been pushed yet at once, checking only what it >> can to not be too slow to not use (so you don't need to run the check >> with each commit but for all of them once you commit - and if issues, >> again, git interactive rebase). > > Speed is really not a big issue for me. I run repoman from my amd64 dev > box, and it's like, maybe 10-13 seconds at most during 'repoman full'? And > my MIPS systems, while not the slowest of slow of that arch, they do teach > you patience over the years. > > The other bits you mention about pkgcheck do sound useful, though. But I am > a stickler for official documentation, because my risk aversion level when > committing to a public repo that can affect hundreds of thousands of users > is *extremely* high. When I first signed up as a dev and we had the It is already mentioned in the devmanual, but we can add it in more places if you specify which. signature.asc Description: Message signed with OpenPGP
