Re: [gentoo-dev] Deprecating repoman

2022-03-19 Thread Michał Górny
On Sat, 2022-03-19 at 08:15 +0100, Ulrich Mueller wrote:
> > > > > > On Sat, 19 Mar 2022, Zoltan Puskas wrote:
> 
> [Please don't top-post.]
> 
> > I've been using both repoman _and_ pkgcheck becasue I was not sure
> > which
> > one covers all the checks I need to make. In fact [Pull Requests
> > wiki](https://wiki.gentoo.org/wiki/Project:GitHub/Pull_requests) has
> > a
> > big red warning box with the following message, and I quote:
> 
> > "CI is not an excuse not to run repoman full, at least for the
> > packages
> > being committed."
> 
> That warning box also says: "Furthermore, pkgcheck is imperfect (yet
> very fast) and may fail to notice some subtle breakages (especially if
> USE flag masking/forcing) is involved."
> 
> Looks like the warning was added by mgorny in 2016 [1]. Have these
> issues with pkgcheck been fixed in the meantime?

Yes, I think all the blockers were fixed since.

-- 
Best regards,
Michał Górny




Re: [gentoo-dev] Deprecating repoman

2022-03-19 Thread Ulrich Mueller
> On Sat, 19 Mar 2022, Zoltan Puskas wrote:

[Please don't top-post.]

> I've been using both repoman _and_ pkgcheck becasue I was not sure which
> one covers all the checks I need to make. In fact [Pull Requests
> wiki](https://wiki.gentoo.org/wiki/Project:GitHub/Pull_requests) has a
> big red warning box with the following message, and I quote:

> "CI is not an excuse not to run repoman full, at least for the packages
> being committed."

That warning box also says: "Furthermore, pkgcheck is imperfect (yet
very fast) and may fail to notice some subtle breakages (especially if
USE flag masking/forcing) is involved."

Looks like the warning was added by mgorny in 2016 [1]. Have these
issues with pkgcheck been fixed in the meantime?

Ulrich

[1] 
https://wiki.gentoo.org/index.php?title=Project%3AGitHub%2FPull_requests=revision=575482=575480


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating repoman

2022-03-12 Thread Mart Raudsepp
Ühel kenal päeval, R, 11.03.2022 kell 21:53, kirjutas Joshua Kinard:
> the extra checks that pkgcheck runs haven't really applied to me.

Looks to be mostly true, as you maintain only a few packages, but you
might find these links useful to filter the CI output per maintainer
regardless:

https://qa-reports.gentoo.org/output/gentoo-ci/output.html;maintainer=kumba
https://qa-reports.gentoo.org/output/gentoo-ci/output.html;maintainer=kumba;include-projects

Or some of these:

https://packages.gentoo.org/maintainer/ku...@gentoo.org/stabilization
https://packages.gentoo.org/maintainer/ku...@gentoo.org/outdated


Mart



Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Joshua Kinard
On 3/11/2022 20:57, Sam James wrote:
> 
> 
>> On 10 Mar 2022, at 23:18, Joshua Kinard  wrote:
>>
>> On 3/10/2022 14:58, Alec Warner wrote:
>>> On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard  wrote:

 On 3/9/2022 16:00, Matt Turner wrote:
> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> are far superior replacements, and it makes sense to have people using
> the same tool and seeing the same warnings as in the CI.
>
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?
>
> Thanks,
> Matt

 repoman has been //the// goto tool for checking in a change since before I
 was a developer in 2003.  It does everything we need, in one simple tool.
 Your proposal looks to replace repoman's functionality (and snark) with two
 or more packages, including tools from a package named after a fellow 
 developer.

 Additionally, "I believe that  are far superior replacements" is an
 entirely subjective opinion and, frankly, completely invalid as
 justification to replace a tool that has worked fine for the last 20+ 
 years.
 What is so fundamentally broken about repoman that cannot be fixed such
 that it needs total replacement by multiple independent tools?  Please
 document. the pros and cons here so that we can all make an informed 
 decision.
> 
> Matt could've given more details about why pkgcheck is superior but I think
> this is actually showing/exposing the problem again: we've assumed that
> everybody should really know repoman lacks a significant number of the checks
> pkgcheck has, as well as being much slower.
> 
> Those are the two reasons why it's superior.

But again, those are subjective observations.  Maybe it's my longer
experience with the project, or maybe it's because I usually refrain from
poking the more complex bits in the tree, or maybe it's the particular niche
I've stuck to, the extra checks that pkgcheck runs haven't really applied to
me.  If I do too many significant changes to an ebuild, I'll re-read its
source a half-dozen times or more to make ***sure*** that I didn't miss
something.  I will grep the tree for similar bits of code to make sure I'm
doing something reasonably sane, I will test that ebuild on at least two
different arches (amd64 and mips), etc.  Maybe I over think it, or maybe I
have some form of hyperpedanticism.  Or maybe I've just been really lucky.
:: shrugs ::

And speed, again, is really quite far down on my list of concerns most of
the time.

That said, yes, I agree with you wholeheartedly that there was a failure at
the Project/Distro level to explain the benefits of using pkgcheck over
repoman scan/full.  That's a communications failure, and it is really
symbolic of a larger issue that projects like ours often struggle with.
Each of us tends to operate off in their own little fiefdom, something I am
just as guilty of as anyone else, and we don't always know what other devs
are doing or how they're doing it.  I wish I had suggestions to offer up at
the moment on fixing this, but, alas...


>>>
>>> So here is the more basic argument, you can agree or disagree.
>>>
>>> *The goal I want is for people to commit to the tree and not break things.*
>>
>> This goal I agree with, and I am rather pedantic about.  If not mildly
>> fearful of.  We've all been there at least once in our dev-lives.  It's
>> almost a rite of passage, if you will, to break the tree with a dumb commit
>> at least once.  Goal is to never repeat that mistake again.
>>
> 
> Right. I spend a fair amount of time fixing issues with repoman doesn't
> find but pkgcheck / CI does, or filing bugs for them.
> 
>>
>>> If we could accomplish this with no tooling at all, that would be
>>> great. Sadly humans are fallible and so we have tools to check their
>>> work. Currently this tooling has two parts:
>>> - pre-push tooling that you run prior to pushing.
>>> - post-push CI tooling that informs you when you break the tree.
>>>
>>> So holding to our goal of not breaking the tree, it's better for
>>> developers to run the same QA tool in pre-push that CI is using,
>>> because our metric for 'breaking the tree' is the CI tool and if the
>>> CI tool passes locally, there is a strong likelihood it will not break
>>> in CI either. Note this argument is generic, I'm not even saying which
>>> tools are in use, or which tools are better, or anything like that.
>>>
>>> Here we see Matt discussing a workflow he finds frustrating.
>>> - A developer does a push.
>>> - Their push breaks CI.
>>> - He inquires about their workflow.
>>> - He learns they did not run the CI tool in their pre-push workflow.
>>> - He tells them they should run the CI tool in their pre-push workflow.
>>> - This happens many times, causing this thread.
>>>
>>> The point of the thread then is to convince people 

Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Joshua Kinard
On 3/11/2022 20:54, Sam James wrote:
> 
> 
>> On 11 Mar 2022, at 19:39, Joshua Kinard  wrote:
>>
>> On 3/11/2022 03:54, Mart Raudsepp wrote:> Ühel kenal päeval, N, 10.03.2022
>> kell 18:18, kirjutas Joshua Kinard:
 I stick to the officially-published method of checking and committing
 changes:
 https://devmanual.gentoo.org/ebuild-maintenance/git/index.html

 The two tools highlighted there for the bulk of the work is repoman
 and pkgdev.  repoman is cited twelve times, pkgdev is cited six times.
 pkgcheck is mentioned once.  pkgcommit has no mentions.

 From that, one should not be faulted for assuming that repoman is the
 more important tool, if not preferred tool, with pkgdev coming in
 second place. pkgcheck comes across as entirely optional and even
 seems equivalent to 'repoman full', and how would one know that
 pkgcommit even exists?
>>>
>>> I believe the very purpose of this thread is to have a consensus/pre-
>>> announcement before actually editing the official documentation as part
>>> of the process of deprecating repoman.
>>
>> I feel that the documentation should have had more mentions of these newer
>> tools as their adoption by other developers accelerated.  Documentation
>> doesn't have to have a fixed point in time when it fully changes over.  It
>> can change organically, like almost everything else in the project.
> 
> Well, I've done that. I've been adding pkgcheck and pkgdev to the devmanual
> over time, and to the wiki.

Yes, I'd seen that, though at least in the ebuild maintenance section on
git, pkgdev's mentions were added in such a way that it was implied to be
comparable to repoman.  Not better than repoman, as many people here are
stating.

People like me, when we see that, are going to conduct an internal risk
assessment.  Use what I know works and get results I know I will get, or use
a completely new tool that I know nothing about, and maybe get the same
result with no perceived benefit or gain.


>> [snip]
> 
>>> Also the benefit of using pkgcheck is to actually be able to make the
>>> same checks that CI would do before you push, so you can amend your
>>> commits to fix issues before they hit the server and CI and break the
>>> tree. pkgcheck is so fast that it can do full tree checks in a
>>> reasonable time (repoman would take days on a radiator mips when you go
>>> outside single package), and I believe has features to have it check all
>>> your commits that haven't been pushed yet at once, checking only what it
>>> can to not be too slow to not use (so you don't need to run the check
>>> with each commit but for all of them once you commit - and if issues,
>>> again, git interactive rebase).
>>
>> Speed is really not a big issue for me.  I run repoman from my amd64 dev
>> box, and it's like, maybe 10-13 seconds at most during 'repoman full'?  And
>> my MIPS systems, while not the slowest of slow of that arch, they do teach
>> you patience over the years.
>>
>> The other bits you mention about pkgcheck do sound useful, though.  But I am
>> a stickler for official documentation, because my risk aversion level when
>> committing to a public repo that can affect hundreds of thousands of users
>> is *extremely* high.  When I first signed up as a dev and we had the
> 
> It is already mentioned in the devmanual, but we can add it in more places
> if you specify which.

Matt's posted a pull request in GH that looks to eliminate all remaining
references of repoman and replace them with pkgdev.  So I think we're done
with this point now once that request is merged in.

-- 
Joshua Kinard
Gentoo/MIPS
ku...@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic



Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Sam James


> On 10 Mar 2022, at 23:18, Joshua Kinard  wrote:
> 
> On 3/10/2022 14:58, Alec Warner wrote:
>> On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard  wrote:
>>> 
>>> On 3/9/2022 16:00, Matt Turner wrote:
 I'd like to deprecate and ultimately remove repoman. I believe that
 dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
 are far superior replacements, and it makes sense to have people using
 the same tool and seeing the same warnings as in the CI.
 
 Are there any useful checks or behaviors of repoman that are missing
 from pkgcheck and pkgcommit?
 
 Thanks,
 Matt
>>> 
>>> repoman has been //the// goto tool for checking in a change since before I
>>> was a developer in 2003.  It does everything we need, in one simple tool.
>>> Your proposal looks to replace repoman's functionality (and snark) with two
>>> or more packages, including tools from a package named after a fellow 
>>> developer.
>>> 
>>> Additionally, "I believe that  are far superior replacements" is an
>>> entirely subjective opinion and, frankly, completely invalid as
>>> justification to replace a tool that has worked fine for the last 20+ years.
>>> What is so fundamentally broken about repoman that cannot be fixed such
>>> that it needs total replacement by multiple independent tools?  Please
>>> document. the pros and cons here so that we can all make an informed 
>>> decision.

Matt could've given more details about why pkgcheck is superior but I think
this is actually showing/exposing the problem again: we've assumed that
everybody should really know repoman lacks a significant number of the checks
pkgcheck has, as well as being much slower.

Those are the two reasons why it's superior.

>> 
>> So here is the more basic argument, you can agree or disagree.
>> 
>> *The goal I want is for people to commit to the tree and not break things.*
> 
> This goal I agree with, and I am rather pedantic about.  If not mildly
> fearful of.  We've all been there at least once in our dev-lives.  It's
> almost a rite of passage, if you will, to break the tree with a dumb commit
> at least once.  Goal is to never repeat that mistake again.
> 

Right. I spend a fair amount of time fixing issues with repoman doesn't
find but pkgcheck / CI does, or filing bugs for them.

> 
>> If we could accomplish this with no tooling at all, that would be
>> great. Sadly humans are fallible and so we have tools to check their
>> work. Currently this tooling has two parts:
>> - pre-push tooling that you run prior to pushing.
>> - post-push CI tooling that informs you when you break the tree.
>> 
>> So holding to our goal of not breaking the tree, it's better for
>> developers to run the same QA tool in pre-push that CI is using,
>> because our metric for 'breaking the tree' is the CI tool and if the
>> CI tool passes locally, there is a strong likelihood it will not break
>> in CI either. Note this argument is generic, I'm not even saying which
>> tools are in use, or which tools are better, or anything like that.
>> 
>> Here we see Matt discussing a workflow he finds frustrating.
>> - A developer does a push.
>> - Their push breaks CI.
>> - He inquires about their workflow.
>> - He learns they did not run the CI tool in their pre-push workflow.
>> - He tells them they should run the CI tool in their pre-push workflow.
>> - This happens many times, causing this thread.
>> 
>> The point of the thread then is to convince people to run the CI tool
>> in pre-push, as a matter of policy, to reduce tree breakage and reduce
>> the occurrence of the above conversation.
> 
> I stick to the officially-published method of checking and committing changes:
> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html
> 
> The two tools highlighted there for the bulk of the work is repoman and
> pkgdev.  repoman is cited twelve times, pkgdev is cited six times.  pkgcheck
> is mentioned once.  pkgcommit has no mentions.

pkgcommit is just a wrapper around git and pkgcheck, so it is there.

It's not like you aren't allowed to make your own wrappers or aliases or 
scripts, right?


>> 
> 
> Back in mid-2002, it was exactly Gentoo's snarkness that tipped the scales
> for me.  I'd just given up on FreeBSD-4.x on a sparc64 machine (old Sun
> Blade 100) and LFS turned out to not be a lot of fun, but Gentoo worked fine
> on it.  All of the colors on the terminal gave it zing and pop, and made it
> rather fun to work with.  rpm and apt-get were dull;  emerge was cheeky and
> playful!  Still is to this day.
> 

I have no objection to (and in fact would rather welcome) contributions
to make other tools more "Gentoo-like". Could you make a PR or provide
some patch to add some more personality to them?


signature.asc
Description: Message signed with OpenPGP


Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Sam James


> On 11 Mar 2022, at 17:14, Peter Stuge  wrote:
> [snip]
> Looking into the future then maybe portage could even come to use
> pkgcore for the low-level things that pkgcore does, then even users
> could enjoy improved performance.
> 

Are you volunteering to work on this substantial task?



signature.asc
Description: Message signed with OpenPGP


Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Sam James


> On 11 Mar 2022, at 19:39, Joshua Kinard  wrote:
> 
> On 3/11/2022 03:54, Mart Raudsepp wrote:> Ühel kenal päeval, N, 10.03.2022
> kell 18:18, kirjutas Joshua Kinard:
>>> I stick to the officially-published method of checking and committing
>>> changes:
>>> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html
>>> 
>>> The two tools highlighted there for the bulk of the work is repoman
>>> and pkgdev.  repoman is cited twelve times, pkgdev is cited six times.
>>> pkgcheck is mentioned once.  pkgcommit has no mentions.
>>> 
>>> From that, one should not be faulted for assuming that repoman is the
>>> more important tool, if not preferred tool, with pkgdev coming in
>>> second place. pkgcheck comes across as entirely optional and even
>>> seems equivalent to 'repoman full', and how would one know that
>>> pkgcommit even exists?
>> 
>> I believe the very purpose of this thread is to have a consensus/pre-
>> announcement before actually editing the official documentation as part
>> of the process of deprecating repoman.
> 
> I feel that the documentation should have had more mentions of these newer
> tools as their adoption by other developers accelerated.  Documentation
> doesn't have to have a fixed point in time when it fully changes over.  It
> can change organically, like almost everything else in the project.

Well, I've done that. I've been adding pkgcheck and pkgdev to the devmanual
over time, and to the wiki.

> [snip]

>> Also the benefit of using pkgcheck is to actually be able to make the
>> same checks that CI would do before you push, so you can amend your
>> commits to fix issues before they hit the server and CI and break the
>> tree. pkgcheck is so fast that it can do full tree checks in a
>> reasonable time (repoman would take days on a radiator mips when you go
>> outside single package), and I believe has features to have it check all
>> your commits that haven't been pushed yet at once, checking only what it
>> can to not be too slow to not use (so you don't need to run the check
>> with each commit but for all of them once you commit - and if issues,
>> again, git interactive rebase).
> 
> Speed is really not a big issue for me.  I run repoman from my amd64 dev
> box, and it's like, maybe 10-13 seconds at most during 'repoman full'?  And
> my MIPS systems, while not the slowest of slow of that arch, they do teach
> you patience over the years.
> 
> The other bits you mention about pkgcheck do sound useful, though.  But I am
> a stickler for official documentation, because my risk aversion level when
> committing to a public repo that can affect hundreds of thousands of users
> is *extremely* high.  When I first signed up as a dev and we had the

It is already mentioned in the devmanual, but we can add it in more places
if you specify which.


signature.asc
Description: Message signed with OpenPGP


Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Sam James


> On 11 Mar 2022, at 19:51, Joshua Kinard  wrote:
> 
> On 3/11/2022 13:25, Alec Warner wrote:
> 
> [snip]
> 
>> 
>> The new workflow with pkgcheck was announced at the end of 2019:
>> https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck
>> 
>> It's been 2 years, I think we can bring everyone into the fold here.
> 
> I've searched my -dev archives for part of that URL, and the only hits I am
> getting is this e-mail thread.  Was this URL previously shared on this
> mailing list or another?  I do not track the Gentoo Blogs, so unless
> something is shared to the mailing lists, I will likely miss it.

I think you may be latching on a bit to the pkgcommit thing. Nobody is making
you use mgorny's scripts. pkgdev has been advertised on this very ML:

https://marc.info/?l=gentoo-dev=161443741531808=2

> 
> That said, I will admit I am uncomfortable with post-commit, pre-push
> validation.   I get that git is vastly different, and vastly superior, to
> CVS.  Get it right the first time, and you don't have to worry about fixing
> it later -- CVS teaches you that very well, and it still works well for git
> workflows.  Going back into git post-commit to fix things is still something
> I need to learn more about, as my git-fu is still pretty amateurish beyond
> the common basics.  Especially when dealing with kernel patch maintenance
> and maintaining lots of small, discrete changes that kernel upstream prefers.

You can always do 'pkgcheck scan' before committing, or, I think
'pkgdev commit -A' might work.


signature.asc
Description: Message signed with OpenPGP


Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Matt Turner
On Fri, Mar 11, 2022 at 4:43 PM Francesco Riosa  wrote:
>
> Il giorno mer 9 mar 2022 alle ore 22:01 Matt Turner  ha 
> scritto:
>>
>> I'd like to deprecate and ultimately remove repoman. I believe that
>> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
>
>
> Hi using  `repoman manifest` in scripts here, what would be the correct 
> replacement for that?

pkgdev manifest



Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Francesco Riosa
Il giorno mer 9 mar 2022 alle ore 22:01 Matt Turner 
ha scritto:

> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
>

Hi using  `repoman manifest` in scripts here, what would be the correct
replacement for that?


Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Arthur Zamarin
On 11/03/2022 21.51, Joshua Kinard wrote:
> On 3/11/2022 13:25, Alec Warner wrote:
> 
> [snip]
> 
>>
>> The new workflow with pkgcheck was announced at the end of 2019:
>> https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck
>>
>> It's been 2 years, I think we can bring everyone into the fold here.
> 
> I've searched my -dev archives for part of that URL, and the only hits I am
> getting is this e-mail thread.  Was this URL previously shared on this
> mailing list or another?  I do not track the Gentoo Blogs, so unless
> something is shared to the mailing lists, I will likely miss it.
> 
> That said, I will admit I am uncomfortable with post-commit, pre-push
> validation.   I get that git is vastly different, and vastly superior, to
> CVS.  Get it right the first time, and you don't have to worry about fixing
> it later -- CVS teaches you that very well, and it still works well for git
> workflows.  Going back into git post-commit to fix things is still something
> I need to learn more about, as my git-fu is still pretty amateurish beyond
> the common basics.  Especially when dealing with kernel patch maintenance
> and maintaining lots of small, discrete changes that kernel upstream prefers.
> 

Just to note, when using pkgdev, I use mainly the command `pkgdev commit
-as`, which stands for auto add files in current working pkg and run
scan *before commit* - which is identical to `repoman full -dx`.

I think I'm going to add configuration mode to pkgdev, so you can set
scan mode to on by default (you can pass `--scan false` to disable it).

-- 
Arthur Zamarin
arthur...@gentoo.org
Gentoo Linux developer (Python, GURU)


OpenPGP_signature
Description: OpenPGP digital signature


Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Joshua Kinard
On 3/11/2022 13:25, Alec Warner wrote:

[snip]

> 
> The new workflow with pkgcheck was announced at the end of 2019:
> https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck
> 
> It's been 2 years, I think we can bring everyone into the fold here.

I've searched my -dev archives for part of that URL, and the only hits I am
getting is this e-mail thread.  Was this URL previously shared on this
mailing list or another?  I do not track the Gentoo Blogs, so unless
something is shared to the mailing lists, I will likely miss it.

That said, I will admit I am uncomfortable with post-commit, pre-push
validation.   I get that git is vastly different, and vastly superior, to
CVS.  Get it right the first time, and you don't have to worry about fixing
it later -- CVS teaches you that very well, and it still works well for git
workflows.  Going back into git post-commit to fix things is still something
I need to learn more about, as my git-fu is still pretty amateurish beyond
the common basics.  Especially when dealing with kernel patch maintenance
and maintaining lots of small, discrete changes that kernel upstream prefers.

-- 
Joshua Kinard
Gentoo/MIPS
ku...@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic



Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Joshua Kinard
On 3/11/2022 03:54, Mart Raudsepp wrote:> Ühel kenal päeval, N, 10.03.2022
kell 18:18, kirjutas Joshua Kinard:
>> I stick to the officially-published method of checking and committing
>> changes:
>> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html
>>
>> The two tools highlighted there for the bulk of the work is repoman
>> and pkgdev.  repoman is cited twelve times, pkgdev is cited six times.
>> pkgcheck is mentioned once.  pkgcommit has no mentions.
>>
>> From that, one should not be faulted for assuming that repoman is the
>> more important tool, if not preferred tool, with pkgdev coming in
>> second place. pkgcheck comes across as entirely optional and even
>> seems equivalent to 'repoman full', and how would one know that
>> pkgcommit even exists?
>
> I believe the very purpose of this thread is to have a consensus/pre-
> announcement before actually editing the official documentation as part
> of the process of deprecating repoman.

I feel that the documentation should have had more mentions of these newer
tools as their adoption by other developers accelerated.  Documentation
doesn't have to have a fixed point in time when it fully changes over.  It
can change organically, like almost everything else in the project.


>
>> That all said, most of my checks are done pre-commit, not pre-push.
>> It's annoying trying to unwind a commit with a mistake in git, so I
>> aim to not have any in the commit itself.  By the time I get to the
>> push stage, everything MUST be perfect before I hit 'git push'. Cause
>> once something is public, there's no going back.  In a handful of
>> cases, I've gone as far as nuking my local copy of the tree, re-
>> cloning it, and re-doing everything again just to fix a mistake that I
>> should have caught pre-commit.
>
> git commit --amend, interactive rebase, etc (all before push). I suggest
> to learn them and embrace them. Re-cloning is not fun, just for
> something that a quick interactive rebase or even a git reset -- hard
> destructive operation instead of re-clone would solve.

The few times I resorted to a re-clone, the damage done was beyond
recoverability of --amend or even rebase.  I don't really remember the
specifics, as the last time I did a full re-clone was maybe 14-17 months
ago.  My developer tree lives on my NAS now with ZFS snapshotting, so
recovery is a lot easier, even if I have a really old snapshot.


> Also the benefit of using pkgcheck is to actually be able to make the
> same checks that CI would do before you push, so you can amend your
> commits to fix issues before they hit the server and CI and break the
> tree. pkgcheck is so fast that it can do full tree checks in a
> reasonable time (repoman would take days on a radiator mips when you go
> outside single package), and I believe has features to have it check all
> your commits that haven't been pushed yet at once, checking only what it
> can to not be too slow to not use (so you don't need to run the check
> with each commit but for all of them once you commit - and if issues,
> again, git interactive rebase).

Speed is really not a big issue for me.  I run repoman from my amd64 dev
box, and it's like, maybe 10-13 seconds at most during 'repoman full'?  And
my MIPS systems, while not the slowest of slow of that arch, they do teach
you patience over the years.

The other bits you mention about pkgcheck do sound useful, though.  But I am
a stickler for official documentation, because my risk aversion level when
committing to a public repo that can affect hundreds of thousands of users
is *extremely* high.  When I first signed up as a dev and we had the
gentoo-x86 CVS repo, there were no CI checks.  It was the responsibility of
the dev committing to get it right the first time, or else.  The fact I
haven't blown the tree up in years, even using archaic workflows, should be
proof enough that what I do does work, even if it is sub-optimal in the eyes
of others.

If we do deprecate repoman, that's fine.  I'll learn the new tooling.  My
initial beef was the use of subjective opinion in proposing the initial
change in the first place.  Even if it is blatantly obvious to many that A >
B, these mailing lists serve as a public archive of our work, so when
proposing key changes, regardless of how many people know/don't know about
something, full justification needs to be stated clearly and openly.

-- 
Joshua Kinard
Gentoo/MIPS
ku...@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic




Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Ulrich Mueller
> On Thu, 10 Mar 2022, Alec Warner wrote:

> I'm not sure a news item is strictly necessary, we might just p.mask
> repoman with a link to the guide that Matt will need to write about
> how repoman is being replaced.

We should distinguish between deprecating the repoman(-only) workflow
and deprecating the repoman tool. Doing the former doesn't imply masking
and removing the repoman package, at least not while it's working and
being maintained as part of Portage.

Also, what would be the harm in having another QA tool available in
addition to pkgcheck? (Long time ago, we used to have three of them.
Does anyone remember qualudis? :)

Ulrich


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Brian Dolbec
On Fri, 11 Mar 2022 10:25:19 -0800
Alec Warner  wrote:

> On Fri, Mar 11, 2022 at 9:14 AM Peter Stuge  wrote:
> >
> > Matt Turner wrote:  
> > > repoman is inferior to other tooling mentioned. The other tooling
> > > is actually run in CI.  
> >
> > The problem seems to be that CI is running something other than
> > developers run, not the other way around.
> >
> >  
> > > Developers should get the same warnings locally as in CI.  
> >
> > I agree. And developers and their tools should not have to bend to
> > whatever CI does, I think the other way around makes more sense.
> >
> >
> > CI doesn't use repoman because of performance issues.  
> 
> I think the problem is that making committers use the CI tool is
> technically a fairly straightforward change; while everything you
> discuss further in the thread is not; because it implies people will
> work on improving tools that have shown little or no progress on
> improving in the 15 years I've been in Gentoo.
> 

Somewhat incorrect here.  I did the majority of the re-write to make
the code maintainable, extensible not that long ago.  It even has the
ability to be repository configurable and include the ability for
custom repository plugin checks to be added and run.   The CI system was
being developed at the same time using pkgcheck.

See my other reply to WilliamH for more detail about that and
fundamental speed differences.



> >
> > What if pkgcore evolves to provide a portage-compatible API?  
> 
> No API is planned and no one is working on it.
> 

I and a few others did some work to ensure pkgcore had some usable
api's from early in it's development.  But pkgcore is so different from
portage code, it was difficult for early to mediocre python devs to
wrap their head around.   There was even work to make porthole be able
to use pkgcore as it's backend package manager, but it was never fully
utilized due to more work needed on pkgcore for some functionality.


> >
> > Then CI could use repoman without performance problems, and
> > developers would also enjoy improved performance, without spending
> > time on replacing tooling which already works for them.  
> 

No, pkgcore and it's QA tool pkgcheck are designed to work together.
Repoman is designed to work with portage base code.   Checks can be
designed to work on either, but not both.   The missing checks that
the CI does can be added to repoman, but the primary dev(s) creating
them on the CI don't recreate them for repoman.  They want to work on
pkgcheck.

Repoman code will NEVER be as fast as pkgcheck due to the legacy
portage code structure it relies on.   Pkgcore was a re-design from the
ground up to to improve upon the shortcomings of the original portage
structure. This resulted in vast speed improvements and reduction in
lines of code to do the same functionality.   Portage (due to Zac) has
made vast improvements in structure, but still requires a ton more
changes to get to where it should be.


> The benefit of getting people to change their behavior is that the
> benefits (less breakage, better tooling) are achieved now; as opposed
> to in some hypothetical future where repoman is improved.
> Note that we have been waiting for 'improved repoman' for about 8
> years; so..I'm not willing to bet on it when we have better tooling
> that works today and the primary complaint is that...what exactly?
> 
> The new workflow with pkgcheck was announced at the end of 2019:
> https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck
> 
> It's been 2 years, I think we can bring everyone into the fold here.
> 
> >
> > Looking into the future then maybe portage could even come to use
> > pkgcore for the low-level things that pkgcore does, then even users
> > could enjoy improved performance.  
> 
> Many things could happen but again, if you talk to people who work on
> pkgcore; there is no concrete plan for this. So I don't see why we are
> betting on it happening.
> 
> If you read radhermit's blog:
> https://pkgcraft.github.io/posts/timesink-chronicles/ you can get one
> take on the project and why it struggled.
> 
> -A
> 
> >
> >
> > Right?
> >
> > //Peter
> >  
> 




Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Alec Warner
On Fri, Mar 11, 2022 at 9:14 AM Peter Stuge  wrote:
>
> Matt Turner wrote:
> > repoman is inferior to other tooling mentioned. The other tooling is
> > actually run in CI.
>
> The problem seems to be that CI is running something other than
> developers run, not the other way around.
>
>
> > Developers should get the same warnings locally as in CI.
>
> I agree. And developers and their tools should not have to bend to
> whatever CI does, I think the other way around makes more sense.
>
>
> CI doesn't use repoman because of performance issues.

I think the problem is that making committers use the CI tool is
technically a fairly straightforward change; while everything you
discuss further in the thread is not; because it implies people will
work on improving tools that have shown little or no progress on
improving in the 15 years I've been in Gentoo.

>
> What if pkgcore evolves to provide a portage-compatible API?

No API is planned and no one is working on it.

>
> Then CI could use repoman without performance problems, and
> developers would also enjoy improved performance, without spending
> time on replacing tooling which already works for them.

The benefit of getting people to change their behavior is that the
benefits (less breakage, better tooling) are achieved now; as opposed
to in some hypothetical future where repoman is improved.
Note that we have been waiting for 'improved repoman' for about 8
years; so..I'm not willing to bet on it when we have better tooling
that works today and the primary complaint is that...what exactly?

The new workflow with pkgcheck was announced at the end of 2019:
https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck

It's been 2 years, I think we can bring everyone into the fold here.

>
> Looking into the future then maybe portage could even come to use
> pkgcore for the low-level things that pkgcore does, then even users
> could enjoy improved performance.

Many things could happen but again, if you talk to people who work on
pkgcore; there is no concrete plan for this. So I don't see why we are
betting on it happening.

If you read radhermit's blog:
https://pkgcraft.github.io/posts/timesink-chronicles/ you can get one
take on the project and why it struggled.

-A

>
>
> Right?
>
> //Peter
>



Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Peter Stuge
Matt Turner wrote:
> repoman is inferior to other tooling mentioned. The other tooling is
> actually run in CI.

The problem seems to be that CI is running something other than
developers run, not the other way around.


> Developers should get the same warnings locally as in CI.

I agree. And developers and their tools should not have to bend to
whatever CI does, I think the other way around makes more sense.


CI doesn't use repoman because of performance issues.

What if pkgcore evolves to provide a portage-compatible API?

Then CI could use repoman without performance problems, and
developers would also enjoy improved performance, without spending
time on replacing tooling which already works for them.

Looking into the future then maybe portage could even come to use
pkgcore for the low-level things that pkgcore does, then even users
could enjoy improved performance.


Right?

//Peter



Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Brian Dolbec
On Thu, 10 Mar 2022 12:07:40 -0600
William Hubbs  wrote:

> On Thu, Mar 10, 2022 at 09:29:59AM -0800, Matt Turner wrote:
> > On Wed, Mar 9, 2022 at 11:09 PM Joonas Niilola 
> > wrote:  
> > >
> > > On 9.3.2022 23.00, Matt Turner wrote:  
> > > > I'd like to deprecate and ultimately remove repoman. I believe
> > > > that dev-util/pkgcheck and pkgcommit (from
> > > > app-portage/mgorny-dev-scripts) are far superior replacements,
> > > > and it makes sense to have people using the same tool and
> > > > seeing the same warnings as in the CI.
> > > >
> > > > Are there any useful checks or behaviors of repoman that are
> > > > missing from pkgcheck and pkgcommit?
> > > >
> > > > Thanks,
> > > > Matt
> > > >  
> > >
> > > I still fail to see the "why" in here. Repoman is better than
> > > pure 'git commit' that I know some people still like to use, and
> > > as long as it's kept maintained.  
> > 
> > repoman is inferior to other tooling mentioned. The other tooling is
> > actually run in CI. Developers should get the same warnings locally
> > as in CI. Restated another way: I'm tired of telling people to stop
> > using repoman or "pkgcheck would have caught that".  
> 
> I am going to nit-pick here, but pkgcheck pulls in pkgcore still. As
> far asI know, pkgcore was meant to be a portage-like package manager,
> but it isn't maintained. So, can we break that dependency before we
> make pkgcheck the official tool for qa checks? I would rather not have
> pkgcore landing on everyone's systems unless it is usable. If I am
> wrong about pkgcore, please correct me and I'll be quiet, but if not
> let's make pkgcheck independent from it before we deprecate repoman.
> 
> Thanks,
> 
> William

William.  pkgcheck uses pkgcore base code as it's library for accessing
and evaluating repositories and ebuilds just like repoman uses portage
api's to do the same.   This is why pkgcheck is primarily faster than
repoman.  pkgcore was a re-design from the ground up of how to handle
repositories and ebuilds within them, with efficiency and speed in mind.

Unfortunately, pkgcore never gained enough developer support to to keep
up with EAPI changes and the miriad of portage options that kept
expanding.   So pkgcore never ended up replacing portage as our main
package manager.   This is one reason why Brian eventually gave up on it
and subsequently left gentoo.


While I put a lot of effort into the re-write of repoman so that the
code is maintainable, more easily extendable. It will never be as fast
as pkgcheck due to the legacy portage code it is based on.

Yes, repoman could be updated to do more checks and catch everything
that pkgcheck does.   It can be made to do many of the checks in
parrallel, but it will never be as efficient as pkgcheck due to the
underlying portage code it uses.   I largely stopped working on
repoman/gentoo due to the near constant bitching about nearly everything
I did from another (ahem) gentoo developer which shall remain un-named.


My only concern about the possible deprecation of repoman, is the fact
that is is based on the current official and primary package manager
code used in gentoo.  While pkgcheck is not.   For that reason,
changes made to the core code is mostly automatically carried
over to repoman without need to change repoman as well.   Ideally,
development should be moved to pkgcore and portage be put into
maintenance mode only. With pkgcore taking over as the primary package
manager for gentoo. But Zac is just too damn efficient at
updating/fixing portage bugs!  As well as adding features and
options... (too damn many to remember half of them now)





Re: [gentoo-dev] Deprecating repoman

2022-03-11 Thread Mart Raudsepp
Ühel kenal päeval, N, 10.03.2022 kell 18:18, kirjutas Joshua Kinard:
> I stick to the officially-published method of checking and committing
> changes:
> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html
> 
> The two tools highlighted there for the bulk of the work is repoman
> and pkgdev.  repoman is cited twelve times, pkgdev is cited six
> times. pkgcheck is mentioned once.  pkgcommit has no mentions.
> 
> From that, one should not be faulted for assuming that repoman is the
> more important tool, if not preferred tool, with pkgdev coming in
> second place.
> pkgcheck comes across as entirely optional and even seems equivalent
> to 'repoman full', and how would one know that pkgcommit even exists?

I believe the very purpose of this thread is to have a consensus/pre-
announcement before actually editing the official documentation as part
of the process of deprecating repoman.

> That all said, most of my checks are done pre-commit, not pre-push. 
> It's annoying trying to unwind a commit with a mistake in git, so I
> aim to not have any in the commit itself.  By the time I get to the
> push stage, everything MUST be perfect before I hit 'git push'. 
> Cause once something is public, there's no going back.  In a handful
> of cases, I've gone as far as nuking my local copy of the tree, re-
> cloning it, and re-doing everything again just to fix a mistake that
> I should have caught pre-commit.

git commit --amend, interactive rebase, etc (all before push).
I suggest to learn them and embrace them. Re-cloning is not fun, just
for something that a quick interactive rebase or even a git reset --
hard destructive operation instead of re-clone would solve.

Also the benefit of using pkgcheck is to actually be able to make the
same checks that CI would do before you push, so you can amend your
commits to fix issues before they hit the server and CI and break the
tree. pkgcheck is so fast that it can do full tree checks in a
reasonable time (repoman would take days on a radiator mips when you go
outside single package), and I believe has features to have it check
all your commits that haven't been pushed yet at once, checking only
what it can to not be too slow to not use (so you don't need to run the
check with each commit but for all of them once you commit - and if
issues, again, git interactive rebase).

repoman does not catch many many QA issues that pkgcheck (and therefore
CI) does.

> How about making an edit to repoman to have it throw a deprecation
> warning for itself and to install whatever we all agree should
> replace it?

I would think that's a natural next step after this thread, and why
this thread exists.

> > 

Mart



Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread Joshua Kinard
On 3/10/2022 14:58, Alec Warner wrote:
> On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard  wrote:
>>
>> On 3/9/2022 16:00, Matt Turner wrote:
>>> I'd like to deprecate and ultimately remove repoman. I believe that
>>> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
>>> are far superior replacements, and it makes sense to have people using
>>> the same tool and seeing the same warnings as in the CI.
>>>
>>> Are there any useful checks or behaviors of repoman that are missing
>>> from pkgcheck and pkgcommit?
>>>
>>> Thanks,
>>> Matt
>>
>> repoman has been //the// goto tool for checking in a change since before I
>> was a developer in 2003.  It does everything we need, in one simple tool.
>> Your proposal looks to replace repoman's functionality (and snark) with two
>> or more packages, including tools from a package named after a fellow 
>> developer.
>>
>> Additionally, "I believe that  are far superior replacements" is an
>> entirely subjective opinion and, frankly, completely invalid as
>> justification to replace a tool that has worked fine for the last 20+ years.
>>  What is so fundamentally broken about repoman that cannot be fixed such
>> that it needs total replacement by multiple independent tools?  Please
>> document. the pros and cons here so that we can all make an informed 
>> decision.
> 
> So here is the more basic argument, you can agree or disagree.
> 
> *The goal I want is for people to commit to the tree and not break things.*

This goal I agree with, and I am rather pedantic about.  If not mildly
fearful of.  We've all been there at least once in our dev-lives.  It's
almost a rite of passage, if you will, to break the tree with a dumb commit
at least once.  Goal is to never repeat that mistake again.


> If we could accomplish this with no tooling at all, that would be
> great. Sadly humans are fallible and so we have tools to check their
> work. Currently this tooling has two parts:
>  - pre-push tooling that you run prior to pushing.
>  - post-push CI tooling that informs you when you break the tree.
> 
> So holding to our goal of not breaking the tree, it's better for
> developers to run the same QA tool in pre-push that CI is using,
> because our metric for 'breaking the tree' is the CI tool and if the
> CI tool passes locally, there is a strong likelihood it will not break
> in CI either. Note this argument is generic, I'm not even saying which
> tools are in use, or which tools are better, or anything like that.
> 
> Here we see Matt discussing a workflow he finds frustrating.
>  - A developer does a push.
>  - Their push breaks CI.
>  - He inquires about their workflow.
>  - He learns they did not run the CI tool in their pre-push workflow.
>  - He tells them they should run the CI tool in their pre-push workflow.
>  - This happens many times, causing this thread.
> 
> The point of the thread then is to convince people to run the CI tool
> in pre-push, as a matter of policy, to reduce tree breakage and reduce
> the occurrence of the above conversation.

I stick to the officially-published method of checking and committing changes:
https://devmanual.gentoo.org/ebuild-maintenance/git/index.html

The two tools highlighted there for the bulk of the work is repoman and
pkgdev.  repoman is cited twelve times, pkgdev is cited six times.  pkgcheck
is mentioned once.  pkgcommit has no mentions.

>From that, one should not be faulted for assuming that repoman is the more
important tool, if not preferred tool, with pkgdev coming in second place.
pkgcheck comes across as entirely optional and even seems equivalent to
'repoman full', and how would one know that pkgcommit even exists?

Not all of us devs are involved in the automated CI stuff that goes on
behind the scenes.  Matt and Michał are, so they know how bad it can be and
the likely countless hours spent fixing things or yelling at others when
these mistakes are made.  But if this isn't documented somewhere, how are
those of us not involved in the CI-side able to keep up?  When I am in doubt
over something, I go to the documentation to address things.  Right now,
that documentation is going to more or less tell me to stick with repoman.

That all said, most of my checks are done pre-commit, not pre-push.  It's
annoying trying to unwind a commit with a mistake in git, so I aim to not
have any in the commit itself.  By the time I get to the push stage,
everything MUST be perfect before I hit 'git push'.  Cause once something is
public, there's no going back.  In a handful of cases, I've gone as far as
nuking my local copy of the tree, re-cloning it, and re-doing everything
again just to fix a mistake that I should have caught pre-commit.


> So the generic argument above, we now get into the specifics.
> 
>>
>> I'm not opposed to making our tools better, but I think before anything can
>> replace the RepoMan, several more boxes need to be ticked:
>>
>> - functionality from multiple tools should be packaged into a 

Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread Sam James


> On 10 Mar 2022, at 21:57, Joshua Kinard  wrote:
>> 
>> I'd be interested in hearing your workflow, so we can capture it in
>> the table (mentioned earlier) so its clear how your existing workflow
>> will work with the new tools (or perhaps there is a gap, or we need to
>> craft / add additional tools?) I agree on the face it may not be
>> obvious what workflows look like.
> 
> My workflow is really rather standard when working in the tree itself.  I
> work one package directory at a time, apply changes that I've tested outside
> of the tree in my local repo, eyeball everything a second time to make sure
> I didn't miss something, regenerate the manifest, git add, run 'repoman full
> -d -x', fix any issues it finds (if any) and manifest/git add again, then
> 'repoman commit' and supply a commit message with sign-off.  Lather, rinse,
> repeat for other packages.
> 

Having the same checks applied as in CI (which affects whether changes
are deployed to users too) is important. pkgcheck has more checks than repoman.


signature.asc
Description: Message signed with OpenPGP


Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread John Helmert III
On Thu, Mar 10, 2022 at 04:53:10PM -0500, Joshua Kinard wrote:
> On 3/10/2022 14:44, Andreas K. Huettel wrote:
> >>>
> >>> I wouldn't block anyone from doing this, but it's not something I'm
> >>> personally interested in pursuing. I see very little value here.
> >>
> >> First, you're trying to justify replacing repoman on an entirely subjective
> >> opinion of "I think  is superior" ...
> > 
> > Well, if you've ever tried it you'll notice that  for  != repoman
> > actually finishes the checks within a finite amount of time. Kind of, the 
> > most blatant argument for ditching repoman, actually.
> 
> If this is a concern for some, has anyone looked into whether repoman can be
> fixed to be more efficient?  If so, how was the determination made that it
> cannot be fixed and instead, needs to be replaced?  It's been around for 20+
> years.  Surely someone has gotten annoyed enough to look at any issues it
> has and attempt to fix them?

It's slow enough that Gentoo CI [1] uses pkgcheck to be remotely
useful.

[1] https://qa-reports.gentoo.org/output/gentoo-ci/output.html

> That said, I'm not terribly bothered by it.  It is slow, don't get me wrong,
> but it's not slow enough that my workflow is significantly impacted.  It
> catches most of the mistakes I've ever made before I make them so that I can
> fix them.  For me, that's job well done.
> 
> -- 
> Joshua Kinard
> Gentoo/MIPS
> ku...@gentoo.org
> rsa6144/5C63F4E3F5C6C943 2015-04-27
> 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943
> 
> "The past tempts us, the present confuses us, the future frightens us.  And
> our lives slip away, moment by moment, lost in that vast, terrible 
> in-between."
> 
> --Emperor Turhan, Centauri Republic
> 


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread Joshua Kinard
On 3/10/2022 14:59, Alec Warner wrote:
> On Thu, Mar 10, 2022 at 10:28 AM Joshua Kinard  wrote:
>>
>> On 3/9/2022 16:47, Matt Turner wrote:
>>> On Wed, Mar 9, 2022 at 1:37 PM Matthias Maier  wrote:


 Just a quick though:

 Looking at the man page of repoman it doesn't look to difficult to
 replicate the behavior with pkgcheck. Meaning, we could think of
 creating a drop-in replacement for "repoman [full]" (which would just
 call pkgcheck) and "repoman commit" which actually does much more than
 just prepending the git commit summary line: repoman commit does

- update the manifest
- bail out if files are not correctly "git add"ed
- add the output of [pkgcheck] as a comment to the git commit
  description
>>>
>>> I wouldn't block anyone from doing this, but it's not something I'm
>>> personally interested in pursuing. I see very little value here.
>>
>> First, you're trying to justify replacing repoman on an entirely subjective
>> opinion of "I think  is superior", then when you get feedback on the
>> idea, you dismiss that feedback with more opinion.
>>
>> Why do you not see value here?  The actions described are actually quite a
>> few useful steps in the process of checking a change into the tree.  If you
>> expect developers to do those steps on their own, that increases, not
>> decreases, the chances of making a mistake.  Or are these steps already
>> handled by one of the other scripts in the replacement packages you propose?
>>  If so, please say so and identify which one(s).
>>
>> My opinion is that any tool that replaces repoman should, at a minimum,
>> replace like functionality with like functionality, plus benefits or
>> enhancements.  This looks more like a step backwards, not a step forwards.
> 
> I'd be interested in hearing your workflow, so we can capture it in
> the table (mentioned earlier) so its clear how your existing workflow
> will work with the new tools (or perhaps there is a gap, or we need to
> craft / add additional tools?) I agree on the face it may not be
> obvious what workflows look like.

My workflow is really rather standard when working in the tree itself.  I
work one package directory at a time, apply changes that I've tested outside
of the tree in my local repo, eyeball everything a second time to make sure
I didn't miss something, regenerate the manifest, git add, run 'repoman full
-d -x', fix any issues it finds (if any) and manifest/git add again, then
'repoman commit' and supply a commit message with sign-off.  Lather, rinse,
repeat for other packages.

This is more or less how the dev manual and git for developers guide says we
should be doing it.  Since git is atomic, limit commits to one package at a
time, then, when ready, push the changes in a single batch after rebasing
(to pickup other dev's changes).

If I'm doing something wrong, yell at me in e-mail, open a bug, or call me
out on the mailing list and serve me some crow pie.  That's how I learn not
to repeat it.  So far, though, this process has worked well for me for a
very long time and I've only had to make minor adjustments at key points,
like switching from CVS to git.

-- 
Joshua Kinard
Gentoo/MIPS
ku...@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic



Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread Joshua Kinard
On 3/10/2022 14:44, Andreas K. Huettel wrote:
>>>
>>> I wouldn't block anyone from doing this, but it's not something I'm
>>> personally interested in pursuing. I see very little value here.
>>
>> First, you're trying to justify replacing repoman on an entirely subjective
>> opinion of "I think  is superior" ...
> 
> Well, if you've ever tried it you'll notice that  for  != repoman
> actually finishes the checks within a finite amount of time. Kind of, the 
> most blatant argument for ditching repoman, actually.

If this is a concern for some, has anyone looked into whether repoman can be
fixed to be more efficient?  If so, how was the determination made that it
cannot be fixed and instead, needs to be replaced?  It's been around for 20+
years.  Surely someone has gotten annoyed enough to look at any issues it
has and attempt to fix them?

That said, I'm not terribly bothered by it.  It is slow, don't get me wrong,
but it's not slow enough that my workflow is significantly impacted.  It
catches most of the mistakes I've ever made before I make them so that I can
fix them.  For me, that's job well done.

-- 
Joshua Kinard
Gentoo/MIPS
ku...@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic



Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread Alec Warner
On Thu, Mar 10, 2022 at 10:28 AM Joshua Kinard  wrote:
>
> On 3/9/2022 16:47, Matt Turner wrote:
> > On Wed, Mar 9, 2022 at 1:37 PM Matthias Maier  wrote:
> >>
> >>
> >> Just a quick though:
> >>
> >> Looking at the man page of repoman it doesn't look to difficult to
> >> replicate the behavior with pkgcheck. Meaning, we could think of
> >> creating a drop-in replacement for "repoman [full]" (which would just
> >> call pkgcheck) and "repoman commit" which actually does much more than
> >> just prepending the git commit summary line: repoman commit does
> >>
> >>- update the manifest
> >>- bail out if files are not correctly "git add"ed
> >>- add the output of [pkgcheck] as a comment to the git commit
> >>  description
> >
> > I wouldn't block anyone from doing this, but it's not something I'm
> > personally interested in pursuing. I see very little value here.
>
> First, you're trying to justify replacing repoman on an entirely subjective
> opinion of "I think  is superior", then when you get feedback on the
> idea, you dismiss that feedback with more opinion.
>
> Why do you not see value here?  The actions described are actually quite a
> few useful steps in the process of checking a change into the tree.  If you
> expect developers to do those steps on their own, that increases, not
> decreases, the chances of making a mistake.  Or are these steps already
> handled by one of the other scripts in the replacement packages you propose?
>  If so, please say so and identify which one(s).
>
> My opinion is that any tool that replaces repoman should, at a minimum,
> replace like functionality with like functionality, plus benefits or
> enhancements.  This looks more like a step backwards, not a step forwards.

I'd be interested in hearing your workflow, so we can capture it in
the table (mentioned earlier) so its clear how your existing workflow
will work with the new tools (or perhaps there is a gap, or we need to
craft / add additional tools?) I agree on the face it may not be
obvious what workflows look like.

-A

>
> --
> Joshua Kinard
> Gentoo/MIPS
> ku...@gentoo.org
> rsa6144/5C63F4E3F5C6C943 2015-04-27
> 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943
>
> "The past tempts us, the present confuses us, the future frightens us.  And
> our lives slip away, moment by moment, lost in that vast, terrible 
> in-between."
>
> --Emperor Turhan, Centauri Republic
>



Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread Alec Warner
On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard  wrote:
>
> On 3/9/2022 16:00, Matt Turner wrote:
> > I'd like to deprecate and ultimately remove repoman. I believe that
> > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> > are far superior replacements, and it makes sense to have people using
> > the same tool and seeing the same warnings as in the CI.
> >
> > Are there any useful checks or behaviors of repoman that are missing
> > from pkgcheck and pkgcommit?
> >
> > Thanks,
> > Matt
>
> repoman has been //the// goto tool for checking in a change since before I
> was a developer in 2003.  It does everything we need, in one simple tool.
> Your proposal looks to replace repoman's functionality (and snark) with two
> or more packages, including tools from a package named after a fellow 
> developer.
>
> Additionally, "I believe that  are far superior replacements" is an
> entirely subjective opinion and, frankly, completely invalid as
> justification to replace a tool that has worked fine for the last 20+ years.
>  What is so fundamentally broken about repoman that cannot be fixed such
> that it needs total replacement by multiple independent tools?  Please
> document. the pros and cons here so that we can all make an informed decision.

So here is the more basic argument, you can agree or disagree.

*The goal I want is for people to commit to the tree and not break things.*

If we could accomplish this with no tooling at all, that would be
great. Sadly humans are fallible and so we have tools to check their
work. Currently this tooling has two parts:
 - pre-push tooling that you run prior to pushing.
 - post-push CI tooling that informs you when you break the tree.

So holding to our goal of not breaking the tree, it's better for
developers to run the same QA tool in pre-push that CI is using,
because our metric for 'breaking the tree' is the CI tool and if the
CI tool passes locally, there is a strong likelihood it will not break
in CI either. Note this argument is generic, I'm not even saying which
tools are in use, or which tools are better, or anything like that.

Here we see Matt discussing a workflow he finds frustrating.
 - A developer does a push.
 - Their push breaks CI.
 - He inquires about their workflow.
 - He learns they did not run the CI tool in their pre-push workflow.
 - He tells them they should run the CI tool in their pre-push workflow.
 - This happens many times, causing this thread.

The point of the thread then is to convince people to run the CI tool
in pre-push, as a matter of policy, to reduce tree breakage and reduce
the occurrence of the above conversation.

So the generic argument above, we now get into the specifics.

>
> I'm not opposed to making our tools better, but I think before anything can
> replace the RepoMan, several more boxes need to be ticked:
>
> - functionality from multiple tools should be packaged into a single tool.
>   * caveat: at least provide a wrapper that, using args, can invoke the
> individual tools if needed.

I do not believe it's reasonable to provide a 'drop-in' backwards
compatibility tool guarantee.
I believe we should provide a table of common repoman actions, with a
mapping to their replacements; so that users can understand how to do
similar actions.

>
> - app-portage/mgorny-dev-scripts needs a new name.  It's fine if it's
> intended to only be a collection of useful scripts for individual developers
> on an as-needed basis, but if it's to be the Official Tool(TM), then it
> needs to have a proper name.  If not all of the scripts contained within it
> are applicable to the sole function of checking a change into the tree, then
> only the scripts that deal with change validation and committing should be
> broken out into a separate package, and ostensibly combined with any other
> tools/scripts into a single package, and preferably a single tool, to get
> the job done.

I don't consider this a blocker, but I think it's mostly a discussion
between mattst88 and mgorny.

>
> - all of our developer documentation needs to be updated to reference the
> new tool and the new way of doing things, as well as a warning not to use
> repoman any further after a set date.  Additionally, a news item is probably
> advisable so that developers and proxy maintainers alike can get the message.

We should hold Matt accountable for updating any relevant development
documentation, including the table I mentioned above.
I'm not sure a news item is strictly necessary, we might just p.mask
repoman with a link to the guide that Matt will need to write about
how repoman is being replaced.

>
> - we need the snark.  Gentoo is all about personality, and RepoMan has been
> scouring our neighborhoods for two decades and change, and while some may
> think this is utterly frivolous, I will actually miss seeing those messages
> on the console every time I commit a change.  They give the RepoMan
> personality and a soul, and thus, contribute to the 

Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread Andreas K. Huettel
> > 
> > I wouldn't block anyone from doing this, but it's not something I'm
> > personally interested in pursuing. I see very little value here.
> 
> First, you're trying to justify replacing repoman on an entirely subjective
> opinion of "I think  is superior" ...

Well, if you've ever tried it you'll notice that  for  != repoman
actually finishes the checks within a finite amount of time. Kind of, the 
most blatant argument for ditching repoman, actually.


-- 
Andreas K. Hüttel
dilfri...@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread Joshua Kinard
On 3/9/2022 16:47, Matt Turner wrote:
> On Wed, Mar 9, 2022 at 1:37 PM Matthias Maier  wrote:
>>
>>
>> Just a quick though:
>>
>> Looking at the man page of repoman it doesn't look to difficult to
>> replicate the behavior with pkgcheck. Meaning, we could think of
>> creating a drop-in replacement for "repoman [full]" (which would just
>> call pkgcheck) and "repoman commit" which actually does much more than
>> just prepending the git commit summary line: repoman commit does
>>
>>- update the manifest
>>- bail out if files are not correctly "git add"ed
>>- add the output of [pkgcheck] as a comment to the git commit
>>  description
> 
> I wouldn't block anyone from doing this, but it's not something I'm
> personally interested in pursuing. I see very little value here.

First, you're trying to justify replacing repoman on an entirely subjective
opinion of "I think  is superior", then when you get feedback on the
idea, you dismiss that feedback with more opinion.

Why do you not see value here?  The actions described are actually quite a
few useful steps in the process of checking a change into the tree.  If you
expect developers to do those steps on their own, that increases, not
decreases, the chances of making a mistake.  Or are these steps already
handled by one of the other scripts in the replacement packages you propose?
 If so, please say so and identify which one(s).

My opinion is that any tool that replaces repoman should, at a minimum,
replace like functionality with like functionality, plus benefits or
enhancements.  This looks more like a step backwards, not a step forwards.

-- 
Joshua Kinard
Gentoo/MIPS
ku...@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic



Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread Joshua Kinard
On 3/9/2022 16:00, Matt Turner wrote:
> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> are far superior replacements, and it makes sense to have people using
> the same tool and seeing the same warnings as in the CI.
> 
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?
> 
> Thanks,
> Matt

repoman has been //the// goto tool for checking in a change since before I
was a developer in 2003.  It does everything we need, in one simple tool.
Your proposal looks to replace repoman's functionality (and snark) with two
or more packages, including tools from a package named after a fellow developer.

Additionally, "I believe that  are far superior replacements" is an
entirely subjective opinion and, frankly, completely invalid as
justification to replace a tool that has worked fine for the last 20+ years.
 What is so fundamentally broken about repoman that cannot be fixed such
that it needs total replacement by multiple independent tools?  Please
document. the pros and cons here so that we can all make an informed decision.

I'm not opposed to making our tools better, but I think before anything can
replace the RepoMan, several more boxes need to be ticked:

- functionality from multiple tools should be packaged into a single tool.
  * caveat: at least provide a wrapper that, using args, can invoke the
individual tools if needed.

- app-portage/mgorny-dev-scripts needs a new name.  It's fine if it's
intended to only be a collection of useful scripts for individual developers
on an as-needed basis, but if it's to be the Official Tool(TM), then it
needs to have a proper name.  If not all of the scripts contained within it
are applicable to the sole function of checking a change into the tree, then
only the scripts that deal with change validation and committing should be
broken out into a separate package, and ostensibly combined with any other
tools/scripts into a single package, and preferably a single tool, to get
the job done.

- all of our developer documentation needs to be updated to reference the
new tool and the new way of doing things, as well as a warning not to use
repoman any further after a set date.  Additionally, a news item is probably
advisable so that developers and proxy maintainers alike can get the message.

- we need the snark.  Gentoo is all about personality, and RepoMan has been
scouring our neighborhoods for two decades and change, and while some may
think this is utterly frivolous, I will actually miss seeing those messages
on the console every time I commit a change.  They give the RepoMan
personality and a soul, and thus, contribute to the uniqueness that is Gentoo.

-- 
Joshua Kinard
Gentoo/MIPS
ku...@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic



Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread John Helmert III
On Thu, Mar 10, 2022 at 12:07:40PM -0600, William Hubbs wrote:
> On Thu, Mar 10, 2022 at 09:29:59AM -0800, Matt Turner wrote:
> > On Wed, Mar 9, 2022 at 11:09 PM Joonas Niilola  wrote:
> > >
> > > On 9.3.2022 23.00, Matt Turner wrote:
> > > > I'd like to deprecate and ultimately remove repoman. I believe that
> > > > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> > > > are far superior replacements, and it makes sense to have people using
> > > > the same tool and seeing the same warnings as in the CI.
> > > >
> > > > Are there any useful checks or behaviors of repoman that are missing
> > > > from pkgcheck and pkgcommit?
> > > >
> > > > Thanks,
> > > > Matt
> > > >
> > >
> > > I still fail to see the "why" in here. Repoman is better than pure 'git
> > > commit' that I know some people still like to use, and as long as it's
> > > kept maintained.
> > 
> > repoman is inferior to other tooling mentioned. The other tooling is
> > actually run in CI. Developers should get the same warnings locally as
> > in CI. Restated another way: I'm tired of telling people to stop using
> > repoman or "pkgcheck would have caught that".
> 
> I am going to nit-pick here, but pkgcheck pulls in pkgcore still. As far
> asI know, pkgcore was meant to be a portage-like package manager, but it
> isn't maintained. So, can we break that dependency before we make
> pkgcheck the official tool for qa checks? I would rather not have
> pkgcore landing on everyone's systems unless it is usable. If I am wrong
> about pkgcore, please correct me and I'll be quiet, but if not let's
> make pkgcheck independent from it before we deprecate repoman.

Yes, pkgcheck pulls in pkgcore, and yes, pkgcore wants to function as
a package manager, but it doesn't conflict with Portage, so there's no
concern in pulling it in. As long as you don't call the executables it
installs (notably pmerge, maybe others), it won't cause any
problems. pkgcheck can also already be considered our official CI
tool, since it's what does our CI.

signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread William Hubbs
On Thu, Mar 10, 2022 at 09:29:59AM -0800, Matt Turner wrote:
> On Wed, Mar 9, 2022 at 11:09 PM Joonas Niilola  wrote:
> >
> > On 9.3.2022 23.00, Matt Turner wrote:
> > > I'd like to deprecate and ultimately remove repoman. I believe that
> > > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> > > are far superior replacements, and it makes sense to have people using
> > > the same tool and seeing the same warnings as in the CI.
> > >
> > > Are there any useful checks or behaviors of repoman that are missing
> > > from pkgcheck and pkgcommit?
> > >
> > > Thanks,
> > > Matt
> > >
> >
> > I still fail to see the "why" in here. Repoman is better than pure 'git
> > commit' that I know some people still like to use, and as long as it's
> > kept maintained.
> 
> repoman is inferior to other tooling mentioned. The other tooling is
> actually run in CI. Developers should get the same warnings locally as
> in CI. Restated another way: I'm tired of telling people to stop using
> repoman or "pkgcheck would have caught that".

I am going to nit-pick here, but pkgcheck pulls in pkgcore still. As far
asI know, pkgcore was meant to be a portage-like package manager, but it
isn't maintained. So, can we break that dependency before we make
pkgcheck the official tool for qa checks? I would rather not have
pkgcore landing on everyone's systems unless it is usable. If I am wrong
about pkgcore, please correct me and I'll be quiet, but if not let's
make pkgcheck independent from it before we deprecate repoman.

Thanks,

William


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating repoman

2022-03-10 Thread Matt Turner
On Wed, Mar 9, 2022 at 11:09 PM Joonas Niilola  wrote:
>
> On 9.3.2022 23.00, Matt Turner wrote:
> > I'd like to deprecate and ultimately remove repoman. I believe that
> > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> > are far superior replacements, and it makes sense to have people using
> > the same tool and seeing the same warnings as in the CI.
> >
> > Are there any useful checks or behaviors of repoman that are missing
> > from pkgcheck and pkgcommit?
> >
> > Thanks,
> > Matt
> >
>
> I still fail to see the "why" in here. Repoman is better than pure 'git
> commit' that I know some people still like to use, and as long as it's
> kept maintained.

repoman is inferior to other tooling mentioned. The other tooling is
actually run in CI. Developers should get the same warnings locally as
in CI. Restated another way: I'm tired of telling people to stop using
repoman or "pkgcheck would have caught that".

> My only worry is: are pkgcheck and pkgdev _really_ maintained anymore?
> More than repoman is?

More than repoman is, definitely.



Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Joonas Niilola
On 9.3.2022 23.00, Matt Turner wrote:
> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> are far superior replacements, and it makes sense to have people using
> the same tool and seeing the same warnings as in the CI.
> 
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?
> 
> Thanks,
> Matt
> 

I still fail to see the "why" in here. Repoman is better than pure 'git
commit' that I know some people still like to use, and as long as it's
kept maintained.
We should be teaching people about the alternatives, and let them choose
whatever they like more.
https://wiki.gentoo.org/wiki/Package_maintainer%27s_responsibilities#Ebuilds_and_git_workflow

Repoman is a very lightweight tool. All that being said, I can't think
of a single feature that pkgdev+pkgcheck don't cover when switching from
repoman.

The quick global CI checks after each commit saves us from a lot of
trouble. If you do bad commits, you get immediately noticed about it and
you can fix it rather quickly usually. When you "get hit", you also
learn about pkgcheck and we've seen that this is the point when people
usually integrate it to their workflows. I'd also like to point out
whenever doing more complicated pushs, one can trigger the automatic CI
check in our Github mirror via a pull request before pushing.

My only worry is: are pkgcheck and pkgdev _really_ maintained anymore?
More than repoman is?

-- juippis


OpenPGP_signature
Description: OpenPGP digital signature


Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Matt Turner
On Wed, Mar 9, 2022 at 4:18 PM Matthias Maier  wrote:
>
>
> On Wed, Mar  9, 2022, at 17:32 CST, Matt Turner  wrote:
>
> > I think you just made that number up :)
>
> Nah. My sample was just bad (ago just made a sizable number of commits.
>
> I would estimate the current usage of repoman to be about 20-25%, down
> from well over 80-90% back when we just switched to git:
>
>   https://tamiko.43-1.org/temp/repoman-usage.pdf

Very interesting. Thanks!



Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Matthias Maier


On Wed, Mar  9, 2022, at 17:32 CST, Matt Turner  wrote:

> I think you just made that number up :)

Nah. My sample was just bad (ago just made a sizable number of commits.

I would estimate the current usage of repoman to be about 20-25%, down
from well over 80-90% back when we just switched to git:

  https://tamiko.43-1.org/temp/repoman-usage.pdf

>> For example, dev-util/pkgdev was suggested as an alternative, but I
>> cannot assess whether this is indeed a viable alternative.
>
> Why not?

Because I haven't used it so far.

> Give it a try.

Absolutely, the faster I can ditch repoman, the better.

Best,
Matthias



Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Matt Turner
On Wed, Mar 9, 2022 at 2:34 PM Matthias Maier  wrote:
>
> On Wed, Mar  9, 2022, at 15:47 CST, Matt Turner  wrote:
>
> > [...]
> >
> > I wouldn't block anyone from doing this, but it's not something I'm
> > personally interested in pursuing. I see very little value here.
>
> I did not intend to imply that you should do anything.
>
> I just want to point out that we had been educating developers for using
> "repoman" to commit for a long time and looking at the "git log" history
> shows that repoman is still used by a sizable number of developers
> (around 50%-ish).

I think you just made that number up :)

> So, it would be very nice to have a somewhat official "blessed"
> alternative in place.
>
>
> For example, dev-util/pkgdev was suggested as an alternative, but I
> cannot assess whether this is indeed a viable alternative.

Why not? Give it a try.



Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Matthias Maier
On Wed, Mar  9, 2022, at 15:47 CST, Matt Turner  wrote:

> [...]
>
> I wouldn't block anyone from doing this, but it's not something I'm
> personally interested in pursuing. I see very little value here.

I did not intend to imply that you should do anything.

I just want to point out that we had been educating developers for using
"repoman" to commit for a long time and looking at the "git log" history
shows that repoman is still used by a sizable number of developers
(around 50%-ish).

So, it would be very nice to have a somewhat official "blessed"
alternative in place.


For example, dev-util/pkgdev was suggested as an alternative, but I
cannot assess whether this is indeed a viable alternative.


Best,
Matthias



Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Matt Turner
On Wed, Mar 9, 2022 at 1:37 PM Matthias Maier  wrote:
>
>
> Just a quick though:
>
> Looking at the man page of repoman it doesn't look to difficult to
> replicate the behavior with pkgcheck. Meaning, we could think of
> creating a drop-in replacement for "repoman [full]" (which would just
> call pkgcheck) and "repoman commit" which actually does much more than
> just prepending the git commit summary line: repoman commit does
>
>- update the manifest
>- bail out if files are not correctly "git add"ed
>- add the output of [pkgcheck] as a comment to the git commit
>  description

I wouldn't block anyone from doing this, but it's not something I'm
personally interested in pursuing. I see very little value here.



Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Matt Turner
On Wed, Mar 9, 2022 at 1:32 PM Brian Evans  wrote:
>
> On 3/9/2022 4:28 PM, Matt Turner wrote:
> > On Wed, Mar 9, 2022 at 1:27 PM Maciej Barć  wrote:
> >>
> >>> Are there any useful checks or behaviors of repoman that are missing
> >>> from pkgcheck and pkgcommit?
> >>
> >> Fixing ebuild copyright date is the first one that comes to mind.
> >
> > pkgcommit does this.
> >
> I highly doubt it as it sits in the github repo
> https://github.com/mgorny/mgorny-dev-scripts/blob/master/pkgcommit
> basically sets up $EDITOR with "cat/pkg:" with an optional message in
> front then passes to 'git commit'

Ah, sorry. copybump (called by pkgbump) is actually what updates the copyright:

https://github.com/mgorny/mgorny-dev-scripts/blob/master/copybump



Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Matthias Maier


Just a quick though:

Looking at the man page of repoman it doesn't look to difficult to
replicate the behavior with pkgcheck. Meaning, we could think of
creating a drop-in replacement for "repoman [full]" (which would just
call pkgcheck) and "repoman commit" which actually does much more than
just prepending the git commit summary line: repoman commit does

   - update the manifest
   - bail out if files are not correctly "git add"ed
   - add the output of [pkgcheck] as a comment to the git commit
 description

Best,
Matthias



Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Brian Evans

On 3/9/2022 4:28 PM, Matt Turner wrote:

On Wed, Mar 9, 2022 at 1:27 PM Maciej Barć  wrote:



Are there any useful checks or behaviors of repoman that are missing
from pkgcheck and pkgcommit?


Fixing ebuild copyright date is the first one that comes to mind.


pkgcommit does this.


I highly doubt it as it sits in the github repo
https://github.com/mgorny/mgorny-dev-scripts/blob/master/pkgcommit 
basically sets up $EDITOR with "cat/pkg:" with an optional message in 
front then passes to 'git commit'


OpenPGP_signature
Description: OpenPGP digital signature


Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Matt Turner
On Wed, Mar 9, 2022 at 1:27 PM Maciej Barć  wrote:
>
> > Are there any useful checks or behaviors of repoman that are missing
> > from pkgcheck and pkgcommit?
>
> Fixing ebuild copyright date is the first one that comes to mind.

pkgcommit does this.



Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Maciej Barć

Are there any useful checks or behaviors of repoman that are missing
from pkgcheck and pkgcommit?


Fixing ebuild copyright date is the first one that comes to mind.


On 3/9/22 10:00 PM, Matt Turner wrote:

I'd like to deprecate and ultimately remove repoman. I believe that
dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
are far superior replacements, and it makes sense to have people using
the same tool and seeing the same warnings as in the CI.

Are there any useful checks or behaviors of repoman that are missing
from pkgcheck and pkgcommit?

Thanks,
Matt



--
Have a great day!

~ Maciej XGQT Barć


OpenPGP_0x14D74A1F43A6AC3C.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature


Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Ionen Wolkens
On Wed, Mar 09, 2022 at 01:00:37PM -0800, Matt Turner wrote:
> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)

There's also dev-util/pkgdev as an alternative to pkgcommit with more
features for those that want them that will feel similar to repoman
- pkgdev commit -c/-b bugnum (repoman commit -c/-b bugnum)
- pkgdev manifest (repoman manifest)
- pkgdev push (runs pkgcheck scan --commits before pushing)
- updates copyright year / manifest on commit
- basic commit message templates, like auto "cat/pkg: add " on
  bumps without manual editing -- prepend cat/pkg: like other tools
  if nothing specific

And other handy features like pkgdev mask --rites days

> are far superior replacements, and it makes sense to have people using
> the same tool and seeing the same warnings as in the CI.
> 
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?
> 
> Thanks,
> Matt
> 

-- 
ionen


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Brian Evans

On 3/9/2022 4:00 PM, Matt Turner wrote:

I'd like to deprecate and ultimately remove repoman. I believe that
dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
are far superior replacements, and it makes sense to have people using
the same tool and seeing the same warnings as in the CI.

Are there any useful checks or behaviors of repoman that are missing
from pkgcheck and pkgcommit?


I certainly would miss 'repoman manifest'.  I bet other scripts would 
too including at least one in the mentioned app-portage/mgonry-dev-scripts.


Brian


OpenPGP_signature
Description: OpenPGP digital signature


Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Matt Turner
On Wed, Mar 9, 2022 at 1:15 PM Rich Freeman  wrote:
>
> On Wed, Mar 9, 2022 at 4:00 PM Matt Turner  wrote:
> >
> > Are there any useful checks or behaviors of repoman that are missing
> > from pkgcheck and pkgcommit?
>
> Would it make sense to package pkgcommit or otherwise stick it
> somewhere official?  I know there is a copy on mgorny's github
> repo/blog, but if this is going to become the recommended way to do
> commits we might want to stick it someplace a bit more canonical, and
> having it in a package would ensure that if there are any changes they
> get distributed.

I agree, this would be nice. (And it's already in a package,
app-portage/mgorny-dev-scripts as I mentioned)

> Obviously there are other ways to set it, but it lacks obtaining the
> gpg key to use from make.conf.  I'm not sure that is really the best
> place to put it in any case.  Just figured I'd point it out.

I think some bit of this message was lost? Looks like you're asking
about where we should set the GPG key used for signing commits. IMO
.git/config is the right place. Configuring it in make.conf was not a
good idea, and as a follow-on I'd like to remove that.



Re: [gentoo-dev] Deprecating repoman

2022-03-09 Thread Rich Freeman
On Wed, Mar 9, 2022 at 4:00 PM Matt Turner  wrote:
>
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?

Would it make sense to package pkgcommit or otherwise stick it
somewhere official?  I know there is a copy on mgorny's github
repo/blog, but if this is going to become the recommended way to do
commits we might want to stick it someplace a bit more canonical, and
having it in a package would ensure that if there are any changes they
get distributed.

Obviously there are other ways to set it, but it lacks obtaining the
gpg key to use from make.conf.  I'm not sure that is really the best
place to put it in any case.  Just figured I'd point it out.

-- 
Rich