Re: Port and patch submissions via ticket vs. via PR.

[Jan Stary]

> > Also, Mojca brought the question of the hundreds of open tickets with
> > port submissions a while back. It might be neat if we had some code
> > that caused a special Github account to generate a PR for a port
> > submission in trac. I wouldn't want this automatically invoked (at
> > least not yet!), but if a human (like me!) could manually invoke it on
> > a few ports at a time when they had free time, it might help to
> > clear out the backlog.
> 
> I'm a afraid this requires human labor that cannot be much helped
> by any automation. The gist of it is to work on the port itself,
> motivatad by need and/or love. The only way for any of
> https://trac.macports.org/query?status=new=submission
> to become an actual port is that _some_guy_ cares enough
> to _have_that_ in MP to turn it into a port, test etc,
> and prefrably create a PR to be merged. Whether this guy
> downloads the attached Portfile from a trac ticket or
> clones somebody's branch is a technicality compared to that.
> 
> For example, if anyone cared about dbxml
> https://trac.macports.org/ticket/24429
> we would have a dbxml port by now. The reason we don't
> is not that it's a trac submission, instead of a PR.

Come to think of it, automatically turning those trac port submissions
that have been rotting there for years into github PRs now
would only make it look like they are not rotting for years.

For example, https://trac.macports.org/ticket/22255
has been dead for eight years. But turn it into a PR,
and hey, there's a PR opened today!
But nothing has changed, really.

Re: Port and patch submissions via ticket vs. via PR.

[Jan Stary]

(Not a member, speaking from a user point of view.)

On Apr 01 12:02:23, pe...@piermont.com wrote:
> As some of you have noticed, I've been trying to keep the PR queue
> relatively clear.

Thank you for that! It really is rewarding
to see the PRs being acted upon in a timely manner,
as opposed to oh so many other projects.

> Part of this is because it is significantly easier to handle new ports
> and submitted patches that come in via pull requests on github than in
> trac tickets.
> 
> 1. You can comment on individual lines in a PR, which is really nice
>for code review purposes, and easily track which comments
>have been addressed.
> 2. When something arrives via PR, it is automatically put through the
>CI system, which, although flawed, provides reasonable assurance
>things build if it works.
> 3. It's easy to invoke (even with automation) appropriate people to
>look at a PR.
> 4. Merging the PR when it is ready is easy.

Also, it is easy to merge _other_ changes into the proposed PR,
including the master changes newer than the PR (rebase).

> I'm wondering if we should therefore encourage people to submit github
> PRs if they have working patches (or are submitting new ports) rather
> than trac tickets. (Not suggesting we should _prevent_ the latter,
> just that maybe we should explicitly encourage the former.)

I have switched to exclusively use github months ago.
I only deal with TRAC tickets if I have to; similarly,
I create github issues, not TRAC tickets.

> Also, Mojca brought the question of the hundreds of open tickets with
> port submissions a while back. It might be neat if we had some code
> that caused a special Github account to generate a PR for a port
> submission in trac. I wouldn't want this automatically invoked (at
> least not yet!), but if a human (like me!) could manually invoke it on
> a few ports at a time when they had free time, it might help to
> clear out the backlog.

I'm a afraid this requires human labor that cannot be much helped
by any automation. The gist of it is to work on the port itself,
motivatad by need and/or love. The only way for any of
https://trac.macports.org/query?status=new=submission
to become an actual port is that _some_guy_ cares enough
to _have_that_ in MP to turn it into a port, test etc,
and prefrably create a PR to be merged. Whether this guy
downloads the attached Portfile from a trac ticket or
clones somebody's branch is a technicality compared to that.

For example, if anyone cared about dbxml
https://trac.macports.org/ticket/24429
we would have a dbxml port by now. The reason we don't
is not that it's a trac submission, instead of a PR.

On Apr 04 06:15:48, ryandes...@macports.org wrote:
> In my experience, port submissions, especially by new contributors, require 
> more cleanup than port updates. It's not uncommon for a submission to have 
> half a dozen or more things wrong with it.

Speaking as a user, I can attest to the fact that my submissions
usually need a fair amount of cleaning before being ready to merge.

> So you could either develop an automated system for copying a Portfile 
> submitted on Trac into a pull request -- and then have to check out that PR 
> branch into your git clone, fix all the problems, amend the commit, force 
> push it -- or you could just download the Portfile from Trac yourself, put it 
> into your git clone, fix it, commit it, and either send a PR or push to 
> master. And either way, you're going to want to make sure it builds locally 
> first. It doesn't seem like the former saves you that much time over the 
> latter, plus of course you've had to spend all that time developing the 
> automated Trac-to-PR system.

Exactly. Turning the trac into a PR _before_ the actual work is done
saves nothing.

On Apr 04 14:18:42, mo...@macports.org wrote:
> The time saved comes from the author doing the cleanup based on
> feedback (which you can leave while waiting for a bus or while going
> for a walk :) vs. one of us having to do all the changes, fiddling
> with branches.

That has nothing to do with trac vs PR:
the author can get feedback on the PR too (I do).

> Alternatively / in addition, what we currently lack is a page
> explaining how random unskilled users willing to spend some time
> helping us could do. If we were able to communicate clearly what such
> users can do when they just feel like helping us, this could be one of
> the suitable tasks.

That is imho a good way to get involved with MP.
The motivation has to come from within though:
nobody's gonna fix ports they neither use nor care about as a first step.

For example, I like audio. If there is a a problem with an audio port
that does something interesting, I will look at it. I will not do the
same for graphics, because I simply don't care. The random unskilled
user will need _some_ motivation beside "I feel like helping".

> checksums after github switch their algorithm etc.)

That's a good example of what I mean: what could 

Re: Port and patch submissions via ticket vs. via PR.

[Mojca Miklavec]

(I noticed that my previous email was flying to some strange address.
I'm sorry if this is the second post, I cannot see the other one in
mail archive, so I'm not sure if it went through.)

On 4 April 2018 at 16:28, Jonathan Stickel wrote:
>
> Some instructions about
> recommended procedures for pull requests would be helpful to me to
> transition to that approach. If anything already exists, please point me to
> it!

I'm not sure if we have written anything. This is upstream documentation:

https://help.github.com/articles/creating-a-pull-request/

Now that I read what I wrote below it sounds more complicated than it
actually is.


The basic idea is that:

- You need an account on GitHub, make sure you add your ssh public key there
  (if you use website for editing, your primary email set there will
appear as commit author)
- Log in to GitHub, fork https://github.com/macports/macports-ports
- (You might want to delete any branch in your  fork that's not
"master", but that's optional)

- Clone repo to your machine:
git clone g...@github.com:macports/macports-ports.git
git remote add some-easy-name
g...@github.com:your-username/macports-ports.git
the name of remote can be changed in .git/config.

- It might help to use that git checkout as the main source specified in
/opt/local/etc/macports/sources.conf

- Then try to remember not to ever commit to the master branch, but
always use a separate branch for each pull request.
# each time before changing anything make sure you have everything
up to date
git fetch --all # optional
git checkout master # make sure you switch to master branch
git pull origin master [--rebase] # update, you don't need rebase
if you don't pollute your master

# make the edits, do all the testing
git branch update-2.19 # create a branch
git checkout update-2.19 # switch to that branch (you may combine
these two steps)
git add path/to/Portfile
git commit # write commit message
git push some-easy-name # use same name as above; this will push
the changes to your fork

- Usually, when you navigate back to
https://github.com/macports/macports-ports you will see a yellow box
asking you if you want to open a pull request.

There are a few other useful commands:
git commit --amend # to make changes to last commit
git push -f some-easy-name # will force-push changes to the PR in
case you fix existing commits
git rebase -i HEAD~N # where N is the number of the last few
commits you want to change

If you opened a pull request which takes a long time to merge for one
reason or another and you want to bring that branch in sync with
master, it's
git checkout update-2.19
git rebase master
Then you'll have the latest version from master with that commit with
update on top.

Mojca

Re: Port and patch submissions via ticket vs. via PR.

[Ryan Schmidt]

On Apr 4, 2018, at 07:18, Mojca Miklavec wrote:

>>> Also, Mojca brought the question of the hundreds of open tickets with
>>> port submissions a while back. It might be neat if we had some code
>>> that caused a special Github account to generate a PR for a port
>>> submission in trac. I wouldn't want this automatically invoked (at
>>> least not yet!), but if a human (like me!) could manually invoke it on
>>> a few ports at a time when they had free time, it might help to
>>> clear out the backlog.
>> 
>> In my experience, port submissions, especially by new contributors, require 
>> more cleanup than port updates. It's not uncommon for a submission to have 
>> half a dozen or more things wrong with it. So you could either develop an 
>> automated system for copying a Portfile submitted on Trac into a pull 
>> request -- and then have to check out that PR branch into your git clone, 
>> fix all the problems, amend the commit, force push it -- or you could just 
>> download the Portfile from Trac yourself, put it into your git clone, fix 
>> it, commit it, and either send a PR or push to master. And either way, 
>> you're going to want to make sure it builds locally first. It doesn't seem 
>> like the former saves you that much time over the latter, plus of course 
>> you've had to spend all that time developing the automated Trac-to-PR system.
> 
> The time saved comes from the author doing the cleanup based on
> feedback (which you can leave while waiting for a bus or while going
> for a walk :) vs. one of us having to do all the changes, fiddling
> with branches.

Fiddling with branches only becomes necessary if we implement this Trac-to-PR 
plan! Fiddling with branches wasn't necessary when we committed to our old 
Subversion repository, and it's not necessary if a committer just downloads the 
patches from Trac into their Git clone of macports-ports, fixes the port, and 
commits it to master.


> In my opinion we should:
> (a) create a page (either or Trac or on GitHub) explaining benefits of
> Pull requests (+ excuse for the delay processing trac tickets)

It should be in the guide, along with our other documentation.

> (b) leave a comment on all submission tickets pointing to that page (I
> guess this can be done automatically)
> 
> I expect some 10-20% of those submissions to be turned into pull
> requests by original authors, but that's already a lot. If we end up
> with 20 submissions one day, I don't mind going over all the twenty of
> them manually, but this is "hopeless" with hundreds of them.
> 
> Alternatively / in addition, what we currently lack is a page
> explaining how random unskilled users willing to spend some time
> helping us could do. If we were able to communicate clearly what such
> users can do when they just feel like helping us, this could be one of
> the suitable tasks. (Other tasks include fixing ports with wrong
> checksums after github switch their algorithm etc.)

How would an automated Trac-to-PR tool write a good commit message? If we're 
just talking about submissions, maybe that's easier. But what if the ticket 
contains multiple revisions of the Portfile? Trac automatically appends a 
number to the file if that filename has already been used, unless the attacher 
clicks the "Replace existing attachment of the same name" checkbox. What about 
multiple patchfiles? Multiple revisions of multiple patchfiles? What if some 
patchfiles became unnecessary during the development of the ticket?

I'm dubious of the quality of some of the submissions in Trac. Certainly some 
are good and have simply been forgotten. But I feel like many have had reviews 
already posted, and the required changes have not been made. Creating a PR for 
the incomplete work would disassociate the work from the existing reviews. 
Would you remember to click the link for each auto-created PR to see what 
existing discussion had occurred? Would repeating that feedback in the PR have 
any effect -- if the submitter ignored it originally, why will posting it again 
in a PR make them respond to it now?

Re: Port and patch submissions via ticket vs. via PR.

[Mark Anderson]

I like the idea of a trac to PR script or something to make things easier.
The github API is pretty easy to use, but I’m not a trac expert by any
means. That said, I’d be willing to help.

—Mark

On Wed, Apr 4, 2018 at 8:18 AM Mojca Miklavec  wrote:

> On 4 April 2018 at 13:15, Ryan Schmidt wrote:
> > On Apr 1, 2018, at 11:02, Perry E. Metzger wrote:
> >
> >> As some of you have noticed, I've been trying to keep the PR queue
> >> relatively clear.
> >
> > Thanks very much for that!
>
> Indeed. What you are doing is simply amazing.
>
> >> Also, Mojca brought the question of the hundreds of open tickets with
> >> port submissions a while back. It might be neat if we had some code
> >> that caused a special Github account to generate a PR for a port
> >> submission in trac. I wouldn't want this automatically invoked (at
> >> least not yet!), but if a human (like me!) could manually invoke it on
> >> a few ports at a time when they had free time, it might help to
> >> clear out the backlog.
> >
> > In my experience, port submissions, especially by new contributors,
> require more cleanup than port updates. It's not uncommon for a submission
> to have half a dozen or more things wrong with it. So you could either
> develop an automated system for copying a Portfile submitted on Trac into a
> pull request -- and then have to check out that PR branch into your git
> clone, fix all the problems, amend the commit, force push it -- or you
> could just download the Portfile from Trac yourself, put it into your git
> clone, fix it, commit it, and either send a PR or push to master. And
> either way, you're going to want to make sure it builds locally first. It
> doesn't seem like the former saves you that much time over the latter, plus
> of course you've had to spend all that time developing the automated
> Trac-to-PR system.
>
> The time saved comes from the author doing the cleanup based on
> feedback (which you can leave while waiting for a bus or while going
> for a walk :) vs. one of us having to do all the changes, fiddling
> with branches.
>
> In my opinion we should:
> (a) create a page (either or Trac or on GitHub) explaining benefits of
> Pull requests (+ excuse for the delay processing trac tickets)
> (b) leave a comment on all submission tickets pointing to that page (I
> guess this can be done automatically)
>
> I expect some 10-20% of those submissions to be turned into pull
> requests by original authors, but that's already a lot. If we end up
> with 20 submissions one day, I don't mind going over all the twenty of
> them manually, but this is "hopeless" with hundreds of them.
>
> Alternatively / in addition, what we currently lack is a page
> explaining how random unskilled users willing to spend some time
> helping us could do. If we were able to communicate clearly what such
> users can do when they just feel like helping us, this could be one of
> the suitable tasks. (Other tasks include fixing ports with wrong
> checksums after github switch their algorithm etc.)
>
> The pull request queue is now amazingly short. Cutting down all other
> queues one by one could be our next goal.
>
> Mojca
>
-- 
Sent from Gmail Mobile on iPhone

Re: Port and patch submissions via ticket vs. via PR.

[Mojca Miklavec]

On 4 April 2018 at 13:15, Ryan Schmidt wrote:
> On Apr 1, 2018, at 11:02, Perry E. Metzger wrote:
>
>> As some of you have noticed, I've been trying to keep the PR queue
>> relatively clear.
>
> Thanks very much for that!

Indeed. What you are doing is simply amazing.

>> Also, Mojca brought the question of the hundreds of open tickets with
>> port submissions a while back. It might be neat if we had some code
>> that caused a special Github account to generate a PR for a port
>> submission in trac. I wouldn't want this automatically invoked (at
>> least not yet!), but if a human (like me!) could manually invoke it on
>> a few ports at a time when they had free time, it might help to
>> clear out the backlog.
>
> In my experience, port submissions, especially by new contributors, require 
> more cleanup than port updates. It's not uncommon for a submission to have 
> half a dozen or more things wrong with it. So you could either develop an 
> automated system for copying a Portfile submitted on Trac into a pull request 
> -- and then have to check out that PR branch into your git clone, fix all the 
> problems, amend the commit, force push it -- or you could just download the 
> Portfile from Trac yourself, put it into your git clone, fix it, commit it, 
> and either send a PR or push to master. And either way, you're going to want 
> to make sure it builds locally first. It doesn't seem like the former saves 
> you that much time over the latter, plus of course you've had to spend all 
> that time developing the automated Trac-to-PR system.

The time saved comes from the author doing the cleanup based on
feedback (which you can leave while waiting for a bus or while going
for a walk :) vs. one of us having to do all the changes, fiddling
with branches.

In my opinion we should:
(a) create a page (either or Trac or on GitHub) explaining benefits of
Pull requests (+ excuse for the delay processing trac tickets)
(b) leave a comment on all submission tickets pointing to that page (I
guess this can be done automatically)

I expect some 10-20% of those submissions to be turned into pull
requests by original authors, but that's already a lot. If we end up
with 20 submissions one day, I don't mind going over all the twenty of
them manually, but this is "hopeless" with hundreds of them.

Alternatively / in addition, what we currently lack is a page
explaining how random unskilled users willing to spend some time
helping us could do. If we were able to communicate clearly what such
users can do when they just feel like helping us, this could be one of
the suitable tasks. (Other tasks include fixing ports with wrong
checksums after github switch their algorithm etc.)

The pull request queue is now amazingly short. Cutting down all other
queues one by one could be our next goal.

Mojca

Re: Port and patch submissions via ticket vs. via PR.

[Ryan Schmidt]

On Apr 1, 2018, at 11:02, Perry E. Metzger wrote:

> As some of you have noticed, I've been trying to keep the PR queue
> relatively clear.

Thanks very much for that!


> I'm wondering if we should therefore encourage people to submit github
> PRs if they have working patches (or are submitting new ports) rather
> than trac tickets. (Not suggesting we should _prevent_ the latter,
> just that maybe we should explicitly encourage the former.)

Yes, I think encouraging PRs instead of ticket attachments, when the submitter 
already has specific changes in mind, would be a good idea.


> Also, Mojca brought the question of the hundreds of open tickets with
> port submissions a while back. It might be neat if we had some code
> that caused a special Github account to generate a PR for a port
> submission in trac. I wouldn't want this automatically invoked (at
> least not yet!), but if a human (like me!) could manually invoke it on
> a few ports at a time when they had free time, it might help to
> clear out the backlog.

In my experience, port submissions, especially by new contributors, require 
more cleanup than port updates. It's not uncommon for a submission to have half 
a dozen or more things wrong with it. So you could either develop an automated 
system for copying a Portfile submitted on Trac into a pull request -- and then 
have to check out that PR branch into your git clone, fix all the problems, 
amend the commit, force push it -- or you could just download the Portfile from 
Trac yourself, put it into your git clone, fix it, commit it, and either send a 
PR or push to master. And either way, you're going to want to make sure it 
builds locally first. It doesn't seem like the former saves you that much time 
over the latter, plus of course you've had to spend all that time developing 
the automated Trac-to-PR system.