Re: [webkit-dev] WebKit Transition to Git

[Konstantin Tokarev]

05.10.2020, 13:26, "Frédéric Wang" :
> One thing to take into account is that WebKit's repository is big and
> public GitHub/GitLab prevent creating large repository by default. This
> means it might not be possible for contributors to actually fork
> WebKit's repository on their account and then create a pull request
> (which is the standard way IIUC).

Every repository which is already present on GitHub can be forked by
anyone without any restrictions.

There are restrictions for repositories which are not forks, e.g. if you
create new repo on your account and try to push whole WebKit there,
you'll get an error.

> Instead, we would probably end up
> doing like web-platform-tests and give contributors the permission to
> create branches to the WebKit account and make Pull Request to the
> master branch. 

This is not needed and should be avoided - private development branches
in main repo will confuse users and will be copied into all forks.


>Probably, we should forbid people to commit to the master
> branch directly (I think someone broke WPT's master branch that way last
> year)...

I have to admit that ability to make small unreviewed commit for thing like 
typo fix or change in personal watchlist is rather useful...

-- 
Regards,
Konstantin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git

[Konstantin Tokarev]

05.10.2020, 17:19, "Robert Ma" :
> On Mon, Oct 5, 2020 at 6:22 AM Frédéric Wang  wrote:
>> One thing to take into account is that WebKit's repository is big and
>> public GitHub/GitLab prevent creating large repository by default. This
>> means it might not be possible for contributors to actually fork
>> WebKit's repository on their account and then create a pull request
>> (which is the standard way IIUC). Instead, we would probably end up
>> doing like web-platform-tests and give contributors the permission to
>> create branches to the WebKit account and make Pull Request to the
>> master branch. Probably, we should forbid people to commit to the master
>> branch directly (I think someone broke WPT's master branch that way last
>> year)...
>
> Note that this is not exactly the reason we give people write access in WPT. 
> The WPT repo is not too big to impede forking. Rather, this was to work 
> around some CI setup constraints (e.g. some secrets not accessible from 
> forks), most of which have been resolved so the write access is largely for 
> convenience for active developers (e.g. avoid having to sync the fork).

There is no need to sync the fork for any development activity. You can have 
WPT repo as an upstream for master branch, and push your topic branches to your 
own fork added as another remote. When you pull master it will be updated from 
WPT repo, or you can do git fetch and rebase your topic branch to origin/master.

The only possible downside is that master branch of your fork may become 
outdated, but since you don't need it for anything and use local master which 
is always in sync, it's just cosmetic issue.

-- 
Regards,
Konstantin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git

[Konstantin Tokarev]

05.10.2020, 23:41, "Yusuke Suzuki" :
> I think security component is special in terms of how to handle it already 
> (e.g. not posting a test with the patch etc.)
> To me, handling non-security issues in GitHub and security issues in Bugzilla 
> is OK.
> By nature, security issues are not open. Since one of our motivation of 
> moving to GitHub is openness for feedback collection, security issue in 
> Bugzilla does not matter for this motivation.
> Ideally, handling both in GitHub is better. But to me, rather than continuing 
> using Bugzilla, using GitHub for non security issues sounds improvement.

To me it sounds as a huge step backwards. Asides from situation with security 
issues, it has other significant drawbacks in domain of issue triaging and 
management:

1. Sub-par support for linking issues to each other


Traditional bug tracking systems (like Bugzilla or JIRA) have support of 
"related" or "linked" issues. Most important relations are

* A depends on B (B blocks A) - blockers and umbrella issues
* B is duplicate of A
* A and B are related in other unspecified way

All GitHub can offer here now is mentions (and, to some extent, milestones for 
case of "umbrella issues" [1]). Mention is created every time someone uses 
"#" (e.g. "#123") in the text of issue or in the comment, where number 
is a sequential number of issue or pull request [2]. When comment is published 
in issue A which mentions issue B, there is a pseudo-comment added to B, and 
subscribers of B receive email notification.

At first glance this naive approach seems to work, but

* There is no easily visible list of relations: if you are not closely 
following all activity on A, to find all issues related to it you have to 
browse through all its (pseudo-)comments, which in some cases might be long. 
* There is no *stateful* list of relations: if A was believed to have common 
source with B, but then it was discovered they are not related, you cannot 
delete relationship between A and B because there is no relationship, just a 
set of comments.
* "#" is not a safe reference format. Sometimes users' comments may 
have other data in "#" format with a different meaning than references 
to GitHub issues. For example, may the force be with you if someone pastes gdb 
or lldb backtrace into comment without escaping it into markdown raw text block 
(```). Also, GitHub parses mentions in git commit messages, so care must be 
taken to avoid any occurrences of "#" with a meaning different from 
reference to issue number.

---

[1] Milestones are not issues themselves, but they establish true two-way 
stateful relation between issues and their "parent" milestone.
[2] For some reason they have shared numbering, which means sometimes you may 
not know what kind of reference is in front of you until you look up its URL or 
navigate


2. Sub-par issue categorization and search
--

In traditional bug tracking systems every issue has properties like status, 
resolution, component, severity/issue type, priority. When new bug is created, 
user chooses values of these properties, which can be later adjusted by the 
person doing triaging. They also are used in user-friendly search dialog like 
[1].

All GitHub can offer here are custom labels. While in theory they are 
sufficient to replace pre-defined issue properties, they require disciplined 
use to be efficient, for example issue must not have mutually exclusive labels. 
To avoid chaos, GitHub allows setting issue labels only to contributors. 
However, this puts more work on triagers, and they cannot triage only certain 
kinds of issues which match their interests by going through results of search 
query.

[1] https://bugs.webkit.org/query.cgi


3. Sub-par attachments
--

Traditional bug trackers allow attaching files to issue. GitHub goes further 
and allows to attach files to every comment. Enjoy the progress -  now you can 
look for attached test cases and proposed patches through all comment feed, 
instead of having them in one place at the top.

Also, on test cases. You probably like this feature of Bugzilla when you can 
attach self-contained HTML file to the issue and then simply open it by URL in 
any browser including your build of WebKit to try it out. Forget this - GitHub 
simply forbids HTML or JS attachments (without wrapping them in archive):

"We don’t support that file type. with a GIF, JPEG, JPG, PNG, DOCX, GZ, 
LOG, PDF, PPTX, TXT, XLSX or ZIP."

And yes, take care not to use tar.xz or tar.bz2 or any other unapproved archive 
type.

But you can attach funny cat picture to your comment and it will be displayed 
inline :)


Conclusion
--

You can say this is all small issues which are not really significant. With 
current state of things when Bugzilla is mostly used as a code review tool, and 
very few of issues reported by 

Re: [webkit-dev] WebKit Transition to Git

[Yusuke Suzuki]

I think security component is special in terms of how to handle it already 
(e.g. not posting a test with the patch etc.)
To me, handling non-security issues in GitHub and security issues in Bugzilla 
is OK.
By nature, security issues are not open. Since one of our motivation of moving 
to GitHub is openness for feedback collection, security issue in Bugzilla does 
not matter for this motivation.
Ideally, handling both in GitHub is better. But to me, rather than continuing 
using Bugzilla, using GitHub for non security issues sounds improvement.

-Yusuke

> On Oct 5, 2020, at 12:58 PM, Fujii Hironori  wrote:
> 
> 
> 
> On Tue, Oct 6, 2020 at 12:40 AM Jonathan Bedard  > wrote:
> 
> That's one solution, but even that is somewhat insufficient because we don’t 
> want to give someone access to every security issue just to give access to a 
> single one. One of the solutions we’ve discussed is to migrate bugs component 
> by component, the security component may stay on bugzilla indefinitely.
> 
> 
>  I don't like this idea of using GitHub issue tracker and Bugzilla together. 
> Let's keep using Bugzilla for all components.
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git

[Fujii Hironori]

On Tue, Oct 6, 2020 at 12:40 AM Jonathan Bedard  wrote:

>
> That's one solution, but even that is somewhat insufficient because we
> don’t want to give someone access to every security issue just to give
> access to a single one. One of the solutions we’ve discussed is to migrate
> bugs component by component, the security component may stay on bugzilla
> indefinitely.
>
>
 I don't like this idea of using GitHub issue tracker and Bugzilla
together. Let's keep using Bugzilla for all components.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git

[Michael Catanzaro]

On Mon, Oct 5, 2020 at 8:40 am, Jonathan Bedard  
wrote:
That's one solution, but even that is somewhat insufficient because 
we don’t want to give someone access to every security issue just 
to give access to a single one. One of the solutions we’ve 
discussed is to migrate bugs component by component, the security 
component may stay on bugzilla indefinitely.


Can you grant access to a single issue by CCing the desired user to the 
issue? I expect that would work (not tested).



___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git [Identifiers and Merge Commits]

[Jonathan Bedard]

>> Monotonic Commit Identifiers
>> Of great interest to Apple?s engineers has been retaining some kind
>> of ordered tag we can use to refer to commits to make defending CI
>> and bisection easier. We?ve developed a scheme for this that assigns
>> commits an ordered identifier per-branch, outlined in?
>> https://trac.webkit.org/wiki/commit-identifiers 
>> , designed to be used
>> alongside git hashes. These identifiers can be used in our current
>> Subversion repository, and we would like to start using them before
>> the project has transitions to git.
>> 
> 
> Have you seen how this was handled in LLVM?
> https://releases.llvm.org/9.0.0/docs/Proposals/GitHubMove.html#on-managing-revision-numbers-with-git
>  
> 

We looked at LLVM and Chromium specifically, our approach is pretty similar to 
LLVM’s, but resets the canonical identifiers when branching from the default 
branch.

> 
> Would you also consider preventing merge commits in order to keep a
> clean mainline branch?


Definitely going to be preventing merge commits, they end up breaking a lot, I 
neglected to mention them because I often (perhaps out of wishful thinking) 
forget they exist.

Jonathan

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git

[Jonathan Bedard]

> On Oct 2, 2020, at 5:05 PM, Michael Catanzaro  wrote:
> 
> On Fri, Oct 2, 2020 at 09:43, Jonathan Bedard  wrote:
>> The biggest blocker we are aware of is managing security bugs, since the 
>> security advisory system used by GitHub is essentially the opposite of how 
>> WebKit security bugs work. Moving to GitHub Issues, if it happens, will be 
>> the last part of this transition, and we are interested in soliciting 
>> feedback from our contributors on what the WebKit project´s integration with 
>> GitHub Issues should look like.
> 
> I don't think we need much integration to use the issue tracker? Once we 
> migrate existing bugs from WebKit Bugzilla, we can use it as we would any 
> other issue tracker? Why would it require integration?

We have some tooling attached to bugzilla (webkit-patch is the highest profile, 
but there are a few others), that’s the sort of thing I’m referring to.

> 
> We might need to use a separate repository with more limited permissions to 
> handle security reports. At least in GitLab, all project developers 
> (committers) have access to all confidential issues. I'm not sure about 
> GitHub, but I assume it would be the same.

That's one solution, but even that is somewhat insufficient because we don’t 
want to give someone access to every security issue just to give access to a 
single one. One of the solutions we’ve discussed is to migrate bugs component 
by component, the security component may stay on bugzilla indefinitely.

> 
> What will require integration is pull request merges. If we want to maintain 
> linear version history, we will want a merge bot. On GNOME GitLab, we have a 
> large number of smaller projects and it's we don't need them, but for a one 
> huge project like WebKit there will be too many conflicts otherwise, because 
> every commit going into the main branch will require all other pull requests 
> to be rebased. A merge bot -- e.g. [1] -- will handle that for us. (Not sure 
> what merge bots are common on GitHub. )

We definitely will have a merge bot (basically what the current commit queue 
is). They are pretty common in GitHub, especially because they’re a good way to 
manage access to protected branches. Aakash and I have discussed some of the 
specifics, the #github-migration channel on Slack is going to be the best place 
to discuss the details of this.

> 
> Michael
> 
> [1] https://gitlab.com/fsdk-marge-bot
> 
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git

[Jonathan Bedard]

`git describe` is quite a bit less powerful, unfortunately. First, while the 
string coming from `git-describe` can be used in git commands, that’s only 
because it contains the abbreviated commit hash, which alone is sufficient to 
identify a commit. Second, such identifiers end up being pretty weird on the 
default branch since we often use tags to mark branch points. This means that 
commits on the default branch would reset their count every time we branched.

Jonathan

> On Oct 4, 2020, at 2:41 PM, Konstantin Tokarev  wrote:
> 
> 
> 
> 02.10.2020, 19:46, "Jonathan Bedard" :
>> Monotonic Commit Identifiers
>> Of great interest to Apple’s engineers has been retaining some kind of 
>> ordered tag we can use to refer to commits to make defending CI and 
>> bisection easier. We’ve developed a scheme for this that assigns commits an 
>> ordered identifier per-branch, outlined in 
>> https://trac.webkit.org/wiki/commit-identifiers, designed to be used 
>> alongside git hashes. These identifiers can be used in our current 
>> Subversion repository, and we would like to start using them before the 
>> project has transitions to git.
> 
> AFAIU, this is very close to what `git describe` does: you give it git hash, 
> it gives you new identifier consisting of 3 parts:
> 
> --
> 
> Note that resulting identifier can be used in all git operations which 
> require git reference (like `git log` or `git show`).
> So, if you push git tags to main repository (maybe lightweight tags, if you 
> are planning to have lots of them), there is no need to invent any other 
> identifiers.
> 
> As for bisection, git bisect works just fine when given two commit hashes.
> 
> 
> -- 
> Regards,
> Konstantin

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git

[Frédéric Wang]

On 05/10/2020 16:15, Robert Ma wrote:
> On Mon, Oct 5, 2020 at 6:22 AM Frédéric Wang  > wrote:
>
> One thing to take into account is that WebKit's repository is big and
> public GitHub/GitLab prevent creating large repository by default.
> This
> means it might not be possible for contributors to actually fork
> WebKit's repository on their account and then create a pull request
> (which is the standard way IIUC). Instead, we would probably end up
> doing like web-platform-tests and give contributors the permission to
> create branches to the WebKit account and make Pull Request to the
> master branch. Probably, we should forbid people to commit to the
> master
> branch directly (I think someone broke WPT's master branch that
> way last
> year)...
>
>  
> Note that this is not exactly the reason we give people write access
> in WPT. The WPT repo is not too big to impede forking. Rather, this
> was to work around some CI setup constraints (e.g. some secrets not
> accessible from forks), most of which have been resolved so the write
> access is largely for convenience for active developers (e.g. avoid
> having to sync the fork
> ).
>
Right, I didn't mean that but thanks for clarifying Robert (and for
adding more info about potential issues with this PR/fork-based workflow).

-- 
Frédéric Wang

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git

[Adrien Destugues]

> One thing to take into account is that WebKit's repository is big and
> public GitHub/GitLab prevent creating large repository by default. This
> means it might not be possible for contributors to actually fork
> WebKit's repository on their account and then create a pull request
> (which is the standard way IIUC). Instead, we would probably end up
> doing like web-platform-tests and give contributors the permission to
> create branches to the WebKit account and make Pull Request to the
> master branch. Probably, we should forbid people to commit to the master
> branch directly (I think someone broke WPT's master branch that way last
> year)...
> 
> Similarly, public GitHub/GitLab prevent very large files by default.
> Hopefully, this does not happen in WebKit (we should definitely add a
> hook to prevent someone to land a big file) but for example that mistake
> happened in Chromium last year (
> https://groups.google.com/u/1/a/chromium.org/g/blink-dev/c/tSOmIfXAP2s/m/H07SmWIoCgAJ
> ) and this broke the sync of their GitHub mirror.

There is already a mirror of the existing WebKit repository on Github, and 
there are
about 2000 forks of it already [1]. At least the Haiku version of WebKit is 
maintained this
way, and had no problems with Github so far. So this shouldn't be a problem?

[1] https://github.com/WebKit/webkit/network/members

-- 
Adrien.

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git

[Robert Ma]

On Mon, Oct 5, 2020 at 6:22 AM Frédéric Wang  wrote:

> One thing to take into account is that WebKit's repository is big and
> public GitHub/GitLab prevent creating large repository by default. This
> means it might not be possible for contributors to actually fork
> WebKit's repository on their account and then create a pull request
> (which is the standard way IIUC). Instead, we would probably end up
> doing like web-platform-tests and give contributors the permission to
> create branches to the WebKit account and make Pull Request to the
> master branch. Probably, we should forbid people to commit to the master
> branch directly (I think someone broke WPT's master branch that way last
> year)...
>

Note that this is not exactly the reason we give people write access in
WPT. The WPT repo is not too big to impede forking. Rather, this was to
work around some CI setup constraints (e.g. some secrets not accessible
from forks), most of which have been resolved so the write access is
largely for convenience for active developers (e.g. avoid having to sync
the fork

).

And yeah, do remember to protect the master branch :)
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit Transition to Git

[Frédéric Wang]

On 02/10/2020 18:43, Jonathan Bedard wrote:
Hi all,

This is great to hear! Several people have already made comments about
things that I wanted to say, but I'd just to add my personal point of
view...

I've used both Mozilla's Phabricator and Chromium's Gerrit for
patches/review, and I find both of them great and pretty easy to use. Of
course it can take time to get familiar with the UI, but I don't think
this is a big blocker, the same is true with any tool...

Mozilla and Chromium projects are basically using a separate tracker for
bug reports and a Phabricator's differential / Gerrit's CL are connected
to a bug entry. Mozilla's differential are attached to Bugzilla as a
patch (which is good to make people cc'ed to the bug aware of WIP patch)
while Chromium's Gerrit only posts the commit message when the patch lands.

Both tools generate a increasing ID in the commit message. However, I
think Phabricator's ID is done when the differential is created, not at
commit time so this is not what we want for WebKit I guess. In any case,
I don't think there should be a problem to add a hook to ensure the ID
is created when landing the patch.

When you upload a patch you are basically creating a new differential/CL
or updating an existing one with a new revision and the difference
between revision is calculated automatically. To upload a patch, you
execute a script similar to our "webkit-patch upload" which can do extra
checks (e.g. style, suggesting reviewers, etc). In any case, Mozilla and
Chromium's projects perform these checks when the patch is uploaded too.

I'm not sure for Mozilla's phabricator but in Chromium's Gerrit you can
also make a CL depend on another one (I mean the review tool and bots
will understand that dependency and behave "nicely"), which is helpful
to split a change into several patches.

I personally prefer both of them over GitHub/GitLab's approach with
branches & merge/pull requests. That way you don't have to publish your
branches / rebases / forced pushes / squashes, etc Instead, you would do
whatever you want in your local clone & branches and always only send
the relevant diff as a new revision. Others say similar features are
available/planned for GitHub/GitLab, so maybe it's not a problem.

One thing to take into account is that WebKit's repository is big and
public GitHub/GitLab prevent creating large repository by default. This
means it might not be possible for contributors to actually fork
WebKit's repository on their account and then create a pull request
(which is the standard way IIUC). Instead, we would probably end up
doing like web-platform-tests and give contributors the permission to
create branches to the WebKit account and make Pull Request to the
master branch. Probably, we should forbid people to commit to the master
branch directly (I think someone broke WPT's master branch that way last
year)...

Similarly, public GitHub/GitLab prevent very large files by default.
Hopefully, this does not happen in WebKit (we should definitely add a
hook to prevent someone to land a big file) but for example that mistake
happened in Chromium last year (
https://groups.google.com/u/1/a/chromium.org/g/blink-dev/c/tSOmIfXAP2s/m/H07SmWIoCgAJ
) and this broke the sync of their GitHub mirror.

I don't know the details of the conversation between Google & GitHub,
but it took several weeks and at the end at Igalia we decided to switch
our MathML development branch to an internal GitLab repository on which
we have full control. Apple might want to weight the pros/cons of
managing their own instance VS relying on proprietary / third-party
instances for this kind of issue or other customizations.

In general, I personally believe it would be good if one only require to
use free software tool to contribute to WebKit. But I guess we already
made an exception by switching to Slack, so...

-- 
Frédéric Wang

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev