Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Tom Stellard via lldb-dev]

On 04/29/2020 01:19 PM, David Blaikie wrote:
> Generally sounds pretty good to me - only variation on the theme (& certainly 
> imho dealer's choice at this point - if you/whoever ends up doing this 
> doesn't like the sound of it, they shouldn't feel they have to do it this 
> way) - maybe creating blank issues up to the current bugzilla PR number (& 
> maybe some padding) in a single/quick-ish (no idea how quickly those can be 
> created) window might help reduce the need for race conditions/shutting down 
> bug reporting, etc
> 

I think this is a really good suggestion.  It would take a lot of
pressure off the migration process.  The only potential downside is that
copying into blank issues might be slower than just moving the issues.
I have not tested this, so I don't know yet, but even if copying is slower
it doesn't really matter that much if there is nothing (e.g. re-enabling
repo access) that is waiting on the process to complete.

-Tom

> On Wed, Apr 29, 2020 at 8:25 AM Tom Stellard via cfe-dev 
> mailto:cfe-...@lists.llvm.org>> wrote:
> 
> Hi,
> 
> Thanks to everyone who provided feedback.  I would like to propose a
> more detailed plan based on the everyone's comments.  It seems like there 
> was a strong
> preference to maintain the bug ID numbers, so this proposal tries to 
> address that.
> 
> TLDR; This proposes to maintain bug ID numbers by overwriting existing 
> GitHub issues
> instead of creating new ones.  e.g. github.com/llvm/llvm-project/issues/1 
>  will
> be overwritten with data from llvm.org/PR1 .  There 
> will be some bugs that
> end up having their data copied into pull requests, which may be strange,
> but the data will be preserved and the IDs will be preserved and this 
> would
> only happen to very old bugs.
> 
> Proposal:
> 
> Detailed steps for doing the migration:
> 
> 
> * Weeks or days before the migration:
> 
> 1. Create a new GitHub repository called llvm-bug-archive and import bug
> data from bugzilla.
> 
> This step should not be under any kind of time pressure, so that the 
> conversion
> process can be debugged and refined.
> 
> 2. Install label notification system using GitHub actions and enable web 
> hook
> to send emails to llvm-bugs list.
> 
> * Day before the migration:
> 
> 3. Make bugzilla readonly.
> 
> 4. Import any new bugs created since the last import.
> 
> There may be commit access disruption during the migration, so
> completing these steps the day before will limit the amount of down time.
> 
> 5. Temporarily re-enable issues in the llvm-project repo and copy 
> existing issues
> to the llvm-bug-archive repo so they get higher IDs numbers than any
> existing PR.  Disable issues when done.
> 
> Note that we will copy issues instead of moving them, which means the 
> original
> issue will remain in tact.  This will allow us to retain the bug IDs
> for future use and not 'lose' a bug ID.
> 
> * Day of migration:
> 
> 6. Lockdown the repo as much as possible to prevent people from creating
> issues or pull requests.
> 
> Temporarily making the repo private may be one way to achieve this.  Other
> suggestions welcome.
> 
> 7. Copy issues with overlapping issues IDs from the llvm-bug-archive repo
> into the llvm-project repo.
> 
> Issues from the llvm-bug-archive repo that have the same ID number as
> existing issues in the llvm-project repo will be manually copied from
> the former to the later.  This will allow us to preserve the PR numbers
> from bugzilla.  Here is an example for how this would work:
> 
> - Delete comments and description from llvm-project issue #1.
> - Copy comments and description from llvm-bug-archive issue #1 into
>   llvm-project issue #1.
> 
> Since GitHub issue and pull requests share the same numbering sequence, 
> any
> PR# from bugzilla that maps to a pull request in the llvm-project repo 
> will
> need to have it's comments copied into a pull request.  These issues will 
> look slightly
> strange since there will be random commits attached to the issue.  
> However,
> all data will be preserved and more importantly the bug ID will be 
> preserved.
> 
> The issues URL can be used to access pull requests e.g.
> pull request #84 is accessible via github.com/llvm/llvm-project/issues/84 
> 
> so even with bugzilla data stored in pull requests, we will still be able 
> to do a simple redirect
> from llvm.org/PR###  to 
> github.com/llvm/llvm-project/issues/### 
> 
> 
> 
> 8. Once all the overlapping Issue IDs have been copied.  Move the rest of 
> the issues
> from the llvm-bug-archive repo to the llvm-project repo.
> 
> 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[David Blaikie via lldb-dev]

Generally sounds pretty good to me - only variation on the theme (&
certainly imho dealer's choice at this point - if you/whoever ends up doing
this doesn't like the sound of it, they shouldn't feel they have to do it
this way) - maybe creating blank issues up to the current bugzilla PR
number (& maybe some padding) in a single/quick-ish (no idea how quickly
those can be created) window might help reduce the need for race
conditions/shutting down bug reporting, etc

On Wed, Apr 29, 2020 at 8:25 AM Tom Stellard via cfe-dev <
cfe-...@lists.llvm.org> wrote:

> Hi,
>
> Thanks to everyone who provided feedback.  I would like to propose a
> more detailed plan based on the everyone's comments.  It seems like there
> was a strong
> preference to maintain the bug ID numbers, so this proposal tries to
> address that.
>
> TLDR; This proposes to maintain bug ID numbers by overwriting existing
> GitHub issues
> instead of creating new ones.  e.g. github.com/llvm/llvm-project/issues/1
> will
> be overwritten with data from llvm.org/PR1.  There will be some bugs that
> end up having their data copied into pull requests, which may be strange,
> but the data will be preserved and the IDs will be preserved and this would
> only happen to very old bugs.
>
> Proposal:
>
> Detailed steps for doing the migration:
>
>
> * Weeks or days before the migration:
>
> 1. Create a new GitHub repository called llvm-bug-archive and import bug
> data from bugzilla.
>
> This step should not be under any kind of time pressure, so that the
> conversion
> process can be debugged and refined.
>
> 2. Install label notification system using GitHub actions and enable web
> hook
> to send emails to llvm-bugs list.
>
> * Day before the migration:
>
> 3. Make bugzilla readonly.
>
> 4. Import any new bugs created since the last import.
>
> There may be commit access disruption during the migration, so
> completing these steps the day before will limit the amount of down time.
>
> 5. Temporarily re-enable issues in the llvm-project repo and copy existing
> issues
> to the llvm-bug-archive repo so they get higher IDs numbers than any
> existing PR.  Disable issues when done.
>
> Note that we will copy issues instead of moving them, which means the
> original
> issue will remain in tact.  This will allow us to retain the bug IDs
> for future use and not 'lose' a bug ID.
>
> * Day of migration:
>
> 6. Lockdown the repo as much as possible to prevent people from creating
> issues or pull requests.
>
> Temporarily making the repo private may be one way to achieve this.  Other
> suggestions welcome.
>
> 7. Copy issues with overlapping issues IDs from the llvm-bug-archive repo
> into the llvm-project repo.
>
> Issues from the llvm-bug-archive repo that have the same ID number as
> existing issues in the llvm-project repo will be manually copied from
> the former to the later.  This will allow us to preserve the PR numbers
> from bugzilla.  Here is an example for how this would work:
>
> - Delete comments and description from llvm-project issue #1.
> - Copy comments and description from llvm-bug-archive issue #1 into
>   llvm-project issue #1.
>
> Since GitHub issue and pull requests share the same numbering sequence, any
> PR# from bugzilla that maps to a pull request in the llvm-project repo will
> need to have it's comments copied into a pull request.  These issues will
> look slightly
> strange since there will be random commits attached to the issue.  However,
> all data will be preserved and more importantly the bug ID will be
> preserved.
>
> The issues URL can be used to access pull requests e.g.
> pull request #84 is accessible via github.com/llvm/llvm-project/issues/84
> so even with bugzilla data stored in pull requests, we will still be able
> to do a simple redirect
> from llvm.org/PR###  to
> github.com/llvm/llvm-project/issues/###
> 
>
>
> 8. Once all the overlapping Issue IDs have been copied.  Move the rest of
> the issues
> from the llvm-bug-archive repo to the llvm-project repo.
>
> This should be faster than doing the copies since we do not need to
> overwrite existing
> issues and can just move the issues from one repo to the other.
>
> The end result of this is that we have all the old bugs from bugzilla
> present as issues
> in the llvm-project repository with all of their ID numbers preserved.
>
>
> * Other action items:
>
> - We need volunteers to help create bug templates to simplify the process
> of submitting
>   bugs.  If you are interested in helping with this, let me know.
>
> - Continue to iterate on the set of issue labels.  This should not block
> the migration since
> labels can be changed at any time, but there were some suggested
> improvements that should
> be discussed.
>
>
> Please reply to this proposal with your questions, comments, praise, or
> concerns.
>
> Thanks,
> Tom
>
>
> [1]
> 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Philip Reames via lldb-dev]

On 4/25/20 10:02 PM, Mehdi AMINI via cfe-dev wrote:



On Fri, Apr 24, 2020 at 12:04 PM Tom Stellard via llvm-dev 
mailto:llvm-...@lists.llvm.org>> wrote:


On 04/24/2020 03:24 AM, Sam McCall wrote:
> clangd's experience using github issues to track bugs (in a
separate repo) has been very positive, and I'm glad you're pushing
on this!
>
> Part of this has been that our issue tracker has been scoped to
our subproject only, which is a scope that the tool works well for
(on the user and developer side).
> As such I don't think we should migrate clangd to a using the
monorepo bugtracker. Email subscription to a label is better than
nothing, but worse than a separate repo.
> Removing the clangd label from the monorepo bugtracker seems
like the simplest thing, though I'm happy to work on auto-moving
bugs if that's better.
>
> (I'd suggest considering the same for other subprojects, though
I know that's not a popular opinion here)

I think it's important for everything in the monorepo to use the
same bug tracker.

There are advantages to having code in the monorepo (e.g. free
updates for API changes, a more consistent build experience, etc.).
But there are also costs, as you have pointed out, like having to use
a less than ideal bug tracker.  It's really up to sub-projects
to make the decision about whether these benefits are worth the costs.
The flang developers have just gone through this process and have
had to make some sacrifices to get the code in, but ultimately
felt the
sacrifices were worth it. 



I think it hurts the ability of developers and users to
collaborate effectively,
if the infrastructure for the project is spread across too many
different places.
And good collaboration is key for a project of this size with some
many tightly
connected components.


+1: seems like clangd here is trying a "in-between" approach in being 
halfway into a LLVM project. It was something that was strongly pushed 
back against multiple times during the discussions on Flang 
integration, it isn't clear to me why we'd get into a different 
approach with clangd. I am really in favor of keeping a cohesion in 
the project and not having a "graph of somehow disconnected projects". 
There might be sub-optimality sometimes, but we should address them 
for everyone instead of one-off improvements that may benefit one 
subproject on the short term but I suspect hurt the project on the 
long term.

+1.  Agreed w/Mehdi.


--
Mehdi


Getting back to the proposal we are discussing.  Do you have any
specific feedback
for improvements that might help make it align better with the
kind of experience
the clangd users and developers are looking for?

- Tom





___
LLVM Developers mailing list
llvm-...@lists.llvm.org 
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev


___
cfe-dev mailing list
cfe-...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Richard Smith via lldb-dev]

On Fri, 24 Apr 2020 at 14:13, Sam McCall via cfe-dev 
wrote:

> On Fri, Apr 24, 2020 at 9:03 PM Tom Stellard  wrote:
>
>> On 04/24/2020 03:24 AM, Sam McCall wrote:
>> > clangd's experience using github issues to track bugs (in a separate
>> repo) has been very positive, and I'm glad you're pushing on this!
>> >
>> > Part of this has been that our issue tracker has been scoped to our
>> subproject only, which is a scope that the tool works well for (on the user
>> and developer side).
>> > As such I don't think we should migrate clangd to a using the monorepo
>> bugtracker. Email subscription to a label is better than nothing, but worse
>> than a separate repo.
>> > Removing the clangd label from the monorepo bugtracker seems like the
>> simplest thing, though I'm happy to work on auto-moving bugs if that's
>> better.
>> >
>> > (I'd suggest considering the same for other subprojects, though I know
>> that's not a popular opinion here)
>>
>> I think it's important for everything in the monorepo to use the same bug
>> tracker.
>>
>> There are advantages to having code in the monorepo (e.g. free
>> updates for API changes, a more consistent build experience, etc.).
>> But there are also costs, as you have pointed out, like having to use
>> a less than ideal bug tracker.  It's really up to sub-projects
>> to make the decision about whether these benefits are worth the costs.
>> The flang developers have just gone through this process and have
>> had to make some sacrifices to get the code in, but ultimately felt the
>> sacrifices were worth it.
>>
>> I think it hurts the ability of developers and users to collaborate
>> effectively,
>> if the infrastructure for the project is spread across too many different
>> places.
>> And good collaboration is key for a project of this size with some many
>> tightly
>> connected components.
>>
> (sorry, I should probably not tilt at this windmill. More on-topic stuff
> below, I promise!)
> Right, and I think having a single-project view of the LLVM organization
> is a mistake: it's a graph of projects, some are highly connected and some
> are not.
> The monorepo has a strong technical reason: the graph is connected and
> accepting a CI boundary anywhere is expensive in the absence of stable APIs.
> But this is much less true for bug tracking systems: the cost to crossing
> boundaries is smaller.
> For clangd, the benefit of sharing a tracker with clang AST+Sema is less
> than the cost of sharing a tracker with clang codegen, LLVM proper, LLD,
> flang, MLIR, ... (and the opposite is true for source control/CI).
> Anyway, this is going to depend on what part(s) of the project graph you
> touch: people connected to many parts will want to make coordinating with
> hundreds of people incrementally, while people connected to few parts are
> far better served by communicating only with the people they need to
> (communication famously scales badly).
>

I would assume we will eventually want to open up our github repository to
pull requests. When that happens, if you use a separate bugtracker then
clangd issues will be split across the two: pull requests will be sent to
the monorepo because (as far as I'm aware) a pull request for a repo can
only be sent to that repo's issue tracker, but your "regular" issues will
be elsewhere. And even that is a rosier picture than I expect you'll
actually experience: people will file issues for clangd on the issue
tracker attached to the clangd repository, simply because that's how the
github workflow works in general. So if you keep a separate issue tracker,
I think you will de facto end up with two issue trackers for clangd, and
will spend some amount of ongoing effort managing that. So, I think you
might be underestimating the costs here if you've not taken that into
account already. That said, moving issues between repos is apparently
feasible, so it seems like we could use clangd as an experiment to find out
how major or minor these concerns actually are.

Getting back to the proposal we are discussing.  Do you have any specific
>> feedback
>> for improvements that might help make it align better with the kind of
>> experience
>> the clangd users and developers are looking for?
>>
> Sorry if it seemed I was trying to derail: I think sharding into multiple
> repos *is* a specific improvement that should be considered, though there
> are arguments against it.
> If "the proposal we are discussing" doesn't admit changes, well, I'm +1 on
> its current form too :-)
>
> Other suggestions:
>
> Issue templates: I think you need at least one for each component.
> Users will be less familiar with the bug tracker conventions than
> developers are, especially given that this one is unusual in covering
> multiple products. Forcing a choice between the "component" tags as well as
> guiding them to include relevant info leaves less of an unstructured mess
> to triage. This helps mitigate the fact that the UI won't separate
> component tags like "lld" from 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Philip Reames via lldb-dev]

On 4/22/20 2:35 PM, Richard Smith wrote:
On Wed, 22 Apr 2020 at 09:45, Philip Reames via cfe-dev 
mailto:cfe-...@lists.llvm.org>> wrote:


On 4/21/20 6:50 PM, Richard Smith wrote:


On Tue, 21 Apr 2020 at 17:00, Tom Stellard via llvm-dev
mailto:llvm-...@lists.llvm.org>> wrote:

On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev
mailto:cfe-...@lists.llvm.org>
>> wrote:
>
>     +1 to James's take
>
>     I'd prefer simplicity of implementation over perfection
here.
>
> If we end up with two different bug numbering systems,
that's a problem that we will be paying for for many years.
It's worth some investment now to avoid that problem. And it
doesn't seem like it really requires much investment.
>
> Here's another path we could take:
>
> 1) Fork the llvm repository to a private "bugs" repository.
Mirror the bugzilla issues there. Iterate until we're happy,
as per James's proposal.
> 2) Sync the forked repository to the llvm repository,
delete the llvm repository, rename "bugs" to "llvm", and make
it public.
>
> Then we'll have the first N bugs in llvm-project/llvm being
*exactly* the bugzilla bugs, and we'll have excised the
existing github issues that we want to pretend never existed
anyway.
>
>
> I think we've missed an important step in the planning
here: we've not agreed on a set of goals for the transition.
Here are mine:
>
>  * We end up with one single issue tracking system
containing all issues, both old and new, both open and closed.
>  * All links and references to existing bugs still work.
>  * We have a single bug numbering system covering all bugs,
and old bugs retain their numbers.

Why are the bug numbers important?


These numbers appear all over our codebase. PR[0-9] appears 3592
times in Clang testcases, plus 45 times in Clang source code and
119 times more as the file names of Clang testcases. If we add
inconvenience to looking up all of those, that makes maintenance
harder each time someone wants to look one of those up. (That's
probably a ~weekly occurrence for me.)


For this use case, a simple script and bulk change to update
references in source repo means the numbering can change
arbitrarily.  ~4k small mechanical changes is just not that much
to review for a one time update assuming you trust the number
remapping script and are just looking for overly aggressive regex
matches.

It's not quite as straightforward as you're suggesting: such a simple 
script would break a bunch of our CodeGen tests that match mangled 
names, if the length of any bug identifier changes. A grep for 
'_Z.*PR[0-9]' indicates that there are at least 254 of those that 
might need manual updating if we took this path.
We have an auto-updater for most llc scripts, but point taken.  My main 
point was this was one time, not that the one time was trivial.


(I don't have any quick fixes for your other mentioned cases.)

Another case I didn't think of before, but that seems very important: 
bug numbers appear in commit messages, and are primary context in 
understanding what the commit is doing and why. [We *could* go on a 
bulk history editing spree to fix those commit messages up (git 
filter-branch actually makes this fairly easy) -- but that too would 
create a little churn as everyone would needs to rebase all their work 
in progress on the rewritten master, and honestly, that sounds a lot 
scarier than any of the other things we've considered in this thread :)]
Agreed, history rewrite as a solution here should be rejected out of 
hand.  :)



Also, bug numbers appear in other bugs. I would assume we're not
going to be able to reliably figure out which numbers appearing
in a bug are bug numbers during the import process, so those
numbers will persist into the github issues world.

(In addition, I'm sure multiple groups have their own tracking
systems, web pages, documentation, etc. that contain references
to LLVM PR numbers. But maybe we shouldn't worry too much about
that.)

Could you help give some example use cases that require having
a non-intersecting set of bug numbers for bugzilla bugs and
github issues?


It makes conversing about bug numbers more difficult if you need
to clarify which system you're talking about. As a minor example,
we'd have to avoid saying "PR" for the new system in order to
avoid confusion, and get used to some new terminology, and
probably not use "bug 1234" to describe either system, because
  

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Richard Smith via lldb-dev]

On Wed, 22 Apr 2020 at 09:45, Philip Reames via cfe-dev <
cfe-...@lists.llvm.org> wrote:

> On 4/21/20 6:50 PM, Richard Smith wrote:
>
> On Tue, 21 Apr 2020 at 17:00, Tom Stellard via llvm-dev <
> llvm-...@lists.llvm.org> wrote:
>
>> On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
>> > On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
>> cfe-...@lists.llvm.org > wrote:
>> >
>> > +1 to James's take
>> >
>> > I'd prefer simplicity of implementation over perfection here.
>> >
>> > If we end up with two different bug numbering systems, that's a problem
>> that we will be paying for for many years. It's worth some investment now
>> to avoid that problem. And it doesn't seem like it really requires much
>> investment.
>> >
>> > Here's another path we could take:
>> >
>> > 1) Fork the llvm repository to a private "bugs" repository. Mirror the
>> bugzilla issues there. Iterate until we're happy, as per James's proposal.
>> > 2) Sync the forked repository to the llvm repository, delete the llvm
>> repository, rename "bugs" to "llvm", and make it public.
>> >
>> > Then we'll have the first N bugs in llvm-project/llvm being *exactly*
>> the bugzilla bugs, and we'll have excised the existing github issues that
>> we want to pretend never existed anyway.
>> >
>> >
>> > I think we've missed an important step in the planning here: we've not
>> agreed on a set of goals for the transition. Here are mine:
>> >
>> >  * We end up with one single issue tracking system containing all
>> issues, both old and new, both open and closed.
>> >  * All links and references to existing bugs still work.
>> >  * We have a single bug numbering system covering all bugs, and old
>> bugs retain their numbers.
>>
>> Why are the bug numbers important?
>
>
> These numbers appear all over our codebase. PR[0-9] appears 3592 times in
> Clang testcases, plus 45 times in Clang source code and 119 times more as
> the file names of Clang testcases. If we add inconvenience to looking up
> all of those, that makes maintenance harder each time someone wants to look
> one of those up. (That's probably a ~weekly occurrence for me.)
>
> For this use case, a simple script and bulk change to update references in
> source repo means the numbering can change arbitrarily.  ~4k small
> mechanical changes is just not that much to review for a one time update
> assuming you trust the number remapping script and are just looking for
> overly aggressive regex matches.
>
It's not quite as straightforward as you're suggesting: such a simple
script would break a bunch of our CodeGen tests that match mangled names,
if the length of any bug identifier changes. A grep for '_Z.*PR[0-9]'
indicates that there are at least 254 of those that might need manual
updating if we took this path.

(I don't have any quick fixes for your other mentioned cases.)
>
Another case I didn't think of before, but that seems very important: bug
numbers appear in commit messages, and are primary context in understanding
what the commit is doing and why. [We *could* go on a bulk history editing
spree to fix those commit messages up (git filter-branch actually makes
this fairly easy) -- but that too would create a little churn as everyone
would needs to rebase all their work in progress on the rewritten master,
and honestly, that sounds a lot scarier than any of the other things we've
considered in this thread :)]

Also, bug numbers appear in other bugs. I would assume we're not going to
> be able to reliably figure out which numbers appearing in a bug are bug
> numbers during the import process, so those numbers will persist into the
> github issues world.
>
> (In addition, I'm sure multiple groups have their own tracking systems,
> web pages, documentation, etc. that contain references to LLVM PR numbers.
> But maybe we shouldn't worry too much about that.)
>
> Could you help give some example use cases that require having
>> a non-intersecting set of bug numbers for bugzilla bugs and github issues?
>>
>
> It makes conversing about bug numbers more difficult if you need to
> clarify which system you're talking about. As a minor example, we'd have to
> avoid saying "PR" for the new system in order to avoid confusion, and get
> used to some new terminology, and probably not use "bug 1234" to describe
> either system, because that would be ambiguous. None of these individual
> factors seems like a huge disruption, but they all seem like inconvenience
> we should prefer to avoid if possible.
>
> -Tom
>>
>>
>> >
>> > It sounds like we don't all agree that the last point is important, but
>> if we can achieve it without any significant additional cost, why not do so?
>> >
>> > Philip
>> >
>> > On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
>> >> In a previous discussion, one other suggestion had been to migrate
>> all the bugzilla bugs to a separate initially-private "bug archive"
>> repository in github. This has a few 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[James Henderson via lldb-dev]

Github may do things in a canonical way, but I think you'll find that lots
of people will refer to them by other means in review comments, email
threads, etc. Let's avoid any risk of ambiguity...

Also, there's no guarantee in the future that Github won't decide to start
auto-linking PR1234 as well, which would just get even more confusing.

On Wed, 22 Apr 2020 at 13:14, James Y Knight via lldb-dev <
lldb-dev@lists.llvm.org> wrote:

> GitHub canonically uses "#NNN" to refer to its bugs or pull requests, and
> also supports "GH-NNN". We'll want to switch to one of those schemes, so
> that automatic linking works properly. So, in that case, PR1234 == legacy
> issue, #1234 or GH-1234 == new issue.
>
> (See
> https://help.github.com/en/github/writing-on-github/autolinked-references-and-urls
> )
>
> On Tue, Apr 21, 2020, 10:43 PM Johannes Doerfert via cfe-dev <
> cfe-...@lists.llvm.org> wrote:
>
>>
>> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
>> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
>> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
>> cfe-...@lists.llvm.org > wrote:
>> >>
>> >>  +1 to James's take
>> >>
>> >>  I'd prefer simplicity of implementation over perfection here.
>> >>
>> >> If we end up with two different bug numbering systems, that's a
>> problem that we will be paying for for many years. It's worth some
>> investment now to avoid that problem. And it doesn't seem like it really
>> requires much investment.
>> >>
>> >> Here's another path we could take:
>> >>
>> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the
>> bugzilla issues there. Iterate until we're happy, as per James's proposal.
>> >> 2) Sync the forked repository to the llvm repository, delete the llvm
>> repository, rename "bugs" to "llvm", and make it public.
>> >>
>> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly*
>> the bugzilla bugs, and we'll have excised the existing github issues that
>> we want to pretend never existed anyway.
>> >>
>> >>
>> >> I think we've missed an important step in the planning here: we've not
>> agreed on a set of goals for the transition. Here are mine:
>> >>
>> >>   * We end up with one single issue tracking system containing all
>> issues, both old and new, both open and closed.
>> >>   * All links and references to existing bugs still work.
>> >>   * We have a single bug numbering system covering all bugs, and old
>> bugs retain their numbers.
>> > Why are the bug numbers important?  Could you help give some example
>> use cases that require having
>> > a non-intersecting set of bug numbers for bugzilla bugs and github
>> issues?
>>
>>
>> While I have no experience in bugzilla or github tooling, the two step
>> process described by Richard doesn't seem to be very complicated.
>>
>>
>> As mentioned by others, we have commits and tests (and sometimes source
>> files) that explicitly mention bug numbers. I do regularly look up bugs
>> from a decade ago to determine if a test or some code still has
>> relevance or not. If PR3214 can be one of two bugs, it does not only
>> increase lookup time but also add confusion to everyone involved.
>>
>>
>> Cheers,
>>
>>Johannes
>>
>>
>>
>> > -Tom
>> >
>> >
>> >> It sounds like we don't all agree that the last point is important,
>> but if we can achieve it without any significant additional cost, why not
>> do so?
>> >>
>> >>  Philip
>> >>
>> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
>> >>>  In a previous discussion, one other suggestion had been to
>> migrate all the bugzilla bugs to a separate initially-private "bug archive"
>> repository in github. This has a few benefits:
>> >>>  1. If the migration is messed up, the repo can be deleted, and
>> the process run again, until we get a result we like.
>> >>>  2. The numbering can be fully-controlled.
>> >>>  Once the bugs are migrated to /some/ github repository,
>> individual issues can then be "moved" between repositories, and github will
>> redirect from the movefrom-repository's bug to the target repository's bug.
>> >>>
>> >>>  We could also just have llvm.org/PR###
>>   be the url only
>> for legacy bugzilla issue numbers -- and have it use a file listing the
>> mappings of bugzilla id -> github id to generate the redirects. (GCC just
>> did this recently for svn revision number redirections,
>> https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
>> >>>
>> >>>  Then we could introduce a new naming scheme for github issue
>> shortlinks.
>> >>>
>> >>>  On Mon, Apr 20, 2020 at 3:50 PM Richard Smith via llvm-dev <
>> llvm-...@lists.llvm.org > wrote:
>> >>>
>> >>>  On Mon, 20 Apr 2020 at 12:31, Tom Stellard via llvm-dev <
>> llvm-...@lists.llvm.org > wrote:
>> >>>
>> >>>  Hi,
>> >>>
>> >>> 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[James Y Knight via lldb-dev]

GitHub canonically uses "#NNN" to refer to its bugs or pull requests, and
also supports "GH-NNN". We'll want to switch to one of those schemes, so
that automatic linking works properly. So, in that case, PR1234 == legacy
issue, #1234 or GH-1234 == new issue.

(See
https://help.github.com/en/github/writing-on-github/autolinked-references-and-urls
)

On Tue, Apr 21, 2020, 10:43 PM Johannes Doerfert via cfe-dev <
cfe-...@lists.llvm.org> wrote:

>
> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
> cfe-...@lists.llvm.org > wrote:
> >>
> >>  +1 to James's take
> >>
> >>  I'd prefer simplicity of implementation over perfection here.
> >>
> >> If we end up with two different bug numbering systems, that's a problem
> that we will be paying for for many years. It's worth some investment now
> to avoid that problem. And it doesn't seem like it really requires much
> investment.
> >>
> >> Here's another path we could take:
> >>
> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the
> bugzilla issues there. Iterate until we're happy, as per James's proposal.
> >> 2) Sync the forked repository to the llvm repository, delete the llvm
> repository, rename "bugs" to "llvm", and make it public.
> >>
> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly*
> the bugzilla bugs, and we'll have excised the existing github issues that
> we want to pretend never existed anyway.
> >>
> >>
> >> I think we've missed an important step in the planning here: we've not
> agreed on a set of goals for the transition. Here are mine:
> >>
> >>   * We end up with one single issue tracking system containing all
> issues, both old and new, both open and closed.
> >>   * All links and references to existing bugs still work.
> >>   * We have a single bug numbering system covering all bugs, and old
> bugs retain their numbers.
> > Why are the bug numbers important?  Could you help give some example use
> cases that require having
> > a non-intersecting set of bug numbers for bugzilla bugs and github
> issues?
>
>
> While I have no experience in bugzilla or github tooling, the two step
> process described by Richard doesn't seem to be very complicated.
>
>
> As mentioned by others, we have commits and tests (and sometimes source
> files) that explicitly mention bug numbers. I do regularly look up bugs
> from a decade ago to determine if a test or some code still has
> relevance or not. If PR3214 can be one of two bugs, it does not only
> increase lookup time but also add confusion to everyone involved.
>
>
> Cheers,
>
>Johannes
>
>
>
> > -Tom
> >
> >
> >> It sounds like we don't all agree that the last point is important, but
> if we can achieve it without any significant additional cost, why not do so?
> >>
> >>  Philip
> >>
> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
> >>>  In a previous discussion, one other suggestion had been to
> migrate all the bugzilla bugs to a separate initially-private "bug archive"
> repository in github. This has a few benefits:
> >>>  1. If the migration is messed up, the repo can be deleted, and
> the process run again, until we get a result we like.
> >>>  2. The numbering can be fully-controlled.
> >>>  Once the bugs are migrated to /some/ github repository,
> individual issues can then be "moved" between repositories, and github will
> redirect from the movefrom-repository's bug to the target repository's bug.
> >>>
> >>>  We could also just have llvm.org/PR###
>   be the url only
> for legacy bugzilla issue numbers -- and have it use a file listing the
> mappings of bugzilla id -> github id to generate the redirects. (GCC just
> did this recently for svn revision number redirections,
> https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
> >>>
> >>>  Then we could introduce a new naming scheme for github issue
> shortlinks.
> >>>
> >>>  On Mon, Apr 20, 2020 at 3:50 PM Richard Smith via llvm-dev <
> llvm-...@lists.llvm.org > wrote:
> >>>
> >>>  On Mon, 20 Apr 2020 at 12:31, Tom Stellard via llvm-dev <
> llvm-...@lists.llvm.org > wrote:
> >>>
> >>>  Hi,
> >>>
> >>>  I wanted to continue discussing the plan to migrate from
> Bugzilla to Github.
> >>>  It was suggested that I start a new thread and give a
> summary of the proposal
> >>>  and what has changed since it was originally proposed in
> October.
> >>>
> >>>  == Here is the original proposal:
> >>>
> >>>
> http://lists.llvm.org/pipermail/llvm-dev/2019-October/136162.html
> >>>
> >>>  == What has changed:
> >>>
> >>>  * You will be able to subscribe to notifications for a
> specific 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Anton Korobeynikov via lldb-dev]

Hi Konrad,

Thanks for the scripts – look useful! For the record, here is the
result of previous experiments
https://github.com/asl/llvm-bugzilla/issues

On Wed, Apr 22, 2020 at 2:21 PM Konrad Kleine via cfe-dev
 wrote:
>
> I wanted to try importing llvm bugs into a fresh github repo and here's my 
> result so far (import is still running): 
> https://github.com/kwk/test-llvm-bz-import-4 . I've written the scripts 
> (https://github.com/kwk/bz2gh) myself because I wanted to remain in control 
> and don't make my life more complicated than it needs to be. The README.md 
> describes in great length what's imported and how. For now I import the 
> bugzillas just as placeholder issues. Then I lock those issues in github to 
> avoid messing with them. Before I created labels based on 
> /. Those are assigned to each issue. It should give a 
> good start to at least reserve all github #IDs so they map 1:1 to LLVM BZs.
>
> On Wed, 22 Apr 2020 at 09:23, Dimitry Andric via cfe-dev 
>  wrote:
>>
>> Since Bugzilla numbers are all under 50,000 (at least for now:), can't we 
>> simply bump the GitHub issue/pull request numbers to 50,000, and start from 
>> there?
>>
>> Then it would be easy to identify: < 5 means Bugzilla, >= 5 means 
>> GitHub.
>>
>> Now somebody's only gotta find a way to file 5-200 bogus GitHub tickets. 
>> :)  (Or ask GitHub support to bump the number synthetically.)
>>
>> -Dimitry
>>
>> On 22 Apr 2020, at 09:10, James Henderson via cfe-dev 
>>  wrote:
>>
>> Similar to other people's experiences, I've worked on a common code base 
>> that supported three different platforms, and each platform used a different 
>> bugzilla with it's own numbering scheme. I regularly came across references 
>> to "BZ123456" with no indication as to which of the three systems that 
>> referred to. This would often mean having to go to each in turn and seeing 
>> if the corresponding bug looked like it had anything to do with the related 
>> topic. Fortunately, given that there were many other things using the same 
>> bugzilla instances, this was usually pretty clear, but not always. Typos in 
>> bug numbers sometimes made things even harder, since you had to spend three 
>> times as long trying to guess.
>>
>> In other words +1 to using unique numbers, however we do it.
>>
>> On Wed, 22 Apr 2020 at 03:44, Johannes Doerfert via cfe-dev 
>>  wrote:
>>>
>>>
>>> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
>>> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
>>> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev 
>>> >> mailto:cfe-...@lists.llvm.org>> wrote:
>>> >>
>>> >>  +1 to James's take
>>> >>
>>> >>  I'd prefer simplicity of implementation over perfection here.
>>> >>
>>> >> If we end up with two different bug numbering systems, that's a problem 
>>> >> that we will be paying for for many years. It's worth some investment 
>>> >> now to avoid that problem. And it doesn't seem like it really requires 
>>> >> much investment.
>>> >>
>>> >> Here's another path we could take:
>>> >>
>>> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the 
>>> >> bugzilla issues there. Iterate until we're happy, as per James's 
>>> >> proposal.
>>> >> 2) Sync the forked repository to the llvm repository, delete the llvm 
>>> >> repository, rename "bugs" to "llvm", and make it public.
>>> >>
>>> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly* 
>>> >> the bugzilla bugs, and we'll have excised the existing github issues 
>>> >> that we want to pretend never existed anyway.
>>> >>
>>> >>
>>> >> I think we've missed an important step in the planning here: we've not 
>>> >> agreed on a set of goals for the transition. Here are mine:
>>> >>
>>> >>   * We end up with one single issue tracking system containing all 
>>> >> issues, both old and new, both open and closed.
>>> >>   * All links and references to existing bugs still work.
>>> >>   * We have a single bug numbering system covering all bugs, and old 
>>> >> bugs retain their numbers.
>>> > Why are the bug numbers important?  Could you help give some example use 
>>> > cases that require having
>>> > a non-intersecting set of bug numbers for bugzilla bugs and github issues?
>>>
>>>
>>> While I have no experience in bugzilla or github tooling, the two step
>>> process described by Richard doesn't seem to be very complicated.
>>>
>>>
>>> As mentioned by others, we have commits and tests (and sometimes source
>>> files) that explicitly mention bug numbers. I do regularly look up bugs
>>> from a decade ago to determine if a test or some code still has
>>> relevance or not. If PR3214 can be one of two bugs, it does not only
>>> increase lookup time but also add confusion to everyone involved.
>>>
>>>
>>> Cheers,
>>>
>>>Johannes
>>>
>>>
>>>
>>> > -Tom
>>> >
>>> >
>>> >> It sounds like we don't all agree that the last point is important, but 
>>> >> if we can achieve it without any significant 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Konrad Kleine via lldb-dev]

I wanted to try importing llvm bugs into a fresh github repo and here's my
result so far (import is still running):
https://github.com/kwk/test-llvm-bz-import-4 . I've written the scripts (
https://github.com/kwk/bz2gh) myself because I wanted to remain in control
and don't make my life more complicated than it needs to be. The README.md
describes in great length what's imported and how. For now I import the
bugzillas just as placeholder issues. Then I lock those issues in github to
avoid messing with them. Before I created labels based on
/. Those are assigned to each issue. It should give
a good start to at least reserve all github #IDs so they map 1:1 to LLVM
BZs.

On Wed, 22 Apr 2020 at 09:23, Dimitry Andric via cfe-dev <
cfe-...@lists.llvm.org> wrote:

> Since Bugzilla numbers are all under 50,000 (at least for now:), can't we
> simply bump the GitHub issue/pull request numbers to 50,000, and start from
> there?
>
> Then it would be easy to identify: < 5 means Bugzilla, >= 5 means
> GitHub.
>
> Now somebody's only gotta find a way to file 5-200 bogus GitHub
> tickets. :)  (Or ask GitHub support to bump the number synthetically.)
>
> -Dimitry
>
> On 22 Apr 2020, at 09:10, James Henderson via cfe-dev <
> cfe-...@lists.llvm.org> wrote:
>
> Similar to other people's experiences, I've worked on a common code base
> that supported three different platforms, and each platform used a
> different bugzilla with it's own numbering scheme. I regularly came across
> references to "BZ123456" with no indication as to which of the three
> systems that referred to. This would often mean having to go to each in
> turn and seeing if the corresponding bug looked like it had anything to do
> with the related topic. Fortunately, given that there were many other
> things using the same bugzilla instances, this was usually pretty clear,
> but not always. Typos in bug numbers sometimes made things even harder,
> since you had to spend three times as long trying to guess.
>
> In other words +1 to using unique numbers, however we do it.
>
> On Wed, 22 Apr 2020 at 03:44, Johannes Doerfert via cfe-dev <
> cfe-...@lists.llvm.org> wrote:
>
>>
>> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
>> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
>> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
>> cfe-...@lists.llvm.org > wrote:
>> >>
>> >>  +1 to James's take
>> >>
>> >>  I'd prefer simplicity of implementation over perfection here.
>> >>
>> >> If we end up with two different bug numbering systems, that's a
>> problem that we will be paying for for many years. It's worth some
>> investment now to avoid that problem. And it doesn't seem like it really
>> requires much investment.
>> >>
>> >> Here's another path we could take:
>> >>
>> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the
>> bugzilla issues there. Iterate until we're happy, as per James's proposal.
>> >> 2) Sync the forked repository to the llvm repository, delete the llvm
>> repository, rename "bugs" to "llvm", and make it public.
>> >>
>> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly*
>> the bugzilla bugs, and we'll have excised the existing github issues that
>> we want to pretend never existed anyway.
>> >>
>> >>
>> >> I think we've missed an important step in the planning here: we've not
>> agreed on a set of goals for the transition. Here are mine:
>> >>
>> >>   * We end up with one single issue tracking system containing all
>> issues, both old and new, both open and closed.
>> >>   * All links and references to existing bugs still work.
>> >>   * We have a single bug numbering system covering all bugs, and old
>> bugs retain their numbers.
>> > Why are the bug numbers important?  Could you help give some example
>> use cases that require having
>> > a non-intersecting set of bug numbers for bugzilla bugs and github
>> issues?
>>
>>
>> While I have no experience in bugzilla or github tooling, the two step
>> process described by Richard doesn't seem to be very complicated.
>>
>>
>> As mentioned by others, we have commits and tests (and sometimes source
>> files) that explicitly mention bug numbers. I do regularly look up bugs
>> from a decade ago to determine if a test or some code still has
>> relevance or not. If PR3214 can be one of two bugs, it does not only
>> increase lookup time but also add confusion to everyone involved.
>>
>>
>> Cheers,
>>
>>Johannes
>>
>>
>>
>> > -Tom
>> >
>> >
>> >> It sounds like we don't all agree that the last point is important,
>> but if we can achieve it without any significant additional cost, why not
>> do so?
>> >>
>> >>  Philip
>> >>
>> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
>> >>>  In a previous discussion, one other suggestion had been to
>> migrate all the bugzilla bugs to a separate initially-private "bug archive"
>> repository in github. This has a few 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Dimitry Andric via lldb-dev]

Since Bugzilla numbers are all under 50,000 (at least for now:), can't we 
simply bump the GitHub issue/pull request numbers to 50,000, and start from 
there?

Then it would be easy to identify: < 5 means Bugzilla, >= 5 means 
GitHub.

Now somebody's only gotta find a way to file 5-200 bogus GitHub tickets. :) 
 (Or ask GitHub support to bump the number synthetically.)

-Dimitry

> On 22 Apr 2020, at 09:10, James Henderson via cfe-dev 
>  wrote:
> 
> Similar to other people's experiences, I've worked on a common code base that 
> supported three different platforms, and each platform used a different 
> bugzilla with it's own numbering scheme. I regularly came across references 
> to "BZ123456" with no indication as to which of the three systems that 
> referred to. This would often mean having to go to each in turn and seeing if 
> the corresponding bug looked like it had anything to do with the related 
> topic. Fortunately, given that there were many other things using the same 
> bugzilla instances, this was usually pretty clear, but not always. Typos in 
> bug numbers sometimes made things even harder, since you had to spend three 
> times as long trying to guess.
> 
> In other words +1 to using unique numbers, however we do it.
> 
> On Wed, 22 Apr 2020 at 03:44, Johannes Doerfert via cfe-dev 
> mailto:cfe-...@lists.llvm.org>> wrote:
> 
> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev 
> >> mailto:cfe-...@lists.llvm.org> 
> >> >> wrote:
> >>
> >>  +1 to James's take
> >>
> >>  I'd prefer simplicity of implementation over perfection here.
> >>
> >> If we end up with two different bug numbering systems, that's a problem 
> >> that we will be paying for for many years. It's worth some investment now 
> >> to avoid that problem. And it doesn't seem like it really requires much 
> >> investment.
> >>
> >> Here's another path we could take:
> >>
> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the 
> >> bugzilla issues there. Iterate until we're happy, as per James's proposal.
> >> 2) Sync the forked repository to the llvm repository, delete the llvm 
> >> repository, rename "bugs" to "llvm", and make it public.
> >>
> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly* the 
> >> bugzilla bugs, and we'll have excised the existing github issues that we 
> >> want to pretend never existed anyway.
> >>
> >>
> >> I think we've missed an important step in the planning here: we've not 
> >> agreed on a set of goals for the transition. Here are mine:
> >>
> >>   * We end up with one single issue tracking system containing all issues, 
> >> both old and new, both open and closed.
> >>   * All links and references to existing bugs still work.
> >>   * We have a single bug numbering system covering all bugs, and old bugs 
> >> retain their numbers.
> > Why are the bug numbers important?  Could you help give some example use 
> > cases that require having
> > a non-intersecting set of bug numbers for bugzilla bugs and github issues?
> 
> 
> While I have no experience in bugzilla or github tooling, the two step
> process described by Richard doesn't seem to be very complicated.
> 
> 
> As mentioned by others, we have commits and tests (and sometimes source
> files) that explicitly mention bug numbers. I do regularly look up bugs
> from a decade ago to determine if a test or some code still has
> relevance or not. If PR3214 can be one of two bugs, it does not only
> increase lookup time but also add confusion to everyone involved.
> 
> 
> Cheers,
> 
>Johannes
> 
> 
> 
> > -Tom
> >
> >
> >> It sounds like we don't all agree that the last point is important, but if 
> >> we can achieve it without any significant additional cost, why not do so?
> >>
> >>  Philip
> >>
> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
> >>>  In a previous discussion, one other suggestion had been to migrate 
> >>> all the bugzilla bugs to a separate initially-private "bug archive" 
> >>> repository in github. This has a few benefits:
> >>>  1. If the migration is messed up, the repo can be deleted, and the 
> >>> process run again, until we get a result we like.
> >>>  2. The numbering can be fully-controlled.
> >>>  Once the bugs are migrated to /some/ github repository, individual 
> >>> issues can then be "moved" between repositories, and github will redirect 
> >>> from the movefrom-repository's bug to the target repository's bug.
> >>>
> >>>  We could also just have llvm.org/PR###  
> >>> > be the url only 
> >>> for legacy bugzilla issue numbers -- and have it use a file listing the 
> >>> mappings of bugzilla id -> github id to generate the redirects. (GCC just 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[James Henderson via lldb-dev]

Similar to other people's experiences, I've worked on a common code base
that supported three different platforms, and each platform used a
different bugzilla with it's own numbering scheme. I regularly came across
references to "BZ123456" with no indication as to which of the three
systems that referred to. This would often mean having to go to each in
turn and seeing if the corresponding bug looked like it had anything to do
with the related topic. Fortunately, given that there were many other
things using the same bugzilla instances, this was usually pretty clear,
but not always. Typos in bug numbers sometimes made things even harder,
since you had to spend three times as long trying to guess.

In other words +1 to using unique numbers, however we do it.

On Wed, 22 Apr 2020 at 03:44, Johannes Doerfert via cfe-dev <
cfe-...@lists.llvm.org> wrote:

>
> On 4/21/20 7:00 PM, Tom Stellard via llvm-dev wrote:
> > On 04/21/2020 03:36 PM, Richard Smith via llvm-dev wrote:
> >> On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
> cfe-...@lists.llvm.org > wrote:
> >>
> >>  +1 to James's take
> >>
> >>  I'd prefer simplicity of implementation over perfection here.
> >>
> >> If we end up with two different bug numbering systems, that's a problem
> that we will be paying for for many years. It's worth some investment now
> to avoid that problem. And it doesn't seem like it really requires much
> investment.
> >>
> >> Here's another path we could take:
> >>
> >> 1) Fork the llvm repository to a private "bugs" repository. Mirror the
> bugzilla issues there. Iterate until we're happy, as per James's proposal.
> >> 2) Sync the forked repository to the llvm repository, delete the llvm
> repository, rename "bugs" to "llvm", and make it public.
> >>
> >> Then we'll have the first N bugs in llvm-project/llvm being *exactly*
> the bugzilla bugs, and we'll have excised the existing github issues that
> we want to pretend never existed anyway.
> >>
> >>
> >> I think we've missed an important step in the planning here: we've not
> agreed on a set of goals for the transition. Here are mine:
> >>
> >>   * We end up with one single issue tracking system containing all
> issues, both old and new, both open and closed.
> >>   * All links and references to existing bugs still work.
> >>   * We have a single bug numbering system covering all bugs, and old
> bugs retain their numbers.
> > Why are the bug numbers important?  Could you help give some example use
> cases that require having
> > a non-intersecting set of bug numbers for bugzilla bugs and github
> issues?
>
>
> While I have no experience in bugzilla or github tooling, the two step
> process described by Richard doesn't seem to be very complicated.
>
>
> As mentioned by others, we have commits and tests (and sometimes source
> files) that explicitly mention bug numbers. I do regularly look up bugs
> from a decade ago to determine if a test or some code still has
> relevance or not. If PR3214 can be one of two bugs, it does not only
> increase lookup time but also add confusion to everyone involved.
>
>
> Cheers,
>
>Johannes
>
>
>
> > -Tom
> >
> >
> >> It sounds like we don't all agree that the last point is important, but
> if we can achieve it without any significant additional cost, why not do so?
> >>
> >>  Philip
> >>
> >>  On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
> >>>  In a previous discussion, one other suggestion had been to
> migrate all the bugzilla bugs to a separate initially-private "bug archive"
> repository in github. This has a few benefits:
> >>>  1. If the migration is messed up, the repo can be deleted, and
> the process run again, until we get a result we like.
> >>>  2. The numbering can be fully-controlled.
> >>>  Once the bugs are migrated to /some/ github repository,
> individual issues can then be "moved" between repositories, and github will
> redirect from the movefrom-repository's bug to the target repository's bug.
> >>>
> >>>  We could also just have llvm.org/PR###
>   be the url only
> for legacy bugzilla issue numbers -- and have it use a file listing the
> mappings of bugzilla id -> github id to generate the redirects. (GCC just
> did this recently for svn revision number redirections,
> https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
> >>>
> >>>  Then we could introduce a new naming scheme for github issue
> shortlinks.
> >>>
> >>>  On Mon, Apr 20, 2020 at 3:50 PM Richard Smith via llvm-dev <
> llvm-...@lists.llvm.org > wrote:
> >>>
> >>>  On Mon, 20 Apr 2020 at 12:31, Tom Stellard via llvm-dev <
> llvm-...@lists.llvm.org > wrote:
> >>>
> >>>  Hi,
> >>>
> >>>  I wanted to continue discussing the plan to migrate from
> Bugzilla to Github.
> >>>  It was suggested that I 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Philip Reames via lldb-dev]

On 4/21/20 3:36 PM, Richard Smith wrote:
On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev 
mailto:cfe-...@lists.llvm.org>> wrote:


+1 to James's take

I'd prefer simplicity of implementation over perfection here.

If we end up with two different bug numbering systems, that's a 
problem that we will be paying for for many years. It's worth some 
investment now to avoid that problem. And it doesn't seem like it 
really requires much investment.


I used to think this was super important, but I've now been through a 
couple of conversions which didn't provide a 1-to-1 mapping.  It's 
annoying for about 6 months, and after that, you basically forget it 
happened.  As long as old bugs are searchable in the new system, and you 
can find the new ID from the old system, the exact identifier isn't as 
important.


Anyways, this is all subjective and I'm certainty not volunteering to 
work on this, so IMHO my own opinion doesn't really count.  :)





Here's another path we could take:

1) Fork the llvm repository to a private "bugs" repository. Mirror the 
bugzilla issues there. Iterate until we're happy, as per James's proposal.
2) Sync the forked repository to the llvm repository, delete the llvm 
repository, rename "bugs" to "llvm", and make it public.


Then we'll have the first N bugs in llvm-project/llvm being *exactly* 
the bugzilla bugs, and we'll have excised the existing github issues 
that we want to pretend never existed anyway.



I think we've missed an important step in the planning here: we've not 
agreed on a set of goals for the transition. Here are mine:


 * We end up with one single issue tracking system containing all 
issues, both old and new, both open and closed.

 * All links and references to existing bugs still work.
 * We have a single bug numbering system covering all bugs, and old 
bugs retain their numbers.


It sounds like we don't all agree that the last point is important, 
but if we can achieve it without any significant additional cost, why 
not do so?


Philip

On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:

In a previous discussion, one other suggestion had been to
migrate all the bugzilla bugs to a separate initially-private
"bug archive" repository in github. This has a few benefits:
1. If the migration is messed up, the repo can be deleted, and
the process run again, until we get a result we like.
2. The numbering can be fully-controlled.
Once the bugs are migrated to /some/ github repository,
individual issues can then be "moved" between repositories, and
github will redirect from the movefrom-repository's bug to the
target repository's bug.

We could also just have llvm.org/PR###
 be the url only for legacy bugzilla
issue numbers -- and have it use a file listing the mappings of
bugzilla id -> github id to generate the redirects. (GCC just did
this recently for svn revision number redirections,
https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).

Then we could introduce a new naming scheme for github issue
shortlinks.

On Mon, Apr 20, 2020 at 3:50 PM Richard Smith via llvm-dev
mailto:llvm-...@lists.llvm.org>> wrote:

On Mon, 20 Apr 2020 at 12:31, Tom Stellard via llvm-dev
mailto:llvm-...@lists.llvm.org>> wrote:

Hi,

I wanted to continue discussing the plan to migrate from
Bugzilla to Github.
It was suggested that I start a new thread and give a
summary of the proposal
and what has changed since it was originally proposed in
October.

== Here is the original proposal:

http://lists.llvm.org/pipermail/llvm-dev/2019-October/136162.html

== What has changed:

* You will be able to subscribe to notifications for a
specific issue
  labels.  We have a proof of concept notification system
using github actions
  that will be used for this.

* Emails will be sent to llvm-bugs when issues are opened
or closed.

* We have the initial list of labels:
https://github.com/llvm/llvm-project/labels

== Remaining issue:

* There is one remaining issue that I don't feel we have
consensus on,
and that is what to do with bugs in the existing
bugzilla.  Here are some options
that we have discussed:

1. Switch to GitHub issues for new bugs only. Bugs filed
in bugzilla that are
still active will be updated there until they are
closed.  This means that over
time the number of active bugs in bugzilla will slowly
decrease as bugs are closed
out.  Then at some point in the future, all of the bugs
from bugzilla will be archived
into their 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Richard Smith via lldb-dev]

On Tue, 21 Apr 2020 at 11:04, Philip Reames via cfe-dev <
cfe-...@lists.llvm.org> wrote:

> +1 to James's take
>
> I'd prefer simplicity of implementation over perfection here.
>
If we end up with two different bug numbering systems, that's a problem
that we will be paying for for many years. It's worth some investment now
to avoid that problem. And it doesn't seem like it really requires much
investment.

Here's another path we could take:

1) Fork the llvm repository to a private "bugs" repository. Mirror the
bugzilla issues there. Iterate until we're happy, as per James's proposal.
2) Sync the forked repository to the llvm repository, delete the llvm
repository, rename "bugs" to "llvm", and make it public.

Then we'll have the first N bugs in llvm-project/llvm being *exactly* the
bugzilla bugs, and we'll have excised the existing github issues that we
want to pretend never existed anyway.


I think we've missed an important step in the planning here: we've not
agreed on a set of goals for the transition. Here are mine:

 * We end up with one single issue tracking system containing all issues,
both old and new, both open and closed.
 * All links and references to existing bugs still work.
 * We have a single bug numbering system covering all bugs, and old bugs
retain their numbers.

It sounds like we don't all agree that the last point is important, but if
we can achieve it without any significant additional cost, why not do so?

> Philip
> On 4/20/20 4:08 PM, James Y Knight via llvm-dev wrote:
>
> In a previous discussion, one other suggestion had been to migrate all the
> bugzilla bugs to a separate initially-private "bug archive" repository in
> github. This has a few benefits:
> 1. If the migration is messed up, the repo can be deleted, and the process
> run again, until we get a result we like.
> 2. The numbering can be fully-controlled.
> Once the bugs are migrated to *some* github repository, individual issues
> can then be "moved" between repositories, and github will redirect from the
> movefrom-repository's bug to the target repository's bug.
>
> We could also just have llvm.org/PR###  be the
> url only for legacy bugzilla issue numbers -- and have it use a file
> listing the mappings of bugzilla id -> github id to generate the redirects.
> (GCC just did this recently for svn revision number redirections,
> https://gcc.gnu.org/pipermail/gcc/2020-April/232030.html).
>
> Then we could introduce a new naming scheme for github issue shortlinks.
>
> On Mon, Apr 20, 2020 at 3:50 PM Richard Smith via llvm-dev <
> llvm-...@lists.llvm.org> wrote:
>
>> On Mon, 20 Apr 2020 at 12:31, Tom Stellard via llvm-dev <
>> llvm-...@lists.llvm.org> wrote:
>>
>>> Hi,
>>>
>>> I wanted to continue discussing the plan to migrate from Bugzilla to
>>> Github.
>>> It was suggested that I start a new thread and give a summary of the
>>> proposal
>>> and what has changed since it was originally proposed in October.
>>>
>>> == Here is the original proposal:
>>>
>>> http://lists.llvm.org/pipermail/llvm-dev/2019-October/136162.html
>>>
>>> == What has changed:
>>>
>>> * You will be able to subscribe to notifications for a specific issue
>>>   labels.  We have a proof of concept notification system using github
>>> actions
>>>   that will be used for this.
>>>
>>> * Emails will be sent to llvm-bugs when issues are opened or closed.
>>>
>>> * We have the initial list of labels:
>>> https://github.com/llvm/llvm-project/labels
>>>
>>> == Remaining issue:
>>>
>>> * There is one remaining issue that I don't feel we have consensus on,
>>> and that is what to do with bugs in the existing bugzilla.  Here are
>>> some options
>>> that we have discussed:
>>>
>>> 1. Switch to GitHub issues for new bugs only.  Bugs filed in bugzilla
>>> that are
>>> still active will be updated there until they are closed.  This means
>>> that over
>>> time the number of active bugs in bugzilla will slowly decrease as bugs
>>> are closed
>>> out.  Then at some point in the future, all of the bugs from bugzilla
>>> will be archived
>>> into their own GitHub repository that is separate from the llvm-project
>>> repo.
>>>
>>> 2. Same as 1, but also create a migration script that would allow anyone
>>> to
>>> manually migrate an active bug from bugzilla to a GitHub issue in the
>>> llvm-project
>>> repo.  The intention with this script is that it would be used to
>>> migrate high-traffic
>>> or important bugs from bugzilla to GitHub to help increase the
>>> visibility of the bug.
>>> This would not be used for mass migration of all the bugs.
>>>
>>> 3. Do a mass bug migration from bugzilla to GitHub and enable GitHub
>>> issues at the same time.
>>> Closed or inactive bugs would be archived into their own GitHub
>>> repository, and active bugs
>>> would be migrated to the llvm-project repo.
>>>
>>
>> Can we preserve the existing bug numbers if we migrate this way? There
>> are lots of references to "PRx" in checked in LLVM 

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Fangrui Song via lldb-dev]

On 2020-04-20, Richard Smith via cfe-dev wrote:

On Mon, 20 Apr 2020 at 13:57, Anton Korobeynikov via cfe-dev <
cfe-...@lists.llvm.org> wrote:


> If we are reasonably certain that no one would be opening new issues on
GitHub while the migration is running...
And pull requests (the numbering is common for issues and pull
requests) as well. And we cannot disable pull requests at all. And I'm
afraid the issues will need to be opened as well during the migration.
And now the real problem: should an "extra" pull request or issue
intervene in the migration there is no way to "reset" the counter
besides deleting the project and creating it once again. We could only
sacrifice some bugzilla issues to restore the numbering...



We can edit summaries and comments, presumably by API calls as well as by
the web UI. So how about this:

Step 1: Preallocate sufficient github issue numbers. Make a bot account and
with it file empty placeholder issues until we have N issues total, where N
is the number of bugzilla bugs.
Step 2: Shut down the ability to file new bugzilla bugs, double-check we
have enough placeholder bugs, then open up github for new issue
submissions. Change llvm.org/PRx to redirect to github for x's
higher than the transition point.
Step 3a: For each issue filed by the bot account, sync that bug's contents
to the bugzilla entry: copy across the subject, all the comments, tags, and
so on.
Step 3b: For each issue not filed by the bot account whose number is that
of a bugzilla entry, file a new github bug and sync it with the bugzilla
bug then add a tracking comment to the github bug saying what the new
number is.

All we would need to do before we switch systems is steps 1 and 2. The data
import can be done incrementally with no time constraints. Once transition
is complete, we can change llvm.org/PRx (and existing bugzilla links,
since I'm sure they exist in the wild too) to redirect to github, and shut
down our bugzilla instance.

If new PRs are filed during step 1, we get new "problem" issue numbers,
which we deal with in the same way as the 210 existing github issue numbers.


Not sure about the API throttling, ideally we can create a new
repository, write a script and allocate a github issue X for each
bugzilla issue X in some repository. We can then gradually copy comments
to that repository.

After contents are migrated, add a redirector llvm.org/PRx -> 
https://github.com/llvm/some-repository/issue/x
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Richard Smith via lldb-dev]

On Mon, 20 Apr 2020 at 13:57, Anton Korobeynikov via cfe-dev <
cfe-...@lists.llvm.org> wrote:

> > If we are reasonably certain that no one would be opening new issues on
> GitHub while the migration is running...
> And pull requests (the numbering is common for issues and pull
> requests) as well. And we cannot disable pull requests at all. And I'm
> afraid the issues will need to be opened as well during the migration.
> And now the real problem: should an "extra" pull request or issue
> intervene in the migration there is no way to "reset" the counter
> besides deleting the project and creating it once again. We could only
> sacrifice some bugzilla issues to restore the numbering...
>

We can edit summaries and comments, presumably by API calls as well as by
the web UI. So how about this:

Step 1: Preallocate sufficient github issue numbers. Make a bot account and
with it file empty placeholder issues until we have N issues total, where N
is the number of bugzilla bugs.
Step 2: Shut down the ability to file new bugzilla bugs, double-check we
have enough placeholder bugs, then open up github for new issue
submissions. Change llvm.org/PRx to redirect to github for x's
higher than the transition point.
Step 3a: For each issue filed by the bot account, sync that bug's contents
to the bugzilla entry: copy across the subject, all the comments, tags, and
so on.
Step 3b: For each issue not filed by the bot account whose number is that
of a bugzilla entry, file a new github bug and sync it with the bugzilla
bug then add a tracking comment to the github bug saying what the new
number is.

All we would need to do before we switch systems is steps 1 and 2. The data
import can be done incrementally with no time constraints. Once transition
is complete, we can change llvm.org/PRx (and existing bugzilla links,
since I'm sure they exist in the wild too) to redirect to github, and shut
down our bugzilla instance.

If new PRs are filed during step 1, we get new "problem" issue numbers,
which we deal with in the same way as the 210 existing github issue numbers.

-- 
> With best regards, Anton Korobeynikov
> Department of Statistical Modelling, Saint Petersburg State University
> ___
> cfe-dev mailing list
> cfe-...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Anton Korobeynikov via lldb-dev]

> If we are reasonably certain that no one would be opening new issues on 
> GitHub while the migration is running...
And pull requests (the numbering is common for issues and pull
requests) as well. And we cannot disable pull requests at all. And I'm
afraid the issues will need to be opened as well during the migration.
And now the real problem: should an "extra" pull request or issue
intervene in the migration there is no way to "reset" the counter
besides deleting the project and creating it once again. We could only
sacrifice some bugzilla issues to restore the numbering...


-- 
With best regards, Anton Korobeynikov
Department of Statistical Modelling, Saint Petersburg State University
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [cfe-dev] [llvm-dev] RFC: Switching from Bugzilla to Github Issues [UPDATED]

[Richard Smith via lldb-dev]

210 issues have been filed on github so far. That's negligible compared to
the total number we have, so a minor additional effort for those seems
acceptable if we can't actually clean them out and reuse the numbers.

So suppose we start with bugzilla issue #211 and migrate the issues to
github one at a time, in order. That would preserve the existing bug
numbering and all existing bugs, other than those first 210. For those 210,
I'd suggest we file new issues on github, and add comments to github issue
1-210 indicating they've been migrated and the new issue number. (I'd be
inclined to delete as much contents from those issues as possible and
retain only the redirect to the new number.)

Would that work?

On Mon, 20 Apr 2020 at 13:33, Anton Korobeynikov via cfe-dev <
cfe-...@lists.llvm.org> wrote:

> Just to clarify a bit: what I wanted to say is that it's unlikely
> that we will be able to ensure that bugzilla issue numbers after
> migration would coincide with github issue numbers. And therefore
> proper mapping will be necessary. And this mapping would be more
> complex than just rewriting the URL.
>
>
> On Mon, Apr 20, 2020 at 11:25 PM Anton Korobeynikov
>  wrote:
> >
> > > Can we preserve the existing bug numbers if we migrate this way? There
> are lots of references to "PRx" in checked in LLVM artifacts and
> elsewhere in the world, as well as links to llvm.org/PRx, and if we
> can preserve all the issue numbers this would ease the transition pain
> substantially.
> > Well... I hate to say this, but quite unlikely. Unfortunately, there
> > were significant changes in GitHub opensource team and these days they
> > are much less responsive than they used to be during our github
> > migration. I asked this question several times, and unfortunately,
> > there is no answer. I will certainly keep trying.
> >
> > The problem here is there is no way to assign / control issue numbers
> > at all. They are just automatically assigned in sequential order.
> > While it might be possible to utilize this while migrating everything
> > to, say, a special archive project on GitHub, we will not be able to
> > control the numbers assigned should we migrate the issues one-by-one
> > or just move from archive to main project.
> >
> > So, the only viable way seems to be plain big mapping from bugzilla to
> > github issue numbers without anything simple like "llvm.org/PRxx
> > becomes https://github.com/llvm/llvm-project/issues/xx;.
> >
> >
> > --
> > With best regards, Anton Korobeynikov
> > Department of Statistical Modelling, Saint Petersburg State University
>
>
>
> --
> With best regards, Anton Korobeynikov
> Department of Statistical Modelling, Saint Petersburg State University
> ___
> cfe-dev mailing list
> cfe-...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev