I started to play around with the (unofficial) Issue Import API
<https://gist.github.com/jonmagic/5282384165e0f86ef105> and so far so good.
- Original timestamps (created, updated) are preserved
- An issue and its comments can be imported with one post request;
importing is done asynchronously but the background job looks sufficiently
fast.
- Notifications are not triggered with mentions in the issue comments
(e.g. https://github.com/mocobeta/sandbox-lucene-10557/issues/249)

I will do more tests and rewrite the scripts with this API and if it works
as expected, try to migrate whole Jira issues into a test repo next time;
it still needs two-pass migration but could be done in a few hours with the
default rate limit.
Is there anybody who kindly provides a github account to make sure that
"any notifications are never triggered" when executing migration? I
confirmed this with my account, just wanted to test it with other one or
two accounts.

Tomoko

2022年6月24日(金) 21:12 Adrien Grand <jpou...@gmail.com>:

> FWIW I don't mind receiving these one-time notifications for the purpose
> of the migration.
>
> On Fri, Jun 24, 2022 at 12:58 PM Tomoko Uchida <
> tomoko.uchida.1...@gmail.com> wrote:
>
>> > If you could give me some "example mail", I can add a "feed to trash"
>> sieve rule before you start it.
>>
>> If we decide to migrate all Jira issues to GItHub with reporter/author
>> account re-mapping (hyperlinks), and we still cannot find a workaround
>> not to trigger notifications, I'll let you all know the example mail
>> text.
>> I can also trigger some real notifications by simulating a few
>> migrations for people who wish to know what will happen in their
>> mailbox/github UI beforehand. I'm really not sure what will happen to
>> everyone by triggering so many notifications.
>>
>> Precisely speaking, you will receive the same number of notifications
>> to your all activities in Jira so far: reporting an issue, assigning
>> yourself to an issue, and leaving a comment.
>> GitHub UI and mailers will aggregate them into "issues", I think.
>>
>> > Another question: With the alternative API is it possible to keep
>> original Reporters and Authors of comments, too?
>>
>> I haven't tried the unofficial API yet, but as far as I read the
>> examples, it is not possible to keep original reporters or authors.
>> You cannot change/tweak GitHub issue reporter or author by any means -
>> it looks a sane decision to me.
>>
>>
>> 2022年6月24日(金) 16:18 Uwe Schindler <u...@thetaphi.de>:
>> >
>> > Hi,
>> >
>> > Am 23.06.2022 um 19:09 schrieb Tomoko Uchida:
>> >
>> > uschindler,Uwe Schindler,2474
>> >
>> > If you could give me some "example mail", I can add a "feed to trash"
>> sieve rule before you start it.
>> >
>> > We have 109158 resources (10608 issues + 98550 comments) in total. With
>> the default rate limit of 5000 reqs per hour, it will take 22 hours if
>> there are no failures/retries. With an enterprise account, it will take 7-8
>> hours I think.
>> >
>> > So a sieve rule would be good to not get constant mail all the time,
>> because my 2474 issues (or notifications?) will be distrubuted over the
>> whole day.
>> >
>> > Another question: With the alternative API is it possible to keep
>> original Reporters and Authors of comments, too?
>> >
>> > Uwe
>> >
>> > Tomoko
>> >
>> >
>> > 2022年6月24日(金) 0:39 Tomoko Uchida <tomoko.uchida.1...@gmail.com>:
>> >>
>> >> It seems not new - I don't figure out why this is not published as a
>> public API yet but according to the comments there, it could be
>> buggy/unstable (still worth a try to me).
>> >>
>> >>
>> >>
>> >> 2022年6月24日(金) 0:26 Tomoko Uchida <tomoko.uchida.1...@gmail.com>:
>> >>>
>> >>> I just browsed through this article about the "import issues" API
>> (looks pretty new and on technical preview status?)
>> >>> https://gist.github.com/jonmagic/5282384165e0f86ef105
>> >>>
>> >>> Seems it solves many of our considerations - preserving original
>> timestamp, bulk importing issue comments, and not triggering notifications.
>> >>>
>> >>> I'll try it later; thank you Rob for providing the information.
>> >>>
>> >>>
>> >>> 2022年6月23日(木) 23:18 Michael Sokolov <msoko...@gmail.com>:
>> >>>>
>> >>>> oh phew! glad to hear this was expected
>> >>>>
>> >>>> On Thu, Jun 23, 2022 at 10:17 AM Tomoko Uchida
>> >>>> <tomoko.uchida.1...@gmail.com> wrote:
>> >>>> >
>> >>>> > > Many comments were lost in the transfer. The last one in the
>> copy is
>> >>>> > > only about 1/4 of the way through this gigantic issue. This
>> really is
>> >>>> > > a blocker I think.
>> >>>> >
>> >>>> > I limited the number of comments per issue up to ten for testing.
>> We can migrate literally all comments - one by one.
>> >>>> >
>> >>>> > Tomoko
>> >>>> >
>> >>>> >
>> >>>> > 2022年6月23日(木) 23:09 Tomoko Uchida <tomoko.uchida.1...@gmail.com>:
>> >>>> >>
>> >>>> >> Hi,
>> >>>> >> I have little now to carefully read/respond to all replies right
>> now, but just wanted to answer this.
>> >>>> >>
>> >>>> >> > Will it be possible to preserve links from issues -> pull
>> requests?
>> >>>> >>
>> >>>> >> Yes it's a bit cumbersome (and it could be difficult to make sure
>> that all links to PRs are covered - it's not solid metadata, you need to
>> parse github bot's comments) but surely possible.
>> >>>> >> See https://github.com/mocobeta/sandbox-lucene-10557/issues/188.
>> >>>> >>
>> >>>> >> As for notifications and attachment files, if there are ways to
>> manage these it'd be great.
>> >>>> >>
>> >>>> >>
>> >>>> >> 2022年6月23日(木) 22:59 Dawid Weiss <dawid.we...@gmail.com>:
>> >>>> >>>
>> >>>> >>>
>> >>>> >>> Interesting, thanks Rob. I see the attachments have been ported
>> in that article as well - something the official API is not able to do.
>> >>>> >>>
>> >>>> >>> https://jira.spring.io/browse/DATACMNS-617?redirect=false
>> >>>> >>>
>> https://github.com/spring-projects/spring-data-commons/issues/1080
>> >>>> >>>
>> >>>> >>> Dawid
>> >>>> >>>
>> >>>> >>> On Thu, Jun 23, 2022 at 3:26 PM Rob Audenaerde <
>> rob.audenae...@gmail.com> wrote:
>> >>>> >>>>
>> >>>> >>>> I didn't read the entire thread, so apologies if this is a
>> duplicate:
>> >>>> >>>>
>> >>>> >>>> Did you check
>> https://spring.io/blog/2021/01/07/spring-data-s-migration-from-jira-to-github-issues
>> >>>> >>>>
>> >>>> >>>> They especially write there is an api that doesn't trigger
>> notifications.
>> >>>> >>>>
>> >>>> >>>> It is documented here:
>> https://gist.github.com/jonmagic/5282384165e0f86ef105
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>> On Thu, Jun 23, 2022, 15:14 Michael Sokolov <msoko...@gmail.com>
>> wrote:
>> >>>> >>>>>
>> >>>> >>>>> Yes thank you! You say this is not difficult, but it looks
>> like a big
>> >>>> >>>>> job to me! Here are a bunch of things I noticed that we would
>> ideally
>> >>>> >>>>> address (from looking at one long and complex issue,
>> LUCENE-9004). I
>> >>>> >>>>> wouldn't be so bold as to say these should block us from
>> proceeding if
>> >>>> >>>>> they're not addressed, just want to point out there is
>> potentially a
>> >>>> >>>>> lot to do:
>> >>>> >>>>>
>> >>>> >>>>> Will it be possible to preserve links from issues -> pull
>> requests?
>> >>>> >>>>> That seems like one of the most important pieces of "metadata".
>> >>>> >>>>>
>> >>>> >>>>> As far as attached files go, I see you seem to have made an
>> attempt?
>> >>>> >>>>> There is a link in
>> https://issues.apache.org/jira/browse/LUCENE-9004
>> >>>> >>>>> where you had posted a picture of a graph, for example; in
>> >>>> >>>>> https://github.com/mocobeta/sandbox-lucene-10557/issues/171
>> it is
>> >>>> >>>>> represented as a link. When you click on the link you get an
>> error
>> >>>> >>>>> though. I wonder if it would be possible to link back to the
>> images
>> >>>> >>>>> hosted in JIRA? (Ideally as an <IMG> tag; otherwise a link
>> would be
>> >>>> >>>>> good).
>> >>>> >>>>>
>> >>>> >>>>> I agree with Ryan - I'd be willing to bulk-delete 1000
>> notifications
>> >>>> >>>>> if it means preserving hyperlinks to people
>> >>>> >>>>>
>> >>>> >>>>> Numbered list formatting became giant bold text (see the
>> comment
>> >>>> >>>>> containing "Here's a strategy")
>> >>>> >>>>>
>> >>>> >>>>> Many comments were lost in the transfer. The last one in the
>> copy is
>> >>>> >>>>> only about 1/4 of the way through this gigantic issue. This
>> really is
>> >>>> >>>>> a blocker I think. I wonder what happened? Maybe some API
>> calls failed
>> >>>> >>>>> and we need to retry???
>> >>>> >>>>>
>> >>>> >>>>> I wanted to check other fancy formatting (tables, block
>> comments, code
>> >>>> >>>>> blocks, etc) but haven't looked at these yet...
>> >>>> >>>>>
>> >>>> >>>>> On Wed, Jun 22, 2022 at 8:34 AM Ryan Ernst <r...@iernst.net>
>> wrote:
>> >>>> >>>>> >
>> >>>> >>>>> > This is great work Tomoko! A couple minor thoughts:
>> >>>> >>>>> >
>> >>>> >>>>> > * I don’t think a flood of notifications from the import is
>> a problem. It’s a one time hassle, and having the actual user links is nice
>> for GitHub’s cross linking system.
>> >>>> >>>>> >
>> >>>> >>>>> > * Do you have an estimate for how many api calls are needed?
>> How many total issues+comments exist in jira? I assume the limits you dealt
>> with were the default 5k requests per hour. If that will take too long, we
>> could consider using a user from an enterprise account which has 3x the
>> limit.
>> >>>> >>>>> >
>> >>>> >>>>> > On Tue, Jun 21, 2022 at 15:56 Tomoko Uchida <
>> tomoko.uchida.1...@gmail.com> wrote:
>> >>>> >>>>> >>
>> >>>> >>>>> >> Hi all,
>> >>>> >>>>> >> again - this is about GitHub migration.
>> >>>> >>>>> >>
>> >>>> >>>>> >> We have a large disagreement on whether we should migrate
>> existing Jira issues (including all closed issues) to GitHub or not.
>> >>>> >>>>> >>
>> >>>> >>>>> >> I drafted a tiny migration tool [1] to see how it looks if
>> we move Jira issues to GitHub, and tried to migrate a small portion of Jira
>> issues/comments to a test repo. You can see it here:
>> >>>> >>>>> >> - Closed issues list
>> https://github.com/mocobeta/sandbox-lucene-10557/issues?q=is%3Aissue+is%3Aclosed
>> >>>> >>>>> >> - Unresolved issues list:
>> https://github.com/mocobeta/sandbox-lucene-10557/issues
>> >>>> >>>>> >>
>> >>>> >>>>> >> I don't deserve to have a strong opinion on how we should
>> treat 20+ years of history so I would reserve my opinion - would the
>> prototype be of some help to have a conversation?
>> >>>> >>>>> >> I have to leave for a while, I'd be glad if you have a talk
>> on it while I'm away and hopefully reach an agreement.
>> >>>> >>>>> >>
>> >>>> >>>>> >> Here's a summary of what can be done.
>> >>>> >>>>> >>
>> >>>> >>>>> >> You can:
>> >>>> >>>>> >> * migrate all texts in issue descriptions and comments to
>> GitHub; browsing/searching old issues should work fine.
>> >>>> >>>>> >> * extract every issue metadata from Jira and port it to
>> labels or issue descriptions (as plain text).
>> >>>> >>>>> >> * map Jira cross-issue link "LUCENE-xxx" to GitHub issue
>> mention "#yyy".
>> >>>> >>>>> >>    * see this example:
>> https://github.com/mocobeta/sandbox-lucene-10557/issues/218
>> >>>> >>>>> >> * map Jira user ids to GitHub accounts if the mapping is
>> given.
>> >>>> >>>>> >> * convert Jira markups to Markdown with parser library.
>> >>>> >>>>> >>    * not perfect - there can be many conversion errors
>> >>>> >>>>> >>
>> >>>> >>>>> >> And here are the limitations. (Please correct me if I'm
>> missing something.)
>> >>>> >>>>> >>
>> >>>> >>>>> >> You cannot:
>> >>>> >>>>> >> * simulate original authors and timestamps; they have to be
>> preserved in free-text forms.
>> >>>> >>>>> >> * migrate attached files (patches, images, etc.) to GitHub;
>> these have to remain in Jira.
>> >>>> >>>>> >>    * it's not allowed to programmatically upload files and
>> attach them to issues.
>> >>>> >>>>> >> * create hyperlinks from issues to GitHub accounts
>> (reporters, comment authors, etc.) by mentions; otherwise everyone will
>> receive a huge volume of notifications.
>> >>>> >>>>> >>    * still accounts can be noted with a markup `@xxxx`
>> (without mentioning) in their right place
>> >>>> >>>>> >> * "bulk" import issues/comments. Each resource has to be
>> posted one by one. Migration would take many hours (perhaps days?) due to
>> the severe API call rate limit.
>> >>>> >>>>> >>
>> >>>> >>>>> >> It's not a particularly difficult task, however, there will
>> be other uncontrollable things I haven't noticed yet.
>> >>>> >>>>> >>
>> >>>> >>>>> >> [1]
>> https://github.com/mocobeta/sandbox-lucene-10557/tree/main/migration
>> >>>> >>>>> >>
>> >>>> >>>>> >> Tomoko
>> >>>> >>>>>
>> >>>> >>>>>
>> ---------------------------------------------------------------------
>> >>>> >>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> >>>> >>>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>> >>>> >>>>>
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> >>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>> >>>>
>> > --
>> > Uwe Schindler
>> > Achterdiek 19, D-28357 Bremen
>> > https://www.thetaphi.de
>> > eMail: u...@thetaphi.de
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>
> --
> Adrien
>

Reply via email to