Seems fixed but looks it starts to leave duplicated PR links for some recent JIRAs. Not a big deal but are they being ran in multiple places maybe?
For instance, https://issues.apache.org/jira/browse/SPARK-25579 https://issues.apache.org/jira/browse/SPARK-25574 https://issues.apache.org/jira/browse/SPARK-25564 2018년 9월 17일 (월) 오후 10:09, Ilan Filonenko <ilan....@gmail.com>님이 작성: > Same over here: > > https://issues.apache.org/jira/browse/SPARK-25291 / > https://github.com/apache/spark/pull/22415 > > On Sun, Sep 16, 2018 at 10:09 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: > >> Seems same thing is happening again. >> >> For instance, >> - https://issues.apache.org/jira/browse/SPARK-25440 / >> https://github.com/apache/spark/pull/22429 >> - https://issues.apache.org/jira/browse/SPARK-25429 / >> https://github.com/apache/spark/pull/22420 >> >> >> 2017년 8월 3일 (목) 오전 9:06, Hyukjin Kwon <gurwls...@gmail.com>님이 작성: >> >>> I think it works for anyone who can leave a web link and comment. >>> >>> >>> For "in progress" Resolution, it looks I am unable to manually set. >>> Please let me know if anyone knows. >>> >>> >>> For single JIRA, I manually modified the script to process single item >>> before. I guess you know what the script does but want to rather test, >>> right? >>> >>> This looks requiring Python JIRA client, which I installed by pip >>> install jira==1.0.7 and I personally tested (and just double checked this) >>> after commenting out JIRA related codes: >>> >>> >>> diff --git a/dev/github_jira_sync.py b/dev/github_jira_sync.py >>> index acc9aeabbb9..c8adf0c4ade 100755 >>> --- a/dev/github_jira_sync.py >>> +++ b/dev/github_jira_sync.py >>> @@ -101,8 +101,8 @@ def get_max_pr(): >>> return 0 >>> >>> >>> -jira_client = jira.client.JIRA({'server': JIRA_API_BASE}, >>> - basic_auth=(JIRA_USERNAME, >>> JIRA_PASSWORD)) >>> +# jira_client = jira.client.JIRA({'server': JIRA_API_BASE}, >>> +# basic_auth=(JIRA_USERNAME, >>> JIRA_PASSWORD)) >>> >>> jira_prs = get_jira_prs() >>> >>> @@ -124,14 +124,15 @@ for issue, pr in sorted(jira_prs, key=lambda kv: >>> int(kv[1]['number'])): >>> url = pr['html_url'] >>> title = "[Github] Pull Request #%s (%s)" % (pr['number'], >>> pr['user']['login']) >>> try: >>> - existing_links = map(lambda l: l.raw['object']['url'], >>> jira_client.remote_links(issue)) >>> + # existing_links = map(lambda l: l.raw['object']['url'], >>> jira_client.remote_links(issue)) >>> + pass >>> except: >>> print("Failure reading JIRA %s (does it exist?)" % issue) >>> print(sys.exc_info()[0]) >>> continue >>> >>> - if url in existing_links: >>> - continue >>> + # if url in existing_links: >>> + # continue >>> >>> icon = {"title": "Pull request #%s" % pr['number'], >>> "url16x16": "https://assets-cdn.github.com/favicon.ico"} >>> @@ -139,12 +140,12 @@ for issue, pr in sorted(jira_prs, key=lambda kv: >>> int(kv[1]['number'])): >>> # For all possible fields see: >>> # >>> https://developer.atlassian.com/display/JIRADEV/Fields+in+Remote+Issue+Links >>> # application = {"name": "Github pull requests", "type": >>> "org.apache.spark.jira.github"} >>> - jira_client.add_remote_link(issue, destination) >>> + # jira_client.add_remote_link(issue, destination) >>> >>> comment = "User '%s' has created a pull request for this issue:" % >>> pr['user']['login'] >>> comment += "\n%s" % pr['html_url'] >>> - if pr_num >= MIN_COMMENT_PR: >>> - jira_client.add_comment(issue, comment) >>> + # if pr_num >= MIN_COMMENT_PR: >>> + # jira_client.add_comment(issue, comment) >>> >>> print("Added link %s <-> PR #%s" % (issue, pr['number'])) >>> num_updates += 1 >>> >>> >>> and >>> >>> >>> rm .github-jira-max >>> ./dev/github_jira_sync.py >>> >>> >>> >>> 2017-08-03 5:20 GMT+09:00 Josh Rosen <joshro...@databricks.com>: >>> >>>> Usually the backend of https://spark-prs.appspot.com does the linking >>>> while processing PR update tasks. It appears that the site's connections to >>>> JIRA have started failing: >>>> >>>> ConnectionError: ('Connection aborted.', HTTPException('Deadline >>>> exceeded while waiting for HTTP response from URL: >>>> https://issues.apache.org/jira/rest/api/2/serverInfo',)) >>>> >>>> From Stackdriver's log-based metrics, I can spot that this problem >>>> started around July 24th. We're already using a much-higher-than-default >>>> URL fetch timeout, so it's possible that the problem is related to access >>>> credentials, IP blocks, outdated client libraries, or something else. >>>> >>>> On Wed, Aug 2, 2017 at 1:10 PM Bryan Cutler <cutl...@gmail.com> wrote: >>>> >>>>> Thanks Hyukjin! I didn't see your previous message.. It looks like >>>>> your manual run worked pretty well for the JIRAs I'm following, the only >>>>> thing is that it didn't mark them as "in progress", but that's not a big >>>>> deal. Otherwise that helps until we can find out why it's not doing this >>>>> automatically. I'm not familiar with that script, can anyone run it to >>>>> apply to a single JIRA they are working on? >>>>> >>>>> On Wed, Aug 2, 2017 at 12:09 PM, Hyukjin Kwon <gurwls...@gmail.com> >>>>> wrote: >>>>> >>>>>> I was wondering about this too.. >>>>>> >>>>>> >>>>>> Yes, actually, I have been manually adding some links by resembling >>>>>> the same steps in the script before. >>>>>> >>>>>> I was thinking it'd rather be nicer to run this manually once and >>>>>> then I ran this against single JIRA >>>>>> >>>>>> first - https://issues.apache.org/jira/browse/SPARK-21526 to show >>>>>> how it looks like and check if there >>>>>> >>>>>> is any issue or objection just in case. >>>>>> >>>>>> >>>>>> Will run this manually now once. I will revert all my action manually >>>>>> if there is any issue by doing this. >>>>>> >>>>>> >>>>>> 2017-08-03 3:50 GMT+09:00 Sean Owen <so...@cloudera.com>: >>>>>> >>>>>>> Hyukjin mentioned this here earlier today and had run it manually, >>>>>>> but yeah I'm not sure where it normally runs or why it hasn't. Shane not >>>>>>> sure if you're the person to ask? >>>>>>> >>>>>>> >>>>>>> On Wed, Aug 2, 2017 at 7:47 PM Bryan Cutler <cutl...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Devs, >>>>>>>> >>>>>>>> I've noticed a couple PRs recently have not been automatically >>>>>>>> linked to the related JIRAs. This was one of mine (I linked it >>>>>>>> manually) >>>>>>>> https://issues.apache.org/jira/browse/SPARK-21583, but I've seen >>>>>>>> it happen elsewhere. I think this is the script that does it, but it >>>>>>>> hasn't been changed recently >>>>>>>> https://github.com/apache/spark/blob/master/dev/github_jira_sync.py. >>>>>>>> Anyone else seen this or know what's going on? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Bryan >>>>>>>> >>>>>>> >>>>>> >>>>> >>>
