Thank you, Steve.
Sounds like something I can get started on. Forgive me but I am brand new to the infrastructure used for this project. Is patch_tested.txt generated by the build system or by jenkins-admin? If I'm modifying the format of it, would I need to modify something else besides jenkins-admin? It doesn't look like it writes the file, only reads it. Could you give an example filter URL that is passed to jenkins-admin? I tried various options but couldn't get it to actually call into Jenkins to start any builds (or even print that it was doing since I was not running it in live mode) Thanks, Neal ________________________________ From: Steve Rowe <[email protected]> Sent: Wednesday, July 24, 2019 9:41:50 AM To: [email protected] <[email protected]> Subject: Re: patch review for github PRs Hi Neal, Patches welcome! IMHO the ideal way to choose patch vs. Github PR, if there are both, is to choose the most recent. I'm not sure if timestamp data is available, though. As I understand the problem, the modifications to jenkins-admin.py will be: 1. Modify the format of patch_tested.txt (used by jenkins-admin.py to remember the latest attachment IDs per JIRA issue) to include Github PR references (currently it's just Issue-ID/attachment-ID pairs). 2. Trigger downstream Jenkins jobs using the correct parameters for Github PRs. I'm not sure if this will require modifications to downstream jobs like PreCommit-SOLR-Build and PreCommit-LUCENE-Build. Looking at the build script here https://builds.apache.org/job/PreCommit-SOLR-Build/configure<https://urldefense.proofpoint.com/v2/url?u=https-3A__builds.apache.org_job_PreCommit-2DSOLR-2DBuild_configure&d=DwMFAg&c=KoC5GYBOIefzxGAm2j6cjFf-Gz7ANghQIP9aFG9DuBs&r=x9rnLNSDhF07vxiNUsNpD5aoDlyqlf6gUyQSHurAkS4&m=BJxBfA0aB7kfgE1gcw5jg_3tRwWU-YgAZ4LHZBWN96s&s=6GbONw7GPAV4E6sYXVU3BiobNq-i9Iq3Bcgq9c7GCDc&e=> : when it invokes Yetus's test-patch script, it ignores the $ATTACHMENT_ID parameter currently sent with the build request by jenkins-admin.py, so maybe Yetus's test-patch script will auto-detect Github PRs without any addition effort? (Since it currently auto-detects which attached patch to use in some way.) One wrinkle here: the Yetus version used by PreCommit-SOLR-Build and PreCommit-LUCENE-Build is pinned at v0.7.0, but the latest Yetus version is 0.10.0 . We should almost certainly upgrade the Yetus version used by those jobs as part of this effort. Related: https://issues.apache.org/jira/browse/LUCENE-8515<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_LUCENE-2D8515&d=DwMFAg&c=KoC5GYBOIefzxGAm2j6cjFf-Gz7ANghQIP9aFG9DuBs&r=x9rnLNSDhF07vxiNUsNpD5aoDlyqlf6gUyQSHurAkS4&m=BJxBfA0aB7kfgE1gcw5jg_3tRwWU-YgAZ4LHZBWN96s&s=nUGJQxgtKSbd4o37zHVgdUkgZS8ERYuxOj3bGnfUI60&e=> If you'd like to start work on this, please create a JIRA issue. At a minimum, a YETUS issue will be required for the jenkins-admin.py changes. Thanks, Steve On Jul 24, 2019, at 12:24 PM, Sidhwaney, Neal B <[email protected]<mailto:[email protected]>> wrote: Hello, I can try to take a look at this if nobody else has the time right now. >From a brief glance at the code, it looks like the right thing to do would to >detect Github PR links in addition to the existing attachment logic, and send >those JIRA issues to test-jira. If I understand the email below, it should >prefer the Github PR without having to do anything special? Thank you, Neal ________________________________ From: Steve Rowe <[email protected]<mailto:[email protected]>> Sent: Friday, July 19, 2019 4:01 PM To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Re: patch review for github PRs Hi Mike, I don't think so, based on this SOLR-10912 comment[1] from Allen Wittenauer, who works on Yetus: Github PR support is sort of there. test-patch does. It can take either a github PR directly on the command line or passed via a JIRA. If it gets told to test a JIRA that references a github PR, it will defer to the PR as the source of the patch. In other words, if a JIRA issue references a github PR and has a patch attached, it will use the github PR and ignore the attachments. However! The job on Jenkins that feeds test-patch is NOT github aware. The original version was built before github integration existed. To make matters worse, that code was locked away in a repository no one really had access to modify. As of a month or so ago, that code is now part of Apache Yetus ( https://github.com/apache/yetus/blob/master/precommit/jenkins/jenkins-admin.py<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_yetus_blob_master_precommit_jenkins_jenkins-2Dadmin.py&d=DwMFAg&c=KoC5GYBOIefzxGAm2j6cjFf-Gz7ANghQIP9aFG9DuBs&r=x9rnLNSDhF07vxiNUsNpD5aoDlyqlf6gUyQSHurAkS4&m=7hQz5S2_nAO_957FdHkWCWJK92guxs5gFEPFLYti6EY&s=M-7rPbjwKa9Oka7DQXUInjcKz4gYh34u-bmfHstZoiM&e=> ), so there is an opportunity for us to fix this problem and add better asf<->github integration. jenkins-admin.py has moved here: https://github.com/apache/yetus/blob/master/precommit/src/main/python/jenkins-admin.py<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_yetus_blob_master_precommit_src_main_python_jenkins-2Dadmin.py&d=DwMFAg&c=KoC5GYBOIefzxGAm2j6cjFf-Gz7ANghQIP9aFG9DuBs&r=x9rnLNSDhF07vxiNUsNpD5aoDlyqlf6gUyQSHurAkS4&m=7hQz5S2_nAO_957FdHkWCWJK92guxs5gFEPFLYti6EY&s=0FNvQz5M1KyWRu6jQVlj1cVlcstCINbIffwDAvWhy-k&e=> and has not been modified since to handle Github PRs. Every 10 minutes, ASF Jenkins job https://builds.apache.org/job/PreCommit-Admin/<https://urldefense.proofpoint.com/v2/url?u=https-3A__builds.apache.org_job_PreCommit-2DAdmin_&d=DwMFAg&c=KoC5GYBOIefzxGAm2j6cjFf-Gz7ANghQIP9aFG9DuBs&r=x9rnLNSDhF07vxiNUsNpD5aoDlyqlf6gUyQSHurAkS4&m=7hQz5S2_nAO_957FdHkWCWJK92guxs5gFEPFLYti6EY&s=giDjZDMBUEMUFqxsWkkd90SyMRod2dUjTKYL0mnkeKk&e=> invokes jenkins-admin.py, which triggers builds on these Jenkins jobs for LUCENE and SOLR issues, respectively: https://builds.apache.org/view/L/view/Lucene/job/PreCommit-LUCENE-Build<https://urldefense.proofpoint.com/v2/url?u=https-3A__builds.apache.org_view_L_view_Lucene_job_PreCommit-2DLUCENE-2DBuild&d=DwMFAg&c=KoC5GYBOIefzxGAm2j6cjFf-Gz7ANghQIP9aFG9DuBs&r=x9rnLNSDhF07vxiNUsNpD5aoDlyqlf6gUyQSHurAkS4&m=7hQz5S2_nAO_957FdHkWCWJK92guxs5gFEPFLYti6EY&s=MopyfiPOq4IE9b5-km1Haqi8YX-Nf77sz8yp1ijEaFc&e=> and https://builds.apache.org/view/L/view/Lucene/job/PreCommit-SOLR-Build<https://urldefense.proofpoint.com/v2/url?u=https-3A__builds.apache.org_view_L_view_Lucene_job_PreCommit-2DSOLR-2DBuild&d=DwMFAg&c=KoC5GYBOIefzxGAm2j6cjFf-Gz7ANghQIP9aFG9DuBs&r=x9rnLNSDhF07vxiNUsNpD5aoDlyqlf6gUyQSHurAkS4&m=7hQz5S2_nAO_957FdHkWCWJK92guxs5gFEPFLYti6EY&s=PS6mjqyqHViVZ5J0Ct98Cp7dEqUIf7B8F1bmRLj-vTk&e=> . [1] https://issues.apache.org/jira/browse/SOLR-10912?focusedCommentId=16380775&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16380775<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SOLR-2D10912-3FfocusedCommentId-3D16380775-26page-3Dcom.atlassian.jira.plugin.system.issuetabpanels-3Acomment-2Dtabpanel-23comment-2D16380775&d=DwMFAg&c=KoC5GYBOIefzxGAm2j6cjFf-Gz7ANghQIP9aFG9DuBs&r=x9rnLNSDhF07vxiNUsNpD5aoDlyqlf6gUyQSHurAkS4&m=7hQz5S2_nAO_957FdHkWCWJK92guxs5gFEPFLYti6EY&s=3wK4HRWQ4n8kI6iosLVLEbDv8iB0VxwU2ViNLf6cXSg&e=> -- Steve On Jul 19, 2019, at 6:20 PM, Michael Sokolov <[email protected]<mailto:[email protected]>> wrote: Is there a way to have Yetus build a PR as we can do with patch files? --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected]<mailto:[email protected]> For additional commands, e-mail: [email protected]<mailto:[email protected]> ________________________________ ________________________________ This message is intended for the sole use of the addressee, and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you are not the addressee you are hereby notified that you may not use, copy, disclose, or distribute to anyone the message or any information contained in the message. If you have received this message in error, please immediately advise the sender by reply email and delete this message.
