Joel, I’ve created <https://issues.apache.org/jira/browse/LUCENE-7793> and <https://issues.apache.org/jira/browse/LUCENE-7794> to add precommit checking (or the equivalent) to buildAndPushRelease.py and smokeTestRelease.py, respectively.
-- Steve www.lucidworks.com > On Apr 20, 2017, at 10:29 AM, Steve Rowe <[email protected]> wrote: > > I was thinking that if buildAndPushRelease.py can identify problems early, > the RM won’t have to later redo work when a problem is found, e.g. by the > smoke tester. IMHO the extra 10 minutes (on my laptop anyway) from > auto-running precommit would be worth it. > > But I agree that the smoke tester should be heavy, since it’s possible that > the RM had weird local conditions that weren’t caught by the release > production tools. > > -- > Steve > www.lucidworks.com > >> On Apr 20, 2017, at 10:23 AM, Joel Bernstein <[email protected]> wrote: >> >> They are both quite long processes. Perhaps we should look at eliminating >> some of the overlap? >> >> Since the smokeTesters is run by those voting on the release maybe this >> should be the heavier of the two. >> >> Joel Bernstein >> http://joelsolr.blogspot.com/ >> >> On Thu, Apr 20, 2017 at 10:00 AM, Steve Rowe <[email protected]> wrote: >> Also, I think buildAndPushRelease.py should invoke “precommit" too, (near) >> where it runs 'ant clean test'. >> >> -- >> Steve >> www.lucidworks.com >> >>> On Apr 20, 2017, at 9:51 AM, Steve Rowe <[email protected]> wrote: >>> >>> That’s bizarre. >>> >>> The smoke tester runs ‘ant validate’, which does *not* include >>> "documentation-lint” (which includes "-ecj-javadoc-lint”, where unused >>> imports are looked for). >>> >>> By contrast, ‘ant precommit’ runs “check-working-copy”, “validate”, and >>> “documentation-lint”. >>> >>> Jenkins caught this not because of the smoke tester, but rather because the >>> “nightly-smoke” task it uses runs “-jenkins-base”, which, like “precommit”, >>> runs “check-working-copy”, “validate”, and “documentation-lint” (among >>> others). >>> >>> I think we should change the smoke tester to look at the same things as >>> precommit, >>> >>> -- >>> Steve >>> www.lucidworks.com >>> >>>> On Apr 20, 2017, at 9:29 AM, Joel Bernstein <[email protected]> wrote: >>>> >>>> Smoke testes passed as well >>>> >>>> >>>> Joel Bernstein >>>> http://joelsolr.blogspot.com/ >>>> >>>> On Thu, Apr 20, 2017 at 9:21 AM, Steve Rowe <[email protected]> wrote: >>>> I guess precommit is only run as part of the smoke tester. >>>> >>>>> On Apr 20, 2017, at 9:20 AM, Joel Bernstein <[email protected]> wrote: >>>>> >>>>> Interesting the entire release process completed without complaining. >>>>> >>>>> Joel Bernstein >>>>> http://joelsolr.blogspot.com/ >>>>> >>>>> On Thu, Apr 20, 2017 at 9:10 AM, Steve Rowe <[email protected]> wrote: >>>>> The unused import was the only problem; I’ve removed it on branch_6_5. >>>>> >>>>> Sorry, hope I didn’t cause a delay in the release process. >>>>> >>>>> -- >>>>> Steve >>>>> www.lucidworks.com >>>>> >>>>>> On Apr 20, 2017, at 8:57 AM, Steve Rowe <[email protected]> wrote: >>>>>> >>>>>> Crap, Jenkins found an unused import in my commit on branch_6_5 :(. I >>>>>> forgot to run precommit. Doing so now. >>>>>> >>>>>> -- >>>>>> Steve >>>>>> www.lucidworks.com >>>>>> >>>>>>> On Apr 19, 2017, at 8:07 PM, Steve Rowe <[email protected]> wrote: >>>>>>> >>>>>>> Done. >>>>>>> >>>>>>> -- >>>>>>> Steve >>>>>>> www.lucidworks.com >>>>>>> >>>>>>>> On Apr 19, 2017, at 7:49 PM, Steve Rowe <[email protected]> wrote: >>>>>>>> >>>>>>>> Woohoo! Thanks Joel. Running all tests on branch_6_5 now. >>>>>>>> >>>>>>>> -- >>>>>>>> Steve >>>>>>>> www.lucidworks.com >>>>>>>> >>>>>>>>> On Apr 19, 2017, at 7:20 PM, Joel Bernstein <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> I'm OK with restarting RC2. Let me know when you finish the back port. >>>>>>>>> >>>>>>>>> Joel Bernstein >>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>> >>>>>>>>> On Wed, Apr 19, 2017 at 7:11 PM, Steve Rowe <[email protected]> wrote: >>>>>>>>> Joel, >>>>>>>>> >>>>>>>>> I know you’re in-process with RC2, but if you have to restart for >>>>>>>>> some reason, I’d like to include SOLR-10527. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Steve >>>>>>>>> www.lucidworks.com >>>>>>>>> >>>>>>>>>> On Apr 19, 2017, at 6:57 PM, Joel Bernstein <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> I also had to change the smoke tester because of the SSL issue. >>>>>>>>>> Which in theory means everyone else will have to as well. I'm >>>>>>>>>> pushing out my changes to branch_6_5, so people can just update >>>>>>>>>> their branch before running the smoke tester. >>>>>>>>>> >>>>>>>>>> Joel Bernstein >>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>> >>>>>>>>>> On Wed, Apr 19, 2017 at 5:02 PM, Joel Bernstein <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> Ok, I had to push out three changes to the branch but I seemed to >>>>>>>>>> have moved passed this issue. >>>>>>>>>> >>>>>>>>>> Joel Bernstein >>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>> >>>>>>>>>> On Wed, Apr 19, 2017 at 4:50 PM, Joel Bernstein <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> I'll push out the changes and then revert. Hopefully the changes >>>>>>>>>> will do the trick after they've been pushed. >>>>>>>>>> >>>>>>>>>> Joel Bernstein >>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>> >>>>>>>>>> On Wed, Apr 19, 2017 at 4:37 PM, Steve Rowe <[email protected]> wrote: >>>>>>>>>> Yeah, that is a sucky part of this: you want to make modifications >>>>>>>>>> on the release branch, but you can’t test unless you commit (and >>>>>>>>>> push). >>>>>>>>>> >>>>>>>>>> Another alternative: make modifications in another checkout, then >>>>>>>>>> invoke the script with from an unmodified source tree. >>>>>>>>>> >>>>>>>>>> e.g.: >>>>>>>>>> >>>>>>>>>> cd .. >>>>>>>>>> cp -r lucene-solr lucene-solr-2 # assuming lucene-solr is your >>>>>>>>>> checkout dir >>>>>>>>>> # edit lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py >>>>>>>>>> cd lucene-solr >>>>>>>>>> python3 -u ../lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py >>>>>>>>>> … >>>>>>>>>> >>>>>>>>>> (a shorter form of that is to make a copy the script alone and >>>>>>>>>> invoke the modified version from your checkout) >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Steve >>>>>>>>>> www.lucidworks.com >>>>>>>>>> >>>>>>>>>>> On Apr 19, 2017, at 4:32 PM, Joel Bernstein <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Ha, that doesn't work either. Now I get this error: >>>>>>>>>>> >>>>>>>>>>> RuntimeError: There are unpushed commits - "git log >>>>>>>>>>> origin/branch_6_5.." output is: >>>>>>>>>>> >>>>>>>>>>> So, I'm going to have to push this out. >>>>>>>>>>> >>>>>>>>>>> Let's first decide if this makes sense as the way forward. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Joel Bernstein >>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>> >>>>>>>>>>> On Wed, Apr 19, 2017 at 4:28 PM, Joel Bernstein >>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> Steve, when I change the script to turn off verification I get the >>>>>>>>>>> following error: >>>>>>>>>>> RuntimeError: git clone is dirty: >>>>>>>>>>> >>>>>>>>>>> So as part of the work around I think I'm going to have to commit >>>>>>>>>>> locally and then revert locally. Does that make sense to you as the >>>>>>>>>>> way forward? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Joel Bernstein >>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>> >>>>>>>>>>> On Wed, Apr 19, 2017 at 4:24 PM, Alexandre Rafalovitch >>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> Could it be anything to do with the fact that the signer is Semantic >>>>>>>>>>> and they are being actively distrusted by - at least - Google: >>>>>>>>>>> http://www.securityweek.com/google-stops-trusting-symantec-issued-certificates >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Alex. >>>>>>>>>>> ---- >>>>>>>>>>> http://www.solr-start.com/ - Resources for Solr users, new and >>>>>>>>>>> experienced >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 19 April 2017 at 16:22, Joel Bernstein <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>> Ok, I'll turn off the cert verification. I wasn't sure if cert >>>>>>>>>>>> verification >>>>>>>>>>>> was something that was integral to the process. >>>>>>>>>>>> >>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Apr 19, 2017 at 4:18 PM, Steve Rowe <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Joel, >>>>>>>>>>>>> >>>>>>>>>>>>> Not sure why this is suddenly an issue - I guess “SNI” >>>>>>>>>>>>> <https://en.wikipedia.org/wiki/Server_Name_Indication> has been >>>>>>>>>>>>> enabled on >>>>>>>>>>>>> archive.apache.org? >>>>>>>>>>>>> >>>>>>>>>>>>> Some useful info here (about a “requests” lib, AFAICT an >>>>>>>>>>>>> alternative to >>>>>>>>>>>>> urllib): >>>>>>>>>>>>> <http://docs.python-requests.org/en/master/community/faq/#what-are-hostname-doesn-t-match-errors> >>>>>>>>>>>>> >>>>>>>>>>>>> Short term, you could turn off certificate verification. The >>>>>>>>>>>>> bottom >>>>>>>>>>>>> answer here uses the same lib as the script (urllib instead of >>>>>>>>>>>>> urllib2, >>>>>>>>>>>>> which is assumed in the other answers on the page), to turn off >>>>>>>>>>>>> certificate >>>>>>>>>>>>> verification: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> <http://stackoverflow.com/questions/19268548/python-ignore-certicate-validation-urllib2> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Steve >>>>>>>>>>>>> www.lucidworks.com >>>>>>>>>>>>> >>>>>>>>>>>>>> On Apr 19, 2017, at 3:43 PM, Joel Bernstein <[email protected]> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> It appears that it's failing on verifying the SSL cert for >>>>>>>>>>>>>> https://archive.apache.org/dist/lucene/java/. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Apr 19, 2017 at 3:36 PM, Joel Bernstein >>>>>>>>>>>>>> <[email protected]> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> I started working on 6.5.1 RC2. I ran the following: >>>>>>>>>>>>>> python3 -u dev-tools/scripts/buildAndPushRelease.py --push-local >>>>>>>>>>>>>> /tmp/releases/6.5.1 --rc-num 2 --sign EE64CB1E >>>>>>>>>>>>>> >>>>>>>>>>>>>> I believe this is same basic command I used for RC1. But this >>>>>>>>>>>>>> time I got >>>>>>>>>>>>>> a new error. At first I thought it might be incorrect keystore >>>>>>>>>>>>>> password but >>>>>>>>>>>>>> I changed it and it is correct. Any thoughts what the issue is? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Traceback (most recent call last): >>>>>>>>>>>>>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 313, in >>>>>>>>>>>>>> <module> >>>>>>>>>>>>>> >>>>>>>>>>>>>> main() >>>>>>>>>>>>>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 294, in >>>>>>>>>>>>>> main >>>>>>>>>>>>>> >>>>>>>>>>>>>> rev = prepare(c.root, c.version, c.key_id, c.key_password) >>>>>>>>>>>>>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 98, in >>>>>>>>>>>>>> prepare >>>>>>>>>>>>>> >>>>>>>>>>>>>> checkDOAPfiles(version) >>>>>>>>>>>>>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 143, in >>>>>>>>>>>>>> checkDOAPfiles >>>>>>>>>>>>>> >>>>>>>>>>>>>> distpage = load(url) >>>>>>>>>>>>>> >>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 67, in load >>>>>>>>>>>>>> >>>>>>>>>>>>>> content = >>>>>>>>>>>>>> urllib.request.urlopen(urlString).read().decode('utf-8') >>>>>>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>> line 223, in urlopen >>>>>>>>>>>>>> >>>>>>>>>>>>>> return opener.open(url, data, timeout) >>>>>>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>> line 526, in open >>>>>>>>>>>>>> >>>>>>>>>>>>>> response = self._open(req, data) >>>>>>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>> line 544, in _open >>>>>>>>>>>>>> >>>>>>>>>>>>>> '_open', req) >>>>>>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>> line 504, in _call_chain >>>>>>>>>>>>>> >>>>>>>>>>>>>> result = func(*args) >>>>>>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>> line 1361, in https_open >>>>>>>>>>>>>> >>>>>>>>>>>>>> context=self._context, check_hostname=self._check_hostname) >>>>>>>>>>>>>> >>>>>>>>>>>>>> File >>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>> line 1320, in do_open >>>>>>>>>>>>>> >>>>>>>>>>>>>> raise URLError(err) >>>>>>>>>>>>>> >>>>>>>>>>>>>> urllib.error.URLError: <urlopen error [SSL: >>>>>>>>>>>>>> CERTIFICATE_VERIFY_FAILED] >>>>>>>>>>>>>> certificate verify failed (_ssl.c:749)> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
