Joel, I’ve reverted your SSL changes to smokeTestRelease.py, and I don’t see any problems so far when I run it with my Python 3.4.2 with the 6.5.1RC2 URL. (Still in progress; Lucene distributions have downloaded so far.)
What was the error that caused you to make the SSL changes to smokeTestRelease.py? And what version of Python are you using? -- Steve www.lucidworks.com > On Apr 20, 2017, at 1:01 PM, Steve Rowe <[email protected]> wrote: > > I have Python 3.4.2 on my Linux box, and apparently 3.4.3 is the first that > supports the changes you made, so the smoke tester is failing for me: > > ----- > File "dev-tools/scripts/smokeTestRelease.py", line 165, in load > content = urllib.request.urlopen(urlString, > context=ctx).read().decode('utf-8') > TypeError: urlopen() got an unexpected keyword argument ‘context’ > ------ > > I’ll install a newer Python. > > -- > Steve > www.lucidworks.com > >> On Apr 20, 2017, at 12:51 PM, Joel Bernstein <[email protected]> wrote: >> >> thanks >> >> Joel Bernstein >> http://joelsolr.blogspot.com/ >> >> On Thu, Apr 20, 2017 at 12:50 PM, Steve Rowe <[email protected]> wrote: >> I’ll take a look. >> >> -- >> Steve >> www.lucidworks.com >> >>> On Apr 20, 2017, at 12:46 PM, Joel Bernstein <[email protected]> wrote: >>> >>> I've uploaded the RC2 here >>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-6.5.1-RC2-revdfc0dd67f7fe45fbf62d88652952ec3e2b3e0070/. >>> >>> But the smoke test breaks with SSL error in different spot when you smoke >>> test from this URL. It's failing with a different command this time so the >>> fix I used in other sports of smoke tester doesn't work. >>> >>> Steve, would you mind taking a look at this and pushing out a fix? >>> >>> Joel Bernstein >>> http://joelsolr.blogspot.com/ >>> >>> On Thu, Apr 20, 2017 at 10:29 AM, Steve Rowe <[email protected]> wrote: >>> I was thinking that if buildAndPushRelease.py can identify problems early, >>> the RM won’t have to later redo work when a problem is found, e.g. by the >>> smoke tester. IMHO the extra 10 minutes (on my laptop anyway) from >>> auto-running precommit would be worth it. >>> >>> But I agree that the smoke tester should be heavy, since it’s possible that >>> the RM had weird local conditions that weren’t caught by the release >>> production tools. >>> >>> -- >>> Steve >>> www.lucidworks.com >>> >>>> On Apr 20, 2017, at 10:23 AM, Joel Bernstein <[email protected]> wrote: >>>> >>>> They are both quite long processes. Perhaps we should look at eliminating >>>> some of the overlap? >>>> >>>> Since the smokeTesters is run by those voting on the release maybe this >>>> should be the heavier of the two. >>>> >>>> Joel Bernstein >>>> http://joelsolr.blogspot.com/ >>>> >>>> On Thu, Apr 20, 2017 at 10:00 AM, Steve Rowe <[email protected]> wrote: >>>> Also, I think buildAndPushRelease.py should invoke “precommit" too, (near) >>>> where it runs 'ant clean test'. >>>> >>>> -- >>>> Steve >>>> www.lucidworks.com >>>> >>>>> On Apr 20, 2017, at 9:51 AM, Steve Rowe <[email protected]> wrote: >>>>> >>>>> That’s bizarre. >>>>> >>>>> The smoke tester runs ‘ant validate’, which does *not* include >>>>> "documentation-lint” (which includes "-ecj-javadoc-lint”, where unused >>>>> imports are looked for). >>>>> >>>>> By contrast, ‘ant precommit’ runs “check-working-copy”, “validate”, and >>>>> “documentation-lint”. >>>>> >>>>> Jenkins caught this not because of the smoke tester, but rather because >>>>> the “nightly-smoke” task it uses runs “-jenkins-base”, which, like >>>>> “precommit”, runs “check-working-copy”, “validate”, and >>>>> “documentation-lint” (among others). >>>>> >>>>> I think we should change the smoke tester to look at the same things as >>>>> precommit, >>>>> >>>>> -- >>>>> Steve >>>>> www.lucidworks.com >>>>> >>>>>> On Apr 20, 2017, at 9:29 AM, Joel Bernstein <[email protected]> wrote: >>>>>> >>>>>> Smoke testes passed as well >>>>>> >>>>>> >>>>>> Joel Bernstein >>>>>> http://joelsolr.blogspot.com/ >>>>>> >>>>>> On Thu, Apr 20, 2017 at 9:21 AM, Steve Rowe <[email protected]> wrote: >>>>>> I guess precommit is only run as part of the smoke tester. >>>>>> >>>>>>> On Apr 20, 2017, at 9:20 AM, Joel Bernstein <[email protected]> wrote: >>>>>>> >>>>>>> Interesting the entire release process completed without complaining. >>>>>>> >>>>>>> Joel Bernstein >>>>>>> http://joelsolr.blogspot.com/ >>>>>>> >>>>>>> On Thu, Apr 20, 2017 at 9:10 AM, Steve Rowe <[email protected]> wrote: >>>>>>> The unused import was the only problem; I’ve removed it on branch_6_5. >>>>>>> >>>>>>> Sorry, hope I didn’t cause a delay in the release process. >>>>>>> >>>>>>> -- >>>>>>> Steve >>>>>>> www.lucidworks.com >>>>>>> >>>>>>>> On Apr 20, 2017, at 8:57 AM, Steve Rowe <[email protected]> wrote: >>>>>>>> >>>>>>>> Crap, Jenkins found an unused import in my commit on branch_6_5 :(. I >>>>>>>> forgot to run precommit. Doing so now. >>>>>>>> >>>>>>>> -- >>>>>>>> Steve >>>>>>>> www.lucidworks.com >>>>>>>> >>>>>>>>> On Apr 19, 2017, at 8:07 PM, Steve Rowe <[email protected]> wrote: >>>>>>>>> >>>>>>>>> Done. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Steve >>>>>>>>> www.lucidworks.com >>>>>>>>> >>>>>>>>>> On Apr 19, 2017, at 7:49 PM, Steve Rowe <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>> Woohoo! Thanks Joel. Running all tests on branch_6_5 now. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Steve >>>>>>>>>> www.lucidworks.com >>>>>>>>>> >>>>>>>>>>> On Apr 19, 2017, at 7:20 PM, Joel Bernstein <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> I'm OK with restarting RC2. Let me know when you finish the back >>>>>>>>>>> port. >>>>>>>>>>> >>>>>>>>>>> Joel Bernstein >>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>> >>>>>>>>>>> On Wed, Apr 19, 2017 at 7:11 PM, Steve Rowe <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> Joel, >>>>>>>>>>> >>>>>>>>>>> I know you’re in-process with RC2, but if you have to restart for >>>>>>>>>>> some reason, I’d like to include SOLR-10527. >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Steve >>>>>>>>>>> www.lucidworks.com >>>>>>>>>>> >>>>>>>>>>>> On Apr 19, 2017, at 6:57 PM, Joel Bernstein <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> I also had to change the smoke tester because of the SSL issue. >>>>>>>>>>>> Which in theory means everyone else will have to as well. I'm >>>>>>>>>>>> pushing out my changes to branch_6_5, so people can just update >>>>>>>>>>>> their branch before running the smoke tester. >>>>>>>>>>>> >>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Apr 19, 2017 at 5:02 PM, Joel Bernstein >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>> Ok, I had to push out three changes to the branch but I seemed to >>>>>>>>>>>> have moved passed this issue. >>>>>>>>>>>> >>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Apr 19, 2017 at 4:50 PM, Joel Bernstein >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>> I'll push out the changes and then revert. Hopefully the changes >>>>>>>>>>>> will do the trick after they've been pushed. >>>>>>>>>>>> >>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Apr 19, 2017 at 4:37 PM, Steve Rowe <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> Yeah, that is a sucky part of this: you want to make modifications >>>>>>>>>>>> on the release branch, but you can’t test unless you commit (and >>>>>>>>>>>> push). >>>>>>>>>>>> >>>>>>>>>>>> Another alternative: make modifications in another checkout, then >>>>>>>>>>>> invoke the script with from an unmodified source tree. >>>>>>>>>>>> >>>>>>>>>>>> e.g.: >>>>>>>>>>>> >>>>>>>>>>>> cd .. >>>>>>>>>>>> cp -r lucene-solr lucene-solr-2 # assuming lucene-solr is your >>>>>>>>>>>> checkout dir >>>>>>>>>>>> # edit lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py >>>>>>>>>>>> cd lucene-solr >>>>>>>>>>>> python3 -u >>>>>>>>>>>> ../lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py … >>>>>>>>>>>> >>>>>>>>>>>> (a shorter form of that is to make a copy the script alone and >>>>>>>>>>>> invoke the modified version from your checkout) >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Steve >>>>>>>>>>>> www.lucidworks.com >>>>>>>>>>>> >>>>>>>>>>>>> On Apr 19, 2017, at 4:32 PM, Joel Bernstein <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Ha, that doesn't work either. Now I get this error: >>>>>>>>>>>>> >>>>>>>>>>>>> RuntimeError: There are unpushed commits - "git log >>>>>>>>>>>>> origin/branch_6_5.." output is: >>>>>>>>>>>>> >>>>>>>>>>>>> So, I'm going to have to push this out. >>>>>>>>>>>>> >>>>>>>>>>>>> Let's first decide if this makes sense as the way forward. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Apr 19, 2017 at 4:28 PM, Joel Bernstein >>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>> Steve, when I change the script to turn off verification I get >>>>>>>>>>>>> the following error: >>>>>>>>>>>>> RuntimeError: git clone is dirty: >>>>>>>>>>>>> >>>>>>>>>>>>> So as part of the work around I think I'm going to have to commit >>>>>>>>>>>>> locally and then revert locally. Does that make sense to you as >>>>>>>>>>>>> the way forward? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Apr 19, 2017 at 4:24 PM, Alexandre Rafalovitch >>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>> Could it be anything to do with the fact that the signer is >>>>>>>>>>>>> Semantic >>>>>>>>>>>>> and they are being actively distrusted by - at least - Google: >>>>>>>>>>>>> http://www.securityweek.com/google-stops-trusting-symantec-issued-certificates >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Alex. >>>>>>>>>>>>> ---- >>>>>>>>>>>>> http://www.solr-start.com/ - Resources for Solr users, new and >>>>>>>>>>>>> experienced >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 19 April 2017 at 16:22, Joel Bernstein <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> Ok, I'll turn off the cert verification. I wasn't sure if cert >>>>>>>>>>>>>> verification >>>>>>>>>>>>>> was something that was integral to the process. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Apr 19, 2017 at 4:18 PM, Steve Rowe <[email protected]> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Joel, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Not sure why this is suddenly an issue - I guess “SNI” >>>>>>>>>>>>>>> <https://en.wikipedia.org/wiki/Server_Name_Indication> has been >>>>>>>>>>>>>>> enabled on >>>>>>>>>>>>>>> archive.apache.org? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Some useful info here (about a “requests” lib, AFAICT an >>>>>>>>>>>>>>> alternative to >>>>>>>>>>>>>>> urllib): >>>>>>>>>>>>>>> <http://docs.python-requests.org/en/master/community/faq/#what-are-hostname-doesn-t-match-errors> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Short term, you could turn off certificate verification. The >>>>>>>>>>>>>>> bottom >>>>>>>>>>>>>>> answer here uses the same lib as the script (urllib instead of >>>>>>>>>>>>>>> urllib2, >>>>>>>>>>>>>>> which is assumed in the other answers on the page), to turn off >>>>>>>>>>>>>>> certificate >>>>>>>>>>>>>>> verification: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> <http://stackoverflow.com/questions/19268548/python-ignore-certicate-validation-urllib2> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Steve >>>>>>>>>>>>>>> www.lucidworks.com >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Apr 19, 2017, at 3:43 PM, Joel Bernstein >>>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> It appears that it's failing on verifying the SSL cert for >>>>>>>>>>>>>>>> https://archive.apache.org/dist/lucene/java/. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Wed, Apr 19, 2017 at 3:36 PM, Joel Bernstein >>>>>>>>>>>>>>>> <[email protected]> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> I started working on 6.5.1 RC2. I ran the following: >>>>>>>>>>>>>>>> python3 -u dev-tools/scripts/buildAndPushRelease.py >>>>>>>>>>>>>>>> --push-local >>>>>>>>>>>>>>>> /tmp/releases/6.5.1 --rc-num 2 --sign EE64CB1E >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I believe this is same basic command I used for RC1. But this >>>>>>>>>>>>>>>> time I got >>>>>>>>>>>>>>>> a new error. At first I thought it might be incorrect keystore >>>>>>>>>>>>>>>> password but >>>>>>>>>>>>>>>> I changed it and it is correct. Any thoughts what the issue is? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Traceback (most recent call last): >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 313, in >>>>>>>>>>>>>>>> <module> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> main() >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 294, in >>>>>>>>>>>>>>>> main >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> rev = prepare(c.root, c.version, c.key_id, c.key_password) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 98, in >>>>>>>>>>>>>>>> prepare >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> checkDOAPfiles(version) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 143, in >>>>>>>>>>>>>>>> checkDOAPfiles >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> distpage = load(url) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 67, in >>>>>>>>>>>>>>>> load >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> content = >>>>>>>>>>>>>>>> urllib.request.urlopen(urlString).read().decode('utf-8') >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>>>> line 223, in urlopen >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> return opener.open(url, data, timeout) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>>>> line 526, in open >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> response = self._open(req, data) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>>>> line 544, in _open >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> '_open', req) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>>>> line 504, in _call_chain >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> result = func(*args) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>>>> line 1361, in https_open >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> context=self._context, check_hostname=self._check_hostname) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> File >>>>>>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", >>>>>>>>>>>>>>>> line 1320, in do_open >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> raise URLError(err) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> urllib.error.URLError: <urlopen error [SSL: >>>>>>>>>>>>>>>> CERTIFICATE_VERIFY_FAILED] >>>>>>>>>>>>>>>> certificate verify failed (_ssl.c:749)> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Joel Bernstein >>>>>>>>>>>>>>>> http://joelsolr.blogspot.com/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>> For additional commands, e-mail: [email protected] >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>> >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
