I was thinking that if buildAndPushRelease.py can identify problems early, the RM won’t have to later redo work when a problem is found, e.g. by the smoke tester. IMHO the extra 10 minutes (on my laptop anyway) from auto-running precommit would be worth it.
But I agree that the smoke tester should be heavy, since it’s possible that the RM had weird local conditions that weren’t caught by the release production tools. -- Steve www.lucidworks.com > On Apr 20, 2017, at 10:23 AM, Joel Bernstein <[email protected]> wrote: > > They are both quite long processes. Perhaps we should look at eliminating > some of the overlap? > > Since the smokeTesters is run by those voting on the release maybe this > should be the heavier of the two. > > Joel Bernstein > http://joelsolr.blogspot.com/ > > On Thu, Apr 20, 2017 at 10:00 AM, Steve Rowe <[email protected]> wrote: > Also, I think buildAndPushRelease.py should invoke “precommit" too, (near) > where it runs 'ant clean test'. > > -- > Steve > www.lucidworks.com > > > On Apr 20, 2017, at 9:51 AM, Steve Rowe <[email protected]> wrote: > > > > That’s bizarre. > > > > The smoke tester runs ‘ant validate’, which does *not* include > > "documentation-lint” (which includes "-ecj-javadoc-lint”, where unused > > imports are looked for). > > > > By contrast, ‘ant precommit’ runs “check-working-copy”, “validate”, and > > “documentation-lint”. > > > > Jenkins caught this not because of the smoke tester, but rather because the > > “nightly-smoke” task it uses runs “-jenkins-base”, which, like “precommit”, > > runs “check-working-copy”, “validate”, and “documentation-lint” (among > > others). > > > > I think we should change the smoke tester to look at the same things as > > precommit, > > > > -- > > Steve > > www.lucidworks.com > > > >> On Apr 20, 2017, at 9:29 AM, Joel Bernstein <[email protected]> wrote: > >> > >> Smoke testes passed as well > >> > >> > >> Joel Bernstein > >> http://joelsolr.blogspot.com/ > >> > >> On Thu, Apr 20, 2017 at 9:21 AM, Steve Rowe <[email protected]> wrote: > >> I guess precommit is only run as part of the smoke tester. > >> > >>> On Apr 20, 2017, at 9:20 AM, Joel Bernstein <[email protected]> wrote: > >>> > >>> Interesting the entire release process completed without complaining. > >>> > >>> Joel Bernstein > >>> http://joelsolr.blogspot.com/ > >>> > >>> On Thu, Apr 20, 2017 at 9:10 AM, Steve Rowe <[email protected]> wrote: > >>> The unused import was the only problem; I’ve removed it on branch_6_5. > >>> > >>> Sorry, hope I didn’t cause a delay in the release process. > >>> > >>> -- > >>> Steve > >>> www.lucidworks.com > >>> > >>>> On Apr 20, 2017, at 8:57 AM, Steve Rowe <[email protected]> wrote: > >>>> > >>>> Crap, Jenkins found an unused import in my commit on branch_6_5 :(. I > >>>> forgot to run precommit. Doing so now. > >>>> > >>>> -- > >>>> Steve > >>>> www.lucidworks.com > >>>> > >>>>> On Apr 19, 2017, at 8:07 PM, Steve Rowe <[email protected]> wrote: > >>>>> > >>>>> Done. > >>>>> > >>>>> -- > >>>>> Steve > >>>>> www.lucidworks.com > >>>>> > >>>>>> On Apr 19, 2017, at 7:49 PM, Steve Rowe <[email protected]> wrote: > >>>>>> > >>>>>> Woohoo! Thanks Joel. Running all tests on branch_6_5 now. > >>>>>> > >>>>>> -- > >>>>>> Steve > >>>>>> www.lucidworks.com > >>>>>> > >>>>>>> On Apr 19, 2017, at 7:20 PM, Joel Bernstein <[email protected]> > >>>>>>> wrote: > >>>>>>> > >>>>>>> I'm OK with restarting RC2. Let me know when you finish the back port. > >>>>>>> > >>>>>>> Joel Bernstein > >>>>>>> http://joelsolr.blogspot.com/ > >>>>>>> > >>>>>>> On Wed, Apr 19, 2017 at 7:11 PM, Steve Rowe <[email protected]> wrote: > >>>>>>> Joel, > >>>>>>> > >>>>>>> I know you’re in-process with RC2, but if you have to restart for > >>>>>>> some reason, I’d like to include SOLR-10527. > >>>>>>> > >>>>>>> -- > >>>>>>> Steve > >>>>>>> www.lucidworks.com > >>>>>>> > >>>>>>>> On Apr 19, 2017, at 6:57 PM, Joel Bernstein <[email protected]> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>> I also had to change the smoke tester because of the SSL issue. > >>>>>>>> Which in theory means everyone else will have to as well. I'm > >>>>>>>> pushing out my changes to branch_6_5, so people can just update > >>>>>>>> their branch before running the smoke tester. > >>>>>>>> > >>>>>>>> Joel Bernstein > >>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>> > >>>>>>>> On Wed, Apr 19, 2017 at 5:02 PM, Joel Bernstein <[email protected]> > >>>>>>>> wrote: > >>>>>>>> Ok, I had to push out three changes to the branch but I seemed to > >>>>>>>> have moved passed this issue. > >>>>>>>> > >>>>>>>> Joel Bernstein > >>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>> > >>>>>>>> On Wed, Apr 19, 2017 at 4:50 PM, Joel Bernstein <[email protected]> > >>>>>>>> wrote: > >>>>>>>> I'll push out the changes and then revert. Hopefully the changes > >>>>>>>> will do the trick after they've been pushed. > >>>>>>>> > >>>>>>>> Joel Bernstein > >>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>> > >>>>>>>> On Wed, Apr 19, 2017 at 4:37 PM, Steve Rowe <[email protected]> wrote: > >>>>>>>> Yeah, that is a sucky part of this: you want to make modifications > >>>>>>>> on the release branch, but you can’t test unless you commit (and > >>>>>>>> push). > >>>>>>>> > >>>>>>>> Another alternative: make modifications in another checkout, then > >>>>>>>> invoke the script with from an unmodified source tree. > >>>>>>>> > >>>>>>>> e.g.: > >>>>>>>> > >>>>>>>> cd .. > >>>>>>>> cp -r lucene-solr lucene-solr-2 # assuming lucene-solr is your > >>>>>>>> checkout dir > >>>>>>>> # edit lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py > >>>>>>>> cd lucene-solr > >>>>>>>> python3 -u ../lucene-solr-2/dev-tools/scripts/buildAndPushRelease.py > >>>>>>>> … > >>>>>>>> > >>>>>>>> (a shorter form of that is to make a copy the script alone and > >>>>>>>> invoke the modified version from your checkout) > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Steve > >>>>>>>> www.lucidworks.com > >>>>>>>> > >>>>>>>>> On Apr 19, 2017, at 4:32 PM, Joel Bernstein <[email protected]> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> Ha, that doesn't work either. Now I get this error: > >>>>>>>>> > >>>>>>>>> RuntimeError: There are unpushed commits - "git log > >>>>>>>>> origin/branch_6_5.." output is: > >>>>>>>>> > >>>>>>>>> So, I'm going to have to push this out. > >>>>>>>>> > >>>>>>>>> Let's first decide if this makes sense as the way forward. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Joel Bernstein > >>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>> > >>>>>>>>> On Wed, Apr 19, 2017 at 4:28 PM, Joel Bernstein > >>>>>>>>> <[email protected]> wrote: > >>>>>>>>> Steve, when I change the script to turn off verification I get the > >>>>>>>>> following error: > >>>>>>>>> RuntimeError: git clone is dirty: > >>>>>>>>> > >>>>>>>>> So as part of the work around I think I'm going to have to commit > >>>>>>>>> locally and then revert locally. Does that make sense to you as the > >>>>>>>>> way forward? > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Joel Bernstein > >>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>> > >>>>>>>>> On Wed, Apr 19, 2017 at 4:24 PM, Alexandre Rafalovitch > >>>>>>>>> <[email protected]> wrote: > >>>>>>>>> Could it be anything to do with the fact that the signer is Semantic > >>>>>>>>> and they are being actively distrusted by - at least - Google: > >>>>>>>>> http://www.securityweek.com/google-stops-trusting-symantec-issued-certificates > >>>>>>>>> > >>>>>>>>> Regards, > >>>>>>>>> Alex. > >>>>>>>>> ---- > >>>>>>>>> http://www.solr-start.com/ - Resources for Solr users, new and > >>>>>>>>> experienced > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On 19 April 2017 at 16:22, Joel Bernstein <[email protected]> > >>>>>>>>> wrote: > >>>>>>>>>> Ok, I'll turn off the cert verification. I wasn't sure if cert > >>>>>>>>>> verification > >>>>>>>>>> was something that was integral to the process. > >>>>>>>>>> > >>>>>>>>>> Joel Bernstein > >>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>> > >>>>>>>>>> On Wed, Apr 19, 2017 at 4:18 PM, Steve Rowe <[email protected]> > >>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Hi Joel, > >>>>>>>>>>> > >>>>>>>>>>> Not sure why this is suddenly an issue - I guess “SNI” > >>>>>>>>>>> <https://en.wikipedia.org/wiki/Server_Name_Indication> has been > >>>>>>>>>>> enabled on > >>>>>>>>>>> archive.apache.org? > >>>>>>>>>>> > >>>>>>>>>>> Some useful info here (about a “requests” lib, AFAICT an > >>>>>>>>>>> alternative to > >>>>>>>>>>> urllib): > >>>>>>>>>>> <http://docs.python-requests.org/en/master/community/faq/#what-are-hostname-doesn-t-match-errors> > >>>>>>>>>>> > >>>>>>>>>>> Short term, you could turn off certificate verification. The > >>>>>>>>>>> bottom > >>>>>>>>>>> answer here uses the same lib as the script (urllib instead of > >>>>>>>>>>> urllib2, > >>>>>>>>>>> which is assumed in the other answers on the page), to turn off > >>>>>>>>>>> certificate > >>>>>>>>>>> verification: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> <http://stackoverflow.com/questions/19268548/python-ignore-certicate-validation-urllib2> > >>>>>>>>>>> > >>>>>>>>>>> -- > >>>>>>>>>>> Steve > >>>>>>>>>>> www.lucidworks.com > >>>>>>>>>>> > >>>>>>>>>>>> On Apr 19, 2017, at 3:43 PM, Joel Bernstein <[email protected]> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> It appears that it's failing on verifying the SSL cert for > >>>>>>>>>>>> https://archive.apache.org/dist/lucene/java/. > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Joel Bernstein > >>>>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>>>> > >>>>>>>>>>>> On Wed, Apr 19, 2017 at 3:36 PM, Joel Bernstein > >>>>>>>>>>>> <[email protected]> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>> I started working on 6.5.1 RC2. I ran the following: > >>>>>>>>>>>> python3 -u dev-tools/scripts/buildAndPushRelease.py --push-local > >>>>>>>>>>>> /tmp/releases/6.5.1 --rc-num 2 --sign EE64CB1E > >>>>>>>>>>>> > >>>>>>>>>>>> I believe this is same basic command I used for RC1. But this > >>>>>>>>>>>> time I got > >>>>>>>>>>>> a new error. At first I thought it might be incorrect keystore > >>>>>>>>>>>> password but > >>>>>>>>>>>> I changed it and it is correct. Any thoughts what the issue is? > >>>>>>>>>>>> > >>>>>>>>>>>> Traceback (most recent call last): > >>>>>>>>>>>> > >>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 313, in > >>>>>>>>>>>> <module> > >>>>>>>>>>>> > >>>>>>>>>>>> main() > >>>>>>>>>>>> > >>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 294, in > >>>>>>>>>>>> main > >>>>>>>>>>>> > >>>>>>>>>>>> rev = prepare(c.root, c.version, c.key_id, c.key_password) > >>>>>>>>>>>> > >>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 98, in > >>>>>>>>>>>> prepare > >>>>>>>>>>>> > >>>>>>>>>>>> checkDOAPfiles(version) > >>>>>>>>>>>> > >>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 143, in > >>>>>>>>>>>> checkDOAPfiles > >>>>>>>>>>>> > >>>>>>>>>>>> distpage = load(url) > >>>>>>>>>>>> > >>>>>>>>>>>> File "dev-tools/scripts/buildAndPushRelease.py", line 67, in load > >>>>>>>>>>>> > >>>>>>>>>>>> content = > >>>>>>>>>>>> urllib.request.urlopen(urlString).read().decode('utf-8') > >>>>>>>>>>>> > >>>>>>>>>>>> File > >>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", > >>>>>>>>>>>> line 223, in urlopen > >>>>>>>>>>>> > >>>>>>>>>>>> return opener.open(url, data, timeout) > >>>>>>>>>>>> > >>>>>>>>>>>> File > >>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", > >>>>>>>>>>>> line 526, in open > >>>>>>>>>>>> > >>>>>>>>>>>> response = self._open(req, data) > >>>>>>>>>>>> > >>>>>>>>>>>> File > >>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", > >>>>>>>>>>>> line 544, in _open > >>>>>>>>>>>> > >>>>>>>>>>>> '_open', req) > >>>>>>>>>>>> > >>>>>>>>>>>> File > >>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", > >>>>>>>>>>>> line 504, in _call_chain > >>>>>>>>>>>> > >>>>>>>>>>>> result = func(*args) > >>>>>>>>>>>> > >>>>>>>>>>>> File > >>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", > >>>>>>>>>>>> line 1361, in https_open > >>>>>>>>>>>> > >>>>>>>>>>>> context=self._context, check_hostname=self._check_hostname) > >>>>>>>>>>>> > >>>>>>>>>>>> File > >>>>>>>>>>>> "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", > >>>>>>>>>>>> line 1320, in do_open > >>>>>>>>>>>> > >>>>>>>>>>>> raise URLError(err) > >>>>>>>>>>>> > >>>>>>>>>>>> urllib.error.URLError: <urlopen error [SSL: > >>>>>>>>>>>> CERTIFICATE_VERIFY_FAILED] > >>>>>>>>>>>> certificate verify failed (_ssl.c:749)> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Joel Bernstein > >>>>>>>>>>>> http://joelsolr.blogspot.com/ > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> --------------------------------------------------------------------- > >>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> --------------------------------------------------------------------- > >>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>> For additional commands, e-mail: [email protected] > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: [email protected] > >>> For additional commands, e-mail: [email protected] > >>> > >>> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
