+1 (binding) for RC2 Thanks for looking into this, Jim and Bharath. I agree that the issues is likely environmental. There was no concrete evidence of an ASAN-related product bug, and ASAN exhaustive is known to be flaky: https://issues.apache.org/jira/browse/IMPALA-5218
I ran release tests of my own here: https://jenkins.impala.io/job/release-test/21/ On Fri, Sep 1, 2017 at 10:59 PM, Jim Apple <[email protected]> wrote: > Yeah, this jenkins.impala.io environment is pretty stable for DEBUG > builds with core test exploration, but ASAN builds are infrequently > done on it, so there might be a flaky test situation that is either > Jenkins-related or test-related, which doesn't directly implicate the > RC. > > I'm still OK with the +1 I cast above. > > On Fri, Sep 1, 2017 at 10:54 PM, Bharath Vissapragada > <[email protected]> wrote: > > Jim, thanks for testing the ASAN configuration. I tested it in my own > > environment (centos6) and it has passed. I'm suspecting that either > > something is flaky or probably related to the build environment. > > > > On Thu, Aug 31, 2017 at 11:23 PM, Jim Apple <[email protected]> > wrote: > > > >> I think ASAN doesn't work with FE tests - I mistakenly left them in > >> for this run. > >> > >> We have never before seen two jobs on the same worker in > jenkins.impala.io > >> . > >> > >> Here's an ASAN rerun that is failing other e2e tests: > >> https://jenkins.impala.io/view/Utility/job/ubuntu-16.04- > >> from-scratch/222/parameters/ > >> > >> With this one, because the machine is still up, you can see no other > >> builds have run on it: > >> https://jenkins.impala.io/computer/ubuntu-16.04%20(i- > >> 0c375c963694b400b)/builds > >> > >> exhaustive DEBUG and RELEASE runs passed. > >> > >> On Thu, Aug 31, 2017 at 10:27 PM, Alexander Behm < > [email protected]> > >> wrote: > >> > Thanks for testing, Jim. I looked into your build but could not > determine > >> > what happened. I found no ASAN output in the logs. It's interesting > that > >> > this test also fails with a crash: > >> > > >> > ERROR at teardown of TestGrantRevoke.test_role_update[exec_option: > >> > {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': > 0, > >> > 'disable_codegen': False, 'abort_on_error': 1, > >> > 'exec_single_node_rows_threshold': 0} | table_format: text/none] > >> > > >> > Looks like these two completely unrelated tests both failed in a > strange > >> > way. > >> > > >> > It's also interesting that many FE unit tests failed with > NoClassDefFound > >> > (very unusual). > >> > > >> > Could it be that two jobs were scheduled on the same worker? > >> Alternatively, > >> > maybe another job did not clean up after itself and your run landed > on an > >> > unclean workspace leading to problems? This smells like an infra > problem > >> to > >> > me, > >> > Maybe do another run? > >> > > >> > > >> > > >> > > >> > > >> > On Thu, Aug 31, 2017 at 8:02 PM, Jim Apple <[email protected]> > wrote: > >> > > >> >> This ASAN testing failed: > >> >> > >> >> https://jenkins.impala.io/view/Utility/job/ubuntu-16.04- > >> >> from-scratch/218/consoleFull > >> >> failed in query_test/test_udfs.py::TestUdfExecution::test_ir_ > >> >> functions[exec_option: > >> >> {'disable_codegen_rows_threshold': 0, 'disable_codegen': False, > >> >> 'exec_single_node_rows_threshold': 0, 'enable_expr_rewrites': > False} | > >> >> table_format: text/none]. Looks like a crash to me. > >> >> > >> >> I didn't see a corresponding bug. Has anyone else seen something like > >> >> this before? > >> >> > >> >> On Thu, Aug 31, 2017 at 1:15 PM, Jim Apple <[email protected]> > >> wrote: > >> >> > BTW, this Jenkins job includes the log of what it tested, which > >> >> > follows the Release Guide, so you should be able to follow along > OK. > >> >> > All committers should have access to run that job, too, if you > don't > >> >> > trust my result. > >> >> > > >> >> > I am also testing exhaustive (not just core) tests at > >> >> > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/, builds > >> >> > 218-220 (once the instances come up). > >> >> > > >> >> > I tested with ImpalaLZO at this commit: > >> >> > https://github.com/cloudera/impala-lzo/tree/ > >> >> 62c4b94ed6e89f0ce2068280864546ebccfb0729 > >> >> > > >> >> > On Thu, Aug 31, 2017 at 6:19 AM, Jim Apple <[email protected]> > >> wrote: > >> >> >> +1 > >> >> >> > >> >> >> https://jenkins.impala.io/job/release-test/20/console > >> >> >> > >> >> >> This tested following > >> >> >> https://cwiki.apache.org/confluence/display/IMPALA/How+ > >> >> to+load+and+run+Impala+tests > >> >> >> and https://cwiki.apache.org/confluence/display/IMPALA/How+ > >> >> to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate. > >> >> >> > >> >> >> On Wed, Aug 30, 2017 at 11:35 PM, Bharath Vissapragada > >> >> >> <[email protected]> wrote: > >> >> >>> This is a vote to release Impala 2.10.0. > >> >> >>> > >> >> >>> - The artefacts for testing can be downloaded from < > >> >> >>> https://dist.apache.org/repos/dist/dev/incubator/impala/2. > 10.0/RC2/ > >> > > >> >> >>> > >> >> >>> - The git tag for this release candidate is 2.10.0-rc2 and > treehash > >> is > >> >> >>> visible at > >> >> >>> < > >> >> >>> https://git-wip-us.apache.org/repos/asf?p=incubator-impala. > >> >> git;a=tree;hb=23d79462da5d0108709e8b1399c97606f4ebdf92 > >> >> >>>> > >> >> >>> > >> >> >>> Please vote +1 or -1. -1 votes should be accompanied by an > >> explanation > >> >> of > >> >> >>> the reason. Only PPMC members and mentors have binding votes, but > >> other > >> >> >>> community members are encouraged to cast non-binding votes. This > >> vote > >> >> will > >> >> >>> pass if there are 3 binding +1 votes and more binding +1 votes > than > >> -1 > >> >> >>> votes. > >> >> >>> > >> >> >>> This wiki page describes how to check the release before you > vote: > >> >> >>> *https://cwiki.apache.org/confluence/display/IMPALA/How+ > >> >> to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate > >> >> >>> <https://cwiki.apache.org/confluence/display/IMPALA/How+ > >> >> to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate>* > >> >> >>> > >> >> >>> The vote will be open until the end of day, September 5th, > Pacific > >> time > >> >> >>> zone (UTC-08:00). > >> >> >>> Once the vote passes the Impala PPMC vote, it still must pass the > >> >> incubator > >> >> >>> PMC vote before a release is made. > >> >> > >> >
