Re: Build failed in 2.x and 2.12.0-release due to updates of sentry-1.5.1-cdh5.16.0-SNAPSHOT

2018-08-30 Thread Fredy Wijaya
Hi Quanglong, I'm working on a fix: https://issues.apache.org/jira/browse/IMPALA-7513. CR is here: https://gerrit.cloudera.org/c/11362/ I agree that the dependencies should be frozen. Feel free to file a JIRA for the 2.x branch to use the released versions instead of the SNAPSHOT versions to make

Issues with query/fragment lifecycle on trunk?

2018-08-30 Thread Todd Lipcon
Hey folks, I ran into some issues with a local core test run last night. This is on my own branch with some uncommitted work, but I haven't touched the backend and these issues seem to be backend-focused. Initially, I noticed the problem because a test run that I started last night was still

Re: Issues with query/fragment lifecycle on trunk?

2018-08-30 Thread Todd Lipcon
I spent another 45 minutes looking at the core and think I figured this out. I found that for the stuck scanner, both of its scanner threads and run and exited, but for some reason it was never marked as "done". I think this might be a regression due to IMPALA-7096

Re: Issues with query/fragment lifecycle on trunk?

2018-08-30 Thread Michael Ho
FWIW, please see the discussion at https://issues.apache.org/jira/browse/IMPALA-6194 about how "stuck IO" may render cancellation in HDFS scan node ineffective. We have seen users reported similar backtraces in the past due to hung RPCs to HDFS name node. Not saying this is the necessarily the

Re: Issues with query/fragment lifecycle on trunk?

2018-08-30 Thread Bikramjeet Vig
On Thu, Aug 30, 2018 at 12:25 PM Todd Lipcon wrote: > Hey folks, > > I ran into some issues with a local core test run last night. This is on my > own branch with some uncommitted work, but I haven't touched the backend > and these issues seem to be backend-focused. > > Initially, I noticed the

Re: Improving Kudu Build Support

2018-08-30 Thread Thomas Tauber-Marshall
The patch for this is finally out: https://gerrit.cloudera.org/#/c/11363/ I put some work into trying to make sure this will be as seamless as possible for everyone. - For people on Ubuntu 16 or the other CDH supported distros, you'll automatically start using the CDH Kudu. - For people on Ubuntu

Re: Issues with query/fragment lifecycle on trunk?

2018-08-30 Thread Todd Lipcon
On Thu, Aug 30, 2018 at 2:44 PM, Pooja Nilangekar < pooja.nilange...@cloudera.com> wrote: > Hi Todd, > > I believe you are right. There are a couple of other race conditions in the > HdfsScanNode which have resulted in broken-builds in the last couple of > weeks. (IMPALA-7418 and IMPALA-7335).

Re: Issues with query/fragment lifecycle on trunk?

2018-08-30 Thread Pooja Nilangekar
Hi Todd, I believe you are right. There are a couple of other race conditions in the HdfsScanNode which have resulted in broken-builds in the last couple of weeks. (IMPALA-7418 and IMPALA-7335). While reviewing the change for IMPALA-7335, Tim and Dan suggested that we need to cleanup the state

Re: Build failed in 2.x and 2.12.0-release due to updates of sentry-1.5.1-cdh5.16.0-SNAPSHOT

2018-08-30 Thread Quanlong Huang
Thank you, Fredy! We should release a compilable 2.12.1 to replace 2.12.0 by whether cherry-pick this patch or downgrade the CDH dependencies to cdh5.15 (which already contains Impala-2.12 thus is compatible with 2.12). I filed a JIRA for this (IMPALA-7518). I'm in favor with the second choice

Re: Issues with query/fragment lifecycle on trunk?

2018-08-30 Thread Todd Lipcon
On Thu, Aug 30, 2018 at 2:48 PM, Todd Lipcon wrote: > On Thu, Aug 30, 2018 at 2:44 PM, Pooja Nilangekar < > pooja.nilange...@cloudera.com> wrote: > >> Hi Todd, >> >> I believe you are right. There are a couple of other race conditions in >> the >> HdfsScanNode which have resulted in

Build failed in 2.x and 2.12.0-release due to updates of sentry-1.5.1-cdh5.16.0-SNAPSHOT

2018-08-30 Thread Quanlong Huang
Hi all, Recently, Sentry in version 1.5.1-cdh5.16.0-SNAPSHOT is updated, which breaks the build of 2.x branch and even 2.12.0-release. The error is [ERROR] /tmp/jenkins/workspace/impala-compile-and-publish/fe/src/main/java/org/apache/impala/catalog/AuthorizationPolicy.java:[58,8]