[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9822 )

Change subject: IMPALA-6338: Disable flaky bloom filter test
..


Patch Set 2: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Mon, 09 Apr 2018 21:52:56 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9822 )

Change subject: IMPALA-6338: Disable flaky bloom filter test
..

IMPALA-6338: Disable flaky bloom filter test

The underlying issue in IMPALA-6338 causes successful queries that are
cancelled internally due to all results having been returned to, in
rare cases, have info missing from the profile. This has caused flaky
tests but has low impact on users, and unfortunately with the current
query lifecycle logic in the coordinator, there is no simple solution.

There is ongoing work to improve query lifecycle logic in the
coordinator holistically, see IMPALA-5384. This work will eventually
address the underlying cause of IMPALA-6338. Until then, we disable
the tests that have been flaky.

Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Reviewed-on: http://gerrit.cloudera.org:8080/9822
Reviewed-by: Thomas Tauber-Marshall 
Reviewed-by: Tim Armstrong 
Tested-by: Impala Public Jenkins 
---
M testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test
1 file changed, 3 insertions(+), 2 deletions(-)

Approvals:
  Thomas Tauber-Marshall: Looks good to me, but someone else must approve
  Tim Armstrong: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9822 )

Change subject: IMPALA-6338: Disable flaky bloom filter test
..


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2256/


--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Mon, 09 Apr 2018 17:55:57 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9822 )

Change subject: IMPALA-6338: Disable flaky bloom filter test
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Mon, 09 Apr 2018 17:54:28 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Thomas Tauber-Marshall (Code Review)]

Thomas Tauber-Marshall has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9822 )

Change subject: IMPALA-6338: Disable flaky bloom filter test
..


Patch Set 2: Code-Review+1

(1 comment)

Carrying forward

http://gerrit.cloudera.org:8080/#/c/9822/1/testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test
File testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test:

http://gerrit.cloudera.org:8080/#/c/9822/1/testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test@77
PS1, Line 77: 
> Can we just comment out the the "1 of 1 Runtime Filter Published" line inst
Done



--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Mon, 09 Apr 2018 17:50:50 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Thomas Tauber-Marshall (Code Review)]

Hello Tim Armstrong, Dan Hecht,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9822

to look at the new patch set (#2).

Change subject: IMPALA-6338: Disable flaky bloom filter test
..

IMPALA-6338: Disable flaky bloom filter test

The underlying issue in IMPALA-6338 causes successful queries that are
cancelled internally due to all results having been returned to, in
rare cases, have info missing from the profile. This has caused flaky
tests but has low impact on users, and unfortunately with the current
query lifecycle logic in the coordinator, there is no simple solution.

There is ongoing work to improve query lifecycle logic in the
coordinator holistically, see IMPALA-5384. This work will eventually
address the underlying cause of IMPALA-6338. Until then, we disable
the tests that have been flaky.

Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
---
M testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test
1 file changed, 3 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/9822/2
--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Dan Hecht (Code Review)]

Dan Hecht has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9822 )

Change subject: IMPALA-6338: Disable flaky bloom filter test
..


Patch Set 1:

> (1 comment)
 >
 > Seems ok if we think that this one test is one of the main causes
 > of flakiness.

I'm okay with this too after you respond to Tim's comment.


--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 06 Apr 2018 16:10:04 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9822 )

Change subject: IMPALA-6338: Disable flaky bloom filter test
..


Patch Set 1: Code-Review+1

(1 comment)

Seems ok if we think that this one test is one of the main causes of flakiness.

http://gerrit.cloudera.org:8080/#/c/9822/1/testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test
File testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test:

http://gerrit.cloudera.org:8080/#/c/9822/1/testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test@77
PS1, Line 77: #
> This is the same query as in test_profile_fragment_instances (context is th
Can we just comment out the the "1 of 1 Runtime Filter Published" line instead 
of disabling the whole test.



--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 04 Apr 2018 22:55:10 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Thomas Tauber-Marshall (Code Review)]

Thomas Tauber-Marshall has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9822 )

Change subject: IMPALA-6338: Disable flaky bloom filter test
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9822/1/testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test
File testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test:

http://gerrit.cloudera.org:8080/#/c/9822/1/testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test@77
PS1, Line 77: #
> how do we know this is IMPALA-6338?  And why is this last case any differen
This is the same query as in test_profile_fragment_instances (context is that 
there was a different but related profile race making this query flaky that I 
fixed previously, and I added test_profile_fragment_instances to check for 
actual completeness of the profile, whereas this test only looks for a few 
bloom filter specific things)

This is also the only query that has ever been flaky (afaik) due to this issue, 
over the many months that IMPALA-6338 has been a bug, presumably because 
something about it is timing just right to hit the race.

In theory, basically any e2e test that has a limit and also checks the runtime 
profile (eg. most of the tests in this file) could be vulnerable to 
IMPALA-6338. I'm happy to go through and disable all such tests, but that's 
potentially a lot of coverage lost and again this particular case is the only 
one I know of to ever actually be flaky.



--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Comment-Date: Tue, 27 Mar 2018 22:23:18 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Dan Hecht (Code Review)]

Dan Hecht has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9822 )

Change subject: IMPALA-6338: Disable flaky bloom filter test
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9822/1/testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test
File testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test:

http://gerrit.cloudera.org:8080/#/c/9822/1/testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test@77
PS1, Line 77: #
how do we know this is IMPALA-6338?  And why is this last case any different 
than the first 3?



--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall 
Gerrit-Reviewer: Dan Hecht 
Gerrit-Comment-Date: Tue, 27 Mar 2018 21:53:34 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6338: Disable flaky bloom filter test

[Thomas Tauber-Marshall (Code Review)]

Thomas Tauber-Marshall has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9822


Change subject: IMPALA-6338: Disable flaky bloom filter test
..

IMPALA-6338: Disable flaky bloom filter test

The underlying issue in IMPALA-6338 causes successful queries that are
cancelled internally due to all results having been returned to, in
rare cases, have info missing from the profile. This has caused flaky
tests but has low impact on users, and unfortunately with the current
query lifecycle logic in the coordinator, there is no simple solution.

There is ongoing work to improve query lifecycle logic in the
coordinator holistically, see IMPALA-5384. This work will eventually
address the underlying cause of IMPALA-6338. Until then, we disable
the tests that have been flaky.

Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
---
M testdata/workloads/functional-query/queries/QueryTest/bloom_filters.test
1 file changed, 14 insertions(+), 13 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/9822/1
--
To view, visit http://gerrit.cloudera.org:8080/9822
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie30b88fb8fb7780fc3a7153c05fdc3606145ce35
Gerrit-Change-Number: 9822
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall