[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 5: Verified+1

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has submitted this change and it was merged.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


IMPALA-2605: Omit the sort and mini stress tests

These stress tests were sometimes causing the end-to-end tests to hang
indefinitey, including in the pre-merge testing (sometimes called
"GVO" or "GVM").

This patch also prints to stdout some connections metrics that may
prove useful for debugging stress test hangs in the future. The
metrics are printed before and after stress tests are run when
run-tests.py is used.

Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Reviewed-on: http://gerrit.cloudera.org:8080/5401
Reviewed-by: Jim Apple 
Tested-by: Impala Public Jenkins
---
M tests/run-tests.py
M tests/stress/test_mini_stress.py
2 files changed, 24 insertions(+), 0 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Jim Apple: Looks good to me, approved



-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 6
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Impala Public Jenkins (Code Review)]

Impala Public Jenkins has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 5:

Build started: http://jenkins.impala.io:8080/job/gerrit-verify-dryrun/151/

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 5: Code-Review+2

rebase carry TIm's +2

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 4: Code-Review+2

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/5401/3/tests/stress/test_mini_stress.py
File tests/stress/test_mini_stress.py:

Line 66:   @pytest.mark.skipif(True,
> I think so - xfail with run=False is kind of the same as skip, right?
Yes


-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/5401/3/tests/stress/test_mini_stress.py
File tests/stress/test_mini_stress.py:

Line 66:   @pytest.mark.skipif(True,
> Wait, I actually jst realized I xfailed them, rather than skipingif them. D
I think so - xfail with run=False is kind of the same as skip, right?


-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/5401/3/tests/stress/test_mini_stress.py
File tests/stress/test_mini_stress.py:

Line 66: self.run_test_case('sort_stress', vector)
> Done
Wait, I actually jst realized I xfailed them, rather than skipingif them. Do 
you think they should be skipiffed?


-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/5401/3/tests/stress/test_mini_stress.py
File tests/stress/test_mini_stress.py:

Line 66:   @pytest.mark.skipif(True,
> Nit: We're already skipping this test individually - let's just do the same
Done


-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Hello Alex Behm, Tim Armstrong,

I'd like you to reexamine a change.  Please visit

http://gerrit.cloudera.org:8080/5401

to look at the new patch set (#4).

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..

IMPALA-2605: Omit the sort and mini stress tests

These stress tests were sometimes causing the end-to-end tests to hang
indefinitey, including in the pre-merge testing (sometimes called
"GVO" or "GVM").

This patch also prints to stdout some connections metrics that may
prove useful for debugging stress test hangs in the future. The
metrics are printed before and after stress tests are run when
run-tests.py is used.

Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
---
M tests/run-tests.py
M tests/stress/test_mini_stress.py
2 files changed, 24 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/5401/4
-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 3: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/5401/3/tests/stress/test_mini_stress.py
File tests/stress/test_mini_stress.py:

Line 66:   @pytest.mark.skipif(True,
Nit: We're already skipping this test individually - let's just do the same 
thing for the other tests to be consistent.


-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 3:

> I'm ok with this change but I think we should add the logging in
 > case we hit the problem on the other stress tests.

Done

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Hello Alex Behm, Tim Armstrong,

I'd like you to reexamine a change.  Please visit

http://gerrit.cloudera.org:8080/5401

to look at the new patch set (#3).

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..

IMPALA-2605: Omit the sort and mini stress tests

These stress tests were sometimes causing the end-to-end tests to hang
indefinitey, including in the pre-merge testing (sometimes called
"GVO" or "GVM").

This patch also prints to stdout some connections metrics that may
prove useful for debugging stress test hangs in the future. The
metrics are printed before and after stress tests are run when
run-tests.py is used.

Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
---
M tests/run-tests.py
M tests/stress/test_mini_stress.py
2 files changed, 22 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/5401/3
-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2:

I'm ok with this change but I think we should add the logging in case we hit 
the problem on the other stress tests.

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2:

> > Maybe we should dump out the value of some of these metrics
 > before
 > > running the tests in tests/run-tests.py. That would help diagnose
 > > in future.
 > 
 > So you're suggesting we add logging for, say, 
 > impala.thrift-server.backend.connections-in-use,
 > impala.thrift-server.beeswax-frontend.connections-in-use, and
 > impala.thrift-server.hiveserver2-frontend.connections-in-use,
 > abandon this patch, and see what we can diagnose the next time this
 > happens?

Any thoughts on this, Tim?

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2:

> Maybe we should dump out the value of some of these metrics before
 > running the tests in tests/run-tests.py. That would help diagnose
 > in future.

So you're suggesting we add logging for, say, 
impala.thrift-server.backend.connections-in-use, 
impala.thrift-server.beeswax-frontend.connections-in-use, and 
impala.thrift-server.hiveserver2-frontend.connections-in-use, abandon this 
patch, and see what we can diagnose the next time this happens?

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2:

I guess I'm uneasy that we don't fully understand the cause of the problem. It 
seems like it's probably a test bug and I don't think we have a path to solving 
that. I looked again at the backtraces in the ticket and just see a bunch of 
threads waiting to read from the client connection.

My other thought is that, even if we disable these tests, will the same problem 
just happen with the other enabled stress tests.

Maybe we should dump out the value of some of these metrics before running the 
tests in tests/run-tests.py. That would help diagnose in future.

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2: Code-Review+1

I'm not going to stand in the way of progress, but for the record:
- We have no root cause for the hang
- We are removing test coverage
- We have no plan of restoring the test coverage

The fact that this test has not caught many failures is not necessarily an 
indication that it's useless. By that definition most tests would be useless.

Think about what would happen if this became an accepted procedure for getting 
rid of "inconvenient" test problems. I concede this case might be an exception, 
just wanted to explain my general reluctance given the facts.

I'm much more in favor of deleting code we don't need instead of leaving it and 
disabling it without understanding what is happening.

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2:

> One option we should consider is replacing this test with a small
 > run of our "regular" stress test run. That way we don't lose
 > coverage and arguably our regular stress runs are better.

Do you mean tests/stress/concurrent_select? Which queries do you think would be 
of interest? I ask because the comment seems to indicate it is parameterized, 
with queries that must be passed in by the user.

This is a substantial script (1500 lines) and it seems to be designed to be "a 
slow process", so tuning it to the needs of pre-merge testing may be lower 
impact per hour spent than some of my other Impala projects, especially given 
the risk that it turns out to not be possible to fit it into our current e2e 
tests framework.

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2:

One option we should consider is replacing this test with a small run of our 
"regular" stress test run. That way we don't lose coverage and arguably our 
regular stress runs are better.

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2:

> I'm ok with removing these tests if someone can explain to me what
 > exactly is happening so we can rule out a product issue with 100%
 > certainty.

I have spent a few hours on this without much progress. This patch is sent 
partially as a method of cutting my losses. Certainly there is some point at 
which we just retire the test. Perhaps in your opinion that should be after a 
few days or a week of fruitless investigation. What do you think?

Alternatively, we could move the test to exhaustive just to un-stall our 
pre-merge testing. Thoughts?

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2: Code-Review-1 -Verified

I'm ok with removing these tests if someone can explain to me what exactly is 
happening so we can rule out a product issue with 100% certainty.

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2:

I don't think I've seen these particular tests catch anything. I couldn't find 
any JIRAs where they caught a bug: 
https://issues.cloudera.org/issues/?jql=text%20~%20%22TestMiniStress%22

Have you seen them catch things pre-commit?

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2:

Alex, I think you clicked the wrong button. -1 on Code-Review is for humans, -1 
on Verified is for robots.

The hangs are happening somewhere between 5 and 25% of the time on ec2. This 
will increase our GVM spending (by duplicating jobs) or latency (by waiting 
until jobs fail to retry).

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Alex Behm (Code Review)]

Alex Behm has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2: Verified-1

I'm against disabling these tests since they are good at finding bugs.  Yes, 
hangs do happen, but pretty rarely. I think the cost/benefit is in favor of 
keeping them.

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Alex Behm 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Taras Bobrovytsky 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2: Code-Review+1

I think Taras should take a look too in case I missed some reason why it would 
be important to continue running these - it looks like he added sort_stress

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has uploaded a new patch set (#2).

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..

IMPALA-2605: Omit the sort and mini stress tests

These stress tests were sometimes causing the end-to-end tests to hang
indefinitey, including in the pre-merge testing (sometimes called
"GVO" or "GVM").

Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
---
M tests/stress/test_mini_stress.py
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/5401/2
-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Tim Armstrong 

[Impala-ASF-CR] IMPALA-2605: Omit the sort and mini stress tests

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-2605: Omit the sort and mini stress tests
..


Patch Set 2:

> I think the DDL stress test is still useful, since we don't have
 > duplicate coverage else. Maybe we should scope this more narrow to
 > test_mini_stress and test_sort_stress, since those seem redundant
 > with the tpc-h/tpc-ds stress tests.

Done

-- 
To view, visit http://gerrit.cloudera.org:8080/5401
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd30abf8215415e0f2830b725e43b005daa2bb2d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple 
Gerrit-Reviewer: Jim Apple 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No