[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2020-01-14 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 5: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Tue, 14 Jan 2020 19:50:37 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2020-01-14 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..

IMPALA-9196: Dump jstack and collect logs when tests timeout

This patch augments script-timeout-check.sh to also dump the jstack of
FE when tests timeout.

Tests:
 - Manually test the script with sudo privilege
 - Tested the script in private Jenkins jobs

Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Reviewed-on: http://gerrit.cloudera.org:8080/14794
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M bin/script-timeout-check.sh
1 file changed, 17 insertions(+), 3 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2020-01-14 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 5: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Tue, 14 Jan 2020 15:12:15 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2020-01-14 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5410/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Tue, 14 Jan 2020 15:12:16 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2020-01-13 Thread Joe McDonnell (Code Review)
Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 4: Code-Review+2

This makes sense to me.


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Mon, 13 Jan 2020 23:01:25 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2020-01-01 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 4:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/5355/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Thu, 02 Jan 2020 07:30:31 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2020-01-01 Thread Quanlong Huang (Code Review)
Hello Joe McDonnell, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14794

to look at the new patch set (#4).

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..

IMPALA-9196: Dump jstack and collect logs when tests timeout

This patch augments script-timeout-check.sh to also dump the jstack of
FE when tests timeout.

Tests:
 - Manually test the script with sudo privilege
 - Tested the script in private Jenkins jobs

Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
---
M bin/script-timeout-check.sh
1 file changed, 17 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/14794/4
--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-12-30 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/5351/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Tue, 31 Dec 2019 00:27:43 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-12-30 Thread Quanlong Huang (Code Review)
Quanlong Huang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 3:

> Patch Set 2:
>
> > > Patch Set 2:
>  > >
>  > > > I'm uncertain about how the privileges work. There are ptrace
>  > >  > limitations in Ubuntu that restrict ptrace by the same user to
>  > a
>  > >  > parent process, which I think is why the gdb part of this
>  > script
>  > >  > works. I'm not sure what permissions jstack would need, and if
>  > this
>  > >  > would work.
>  > >  >
>  > >  > If you haven't already, a test that you could run for the
>  > >  > permissions is to run the end to end tests and set
>  > >  > TIMEOUT_FOR_RUN_ALL_TESTS_MINS to some modest value (15 mins)
>  > and
>  > >  > verify you get the logs you want and jstack works.
>  > >  >
>  > >  > Once we verify that the permissions are ok in the normal way
>  > we run
>  > >  > this, the code looks good to me.
>  > >
>  > > Circling back to this review. My guess is that this doesn't work
>  > in its current form on Ubuntu, but it might work on other
>  > platforms.
>  > >
>  > > It looks like it is harmless if these debug commands fail
>  > (because the script doesn't have "set -euo pipefail"). I think any
>  > step forward in this debugging information is ok to merge as long
>  > as it improves some platform without regressing anything. We should
>  > add comments about dump statements that don't work on some
>  > platforms, but that shouldn't stop us from adding statements that
>  > do work on Centos7 or some other platform. Obviously, it would be
>  > nice for these things to work on Ubuntu.
>  >
>  > I'm still testing this script in internal jenkeins jobs. It looks
>  > wired to me that the script fails with "lsof: command not found".
>  > But when installing lsof explictly, it saids it's already
>  > installed:
>  >
>  > ++ sudo yum install -y lsof
>  > Loaded plugins: fastestmirror
>  > Loading mirror speeds from cached hostfile
>  > Package lsof-4.87-4.el7.x86_64 already installed and latest version
>  > Nothing to do
>  > ++ which lsof
>  > which: no lsof in 
> (/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin)
>  >
>  > I think it's the problem with PATH. Will check it later. Internal
>  > job link:
>  > 
> https://master-02.jenkins.cloudera.com/job/impala-private-parameterized/6139
>
> Just as a reminder, it is important not to post links that are not publicly 
> accessible. Reviews need to be conducted in a way that everyone can 
> participate. This also protects any companies that may participate in Apache 
> projects.
>
> About the lsof: Unfortunately, this is an area where different Linux 
> distributions will be different. We've dealt with that in a couple ways:
> 1. Try to find a subset that works. If we are copying some log files, copying 
> too many log files is pretty harmless as long as we get the ones we want. I 
> think if you found a command that listed the most recent 10 logs files that 
> were modified in that directory using basic utilities like find, it would 
> work on all Linux distributions.
> 2. In other scripts like bin/bootstrap_system.sh, we have commands that are 
> conditional on the Linux version.

Sorry for the unpublic link and late for this..

I realize that bin/run-all-tests.sh makes all log dirs under $IMPALA_HOME. The 
logs of custom cluster tests are in /tmp when running manually. So we may don't 
need to collect the log files again.

Remove the lsof stuffs. Still test this patch in private Jenkins jobs.


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Tue, 31 Dec 2019 00:00:20 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-12-30 Thread Quanlong Huang (Code Review)
Hello Joe McDonnell, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14794

to look at the new patch set (#3).

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..

IMPALA-9196: Dump jstack and collect logs when tests timeout

This patch augments script-timeout-check.sh to also dump the jstack of
FE when tests timeout.

Tests:
 - Manually test the script with sudo privilege

Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
---
M bin/script-timeout-check.sh
1 file changed, 15 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/14794/3
--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-12-03 Thread Joe McDonnell (Code Review)
Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 2:

> > Patch Set 2:
 > >
 > > > I'm uncertain about how the privileges work. There are ptrace
 > >  > limitations in Ubuntu that restrict ptrace by the same user to
 > a
 > >  > parent process, which I think is why the gdb part of this
 > script
 > >  > works. I'm not sure what permissions jstack would need, and if
 > this
 > >  > would work.
 > >  >
 > >  > If you haven't already, a test that you could run for the
 > >  > permissions is to run the end to end tests and set
 > >  > TIMEOUT_FOR_RUN_ALL_TESTS_MINS to some modest value (15 mins)
 > and
 > >  > verify you get the logs you want and jstack works.
 > >  >
 > >  > Once we verify that the permissions are ok in the normal way
 > we run
 > >  > this, the code looks good to me.
 > >
 > > Circling back to this review. My guess is that this doesn't work
 > in its current form on Ubuntu, but it might work on other
 > platforms.
 > >
 > > It looks like it is harmless if these debug commands fail
 > (because the script doesn't have "set -euo pipefail"). I think any
 > step forward in this debugging information is ok to merge as long
 > as it improves some platform without regressing anything. We should
 > add comments about dump statements that don't work on some
 > platforms, but that shouldn't stop us from adding statements that
 > do work on Centos7 or some other platform. Obviously, it would be
 > nice for these things to work on Ubuntu.
 >
 > I'm still testing this script in internal jenkeins jobs. It looks
 > wired to me that the script fails with "lsof: command not found".
 > But when installing lsof explictly, it saids it's already
 > installed:
 >
 > ++ sudo yum install -y lsof
 > Loaded plugins: fastestmirror
 > Loading mirror speeds from cached hostfile
 > Package lsof-4.87-4.el7.x86_64 already installed and latest version
 > Nothing to do
 > ++ which lsof
 > which: no lsof in 
 > (/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin)
 >
 > I think it's the problem with PATH. Will check it later. Internal
 > job link:
 > https://master-02.jenkins.cloudera.com/job/impala-private-parameterized/6139

Just as a reminder, it is important not to post links that are not publicly 
accessible. Reviews need to be conducted in a way that everyone can 
participate. This also protects any companies that may participate in Apache 
projects.

About the lsof: Unfortunately, this is an area where different Linux 
distributions will be different. We've dealt with that in a couple ways:
1. Try to find a subset that works. If we are copying some log files, copying 
too many log files is pretty harmless as long as we get the ones we want. I 
think if you found a command that listed the most recent 10 logs files that 
were modified in that directory using basic utilities like find, it would work 
on all Linux distributions.
2. In other scripts like bin/bootstrap_system.sh, we have commands that are 
conditional on the Linux version.


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Wed, 04 Dec 2019 01:14:03 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-12-03 Thread Quanlong Huang (Code Review)
Quanlong Huang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 2:

> Patch Set 2:
>
> > I'm uncertain about how the privileges work. There are ptrace
>  > limitations in Ubuntu that restrict ptrace by the same user to a
>  > parent process, which I think is why the gdb part of this script
>  > works. I'm not sure what permissions jstack would need, and if this
>  > would work.
>  >
>  > If you haven't already, a test that you could run for the
>  > permissions is to run the end to end tests and set
>  > TIMEOUT_FOR_RUN_ALL_TESTS_MINS to some modest value (15 mins) and
>  > verify you get the logs you want and jstack works.
>  >
>  > Once we verify that the permissions are ok in the normal way we run
>  > this, the code looks good to me.
>
> Circling back to this review. My guess is that this doesn't work in its 
> current form on Ubuntu, but it might work on other platforms.
>
> It looks like it is harmless if these debug commands fail (because the script 
> doesn't have "set -euo pipefail"). I think any step forward in this debugging 
> information is ok to merge as long as it improves some platform without 
> regressing anything. We should add comments about dump statements that don't 
> work on some platforms, but that shouldn't stop us from adding statements 
> that do work on Centos7 or some other platform. Obviously, it would be nice 
> for these things to work on Ubuntu.

I'm still testing this script in internal jenkeins jobs. It looks wired to me 
that the script fails with "lsof: command not found". But when installing lsof 
explictly, it saids it's already installed:

 ++ sudo yum install -y lsof
 Loaded plugins: fastestmirror
 Loading mirror speeds from cached hostfile
 Package lsof-4.87-4.el7.x86_64 already installed and latest version
 Nothing to do
 ++ which lsof
 which: no lsof in 
(/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin)

I think it's the problem with PATH. Will check it later. Internal job link:
https://master-02.jenkins.cloudera.com/job/impala-private-parameterized/6139


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Wed, 04 Dec 2019 00:34:54 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-12-03 Thread Joe McDonnell (Code Review)
Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 2:

> I'm uncertain about how the privileges work. There are ptrace
 > limitations in Ubuntu that restrict ptrace by the same user to a
 > parent process, which I think is why the gdb part of this script
 > works. I'm not sure what permissions jstack would need, and if this
 > would work.
 >
 > If you haven't already, a test that you could run for the
 > permissions is to run the end to end tests and set
 > TIMEOUT_FOR_RUN_ALL_TESTS_MINS to some modest value (15 mins) and
 > verify you get the logs you want and jstack works.
 >
 > Once we verify that the permissions are ok in the normal way we run
 > this, the code looks good to me.

Circling back to this review. My guess is that this doesn't work in its current 
form on Ubuntu, but it might work on other platforms.

It looks like it is harmless if these debug commands fail (because the script 
doesn't have "set -euo pipefail"). I think any step forward in this debugging 
information is ok to merge as long as it improves some platform without 
regressing anything. We should add comments about dump statements that don't 
work on some platforms, but that shouldn't stop us from adding statements that 
do work on Centos7 or some other platform. Obviously, it would be nice for 
these things to work on Ubuntu.


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Tue, 03 Dec 2019 18:38:46 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-11-25 Thread Joe McDonnell (Code Review)
Joe McDonnell has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 2:

I'm uncertain about how the privileges work. There are ptrace limitations in 
Ubuntu that restrict ptrace by the same user to a parent process, which I think 
is why the gdb part of this script works. I'm not sure what permissions jstack 
would need, and if this would work.

If you haven't already, a test that you could run for the permissions is to run 
the end to end tests and set TIMEOUT_FOR_RUN_ALL_TESTS_MINS to some modest 
value (15 mins) and verify you get the logs you want and jstack works.

Once we verify that the permissions are ok in the normal way we run this, the 
code looks good to me.


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Tue, 26 Nov 2019 01:32:20 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-11-25 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/5130/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Joe McDonnell 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Mon, 25 Nov 2019 14:47:21 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-11-25 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/5129/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Mon, 25 Nov 2019 14:28:40 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-11-25 Thread Quanlong Huang (Code Review)
Quanlong Huang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14794/1/bin/script-timeout-check.sh
File bin/script-timeout-check.sh:

http://gerrit.cloudera.org:8080/#/c/14794/1/bin/script-timeout-check.sh@46
PS1, Line 46:   for log in $(lsof -p $pid | egrep "INFO|WARN|ERROR" | awk 
'{print $9}' | sort | uniq); do
> line too long (91 > 90)
Done



--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Quanlong Huang 
Gerrit-Comment-Date: Mon, 25 Nov 2019 14:27:39 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-11-25 Thread Quanlong Huang (Code Review)
Hello Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14794

to look at the new patch set (#2).

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..

IMPALA-9196: Dump jstack and collect logs when tests timeout

This patch augments script-timeout-check.sh to also dump the jstack of
FE and collect server logs when tests timeout.

Tests:
 - Manually test the script with sudo privilege

Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
---
M bin/script-timeout-check.sh
1 file changed, 27 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/14794/2
--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-11-25 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14794 )

Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14794/1/bin/script-timeout-check.sh
File bin/script-timeout-check.sh:

http://gerrit.cloudera.org:8080/#/c/14794/1/bin/script-timeout-check.sh@46
PS1, Line 46:   for log in $(lsof -p $pid | egrep "INFO|WARN|ERROR" | awk 
'{print $9}' | sort | uniq); do
line too long (91 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Mon, 25 Nov 2019 13:45:15 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-9196: Dump jstack and collect logs when tests timeout

2019-11-25 Thread Quanlong Huang (Code Review)
Quanlong Huang has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/14794


Change subject: IMPALA-9196: Dump jstack and collect logs when tests timeout
..

IMPALA-9196: Dump jstack and collect logs when tests timeout

This patch augments script-timeout-check.sh to also dump the jstack of
FE and collect server logs when tests timeout.

Tests:
 - Manually test the script with sudo privilege

Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
---
M bin/script-timeout-check.sh
1 file changed, 26 insertions(+), 3 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/94/14794/1
--
To view, visit http://gerrit.cloudera.org:8080/14794
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib8a5b140024c236209c7e44149660189890b9d06
Gerrit-Change-Number: 14794
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang