[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9262 )

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..


Patch Set 6: Code-Review+2

(1 comment)

Carrying forward +2

http://gerrit.cloudera.org:8080/#/c/9262/5/src/kudu/util/debug/unwind_safeness.cc
File src/kudu/util/debug/unwind_safeness.cc:

http://gerrit.cloudera.org:8080/#/c/9262/5/src/kudu/util/debug/unwind_safeness.cc@23
PS5, Line 23: // Based on public domain code by Aliaksey Kandratsenka at
> Do we need to add this to LICENSE?
It's only super loosely based on it, so I don't think so (eg his original was 
ANSI C). Plus public domain has no reporting requirements by ASF policy AFAIK



--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 6
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 21 Feb 2018 06:08:00 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Todd Lipcon (Code Review)]

Todd Lipcon has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9262 )

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..

KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

This follows some recommendations from [1] to avoid the potential for
deadlocks if we attempt to gather a stack trace from a signal handler
context while the thread is inside calls to libdl.

The approach is somewhat ugly: we have to override the sensitive libdl
calls and use them to increment a thread-local counter indicating that
it would be unsafe to collect a stack. We then check this counter before
capturing a stack: if we see that we are inside libdl we just return an
fake stack trace indicating why it could not be captured.

The new test included deadlocks reliably without the fix.

[1] 
https://github.com/gperftools/gperftools/wiki/gperftools'-stacktrace-capturing-methods-and-their-issues
Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Reviewed-on: http://gerrit.cloudera.org:8080/9262
Tested-by: Kudu Jenkins
Reviewed-by: Todd Lipcon 
---
M src/kudu/util/CMakeLists.txt
M src/kudu/util/debug-util-test.cc
M src/kudu/util/debug-util.cc
M src/kudu/util/debug-util.h
A src/kudu/util/debug/unwind_safeness.cc
A src/kudu/util/debug/unwind_safeness.h
6 files changed, 291 insertions(+), 10 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Todd Lipcon: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 7
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Mike Percy (Code Review)]

Mike Percy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9262 )

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..


Patch Set 5: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9262/5/src/kudu/util/debug/unwind_safeness.cc
File src/kudu/util/debug/unwind_safeness.cc:

http://gerrit.cloudera.org:8080/#/c/9262/5/src/kudu/util/debug/unwind_safeness.cc@23
PS5, Line 23: // Based on public domain code by Aliaksey Kandratsenka at
Do we need to add this to LICENSE?



--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 21 Feb 2018 01:17:58 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Mike Percy (Code Review)]

Mike Percy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9262 )

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..


Patch Set 5:

(2 comments)

Nice. How the heck did you even stumble upon this workaround?

http://gerrit.cloudera.org:8080/#/c/9262/5/src/kudu/util/debug/unwind_safeness.cc
File src/kudu/util/debug/unwind_safeness.cc:

http://gerrit.cloudera.org:8080/#/c/9262/5/src/kudu/util/debug/unwind_safeness.cc@53
PS5, Line 53: libdl
nit: period at end


http://gerrit.cloudera.org:8080/#/c/9262/5/src/kudu/util/debug/unwind_safeness.cc@63
PS5, Line 63: *
nit: any reason to move the asterisks around from their usual place in these 
signatures?



--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 5
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 21 Feb 2018 01:12:49 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Will Berkeley (Code Review)]

Will Berkeley has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9262 )

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..


Patch Set 4: Code-Review+1

SGTM. I'd feel better if someone else took a look too, given the fancy hacks.

I checked it builds on macOS and the /stacks page returns a useful error.


--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 14 Feb 2018 01:32:59 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9262 )

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9262/3/src/kudu/util/debug-util-test.cc
File src/kudu/util/debug-util-test.cc:

http://gerrit.cloudera.org:8080/#/c/9262/3/src/kudu/util/debug-util-test.cc@19
PS3, Line 19: #include 
> Linux-specific, it seems.
Done



--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 14 Feb 2018 00:58:31 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Todd Lipcon (Code Review)]

Hello Will Berkeley, Tidy Bot, Mike Percy, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9262

to look at the new patch set (#4).

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..

KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

This follows some recommendations from [1] to avoid the potential for
deadlocks if we attempt to gather a stack trace from a signal handler
context while the thread is inside calls to libdl.

The approach is somewhat ugly: we have to override the sensitive libdl
calls and use them to increment a thread-local counter indicating that
it would be unsafe to collect a stack. We then check this counter before
capturing a stack: if we see that we are inside libdl we just return an
fake stack trace indicating why it could not be captured.

The new test included deadlocks reliably without the fix.

[1] 
https://github.com/gperftools/gperftools/wiki/gperftools'-stacktrace-capturing-methods-and-their-issues
Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
---
M src/kudu/util/CMakeLists.txt
M src/kudu/util/debug-util-test.cc
M src/kudu/util/debug-util.cc
M src/kudu/util/debug-util.h
A src/kudu/util/debug/unwind_safeness.cc
A src/kudu/util/debug/unwind_safeness.h
6 files changed, 291 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/62/9262/4
--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Will Berkeley (Code Review)]

Will Berkeley has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9262 )

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9262/3/src/kudu/util/debug-util-test.cc
File src/kudu/util/debug-util-test.cc:

http://gerrit.cloudera.org:8080/#/c/9262/3/src/kudu/util/debug-util-test.cc@19
PS3, Line 19: #include 
Linux-specific, it seems.



--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 14 Feb 2018 00:20:16 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9262 )

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..


Patch Set 2:

ok, hopefully fixed sanitizer builds and OSX. can you let me know if I got the 
__APPLE__ checks in the right spots?


--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Tue, 13 Feb 2018 23:56:39 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Todd Lipcon (Code Review)]

Hello Will Berkeley, Tidy Bot, Mike Percy, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9262

to look at the new patch set (#3).

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..

KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

This follows some recommendations from [1] to avoid the potential for
deadlocks if we attempt to gather a stack trace from a signal handler
context while the thread is inside calls to libdl.

The approach is somewhat ugly: we have to override the sensitive libdl
calls and use them to increment a thread-local counter indicating that
it would be unsafe to collect a stack. We then check this counter before
capturing a stack: if we see that we are inside libdl we just return an
fake stack trace indicating why it could not be captured.

The new test included deadlocks reliably without the fix.

[1] 
https://github.com/gperftools/gperftools/wiki/gperftools'-stacktrace-capturing-methods-and-their-issues
Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
---
M src/kudu/util/CMakeLists.txt
M src/kudu/util/debug-util-test.cc
M src/kudu/util/debug-util.cc
M src/kudu/util/debug-util.h
A src/kudu/util/debug/unwind_safeness.cc
A src/kudu/util/debug/unwind_safeness.h
6 files changed, 289 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/62/9262/3
--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9262 )

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..


Patch Set 2:

Apparently sanitizer builds are not so keen on this hack. Will investigate.


--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Mon, 12 Feb 2018 23:06:53 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Todd Lipcon (Code Review)]

Hello Will Berkeley, Tidy Bot, Mike Percy, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9262

to look at the new patch set (#2).

Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..

KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

This follows some recommendations from [1] to avoid the potential for
deadlocks if we attempt to gather a stack trace from a signal handler
context while the thread is inside calls to libdl.

The approach is somewhat ugly: we have to override the sensitive libdl
calls and use them to increment a thread-local counter indicating that
it would be unsafe to collect a stack. We then check this counter before
capturing a stack: if we see that we are inside libdl we just return an
fake stack trace indicating why it could not be captured.

The new test included deadlocks reliably without the fix.

[1] 
https://github.com/gperftools/gperftools/wiki/gperftools'-stacktrace-capturing-methods-and-their-issues
Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
---
M src/kudu/util/CMakeLists.txt
M src/kudu/util/debug-util-test.cc
M src/kudu/util/debug-util.cc
M src/kudu/util/debug-util.h
A src/kudu/util/debug/unwind_safeness.cc
A src/kudu/util/debug/unwind_safeness.h
6 files changed, 268 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/62/9262/2
--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Will Berkeley 

[kudu-CR] KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

[Todd Lipcon (Code Review)]

Hello Will Berkeley, Mike Percy,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/9262

to review the following change.


Change subject: KUDU-2291 (part 4): avoid potential for deadlocks in stack 
trace collection
..

KUDU-2291 (part 4): avoid potential for deadlocks in stack trace collection

This follows some recommendations from [1] to avoid the potential for
deadlocks if we attempt to gather a stack trace from a signal handler
context while the thread is inside calls to libdl.

The approach is somewhat ugly: we have to override the sensitive libdl
calls and use them to increment a thread-local counter indicating that
it would be unsafe to collect a stack. We then check this counter before
capturing a stack: if we see that we are inside libdl we just return an
fake stack trace indicating why it could not be captured.

[1] 
https://github.com/gperftools/gperftools/wiki/gperftools'-stacktrace-capturing-methods-and-their-issues
Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
---
M src/kudu/util/CMakeLists.txt
M src/kudu/util/debug-util-test.cc
M src/kudu/util/debug-util.cc
A src/kudu/util/debug/unwind_safeness.cc
4 files changed, 234 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/62/9262/1
--
To view, visit http://gerrit.cloudera.org:8080/9262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I414718a26c4810de86a10397bb9803dc5e386d1a
Gerrit-Change-Number: 9262
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Will Berkeley