Mike Percy has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/11726 )
Change subject: thread: show thread limit info when thread creation fails
......................................................................
thread: show thread limit info when thread creation fails
It seems useful to return the number of outstanding threads and the
ulimit nproc when thread creation fails, to help an administrator
diagnose the cause of the fork()/clone() failure. This patch adds that
info to the error Status returned from Thread::Create().
That necessitated making the ReadThreadsRunning() method of
ThreadManager public.
Tested manually on Linux. Example:
$ ps -efwwwL | grep mpercy | wc -l
2357
$ ulimit -u 2370
$ ./bin/kudu-tserver --fs-wal-dir $(pwd)/wal --logtostderr
...
F1018 14:16:00.312577 21557 service_pool.cc:93] Check failed: _s.ok() Bad
status: Runtime error: Could not create thread (63 Kudu-managed threads running
in this process, 2370 max processes allowed for current user): Resource
temporarily unavailable (error 11)
*** Check failure stack trace: ***
*** Aborted at 1539897360 (unix time) try "date -d @1539897360" if you are
using GNU date ***
PC: @ 0x7f503d9d9e97 gsignal
*** SIGABRT (@0x3e800005435) received by PID 21557 (TID 0x7f503cf34900) from
PID 21557; stack trace: ***
@ 0x7f5042455890 (unknown)
@ 0x7f503d9d9e97 gsignal
@ 0x7f503d9db801 abort
@ 0x7f504024c309 kudu::AbortFailureFunction()
@ 0x7f503f645d0d google::LogMessage::Fail()
@ 0x7f503f647ce4 google::LogMessage::SendToLog()
@ 0x7f503f64582d google::LogMessage::Flush()
@ 0x7f503f6486b9 google::LogMessageFatal::~LogMessageFatal()
@ 0x7f504172688e kudu::rpc::ServicePool::Init()
@ 0x7f50456b3eed kudu::RpcServer::RegisterService()
@ 0x7f50456bfc59 kudu::server::ServerBase::RegisterService()
@ 0x7f50459004ac kudu::tserver::TabletServer::Start()
@ 0x40683e kudu::tserver::TabletServerMain()
@ 0x4060a2 main
@ 0x7f503d9bcb97 __libc_start_main
@ 0x405fba _start
Aborted (core dumped)
Change-Id: I8e0bd0d0776142e8feff18bffe15e61ca1ba5816
Reviewed-on: http://gerrit.cloudera.org:8080/11726
Reviewed-by: Andrew Wong <[email protected]>
Reviewed-by: Adar Dembo <[email protected]>
Tested-by: Mike Percy <[email protected]>
---
M src/kudu/util/thread.cc
1 file changed, 19 insertions(+), 7 deletions(-)
Approvals:
Andrew Wong: Looks good to me, approved
Adar Dembo: Looks good to me, approved
Mike Percy: Verified
--
To view, visit http://gerrit.cloudera.org:8080/11726
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I8e0bd0d0776142e8feff18bffe15e61ca1ba5816
Gerrit-Change-Number: 11726
Gerrit-PatchSet: 4
Gerrit-Owner: Mike Percy <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mike Percy <[email protected]>
Gerrit-Reviewer: Tidy Bot (241)
