[
https://issues.apache.org/jira/browse/KUDU-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612735#comment-17612735
]
Joe McDonnell commented on KUDU-3404:
-------------------------------------
It sounds like using thread local storage would avoid some memory allocations,
so there may be a performance impact to setting WITH_TLS=OFF.
[~MikaelSmith] Impala is capable of changing our build steps for Kudu to handle
it, but right now we would need to patch Kudu source. I'm definitely ok with
some build setting that Impala can use which turns glog's WITH_TLS to OFF. It's
up to Kudu whether WITH_TLS=ON is something that seems useful for Kudu server
binaries.
> glog 0.6.0 increases the TLS usage of libkudu_client.so substantially
> ---------------------------------------------------------------------
>
> Key: KUDU-3404
> URL: https://issues.apache.org/jira/browse/KUDU-3404
> Project: Kudu
> Issue Type: Bug
> Components: client
> Affects Versions: 1.17.0
> Reporter: Joe McDonnell
> Assignee: Marton Greber
> Priority: Critical
> Attachments: 0001-Add-WITH_TLS-OFF-to-glog-build-definition.patch
>
>
> Glog 0.4 introduced supported for using thread local storage for its buffer.
> This feature is controlled by the WITH_TLS CMake variable, and it defaults to
> ON. See
> [https://github.com/google/glog/commit/2df0ca34aa3000dadf76633ca700abf0bf50756d]
> . When Kudu upgraded to glog 0.6.0 as part of the M1 fixes in "[KUDU-3374
> Add support for M1 and macOS
> Monterey|https://github.com/apache/kudu/commit/543e128d473f8f7836e605bba8cd6512fa918550]";,
> it increased the thread local storage usage by >30000 bytes.
> {noformat}
> # Older libkudu_client.so has 0x100 = 256 bytes of TLS:
> $ readelf -l libkudu_client.so | grep "TLS" -A1
> TLS 0x00000000007d14c0 0x00000000007d24c0 0x00000000007d24c0
> 0x0000000000000080 0x0000000000000100 R 0x40
> # Newer libkudu_client.so has 0x77b9 = 30649 bytes of TLS:
> $ readelf -l libkudu_client.so.0 | grep TLS -A1
> TLS 0x0000000000751280 0x0000000000752280 0x0000000000752280
> 0x0000000000000080 0x00000000000077b9 R 40{noformat}
> This is a problem for Impala, because Impala starts a JVM. There are certain
> JVM threads (like the "reaper thread") that have very small stacks (e.g.
> 32KB) and with glibc the TLS space is allocated at the expense of stack
> space. 30k of TLS usage leaves very little for the reaper thread. There are a
> series of bugs where the Java reaper thread hits a StackOverflowException
> because of high TLS usage. This can cause various symptoms including hangs.
> GLIBC message thread: [https://sourceware.org/bugzilla/show_bug.cgi?id=11787]
> JDK bugs:
> [JDK-8217475|https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8217475],
> [JDK-8225035|https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8225035]
> To resolve Impala's problem, it would be useful to build libkudu_client.so
> with glog's WITH_TLS=OFF.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
