[2/2] kudu git commit: thirdparty: patch tcmalloc to improve AllocLarge performance

thirdparty: patch tcmalloc to improve AllocLarge performance This pulls in an upstream tcmalloc pull request[1] which addresses O(n) behavior in the large allocation path. Without this patch, large allocations (>=1MB) do a linear scan of all large spans of free heap to find the best fit, which is

[1/2] kudu git commit: KUDU-2301: (Part-1) Add instrumentation on a per connection level

Repository: kudu Updated Branches: refs/heads/master 0ee40b661 -> 1ae56048b KUDU-2301: (Part-1) Add instrumentation on a per connection level This patch returns the OutboundTransfer queue size on a per connection level and makes them accessible via the DumpRunningRpcs() call. A test is added

kudu git commit: Optionally advance safe time with non-write operations

Repository: kudu Updated Branches: refs/heads/branch-1.5.x e1b8444fc -> 611627f2c Optionally advance safe time with non-write operations We currently have a problem with safe time advancement when there are no write messages in the WAL. Because of KUDU-2233, NO_OP and CHANGE_CONFIG messages ar

kudu git commit: KUDU-2305: Fix local variable usage to handle 2GB messages

Repository: kudu Updated Branches: refs/heads/master 1ae56048b -> 2b0c1c019 KUDU-2305: Fix local variable usage to handle 2GB messages The maximum value for rpc_max_message_size (an int32_t) is INT_MAX. However, when using this maximum value, certain local variables in SerializeMessage() and I

kudu git commit: KUDU-2238. DMS not flush under memory pressure.

Repository: kudu Updated Branches: refs/heads/branch-1.6.x 0c5628f6b -> 8d9eff378 KUDU-2238. DMS not flush under memory pressure. When we choose DMS to flush, now we always pick the DMS with highest log retention. However, as KUDU-2238 shows, in some cases DMS with highest log retention may on

kudu git commit: KUDU-2238. DMS not flush under memory pressure.

Repository: kudu Updated Branches: refs/heads/branch-1.5.x 611627f2c -> 4b3a5a5f4 KUDU-2238. DMS not flush under memory pressure. When we choose DMS to flush, now we always pick the DMS with highest log retention. However, as KUDU-2238 shows, in some cases DMS with highest log retention may on

[1/5] kudu git commit: [consensus_peers] micro-optimization on controller status

Repository: kudu Updated Branches: refs/heads/master 2b0c1c019 -> 40ba6c143 [consensus_peers] micro-optimization on controller status While running raft_consensus_stress-itest where a lot of RPC fail, I found the following stats for operator new[](unsigned long) in tserver with regard of Statu

[2/5] kudu git commit: Make tcmalloc heap sampling more useful

Make tcmalloc heap sampling more useful Previously our /pprof/heap path required that the process be started with lifetime heap profiling enabled. This mode of operation is very slow so we would never be able to use it in production. Even in tests it's of limited utility. The alternative is tcmal

[4/5] kudu git commit: java: provide our own preferred cipher suite list

java: provide our own preferred cipher suite list This updates the Java client to use a cipher suite list which matches the server side. This has a very big performance benefit since it prioritizes GCM-based ciphers which are significantly faster than those that use SHA256 or SHA384 HMAC for integ

[5/5] kudu git commit: row: optimize copying of MRS rows into the Arena

row: optimize copying of MRS rows into the Arena I tried a stress workload using YCSB with 100 columns, each a 10-byte string. I expected this to be roughly the same performance as 10 columns containing 100-byte strings, but in fact it was about 3x as slow. A profile showed most of the CPU consume

[3/5] kudu git commit: Improve tablet status messages during startup

Improve tablet status messages during startup Currently while the tablet server is starting, all of the tablets that are waiting in the 'tablet-open' threadpool have the status 'Tablet initializing...'. This can be confusing to admins since it sounds like it is stuck rather than waiting in a queue

kudu git commit: tablet: don't store row count in delta tracker

Repository: kudu Updated Branches: refs/heads/master 40ba6c143 -> 1f2617674 tablet: don't store row count in delta tracker DeltaTracker's constructor previously required a row-count that can only be obtained by first reading from disk. While useful as a sanity check while tracking updates, it'

kudu git commit: Don't perform compactions when clean time has not been advanced

Repository: kudu Updated Branches: refs/heads/branch-1.5.x 4b3a5a5f4 -> 8c9dcc0ee Don't perform compactions when clean time has not been advanced Due to KUDU-2233 we might not advance safe time, and thus clean time, at bootstrap causing possible corruption or crashes. This patch adds a check

[2/5] kudu git commit: KUDU-2297 (part 1): refactor metrics log out of ServerBase

KUDU-2297 (part 1): refactor metrics log out of ServerBase This is preparing for putting more information into the metrics log instead of just metrics snapshots. The logging code will get more complex as it gains features, so this makes a new class for it. This is slightly incompatible since I al

[4/5] kudu git commit: rw_semaphore: dont include debug-util.h when not necessary

rw_semaphore: dont include debug-util.h when not necessary Just avoids an unnecessary include which was making my compilation slow down every time I modified debug-util.h. Change-Id: I8a194bbffbe413eaffe0449639e3b4aa35a89900 Reviewed-on: http://gerrit.cloudera.org:8080/9328 Reviewed-by: Mike Perc

[1/5] kudu git commit: KUDU-2291 (part 7): fix race in stack trace collection

Repository: kudu Updated Branches: refs/heads/master 1f2617674 -> 1cf8ad189 KUDU-2291 (part 7): fix race in stack trace collection This fixes a narrow race in which StackTraceCollector could return 'TimedOut' but in fact the stack trace would have been collected. See the comments for details.

[3/5] kudu git commit: KUDU-2297 (part 2). Convert diagnostics log to a format closer to glog

KUDU-2297 (part 2). Convert diagnostics log to a format closer to glog Previously the metrics/diagnostics log used a custom log format designed to be readable by scripts. The only timestamps were unix microtimes, which are convenient for computers but not for people. In order to make the log more

[5/5] kudu git commit: Fix move constructors to be noexcept

Fix move constructors to be noexcept I noticed while looking at a profile of a tablet server starting up that I saw a lot of CPU used in stacks like: kudu::subtle::RefCountedThreadSafeBase::AddRef() const void std::vector>:: _M_emplace_back_aux> Digging into the source of std::vector,