[1/2] kudu git commit: KUDU-2113 Segfault because of consensus conflict and missing tablet server

2017-08-29 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 5f34a7d09 -> c9b9315a0 KUDU-2113 Segfault because of consensus conflict and missing tablet server Sometimes the master doesn't report all tablet servers. This can happen, for example, when the master is starting. When this happened and

[2/2] kudu git commit: Add metrics for tablet state and tablet copy

2017-08-29 Thread danburkert
Add metrics for tablet state and tablet copy This patch adds a metric to each tablet that records its state, e.g. RUNNING or TOMBSTONED. Metrics systems will need to aggregate the state strings to give server-level statistics on tablet state. This patch also adds metrics for tablet copy

kudu git commit: Update version to 1.6.0-SNAPSHOT

2017-08-28 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 655cf1d5e -> 15e28176f Update version to 1.6.0-SNAPSHOT Change-Id: I961d9804b34ecdc09884273dbcc62d0de1809d8d Reviewed-on: http://gerrit.cloudera.org:8080/7869 Reviewed-by: Adar Dembo Tested-by: Kudu Jenkins Project:

[kudu] Git Push Summary

2017-08-28 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.5.x [created] 655cf1d5e

kudu git commit: [release notes] add new kudu tool features and metrics improvements

2017-08-28 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master ed76e0b2c -> 655cf1d5e [release notes] add new kudu tool features and metrics improvements Change-Id: Ic9514999a913c52fdfc9ed3cf6a85da50b190c68 Reviewed-on: http://gerrit.cloudera.org:8080/7865 Reviewed-by: Adar Dembo

kudu git commit: docs: Add release note for tombstoned voting

2017-08-28 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 975cb5cdb -> ed76e0b2c docs: Add release note for tombstoned voting Change-Id: I14d06ac34ee43cd5592a5fd77074224997812b4e Reviewed-on: http://gerrit.cloudera.org:8080/7858 Tested-by: Mike Percy Reviewed-by: Dan Burkert

kudu git commit: docs: add 1.5.0 release note for tablet copying and flushing improvements

2017-08-28 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 334dcbe2b -> 975cb5cdb docs: add 1.5.0 release note for tablet copying and flushing improvements Change-Id: Idf254bfcad27b66e8fa515be92285c876332a0be Reviewed-on: http://gerrit.cloudera.org:8080/7862 Tested-by: Kudu Jenkins Reviewed-by: Dan

kudu git commit: docs: add 1.5.0 release note for disk failures

2017-08-28 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master c7a22fa6b -> 31ddbcf0b docs: add 1.5.0 release note for disk failures Change-Id: I3ad45527dc32a4946c817ddbef2149433f3b5b89 Reviewed-on: http://gerrit.cloudera.org:8080/7856 Reviewed-by: Dan Burkert Tested-by: Dan

[1/3] kudu git commit: KUDU-1544: Race in Java client's AsyncKuduSession.apply()

2017-08-25 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master faa0b14ef -> 1c70e5df8 KUDU-1544: Race in Java client's AsyncKuduSession.apply() This fixes a potential race in AsyncKuduSession.apply by acquiring the notification deferred early. See the JIRA for details about the race. Acquiring the

[3/3] kudu git commit: release notes: add contributor counts

2017-08-25 Thread danburkert
release notes: add contributor counts Change-Id: Ib385b93671056c37474367266291370d7893550c Reviewed-on: http://gerrit.cloudera.org:8080/7848 Reviewed-by: Todd Lipcon Tested-by: Todd Lipcon Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit:

[2/3] kudu git commit: release notes: add web UI config dashboard entry

2017-08-25 Thread danburkert
release notes: add web UI config dashboard entry Change-Id: I8812e1643afc50af0633343aa76fbea79bc83d72 Reviewed-on: http://gerrit.cloudera.org:8080/7847 Tested-by: Kudu Jenkins Reviewed-by: Todd Lipcon Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit:

kudu git commit: KUDU-2102: fix PosixRWFile::Sync to guarantee durability when used concurrently

2017-08-25 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 2895538e9 -> 9a07b2fed KUDU-2102: fix PosixRWFile::Sync to guarantee durability when used concurrently PosixRWFile has an AtomicBool to optimize the perfomance of Sync() so that when two threads call Sync() at the same time, only one will

kudu git commit: [release-notes] KUDU-2013 is fixed in Kudu 1.5

2017-08-25 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 64c0b15e9 -> 2895538e9 [release-notes] KUDU-2013 is fixed in Kudu 1.5 Change-Id: Ie3098b05747bc19b5d1a8f931d171cd4048c8bbf Reviewed-on: http://gerrit.cloudera.org:8080/7840 Reviewed-by: Todd Lipcon Reviewed-by: Dan Burkert

[2/2] kudu git commit: KUDU-1726: Avoid fsync-per-block in tablet copy

2017-08-24 Thread danburkert
KUDU-1726: Avoid fsync-per-block in tablet copy This patch incorporates BlockTransaction API into tablet copy, to avoid fsync() on each block during copying. Blocks are synced to disk together once the tablet copy is complete. It also introduces a new block manager metric 'total_disk_sync' to

[1/2] kudu git commit: KUDU-1943: Add BlockTransaction to Block Manager

2017-08-24 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 81fad4da9 -> 3d42a264c KUDU-1943: Add BlockTransaction to Block Manager This adds a new layer of abstraction 'BlockTransaction' to Block Manager, to accumulate new blocks in a logical operation, e.g. flush/compaction. Avoid using fsync()

kudu git commit: gradle: target JRE 7 compatibility

2017-08-24 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master f9a6a1b2d -> 81fad4da9 gradle: target JRE 7 compatibility Change-Id: Ib836a748f0f000b07460dd3141940655dbfb7a29 Reviewed-on: http://gerrit.cloudera.org:8080/7785 Tested-by: Kudu Jenkins Reviewed-by: Todd Lipcon Reviewed-by:

kudu git commit: [java] Include Spark/Scala base version in kudu-spark-tools

2017-08-21 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 7986ee028 -> fa2757763 [java] Include Spark/Scala base version in kudu-spark-tools Renames the kudu-spark-tools artifact to include the spark and scala base version. Including the scala version is standard practice and helps users avoid

kudu git commit: KUDU-2104. Upgrade to Spark 2.2.0

2017-08-21 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 055425245 -> 84096d89b KUDU-2104. Upgrade to Spark 2.2.0 Upgrades to Spark 2.2.0 and documents the Java 8 runtime requirement. Also adjusts the Jenkins scripts to use Java 8 when building and running. Change-Id:

kudu git commit: Web UI: switch /maintenance-manager endpoint to mustache

2017-08-10 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 7eb58663f -> ecd67486b Web UI: switch /maintenance-manager endpoint to mustache This cherry-picks just the changes to switch the /maintenance-manager endpoint to use mustache by Sam Okrent in https://gerrit.cloudera.org/#/c/7570/, with the

[1/2] kudu git commit: Add security dashboard to web UI

2017-07-26 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master cb9a6b84d -> 9285f2b44 Add security dashboard to web UI Adds a new "Configuration" page to the web UI that displays a list of security configuration options, their current values, whether or not they're the most secure settings, and

[2/8] kudu git commit: Update web UI style

2017-07-26 Thread danburkert
http://git-wip-us.apache.org/repos/asf/kudu/blob/cb9a6b84/www/jquery-3.2.1.min.js -- diff --git a/www/jquery-3.2.1.min.js b/www/jquery-3.2.1.min.js new file mode 100644 index 000..644d35e --- /dev/null +++

[7/8] kudu git commit: Update web UI style

2017-07-26 Thread danburkert
http://git-wip-us.apache.org/repos/asf/kudu/blob/cb9a6b84/www/bootstrap/css/bootstrap.css -- diff --git a/www/bootstrap/css/bootstrap.css b/www/bootstrap/css/bootstrap.css deleted file mode 100644 index 1b519e2..000 ---

[5/8] kudu git commit: Update web UI style

2017-07-26 Thread danburkert
http://git-wip-us.apache.org/repos/asf/kudu/blob/cb9a6b84/www/bootstrap/img/glyphicons-halflings-white.png -- diff --git a/www/bootstrap/img/glyphicons-halflings-white.png b/www/bootstrap/img/glyphicons-halflings-white.png

[8/8] kudu git commit: Update web UI style

2017-07-26 Thread danburkert
Update web UI style Update to latest versions of jquery and bootstrap, and make some small stylistic changes to the web UI. Change-Id: I8f9de6f5e555e33cbc2e25801cf59b74640d83ee Reviewed-on: http://gerrit.cloudera.org:8080/7504 Tested-by: Kudu Jenkins Reviewed-by: Dan Burkert

[1/8] kudu git commit: Integrate Mustache templates to webserver

2017-07-26 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 0fe3a288d -> cb9a6b84d Integrate Mustache templates to webserver This commit alters the design of webserver path handlers to use Mustache templates instead of generating HTML in server code. Path handlers now accept a json object as a

[4/8] kudu git commit: Update web UI style

2017-07-26 Thread danburkert
http://git-wip-us.apache.org/repos/asf/kudu/blob/cb9a6b84/www/bootstrap/js/bootstrap.min.js -- diff --git a/www/bootstrap/js/bootstrap.min.js b/www/bootstrap/js/bootstrap.min.js index 55da8c3..9bcd2fc 100644 ---

[3/8] kudu git commit: Update web UI style

2017-07-26 Thread danburkert
http://git-wip-us.apache.org/repos/asf/kudu/blob/cb9a6b84/www/jquery-1.11.1.min.js -- diff --git a/www/jquery-1.11.1.min.js b/www/jquery-1.11.1.min.js deleted file mode 100644 index ab28a24..000 --- a/www/jquery-1.11.1.min.js

[1/2] kudu git commit: thirdparty: use ninja when possible

2017-07-25 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 27854fd49 -> be8e3c22b thirdparty: use ninja when possible Using ninja to build LLVM, etc, is a bit faster than using make, and produces less ugly output. Change-Id: Idacd6d8bc9ad371c3c925f7c628ad69785bb870e Reviewed-on:

kudu git commit: Add json builder utility

2017-07-12 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 69ee17dc7 -> 81939782a Add json builder utility The EasyJson class serves as a wrapper around rapidjson Value objects, to simplify their usage. It is intended solely for building json objects, not writing/parsing. Simplifies code like

[1/3] kudu git commit: java: refactor ITClientStress for more reusable code

2017-07-12 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master bce46e12f -> 69ee17dc7 java: refactor ITClientStress for more reusable code Change-Id: I263b5681b52df0fa496cce2f58c0605c66810e64 Reviewed-on: http://gerrit.cloudera.org:8080/7360 Tested-by: Kudu Jenkins Reviewed-by: Jean-Daniel Cryans

[3/3] kudu git commit: util: add Status::AndThen combinator

2017-07-12 Thread danburkert
util: add Status::AndThen combinator The AndThen combinator makes it easier to chain together Status-returning operations when failure recovery is something other than returning early. RETURN_NOT_OK already handles the return-early case elegantly. Also tweaks Status::CloneAndPrepend and

kudu git commit: [java] update dependencies and clean up some deprecate method usage

2017-06-28 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master bbed78cbd -> 15e33c122 [java] update dependencies and clean up some deprecate method usage Change-Id: I4b9746f70187c5a7ca0dd016db92ae4e6de7ecc3 Reviewed-on: http://gerrit.cloudera.org:8080/7276 Tested-by: Kudu Jenkins Reviewed-by: Dan

kudu git commit: KUDU-1929: [rpc] Allow using encrypted private keys for TLS

2017-06-27 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 5a258508f -> 57a07ae72 KUDU-1929: [rpc] Allow using encrypted private keys for TLS * This patch adds a new flag for a "password command" for the RPC private key. * This also makes the webserver move to using the new function

kudu git commit: [java] Ensure the shaded packages are always in the same location

2017-06-27 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master a85b294fb -> 5a258508f [java] Ensure the shaded packages are always in the same location Currently when a module depends on kudu-clients and is shaded it is possible to get bloat in the jar due to dependencies like guava being included in

kudu git commit: [java] remove unused scalamock dependency

2017-06-27 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 52e63d8aa -> a85b294fb [java] remove unused scalamock dependency Change-Id: I97005bd5211abd0b119373ec265b8e9d01b385cf Reviewed-on: http://gerrit.cloudera.org:8080/7292 Tested-by: Kudu Jenkins Reviewed-by: Dan Burkert

kudu git commit: Add mustache as a dependency

2017-06-26 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master baa8d1716 -> ab07952fb Add mustache as a dependency Mustache is a popular template engine. The version added by this commit is a C++ implementation found at https://github.com/henryr/cpp-mustache. Mustache is intended to be use for

kudu git commit: KUDU-1955 refuse to use world-readable keytabs

2017-06-23 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master f9e51ca63 -> 5121df65e KUDU-1955 refuse to use world-readable keytabs Allowing users to supply keytab files and TLS private keys with world-readable permissions lessens a cluster's security. During Kerberos/TLS initialization, servers now

[1/2] kudu git commit: fs_manager: clean tmp files from all special directories

2017-06-21 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 870343f0b -> a5b7ece1b fs_manager: clean tmp files from all special directories Commit c26e09e regressed tmp file cleanup in that the tablet-meta and cmeta directories were no longer cleaned. This is because I assumed that the cleanup done

[2/2] kudu git commit: maven: remove unused repositories, plugins, and dependencies

2017-06-15 Thread danburkert
maven: remove unused repositories, plugins, and dependencies Change-Id: I06513b301923a89334cd8820b5cd4abf11543425 Reviewed-on: http://gerrit.cloudera.org:8080/7201 Reviewed-by: Adar Dembo Tested-by: Kudu Jenkins Project: http://git-wip-us.apache.org/repos/asf/kudu/repo

[3/4] kudu git commit: Fix all 'misc-string-compare' warnings from clang-tidy

2017-06-09 Thread danburkert
Fix all 'misc-string-compare' warnings from clang-tidy See https://clang.llvm.org/extra/clang-tidy/checks/misc-string-compare.html for details (auto-generated using clang-tidy --fix with only this check enabled) Change-Id: I18e609172eda7793f79ed9bb8ddafdf3b85d5fb0 Reviewed-on:

[1/4] kudu git commit: Fix various use-after-move errors

2017-06-09 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 6dade2aa0 -> 8d2ca982f Fix various use-after-move errors This fixes all of the cases where clang-tidy detects 'misc-use-after-move'. This is necessary before enabling protobuf move constructors to avoid test failures. Change-Id:

[2/4] kudu git commit: Enable move constructors for protobufs

2017-06-09 Thread danburkert
Enable move constructors for protobufs This enables an experimental option for protobuf to generate move constructors for protos. This should make storing protobufs in containers more performant, and also should allow us to avoid some current patterns where we use ::Swap() instead of relying on

[4/4] kudu git commit: Tweak build-and-test.sh to fix spark1 test failures

2017-06-09 Thread danburkert
Tweak build-and-test.sh to fix spark1 test failures Maven seems to be caching old artifacts which is leading to hard-to-diagnose test failures in some branches. This commit changes the spark1 test to do a full clean/rebuild instead of selectively removing the kudu-spark target dir. I've also

kudu git commit: Bump googletest

2017-06-05 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 0f31386d2 -> f329e089f Bump googletest The new clang tidy is issuing warnings about gtest macros, the new version seems to fix the issue. Change-Id: I1cf6eee7007857f66b4177d3bfed665d8702e37f Reviewed-on:

kudu git commit: [tools] Use PrintTable to format ksck's consensus matrix

2017-06-05 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.4.x afcd412ba -> 9adbdd771 [tools] Use PrintTable to format ksck's consensus matrix The first version of ksck's consensus matrix used its own table code, but actually there was already nice table code available in tool_action_common.h.

kudu git commit: [java-client] update protoc maven plugin

2017-06-05 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.3.x a1234c05a -> 713703ea1 [java-client] update protoc maven plugin The current protoc maven plugin is not hosted on maven central and the hosting domain is unavailable. It also has not been updated since 2012. It appears like the most

kudu git commit: Bump LLVM to 4.0.0

2017-06-02 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master e3b7d4dc1 -> ce83b6cef Bump LLVM to 4.0.0 I'm working on integrating Thrift into the Kudu build, and hit https://reviews.llvm.org/D22800, which is fixed in 4.0.0. Change-Id: I64f98bb56d2510ba03ef5ea477a4d7bb4b48c555 Reviewed-on:

kudu git commit: KUDU-1911 improve missing required arg message

2017-05-26 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 7e02ede33 -> 86fac3fa2 KUDU-1911 improve missing required arg message When using a tool that has a required positional argument, excluding that argument or passing it as a flag caused a confusing error message to be printed out ("must

kudu git commit: KUDU-1988: add support for advertised host:port info.

2017-05-24 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master e2097c8b9 -> 12683435f KUDU-1988: add support for advertised host:port info. Prior to this change, it was difficult to deploy a Kudu cluster in containerized environments where each container has a unique, local subnetwork with an

[1/2] kudu git commit: Fix flaky test TestRecoverFromOpIdOverflow (again)

2017-05-24 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master d84fc1072 -> c5fe583e9 Fix flaky test TestRecoverFromOpIdOverflow (again) The previous attempt to fix this in commit f0580499dc50e8a47ff6251301cdc15b9b79edcb had a flaw, but this test really does fix the primary source of the flakiness.

kudu git commit: KUDU-2020: tserver failure causes multiple tablet copy operations per under-replicated tablet

2017-05-23 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master fb6cc895b -> d84fc1072 KUDU-2020: tserver failure causes multiple tablet copy operations per under-replicated tablet The 'active ingredient' in this patch is the change to TsTabletManager::StartTabletCopy that causes an ALREADY_INPROGRESS

kudu git commit: docs: revise security doc based on KUDU-1875 and add release notes for 1.4

2017-05-22 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 95b52 -> d354693c1 docs: revise security doc based on KUDU-1875 and add release notes for 1.4 Change-Id: I2e386203f5ed3ef66e2ec136e67738b8c7eb8b1a Reviewed-on: http://gerrit.cloudera.org:8080/6922 Reviewed-by: Dan Burkert

[2/5] kudu git commit: KUDU-2017. Fix calculation of perf improvement for flushes

2017-05-18 Thread danburkert
KUDU-2017. Fix calculation of perf improvement for flushes The code to calculate the MM "perf improvement score" for a flush had two bugs: (1) we calculated threshold - current_usage instead of current_usage - threshold, which resulted in a negative score. (2) we had an unsigned integer

[3/5] kudu git commit: KUDU-1998. Re-enable tcmalloc on OSX

2017-05-18 Thread danburkert
KUDU-1998. Re-enable tcmalloc on OSX We switched back away from using malloc hooks for memory tracking, so this shouldn't be an issue anymore. Change-Id: I9bb3329aa4c12595dedf27d022558671e90206fb Reviewed-on: http://gerrit.cloudera.org:8080/6917 Reviewed-by: Adar Dembo

[1/5] kudu git commit: process_memory: go back to non-incremental tracking

2017-05-18 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master c178563f6 -> 4347174d7 process_memory: go back to non-incremental tracking The incremental tracking has more overhead than expected, even with our fancy striped counters, etc. In tpch_real_world benchmarks, the LongAdder::IncrementBy() and

[4/5] kudu git commit: [c++ client] less logging about authn token

2017-05-18 Thread danburkert
[c++ client] less logging about authn token LOG(INFO) --> VLOG(2) on receiving and adopting authn token by the Kudu C++ client. Change-Id: I13693bc89e528eb4dfdd0e85d6d5b8faac8edadf Reviewed-on: http://gerrit.cloudera.org:8080/6914 Tested-by: Kudu Jenkins Reviewed-by: Todd Lipcon

[5/5] kudu git commit: Fix broken url to "Building a Better Bloom Filter"

2017-05-18 Thread danburkert
Fix broken url to "Building a Better Bloom Filter" Change-Id: Ic113677c7ff4c2080c0ac07cb8f90db77aaeaa52 Reviewed-on: http://gerrit.cloudera.org:8080/6920 Reviewed-by: Dan Burkert Tested-by: Dan Burkert Project:

[1/2] kudu git commit: spark: add support for fault tolerant scanner

2017-05-15 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 4af33ba13 -> b28a59266 spark: add support for fault tolerant scanner This adds support to use fault tolerant scanner for spark job. By default non fault tolerant scanner is used. To turn on fault tolerant scanner, use job config:

[2/2] kudu git commit: [java client] removes duplicate checks at KuduScanToken

2017-05-15 Thread danburkert
[java client] removes duplicate checks at KuduScanToken KUDU-579 added fault tolerant scanner support at java client. However,it missed TODO for KUDU-1040 and results in duplicate code at KuduScanToken. This patch removes it. Change-Id: I7e036d5fe333009a74294d27c54fe846cdc404e9 Reviewed-on:

kudu git commit: Bump glog to 0.3.5

2017-05-12 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master a7e8994e4 -> 4ec6280c9 Bump glog to 0.3.5 Change-Id: I9f638970abfd1aa214cf0032b46064ff7edf4dc4 Reviewed-on: http://gerrit.cloudera.org:8080/6871 Tested-by: Kudu Jenkins Reviewed-by: Dan Burkert Project:

kudu git commit: cmake: fix protoc dependency typo

2017-05-11 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master b2d247fa4 -> a7e8994e4 cmake: fix protoc dependency typo The effect of this typo was that protoc would always be statically linked. Since it's only a build-time dependency, we never noticed previously. Change-Id:

kudu git commit: KUDU-1999: Spark connector should login with Kerberos credentials on driver

2017-05-09 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.3.x 1aa34a1d2 -> ea87fdd8e KUDU-1999: Spark connector should login with Kerberos credentials on driver Tested on a secure cluster using the Spark ITBLL job: spark2-submit \ --deploy-mode=cluster \ --master=yarn \

kudu git commit: KUDU-1999: Spark connector should login with Kerberos credentials on driver

2017-05-09 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master c4c796a25 -> a877566e9 KUDU-1999: Spark connector should login with Kerberos credentials on driver Tested on a secure cluster using the Spark ITBLL job: spark2-submit \ --deploy-mode=cluster \ --master=yarn \ --principal=dan \

[3/3] kudu git commit: [c++-client] clear non-covered entries from meta cache on table open

2017-04-28 Thread danburkert
[c++-client] clear non-covered entries from meta cache on table open Clearing non-covered range entries from the meta cache on table open makes it easier to share client instances among different contexts without having to worry about polluting the non-covering range cache. Change-Id:

[1/3] kudu git commit: [fs] fix build on OS X

2017-04-28 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 7b582a896 -> fdc022fe6 [fs] fix build on OS X Added kudu_fs dependency for kudu_fs_test_util library. Prior to this patch, the following error was reported if trying to build on OS X: Undefined symbols for architecture x86_64:

[2/3] kudu git commit: KUDU-579 [java_client] Scanner fault tolerance

2017-04-28 Thread danburkert
KUDU-579 [java_client] Scanner fault tolerance This patch adds java client support to restart scanners if they fail in the middle of table scanning. AsyncKuduScanner records the final primary key retrieved in the previous batch. If a tserver fails while scanning, the client marks the tserver as

[2/2] kudu git commit: Add 'kudu master list' tool

2017-04-21 Thread danburkert
Add 'kudu master list' tool This also sneaks in a couple other changes: - kudu tserver list now has a 'heartbeat' column option to display the number of ms since the last heartbeat. - A fix for the pretty table formatter. Change-Id: I9e77d41b244143d1258b26b50c4d2a68dedd8f00 Reviewed-on:

kudu git commit: Add 'kudu tserver list' tool

2017-04-20 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master b2843ee5e -> 319a1bc15 Add 'kudu tserver list' tool This adds a new tool action to list tablet servers, and associated information. There are a few output formatting options: the default is a human-readable table in the psql style; the

kudu git commit: [docs] Revise disk-failure-recovery workflow

2017-04-19 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.3.x 3211d9781 -> 73918d055 [docs] Revise disk-failure-recovery workflow Missed a few comments in the original commit. Change-Id: I35c2ca56fdcf07917e7e0ba7f0d9be7b6bb19568 Reviewed-on: http://gerrit.cloudera.org:8080/6608 Reviewed-by:

kudu git commit: [docs] Add security guide

2017-04-10 Thread danburkert
htly_etl_service_account +--admin-acl=hadoopadmin +``` + +Further information about these flags can be found in the configuration +flag reference. +// TODO(todd) add a link + + +[[known-limitations]] +== Known Limitations + +Kudu has a few known security limitations: + +// TODO(danburkert): add JIR

kudu git commit: Add ksck section to admin guide common workflows

2017-04-10 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master bd24f04fb -> 9d03677e4 Add ksck section to admin guide common workflows I've often wanted this when helping people through ksck. Change-Id: I9631337b113d2c67be0057f728c68f792e8a4fd6 Reviewed-on: http://gerrit.cloudera.org:8080/6598

[2/2] kudu git commit: KUDU-1966: Data directories can be removed erroneously

2017-04-10 Thread danburkert
KUDU-1966: Data directories can be removed erroneously Also revises the error messages in PathInstanceMetadataFile::CheckIntegrity to be in terms of data directories, since this is what end-users will be familiar with. These errors should be rare, but they can happen if a user is monkeying around

kudu git commit: Add --redact=none flag option for specifying no redaction

2017-04-07 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master b95d3a4fa -> f304dd32f Add --redact=none flag option for specifying no redaction It's still possible to specify no redaction via --redact='', but I was only able to determine how to do that by reading the code. I think this makes it a bit

[1/2] kudu git commit: generic_iterators: avoid holding completed iters in merge/union

2017-04-05 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 27b3de7ab -> eca517108 generic_iterators: avoid holding completed iters in merge/union Previously, the MergeIterator and UnionIterator would hold on to all of the iterators they consume until the entire process completed. Each such iterator

[1/4] kudu git commit: rpc-bench: add ability to enable encryption

2017-04-05 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master f4272eb5c -> 34310a8e9 rpc-bench: add ability to enable encryption This adds a --enable-encryption flag for the benchmark which disables the "negotiation-only" TLS optimization and enables SSL on the server. This allows us to benchmark

[3/4] kudu git commit: Spark ITBLL

2017-04-05 Thread danburkert
Spark ITBLL This adds a new end-to-end test for Spark based on the existing MapReduce ITBLL test. Common parts of the two have been abstracted into a utility class. The implementation is only compatible with Spark2/Scala2.11, so the new submodule is deactivated when compiling with the

[4/4] kudu git commit: kudu-spark: switch default profile from spark1/2.10 to spark2/2.11

2017-04-05 Thread danburkert
kudu-spark: switch default profile from spark1/2.10 to spark2/2.11 Now that we have kudu-spark-tools which only works with Spark2, it's probably best if developers are defaulted to running tests with Spark2. This shouldn't affect CI or published artifacts. Change-Id:

kudu git commit: docs: Fix axis lable for hash-range-partitioning

2017-04-03 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master bee8e875e -> 7206041f9 docs: Fix axis lable for hash-range-partitioning Horizontal axis label should show range partition by time according to the example not (host, metric) Change-Id: I619d2082941de257cee4270079b46116944c77a6

kudu git commit: [consensus] add lock coverage for peer queue

2017-03-10 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 80f951236 -> 4a33783ee [consensus] add lock coverage for peer queue A Peer or PeerQueue would lock without taking into account the presence of a peer calling back after TabletCopy and updating its members, which doesn't abide by external

kudu git commit: [util] fixed env_util-test.cc compilation on OS X

2017-03-10 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.3.x dfd9b491f -> d784f04d8 [util] fixed env_util-test.cc compilation on OS X The prior version failed to compile on OS X with clang version 3.9.0 (branches/release_39 277311) with error messaged containing:

kudu git commit: [macOS] fix security test failures

2017-03-10 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 5759a08d3 -> c5ec0ddb0 [macOS] fix security test failures Change-Id: I5c4d464edc09f41151f95ad2f0e694f2bc00b0f0 Reviewed-on: http://gerrit.cloudera.org:8080/6346 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin

[2/2] kudu git commit: java: avoid spewing ClosedChannelException on client-initiated disconnects

2017-03-08 Thread danburkert
java: avoid spewing ClosedChannelException on client-initiated disconnects When submitting an MR job, the submitting program opens a client instance for a short amount of time, just to fetch an authentication token. When it then calls 'close()', any TabletClient connections that the client may

[1/2] kudu git commit: [rpc] Do not send error reply for invalid connection context

2017-03-08 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.3.x 86055e295 -> 27b850469 [rpc] Do not send error reply for invalid connection context We recently introduced a nonce into the KRPC protocol when using Kerberos for authentication. The client sends the encoded nonce to the server in the

[3/3] kudu git commit: Fix Webserver option 'password_file' to pass correct Squeasel option 'global_auth_file' for enabling HTTP authorization.

2017-03-08 Thread danburkert
Fix Webserver option 'password_file' to pass correct Squeasel option 'global_auth_file' for enabling HTTP authorization. Also add test case for web UI .htpasswd support. Change-Id: I2d30f450abfb3d0addc0eef39bcf78c87e4298c5 Reviewed-on: http://gerrit.cloudera.org:8080/6300 Tested-by: Kudu Jenkins

[2/3] kudu git commit: [sys_catalog] no copying into intermediate KuduPartialRow

2017-03-08 Thread danburkert
[sys_catalog] no copying into intermediate KuduPartialRow Use KuduPartialRow::SetStringNoCopy() instead of KuduPartialRow::SetString() while setting fields for the system catalog table. Copying the data into the intermediate KuduPartialRow object is not necessary: the intermediate object is not

[1/3] kudu git commit: KUDU-1887: Add RpcContext::DiscardTransfer()

2017-03-08 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 72895966c -> 937064f91 KUDU-1887: Add RpcContext::DiscardTransfer() DiscardTransfer() allows RPC handlers to release the memory backing an inbound call, without being forced to respond to the RPC at the same time. Change-Id:

[1/2] kudu git commit: [exactly_once_rpc-test] Reduce flakyness of TestExactlyOnceSemanticsGarbageCollection

2017-03-07 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 97831eadf -> f24307db4 [exactly_once_rpc-test] Reduce flakyness of TestExactlyOnceSemanticsGarbageCollection This test is failing frequently in the last call with either: Bad status: Network error: Recv() got EOF from remote (error 108) or

[2/2] kudu git commit: [rpc] Do not send error reply for invalid connection context

2017-03-07 Thread danburkert
[rpc] Do not send error reply for invalid connection context We recently introduced a nonce into the KRPC protocol when using Kerberos for authentication. The client sends the encoded nonce to the server in the connection context message, to which the client does not expect a reply. This commit

kudu git commit: KUDU-1897: disable Kerberos replay cache

2017-03-06 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master f65feff68 -> 45548c90c KUDU-1897: disable Kerberos replay cache Change-Id: Ifbce55a0b12682fdf69e7b2c361c6336495db64d Reviewed-on: http://gerrit.cloudera.org:8080/6254 Reviewed-by: Todd Lipcon Tested-by: Kudu Jenkins

kudu git commit: Java Client: authenticate with login username when using SASL PLAIN

2017-03-01 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master abbd09a57 -> 55c3db350 Java Client: authenticate with login username when using SASL PLAIN Also tweaks up some logging output I noticed when reviewing the negotiation tests for this change. Change-Id:

kudu git commit: Destabilize rpc_authentication and rpc_encryption flags

2017-03-01 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 1716d61ce -> abbd09a57 Destabilize rpc_authentication and rpc_encryption flags These flags are not yet ready to be stabilized, it's possible they will go through some changes in the future. Change-Id:

kudu git commit: Bump gflags to 2.2.0

2017-03-01 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 93b54f0ff -> 1716d61ce Bump gflags to 2.2.0 Flags can now be specified with dashes instead of underscores‼️️ Change-Id: I400dc8eb74cce47848497342dbe5f945e1a98461 Reviewed-on: http://gerrit.cloudera.org:8080/6207 Reviewed-by: Todd

[1/2] kudu git commit: KUDU-1851 - [python] TableAlterer direct instantiation causes SIGSEGV

2017-03-01 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.2.x 32d4d5f3c -> 1af814b09 KUDU-1851 - [python] TableAlterer direct instantiation causes SIGSEGV The python client is currently crashing when a TableAlterer is directly instantiated as opposed to instantiating via the new_table_alterer

[1/2] kudu git commit: KUDU-1901. Fix crash in concurrent OpenSSL usage

2017-03-01 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 0efc1e26a -> 0609277c4 KUDU-1901. Fix crash in concurrent OpenSSL usage This fixes a crash in which a TLSContext's SSL_CTX was being used by a thread to accept a connection at the same time as its underlying certificate was being changed.

[2/2] kudu git commit: KUDU-1852. KuduTableAlterer should not crash with nullptr arguments

2017-03-01 Thread danburkert
KUDU-1852. KuduTableAlterer should not crash with nullptr arguments Change-Id: Ifa7c34b476111ecd33d4fd3ef5cc363a410ad76a Reviewed-on: http://gerrit.cloudera.org:8080/5797 Reviewed-by: Dan Burkert Tested-by: Kudu Jenkins (cherry picked from commit

[2/2] kudu git commit: Workaround a leak in OpenSSL 1.0.0

2017-03-01 Thread danburkert
Workaround a leak in OpenSSL 1.0.0 This fixes another leak seen occasionally as a flaky test on RHEL 6: Direct leak of 56 byte(s) in 1 object(s) allocated from: #2 0x7f21e5fe037c in CRYPTO_malloc src/openssl/crypto/mem.c:306 #3 0x7f21e607f45c in EVP_PKEY_new

kudu git commit: KUDU-1899. Fix support for empty string keys

2017-03-01 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.2.x 7fa56a53a -> 32d4d5f3c KUDU-1899. Fix support for empty string keys We've never prevented users from inserting a row with "" as a key. However, a faulty assertion in diskrowset.cc caused the tablet server to crash on flush in this

[4/5] kudu git commit: Make reduce flake from KUDU-1642 tests in client-test

2017-02-28 Thread danburkert
Make reduce flake from KUDU-1642 tests in client-test This ups the timeouts so the tests should be less flaky. It also removes a couple stray zeroes and improves the flush behavior. Change-Id: Iba01eb7e9976f2fae4061ba918457c946d98c067 Reviewed-on: http://gerrit.cloudera.org:8080/5882 Tested-by:

[3/5] kudu git commit: [client-test] Reduce flakyness of TestWriteWithDeadTabletServer

2017-02-28 Thread danburkert
[client-test] Reduce flakyness of TestWriteWithDeadTabletServer This test fails often due to an unexpected reason for the write timeout. This is related to KUDU-1466. I just commented out the assertion and left a TODO to reenable once we address the jira. Also added some info to the jira itself.

[2/5] kudu git commit: KUDU-1880 Prevent eager ignoring of NULLs

2017-02-28 Thread danburkert
KUDU-1880 Prevent eager ignoring of NULLs During a nullable block scan, if a NULL set of cells is reached before the decoder evaluation is disabled (e.g. if the first rows in the block are NULL), the rows will be prematurely marked as not in the results set. If evaluating an IsNull predicate,

<    1   2   3   4   5   6   7   >