[1/2] kudu git commit: Add ExtractDouble method to JsonReader and small improvements

2018-10-19 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 785490ce5 -> b3486d99a Add ExtractDouble method to JsonReader and small improvements Change-Id: Iccbdc2467746b68ac871f5feeb01e610610b13d6 Reviewed-on: http://gerrit.cloudera.org:8080/11728 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin

[2/2] kudu git commit: Add missing test dependency for Hive tests

2018-10-19 Thread danburkert
Add missing test dependency for Hive tests The HMS client internally calls into the JobConf class, which is provided by the org.apache.hadoop.hadoop-mapreduce-client-core artifact. The HMS only specifies an optional dependency on hadoop-mapreduce-client-core, so we are forced to depend on it

[2/3] kudu git commit: thrift operators: add more test coverage

2018-10-17 Thread danburkert
thrift operators: add more test coverage This is a follow-up to Adar's suggestions in https://gerrit.cloudera.org/c/11693/. Change-Id: Ifdba996a9c4e6e01f55c257bfd10fb3f74f0f894 Reviewed-on: http://gerrit.cloudera.org:8080/11713 Reviewed-by: Adar Dembo Tested-by: Kudu Jenkins Project:

[1/3] kudu git commit: sentry module: introduce using declarations for generated types

2018-10-17 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 36d09f187 -> 082bbfe20 sentry module: introduce using declarations for generated types This cleans up the .cc files in the sentry module with 'using' declarations, as suggested by Adar. Change-Id: I51f61eedc1069d362fa1ce26ce0449cce8f0846b

[3/3] kudu git commit: HMS integration: provide Java API to override owner during table create

2018-10-17 Thread danburkert
HMS integration: provide Java API to override owner during table create This is necessary so that Impala can override the table owner, instead of using the inferred username of the connection (which would be 'impala'). This API is marked as unstable since it's not clear we will want to support it

kudu git commit: Fix thrift operator< implementations

2018-10-16 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 09848055b -> 58ecdd155 Fix thrift operator< implementations The previous implementation didn't account for unset optional fields, and did not short-circuit after finding the first field which was smaller. Buggy operator< is problematic

[1/2] kudu git commit: [compaction] KUDU-2056: Expose a metric for how much a tablet needs to be compacted

2018-10-16 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 8c184e2a9 -> 09848055b [compaction] KUDU-2056: Expose a metric for how much a tablet needs to be compacted This adds a new metric 'average_diskrowset_height' that reflects how uncompacted a tablet replica is. This metric is obtained by

[2/2] kudu git commit: Update Java README Intellij test runner instructions

2018-10-16 Thread danburkert
Update Java README Intellij test runner instructions Change-Id: Ie372417350f50a7d4f5567cd991ac5d886329cdb Reviewed-on: http://gerrit.cloudera.org:8080/11700 Reviewed-by: Grant Henke Tested-by: Grant Henke Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit:

kudu git commit: Implement BloomFilter Predicate in server side.

2018-10-11 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master c8dd7b53f -> 8af288a26 Implement BloomFilter Predicate in server side. Change-Id: I62c2de42667d0255d94e19db773240f7f9ee636c Reviewed-on: http://gerrit.cloudera.org:8080/11100 Tested-by: Kudu Jenkins Reviewed-by: Dan Burkert Project:

kudu git commit: Improve logging for mini HMS and Sentry services

2018-10-09 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 88edfc36a -> 61aed1a58 Improve logging for mini HMS and Sentry services This commit adds custom logging configurations for both mini HMS and Sentry. Both services now log warnings or errors to the console, and info level messages to a file

kudu git commit: Bump Sentry and Hadoop versions

2018-10-08 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 2d6f8ffa9 -> 8d6cfe10d Bump Sentry and Hadoop versions The motivation is to get access to SENTRY-2371, which introduces a new Thrift interface that will be used by Kudu to retrieve user privileges. SENTRY-2371 has not been released yet, so

kudu git commit: Extract connection retrying and HA support from HmsCatalog

2018-10-05 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master f1f192ca3 -> 42b75278a Extract connection retrying and HA support from HmsCatalog Introduces a new abstraction, thrift::HaClient, for providing HA and retrying support on top of Thrift clients like HmsClient and SentryClient. The

[3/4] kudu git commit: KUDU-2541: Fill out basic sentry client API

2018-09-26 Thread danburkert
KUDU-2541: Fill out basic sentry client API This commit contains more plumbing for the upcoming Sentry integration (KUDU-428). The Sentry client class is filled out with enough methods to ensure the Thrift RPC is working correctly, as well as error conversion from Sentry and Thrift's format to

[4/4] kudu git commit: KUDU-2541: add Kerberos support to Sentry client and mini-cluster

2018-09-26 Thread danburkert
KUDU-2541: add Kerberos support to Sentry client and mini-cluster More infrastructure work towards KUDU-428. Change-Id: I3f7e27137135cb9b463a90b98e4aba864cece3c1 Reviewed-on: http://gerrit.cloudera.org:8080/11525 Reviewed-by: Hao Hao Reviewed-by: Adar Dembo Tested-by: Kudu Jenkins Project:

[1/4] kudu git commit: [DOCS] Added a version notice for changing managed Kudu table names in Impala

2018-09-26 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master c9f1772e7 -> ecc4998cb [DOCS] Added a version notice for changing managed Kudu table names in Impala With IMPALA-5654, users can no longer change kudu.table_name property for managed Kudu tables in Impala. Change-Id:

[2/4] kudu git commit: KUDU-428: add Sentry to thirdparty, mini-sentry

2018-09-26 Thread danburkert
KUDU-428: add Sentry to thirdparty, mini-sentry This commit adds Sentry to thirdparty, and fills out the MiniSentry class with an initial implementation. Notable features that aren't implemented: - Stripped Sentry packaging. I've put an unmodified version of Sentry 2.0.1 into thirdparty. It

[2/2] kudu git commit: Add Hive Metastore service principal configuration

2018-09-25 Thread danburkert
Add Hive Metastore service principal configuration This commit adds a new flag, --hive_metastore_kerberos_principal flag which corresponds to the HMS 'hive.metastore.kerberos.principal' configuration. This configuration is rarely overridden, but in cases where it is, having a way to match it in

[1/2] kudu git commit: hybrid_clock: restore SleepFor in WaitUntilAfterLocally

2018-09-25 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master ddaa766e2 -> a09c89abc hybrid_clock: restore SleepFor in WaitUntilAfterLocally This seems to have gone missing from commit f2d96437c. The end result is a busy loop, which was probably unintentional. I also removed the loop, which seemed

kudu git commit: Add thrift module for common thrift utilities

2018-09-21 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 6129467d9 -> ddaa766e2 Add thrift module for common thrift utilities The HMS patch series inlined all necessary Thrift utilities into the hms module, since it was the only use of Thrift in the codebase. Now that we're also planning on

kudu git commit: HMS integration: set table owner field in HMS table metadata

2018-09-11 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master ed60c11a3 -> 97dbffb95 HMS integration: set table owner field in HMS table metadata Other systems use table ownership for purposes like assigning privileges. This patch sets the owner field in the HMS for Kudu tables to the client's user

[1/3] kudu git commit: [master] fix compilation warning of DCHECK_NOTNULL

2018-09-11 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master f0716cc52 -> ed60c11a3 [master] fix compilation warning of DCHECK_NOTNULL Fixed compilation warning in case of RELEASE build: kudu/master/ts_descriptor.cc:103:18: warning: expression result unused [-Wunused-value]

[3/3] kudu git commit: move KUDU_NO_EXPORT attribute on methods

2018-09-11 Thread danburkert
move KUDU_NO_EXPORT attribute on methods The original location was ineffective due to a bug in older versions of GCC, [1] appears to be the best tracking issue. Adding the annotation at the end of the method signature silences the warnings on GCC 4.8, and maintains symbol hiding as checked with

[2/3] kudu git commit: build: add option to ignore test failures

2018-09-11 Thread danburkert
build: add option to ignore test failures Sometimes when running build-and-test.sh, we may not care about the failure of any given test, so long as the tests ran, e.g. if running only for the sake of reporting the results of the tests. As such, this patch adds an option to ignore test failures

[2/2] kudu git commit: KUDU-428: Sentry integration scaffolding

2018-08-28 Thread danburkert
KUDU-428: Sentry integration scaffolding In preparation for KUDU-428 (Sentry integration), this commit introduces some basic module and build-system scaffolding, including a new 'sentry' module, as well as the Sentry service thrift definition files. This is heavily based on the HMS integration

kudu git commit: Add unstable client APIs to fetch HMS integration configuration status

2018-08-24 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master a8d088e11 -> 2c572cbd4 Add unstable client APIs to fetch HMS integration configuration status The Impala/Kudu integration must know whether the Kudu cluster is configured with the HMS integration enabled. This commit adds an unstable API to

[2/6] kudu git commit: [webui] Add partition and on-disk size to /tablet

2018-08-23 Thread danburkert
[webui] Add partition and on-disk size to /tablet Change-Id: If8de8e3232fc608dd31ab7275b29805b0abdc140 Reviewed-on: http://gerrit.cloudera.org:8080/11289 Reviewed-by: Alexey Serbin Tested-by: Will Berkeley Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit:

[5/6] kudu git commit: [dist-test] Fix the descriptions of Java tests

2018-08-23 Thread danburkert
[dist-test] Fix the descriptions of Java tests Removes the extra trailing “.isolate” in the descriptions of Java tests run on dist-test. Change-Id: I6fd893e46b0cb2f3c23900130dc2c904d909a2ae Reviewed-on: http://gerrit.cloudera.org:8080/11300 Tested-by: Kudu Jenkins Reviewed-by: Andrew Wong

[4/6] kudu git commit: [webui] Fancy tables for /maintenance-manager

2018-08-23 Thread danburkert
[webui] Fancy tables for /maintenance-manager Change-Id: I725c13249b2cdc95fa674135c75b1414b2fc9047 Reviewed-on: http://gerrit.cloudera.org:8080/11291 Reviewed-by: Alexey Serbin Tested-by: Will Berkeley Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit:

[1/6] kudu git commit: [webui] Convert /tablet and its offshoots to mustache

2018-08-23 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 9ad90eebe -> 36187ef62 [webui] Convert /tablet and its offshoots to mustache This patch converts the following pages to mustache templates: * /tablet * /tablet-consensus-status * /tablet-rowsetlayout-svg * /log-anchors It does the

[3/6] kudu git commit: [webui] Make the tables on /tablets fancy

2018-08-23 Thread danburkert
[webui] Make the tables on /tablets fancy The /tablets page has live tablets and a tombstoned tablets tables. These tables get big and unwieldy to use. This patch makes the tables paginated and searchable by adding the bootstrap-table plugin[1]. I picked this plugin because it integrates with our

[6/6] kudu git commit: Enable iwyu on hms files

2018-08-23 Thread danburkert
Enable iwyu on hms files Change-Id: I7d4a5b23683b4fbec12fa86c24d997677ee3113a Reviewed-on: http://gerrit.cloudera.org:8080/11301 Reviewed-by: Adar Dembo Tested-by: Kudu Jenkins Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit:

kudu git commit: Bump Hive version, normalize hive package name

2018-08-21 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 368990140 -> 60020cf32 Bump Hive version, normalize hive package name This commit bumps the Hive version in thirdparty, and strips the 'apache' prefix and 'bin' suffix from the thirdparty tarball package, which brings it inline with the

kudu git commit: KUDU-2540: Authorization failures on exactly-once RPCs cause FATAL

2018-08-14 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.7.x cd28c7553 -> b24f166e3 KUDU-2540: Authorization failures on exactly-once RPCs cause FATAL See the associated JIRA for bug details. The fix follows a slightly different approach than suggested by Todd, namely initialization of the RPC

kudu git commit: KUDU-2540: Authorization failures on exactly-once RPCs cause FATAL

2018-08-14 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.6.x 6dfc10ac0 -> c50b7fe69 KUDU-2540: Authorization failures on exactly-once RPCs cause FATAL See the associated JIRA for bug details. The fix follows a slightly different approach than suggested by Todd, namely initialization of the RPC

kudu git commit: KUDU-2540: Authorization failures on exactly-once RPCs cause FATAL

2018-08-14 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 2d92370d9 -> e795d5a82 KUDU-2540: Authorization failures on exactly-once RPCs cause FATAL See the associated JIRA for bug details. The fix follows a slightly different approach than suggested by Todd, namely initialization of the RPC

[1/4] kudu git commit: [tools] --report_only option for 'kudu cluster rebalance'

2018-08-01 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 6c68e64d8 -> b2c642890 [tools] --report_only option for 'kudu cluster rebalance' Added --report_only option for the 'kudu cluster rebalance' CLI tool along with corresponding test. Change-Id: Id8a81448a31ba76bb651b13e052f2c508cd0acd6

[2/4] kudu git commit: hms tools: do not require HMS configuration flags

2018-08-01 Thread danburkert
hms tools: do not require HMS configuration flags The HMS tools will now lookup required HMS configuration (hive_metastore_uris and hive_metastore_sasl_enabled) from the master when it is not provided via command line flags. Looking up the configs on the master is more ergonomic and less

[4/4] kudu git commit: hms-tool: lookup master addresses config from master

2018-08-01 Thread danburkert
hms-tool: lookup master addresses config from master The HMS check and fix tools currently use the master addresses configuration provided on the command line to validate and fix metadata in HMS table entries. This only works correctly if the administrator inputs the master addresses exactly as

[3/4] kudu git commit: hms precheck tool

2018-08-01 Thread danburkert
hms precheck tool This commit adds a 'kudu hms precheck' tool that is meant to be used before enabling the HMS integration on an existing cluster. The tool checks for tables whose names are not unique when compared with Hive's case-insensitive identifier rules. These conflicting tables would

kudu git commit: KUDU-2509 fix use-after-free in case of WAL replay error

2018-08-01 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.7.x cf21f3673 -> 7fba099d5 KUDU-2509 fix use-after-free in case of WAL replay error Fixed use-after-free mistake in case of a failure to apply a pending commit message from the WAL while bootstrapping a tablet. Also, a repro scenario to

[4/4] kudu git commit: hms-tool: refactor check tool and combine upgrade and fix

2018-07-30 Thread danburkert
hms-tool: refactor check tool and combine upgrade and fix This commit combines the 'hms upgrade' and 'hms fix' tools. The tools previously had overlapping responsibilities for checking some types of inconsistencies. The 'fix' tool now has flags which can control whether it attempts certain types

[2/4] kudu git commit: [kudu-admin-test] improvements on error reporting

2018-07-30 Thread danburkert
[kudu-admin-test] improvements on error reporting This changelist introduces improvements on logging in case if an assertion triggers in the rebalancer-related scenarios. I found it's hard to troubleshoot the flakiness reported lately in the

[1/4] kudu git commit: [util] update on ParseVersion()

2018-07-30 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 34b058b8b -> 8e40dfdb9 [util] update on ParseVersion() This changelist updates implementation of the kudu::ParseVersion() utility function to recognize and successfully parse version strings like '1.8.0-cdh6.x-SNAPSHOT'.

[3/4] kudu git commit: hms-tool: refactor check tool and combine upgrade and fix

2018-07-30 Thread danburkert
http://git-wip-us.apache.org/repos/asf/kudu/blob/8e40dfdb/src/kudu/tools/tool_action_hms.cc -- diff --git a/src/kudu/tools/tool_action_hms.cc b/src/kudu/tools/tool_action_hms.cc index 6273ed2..db90ee3 100644 ---

[1/2] kudu git commit: Add kudu-backup module to Maven build

2018-07-26 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 0418b225e -> 34b058b8b Add kudu-backup module to Maven build Change Base64 encode/decode to java 7 compatible versions to fix sniffer errors. Change-Id: Ia1b34b733d8d85106060068e611be7eecc0717f2 Reviewed-on:

[2/2] kudu git commit: [DOCS] Fixed a broken link in administration.adoc

2018-07-26 Thread danburkert
[DOCS] Fixed a broken link in administration.adoc Change-Id: I7ca94cc24b3038659b89c62d1e1ac18e74acf65a Reviewed-on: http://gerrit.cloudera.org:8080/11066 Reviewed-by: Dan Burkert Tested-by: Dan Burkert Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit:

kudu git commit: Add alter_external_catalogs flag to table rename tool

2018-07-24 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 141ee5a52 -> d6ee0047c Add alter_external_catalogs flag to table rename tool It's going to be necessary to be able to alter the Kudu catalog independently of the HMS in order to repair and upgrade tables when the HMS integration is turned

kudu git commit: Allow non-HMS compatible tables to be renamed after turning on HMS integration

2018-07-24 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master ccdb6b557 -> 7fe04438e Allow non-HMS compatible tables to be renamed after turning on HMS integration This is a bug fix followup to cee17c03bc30037bf2a, which introduced automatic downcasing of table names during new table creation and

kudu git commit: hms-tool: replace a few copies with const refs

2018-07-23 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master a68c9355b -> ec1e47f41 hms-tool: replace a few copies with const refs This patch contains no functional changes. Change-Id: Id2ace8469b82c995f6f528aff4322bf4fed22bd2 Reviewed-on: http://gerrit.cloudera.org:8080/11015 Reviewed-by: Hao Hao

[2/2] kudu git commit: [DOCS] Updated the max and recommended numbers of tablets per server

2018-07-18 Thread danburkert
[DOCS] Updated the max and recommended numbers of tablets per server Change-Id: Ida16e45477bd18206f25db8cc2bf890742d4ec63 Reviewed-on: http://gerrit.cloudera.org:8080/10969 Tested-by: Kudu Jenkins Reviewed-by: Dan Burkert Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit:

[1/2] kudu git commit: [DOCS] Removed KUDU-1942 and KUDU-2032 from Known Issues List

2018-07-18 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master c2cecb005 -> 521189744 [DOCS] Removed KUDU-1942 and KUDU-2032 from Known Issues List Change-Id: I84bdf0193df08241ea632e1b1b51476bba40403c Reviewed-on: http://gerrit.cloudera.org:8080/10953 Tested-by: Kudu Jenkins Reviewed-by: Dan Burkert

kudu git commit: KUDU-2312: Scan predicate application ordering is non-deterministic

2018-07-17 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.6.x f0d95c27e -> f43c77bed KUDU-2312: Scan predicate application ordering is non-deterministic This changes the scan predicate evaluation ordering so that it primarily orders by selectivity (as before), but breaks ties by column index.

kudu git commit: KUDU-2447 Fix TS crashed with "NONE predicate can not be pushed into key"

2018-07-17 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.7.x a307f2a31 -> 2a8893e3e KUDU-2447 Fix TS crashed with "NONE predicate can not be pushed into key" tserver crashes on Scan when [lowerPrimaryKey, upperPrimary) and predicates of primary column do not overlap Change-Id:

kudu git commit: hms-tool: filter non-Kudu tables in the HMS

2018-07-12 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master f62bbdd64 -> d074de85d hms-tool: filter non-Kudu tables in the HMS Renames HmsCatalog::RetrieveTables to GetKuduTables and changes the semantics such that only Kudu tables are returned. The only caller was the HMS tool, and it only needs to

kudu git commit: KUDU-2191: downcase/normalize table names during DDL

2018-07-11 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 4eefc8d4e -> cee17c03b KUDU-2191: downcase/normalize table names during DDL This is a followup to 7b048b8dbe which changed the catalog manager to be case preserving, but insensitive on lookup when the HMS integration is enabled. It turns

kudu git commit: KUDU-2492: Make the use of SO_REUSEPORT conditional on it being defined

2018-07-06 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 8c744afa9 -> af8bb879c KUDU-2492: Make the use of SO_REUSEPORT conditional on it being defined A recent commit to Kudu, "rpc: add experimental rpc_reuseport flag", added the use of the rpc flag SO_REUSEREPORT. This flag is not available

kudu git commit: KUDU-2191: support table-name identifiers with upper case chars

2018-07-03 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 1ae050e4d -> 7b048b8db KUDU-2191: support table-name identifiers with upper case chars Summary: When the HMS integration is enabled, Kudu now preserves table name casing, but uses case-insensitive lookups to retrieve tables. Background:

kudu git commit: KUDU-2191: Adjust upgrade tool to tolerate downgraded metadata

2018-07-02 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master f5c802ced -> 95182a6df KUDU-2191: Adjust upgrade tool to tolerate downgraded metadata Currently, TestHmsDowngrade keeps on failing as the metadata upgrade tool does not handle tables being downgraded properly. This issue didn't surface

[2/2] kudu git commit: Add regression test for HIVE-19569

2018-06-27 Thread danburkert
Add regression test for HIVE-19569 HIVE-19569 affects some versions of the Hive 3.x line, and it will break our integration. This additional test case will alert us if our bundeled Hive version becomes susceptible. Change-Id: I560c65c9f56ed64fd13897951d2a719abb5c066b Reviewed-on:

[2/2] kudu git commit: KUDU-2191: reject alter table rename to same table name

2018-06-27 Thread danburkert
KUDU-2191: reject alter table rename to same table name This commit ensures 'ALTER TABLE t RENAME TO t;' operations will fail validation in the catalog manager when the HMS integration is enabled. The HMS allows these types of alterations, however Kudu (without HMS integration) does not, so this

[1/2] kudu git commit: Add a simple metric for cluster skew

2018-06-27 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 9c89c2a9a -> 29c66db48 Add a simple metric for cluster skew This adds a very simple 'cluster_skew' metric to the master that reports on the difference in number of replicas between the most and least loaded tablet servers. This information

kudu git commit: KUDU-2471: Fix ColumnSchema.equals NPE with non-Decimal columns

2018-06-27 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.7.x 8d7221570 -> a307f2a31 KUDU-2471: Fix ColumnSchema.equals NPE with non-Decimal columns Fixes a potential NPE on non-decimal columns and adds tests to prevent the issue in the future. Change-Id:

[2/3] kudu git commit: [consensus] update UnsafeChangeConfig() signature

2018-06-22 Thread danburkert
[consensus] update UnsafeChangeConfig() signature While testing various scenarios involving kudu CLI's 'remote_replica unsafe_change_config' sub-command, it turned out that tserver might end up trying to respond with invalid value of TabletServerErrorPB::Code. In that case, a tserver built in

[1/3] kudu git commit: KUDU-2260: Log block manager should handle null bytes in metadata on crash

2018-06-22 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master c7918e87b -> f6e8fe6c6 KUDU-2260: Log block manager should handle null bytes in metadata on crash On ext4 with data=ordered (the default), it's possible for a write to persist an increase to the filesize without persisting the actual data.

[3/3] kudu git commit: Synchronizer::WaitFor thread-safety

2018-06-22 Thread danburkert
Synchronizer::WaitFor thread-safety WaitFor's implementation previously was not thread safe in the case that the waiter deallocated the Synchronizer after receiving a timeout. Change-Id: I9565a50839ffd23b5bac6986a6fdee41ac21aa3a Reviewed-on: http://gerrit.cloudera.org:8080/10783 Reviewed-by:

kudu git commit: KUDU-2447 Fix TS crashed with "NONE predicate can not be pushed into key"

2018-06-21 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 61fb6dc30 -> c7918e87b KUDU-2447 Fix TS crashed with "NONE predicate can not be pushed into key" tserver crashes on Scan when [lowerPrimaryKey, upperPrimary) and predicates of primary column do not overlap Change-Id:

[1/2] kudu git commit: KUDU-2191 (12/n): Hive Metastore notification log event listener

2018-06-15 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master b5f3d1a10 -> 5f1ca32f3 http://git-wip-us.apache.org/repos/asf/kudu/blob/5f1ca32f/src/kudu/master/hms_notification_log_listener.h -- diff --git

kudu git commit: Bump Hive version

2018-06-12 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master c1c15ad5f -> 930c7f40f Bump Hive version HIVE-16886/HIVE-18526 is causing the notification log listener tests to be flaky, this version bump includes the fix committed upstream. Change-Id: I5f2872da67e718d0801f08a71482a6bf2ecb95d2

kudu git commit: KUDU-2472: address master-stress-test flakiness

2018-06-12 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 37f3a95d8 -> c1c15ad5f KUDU-2472: address master-stress-test flakiness Restores a hack that prevented master-stress-test flakiness, and adds a TODO to remove the hack once KUDU-1358 is addressed. The hack was broken in 1a707b991 when an

kudu git commit: Rename Hive MetaStore to Metastore

2018-05-14 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master ca3e38759 -> 5d113ec3c Rename Hive MetaStore to Metastore Hive isn't consistent about the capitilization so it's not clear which is more correct, but we should at least be consistent ourselves. Change-Id:

kudu git commit: catalog-manager: refactor AlterTable and DeleteTable methods

2018-05-14 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master a65e58ec0 -> ca3e38759 catalog-manager: refactor AlterTable and DeleteTable methods This commit splits CatalogManager::AlterTable and CatalogManager::DeleteTable in two. One method handles RPC specifics, and the second handles applying the

kudu git commit: catalog-manager: return error from FindAndLockTable if table not found

2018-05-14 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 1a707b991 -> a65e58ec0 catalog-manager: return error from FindAndLockTable if table not found This cleans up some repeated boilerplate in the CatalogManager. Change-Id: I4d8176d6886474a84b4c75c80f50763eabe70a6d Reviewed-on:

kudu git commit: catalog-manager: refactor error handling

2018-05-14 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 05d594867 -> 1a707b991 catalog-manager: refactor error handling This commit refactors how the SetupError helper works slightly in order to reduce the prevelance of temporary Status locals. I've found the Status locals to be error prone in

kudu git commit: KUDU-2346: Fix mac pkg-config issue.

2018-05-14 Thread danburkert
Repository: kudu Updated Branches: refs/heads/branch-1.7.x 5c68767d3 -> 5ef577b65 KUDU-2346: Fix mac pkg-config issue. Previously an empty return string from pkg-config --cflags was used as a proxy for testing if the package was installed. The MacOs version of pkg-config returns an error

kudu git commit: [docs] Update macOS build docs

2018-05-11 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master c18e7e1a6 -> 72df429ec [docs] Update macOS build docs * tapping homebrew/dupes is unnecessary now as it's now deprecated and all formulae have been migrated to homebrew-core or deleted Change-Id: Ic2a8aa7f58b7401115f36f42634d80b4d64e9794

[2/3] kudu git commit: Fix int overflow GetClockTimeMicros() on macOS

2018-05-10 Thread danburkert
Fix int overflow GetClockTimeMicros() on macOS On macOS mach_timespec_t.tv_sec is only 4 bytes and we were converting to micros before moving to a bigger var. This would cause all the wall times obtained on a mac (through GetClockTimeMicros() to be wrong. This was likely the cause of KUDU-2435

[1/3] kudu git commit: [tools] Add ids to tables and tablets; table name to tablets

2018-05-10 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 3f43c03a1 -> 3855328b7 [tools] Add ids to tables and tablets; table name to tablets This adds tablet id, table id, and table name to KsckTabletSummary, and table id to KsckTableSummary. This will be tested in the follow-up JSON formatting

[3/3] kudu git commit: thirdparty: tweak clang compiler flags

2018-05-10 Thread danburkert
thirdparty: tweak clang compiler flags Prior to this small tweak the thirdparty clang build output thousands of warnings similar to this one when compiling with clang: Building CXX object lib/Analysis/CMakeFiles/LLVMAnalysis.dir/LazyValueInfo.cpp.o clang: warning:

kudu git commit: [thirdparty] Fix failing openssl check on MacOS

2018-05-09 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master d83cd11a9 -> 60c8e305b [thirdparty] Fix failing openssl check on MacOS Since ea286417af the thirdparty build on Mac was failing as the openssl library path needs to be set. This commit adds this flag by invoking "pkg-config --cflags

[1/2] kudu git commit: [Java] Fix TestSecurity tests on Mac

2018-05-03 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 06afa8f04 -> eb4d88f0a [Java] Fix TestSecurity tests on Mac TestSecurity tests were failing because InetAddress.getCanonicalHostName was returning localhost for the 127.0.0.1 IP address which results Kerberos rejecting as not kinit’d.

[2/2] kudu git commit: mini-cluster: support parallel multi-master clusters

2018-05-03 Thread danburkert
mini-cluster: support parallel multi-master clusters This commit refactors the mini-clusters to internally use reserved sockets for their child daemons. Reserved sockets are simply sockets bound to a random port with SO_REUSEPORT. As a result, master addresses for multi-master mini clusters no

kudu git commit: KUDU-2191 (10/n): PoC: disallow Kudu alter column, create table operations from Hive

2018-05-03 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 7767679a6 -> f54dbb891 KUDU-2191 (10/n): PoC: disallow Kudu alter column, create table operations from Hive This commit makes it more difficult for processes or users who are not the Kudu Master to alter the columns of Kudu table HMS

kudu git commit: rpc: add experimental rpc_reuseport flag

2018-05-03 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 96fa80503 -> eded05723 rpc: add experimental rpc_reuseport flag This option sets the SO_REUSEPORT socket option on a server's bound socket. This socket option is supported on all platforms which Kudu supports, except RHEL 6.4 and below.

kudu git commit: KUDU-2351 Add IP/port for Recv() failure

2018-04-19 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master abbde75e1 -> 1f380f279 KUDU-2351 Add IP/port for Recv() failure Error messages like "Recv() got EOF from remote", "recv error:" and "BlockingRecv error: failed to read from TLS socket" don't contain the remote address, making it hard to

[1/2] kudu git commit: Add krb5-server to SLES prerequisite libraries

2018-04-18 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master bba339b11 -> c25b7cbd8 Add krb5-server to SLES prerequisite libraries I was running a test with Kudu on SLES and realized that the MiniKDC doesn't work due to 'kdb5_util' not being installed. Running "zypper install krb5-server" fixes

[2/2] kudu git commit: [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull option

2018-04-18 Thread danburkert
[spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull option This patch adds the KuduWriteOptions class to allow configuration of writes to the Kudu table when writing with Spark. This allows extensibility via adding more fields in the future. The instance of this class is passed to

kudu git commit: HmsCatalog: fix bug in reconnection backoff logic

2018-04-12 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 133ef7e9d -> 361ee99f0 HmsCatalog: fix bug in reconnection backoff logic This fixes an accidentally shadowed Status variable. The effect of this bug was that the HmsCatalog would not throttle HMS reconnect attempts. The tests actually took

kudu git commit: Add hms_client-test to the IWYU filter

2018-04-11 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 03d1fcb14 -> c09a62961 Add hms_client-test to the IWYU filter This is in keeping with the other HMS files that trigger many false positives on the upstream gerrit. I've verified that the file does pass IWYU on a local instance. Change-Id:

[2/2] kudu git commit: KUDU-2191 (9/n): HMS Catalog should short-circuit no-op alter and create table calls

2018-04-10 Thread danburkert
KUDU-2191 (9/n): HMS Catalog should short-circuit no-op alter and create table calls This tweaks the HMS catalog class introduced in part 7 to be more flexible with how tables are created and altered. In particular, when altering, renaming, or creating a table entry in the HMS, it returns an OK

[1/2] kudu git commit: Add GetFlags endpoint and tool

2018-04-10 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 337a731c4 -> 00c2754ca Add GetFlags endpoint and tool This adds an rpc endpoint that retrieves gflags from servers. It also includes a tool for retrieving flag values from servers. By default, it returns only flags with non-default

kudu git commit: KUDU-2401: External TLS certificate with Intermediate CA in server cert file fails

2018-04-05 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master d1c99e02f -> 109fdc84e KUDU-2401: External TLS certificate with Intermediate CA in server cert file fails Take 2 certificate files: cert.pem and truststore.pem cert.pem has 2 certificates in it: A cert for that node (with CN="hostname",

kudu git commit: Followup to fe4d962c09012e to fix broken test

2018-04-03 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 9e918bf77 -> de067f792 Followup to fe4d962c09012e to fix broken test The newly introduced tests weren't cleaning up the test directory because they didn't inherit from KuduTest. Change-Id: I0e323f5c9082013510cddc6873bd81675e66d644

kudu git commit: KUDU-2191 (8/n): Integrate HmsCatalog into CatalogManager

2018-04-03 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 5fc31db00 -> 02560c188 KUDU-2191 (8/n): Integrate HmsCatalog into CatalogManager This commit connects the CatalogManager to the HMS via the HmsCatalog abstraction. When the HMS integration is enabled (by setting the --hive-metastore-uris

kudu git commit: Followup to bae6170c77 to fix build on macos

2018-04-02 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master fff4185e0 -> 96951f68f Followup to bae6170c77 to fix build on macos RLIM_INFINITY is kint64max on MacOS and kuint64max on Linux. Change-Id: I13780d86d4a286710c1bcf8a5f4266fb7fe162c7 Reviewed-on: http://gerrit.cloudera.org:8080/9888

[3/3] kudu git commit: KUDU-2191 (6/n): Fixups to the C++ HMS client

2018-04-02 Thread danburkert
KUDU-2191 (6/n): Fixups to the C++ HMS client These are changes necessary for subsequent patches in the series, but they are easier to review in isolation. Change-Id: I86a8d984e67fe6b4c004725828b4296476c2389e Reviewed-on: http://gerrit.cloudera.org:8080/9861 Reviewed-by: Adar Dembo

[2/3] kudu git commit: KUDU-2384. Don't collect thread stacks when running under debugger

2018-04-02 Thread danburkert
KUDU-2384. Don't collect thread stacks when running under debugger This changes the stack watchdog and the 'collect all stacks' utility code to not run when it appears that the process is running under a tracer such as gdb or strace. This makes debugging Kudu processes significantly easier

[1/3] kudu git commit: [examples] Add licenses to Python examples

2018-04-02 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master ba955733d -> fff4185e0 [examples] Add licenses to Python examples This also adds a README to the graphite example warning that it's not a real integration, just an example. Change-Id: I8825bfac0416fc19fdb99c972eb104898ba78e7d Reviewed-on:

kudu git commit: KUDU-2259: add real user to AuthenticationCredentialsPB

2018-03-14 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 30cf90028 -> e684de337 KUDU-2259: add real user to AuthenticationCredentialsPB This commit adds the 'real user' to the authn credentials token, which is used when negotiating connections with SASL PLAIN authentication. This is useful when

kudu git commit: Remove namespace hack in hms_client.h

2018-03-07 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master debcb8ea2 -> b525378c5 Remove namespace hack in hms_client.h To avoid having to use the very long Apache.Hadoop.Hive namespace for all of the Thrift-generated HMS types, we aliased the namespace in hms_client.h to 'hive'. This doesn't

kudu git commit: Add CatalogManager::master_consensus() accessor

2018-03-07 Thread danburkert
Repository: kudu Updated Branches: refs/heads/master 74dc5da86 -> 954d9f71b Add CatalogManager::master_consensus() accessor A patch I'm working on is going to start calling Master::GetMasterHostPorts from inside of CatalogManager::Init. GetMasterHostPorts calls into the catalog manager, and

  1   2   3   4   5   6   7   >