[kudu-CR] [Java] Use Yetus annotations in place of @VisibleForTesting

[Grant Henke (Code Review)]

Hello Kudu Jenkins, Adar Dembo, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10077

to look at the new patch set (#7).

Change subject: [Java] Use Yetus annotations in place of @VisibleForTesting
..

[Java] Use Yetus annotations in place of @VisibleForTesting

We use yetus for marking our intended InterfaceAudience.
This patch replaces Guava’s @VisibleForTesting with
@InterfaceAudience.LimitedPrivate("Test”).

kudu-hive is an exception to the rule because it can not
have any dependencies that are not provided by the
hive-metastore. This limitation is noted in the kudu-hive
build files.

I also elimated an uneccesary usage of ImmutableList
and Lists while making this change. Scala has
straightforward syntax for making lists without Guava.

The primary reason for this change is to use
Yetus as the single annotation library for defining
InterfaceAudience, but this has the added benefit
of eliminating any Guava use in kudu-spark,
kudu-spark-tools which didn’t
have Gauva marked as a dependency.

Change-Id: Ic0d8845e8cbbf5ea81345c46c5487121d1098f91
---
M java/kudu-client-tools/build.gradle
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Batch.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectToCluster.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java
M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduPredicate.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduRpc.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanToken.java
M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java
M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java
M java/kudu-client/src/main/java/org/apache/kudu/client/SecurityContext.java
M java/kudu-client/src/main/java/org/apache/kudu/client/Status.java
M java/kudu-client/src/main/java/org/apache/kudu/client/TableLocationsCache.java
M java/kudu-client/src/main/java/org/apache/kudu/util/ByteVec.java
M java/kudu-flume-sink/src/main/java/org/apache/kudu/flume/sink/KuduSink.java
M java/kudu-spark-tools/build.gradle
M java/kudu-spark-tools/pom.xml
M 
java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/ImportExportFiles.scala
M 
java/kudu-spark-tools/src/main/scala/org/apache/kudu/spark/tools/IntegrationTestBigLinkedList.scala
M 
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala
25 files changed, 51 insertions(+), 64 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/77/10077/7
--
To view, visit http://gerrit.cloudera.org:8080/10077
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic0d8845e8cbbf5ea81345c46c5487121d1098f91
Gerrit-Change-Number: 10077
Gerrit-PatchSet: 7
Gerrit-Owner: Grant Henke 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] KUDU-2412: Fix python client compilation in el6 environments

[Grant Henke (Code Review)]

Grant Henke has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10114


Change subject: KUDU-2412: Fix python client compilation in el6 environments
..

KUDU-2412: Fix python client compilation in el6 environments

Identifies if the compiler is gcc < 4.6.0 and uses cython
preprocessors to ignore any code using __int128. This
includes test code.

This is a bit of a hacky approach but I needed to
duplicate the entire cdef anytime I used a preprocessor
to remove methods. The preprocessor statements
do not work inside of a cdef stament and you can’t
break out cdef staments to use the additively.

The cython preprocessors do not have a lot of
documentation. Perhaps there is a better approach I missed
that we could apply in a follow up.

See here for documentation:
http://cython.readthedocs.io/en/latest/src/userguide/language_basics.html#conditional-statements

Change-Id: Ibd93b57020b80597baae9c8d3e0434c46f7fc3d7
---
M python/.gitignore
M python/kudu/__init__.py
M python/kudu/client.pyx
M python/kudu/libkudu_client.pxd
M python/kudu/schema.pyx
M python/kudu/tests/test_scanner.py
M python/kudu/tests/test_scantoken.py
M python/kudu/tests/util.py
M python/setup.py
9 files changed, 401 insertions(+), 159 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/14/10114/1
--
To view, visit http://gerrit.cloudera.org:8080/10114
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibd93b57020b80597baae9c8d3e0434c46f7fc3d7
Gerrit-Change-Number: 10114
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke 

[kudu-CR] Modify fix includes.py for Kudu usage, add a wrapper

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10106 )

Change subject: Modify fix_includes.py for Kudu usage, add a wrapper
..


Patch Set 1:

(1 comment)

Ended up redoing this a bit for rev 2

http://gerrit.cloudera.org:8080/#/c/10106/1/build-support/iwyu/fix_includes.py
File build-support/iwyu/fix_includes.py:

http://gerrit.cloudera.org:8080/#/c/10106/1/build-support/iwyu/fix_includes.py@1220
PS1, Line 1220:   # We also never insert a blank line between C and C++-style 
#includes,
> Update this comment.
Done



--
To view, visit http://gerrit.cloudera.org:8080/10106
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3c286271a39a0d825fb11e5610d8eb7e5b0729b9
Gerrit-Change-Number: 10106
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 19 Apr 2018 02:14:53 +
Gerrit-HasComments: Yes

[kudu-CR] Modify fix includes.py for Kudu usage, add a wrapper

[Todd Lipcon (Code Review)]

Hello Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10106

to look at the new patch set (#2).

Change subject: Modify fix_includes.py for Kudu usage, add a wrapper
..

Modify fix_includes.py for Kudu usage, add a wrapper

This makes a few small changes to the upstream fix_includes.py script:

- We name our tests 'foo-test.cc' instead of 'foo_test.cc'. Expands
  the regex appropriately.

- Add support for 'IWYU pragma: keep' to avoid removing duplicate
  headers. Without this, it was attempting to remove the duplicate
  include in bitshuffle_arch_wrapper.cc which includes the bitshuffle
  header twice with different #defines

- Adds a --blank_line_between_c_and_cxx_includes option which preserves
  the style that we use.

- Adds support for differentiating between thirdparty  includes
  versus system  includes by checking for file existence in a set
  of directories passed on the command line. Google apparently uses
  quoted includes like "protobuf/foo.h" for this purpose whereas we use
  <>.

This also adds a wrapper script build-support/iwyu.py which handles
invoking IWYU and also integrating it with the modified fix_includes.py.

This takes the place of the old 'iwyu.sh' and associated awk script. It
supports automatically piping the IWYU results back through the include
fixer and either producing a diff of errors (default output) or fixing
them in-situ.

Change-Id: I3c286271a39a0d825fb11e5610d8eb7e5b0729b9
---
M CMakeLists.txt
M build-support/build_source_release.py
A build-support/iwyu.py
A build-support/iwyu/__init__.py
M build-support/iwyu/fix_includes.py
D build-support/iwyu/iwyu-filter.awk
D build-support/iwyu/iwyu.sh
M build-support/kudu_util.py
M build-support/release/rat_exclude_files.txt
M src/kudu/cfile/bitshuffle_arch_wrapper.cc
10 files changed, 317 insertions(+), 252 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/06/10106/2
--
To view, visit http://gerrit.cloudera.org:8080/10106
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3c286271a39a0d825fb11e5610d8eb7e5b0729b9
Gerrit-Change-Number: 10106
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] KUDU-2191: Metadata Upgrade Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10075 )

Change subject: KUDU-2191: Metadata Upgrade Tool
..


Patch Set 3:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/10075/2/src/kudu/hms/hms_client.h
File src/kudu/hms/hms_client.h:

http://gerrit.cloudera.org:8080/#/c/10075/2/src/kudu/hms/hms_client.h@90
PS2, Line 90:   static const char* const kLegacyKuduStorageHandler;
> +1 on legacy over deprecated, maybe extend that to all the old key names?
Done


http://gerrit.cloudera.org:8080/#/c/10075/1/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/10075/1/src/kudu/tools/kudu-tool-test.cc@1854
PS1, Line 1854:   table.tableName = table_name;
> Isn't that metadata checking going to be problematic in practice?  If it's
Yeah, I have changed the metadata checking in AlterTable event. But I do not 
think it is good to take out the metadata checking in CreateTable event, since 
legacy tables should not be created anymore once HMS integration is on. Or I am 
missing something?


http://gerrit.cloudera.org:8080/#/c/10075/2/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/10075/2/src/kudu/tools/kudu-tool-test.cc@1897
PS2, Line 1897: void ValidateHmsEntries(HmsClient* hms_client,
> I think it'd be simpler to use ASSERTs internally here, and just call it wi
Done


http://gerrit.cloudera.org:8080/#/c/10075/1/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10075/1/src/kudu/tools/tool_action_hms.cc@94
PS1, Line 94: }
> Yeah, I think in any case where we're going to rename a table it would be b
Sorry, I do not quite follow the last sentence. Not sure how to fill in default 
database name based on the existing table name? I added a param for users to 
specify default database name in the tool, is that what you mean 'kudu table 
name table param'?


http://gerrit.cloudera.org:8080/#/c/10075/1/src/kudu/tools/tool_action_hms.cc@155
PS1, Line 155:   unordered_map hms_tables_map;
> So would this work if the user passes --unlock_experimental_flags?  I think
Looks like this workaround works. Updated the code.


http://gerrit.cloudera.org:8080/#/c/10075/2/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10075/2/src/kudu/tools/tool_action_hms.cc@38
PS2, Line 38: #include "kudu/hms/hms_catalog.h"
> warning: #includes are not sorted properly [llvm-include-order]
Done



--
To view, visit http://gerrit.cloudera.org:8080/10075
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1c8488e1c7b04b8bedc31c83b1496b53bae49cb3
Gerrit-Change-Number: 10075
Gerrit-PatchSet: 3
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Thu, 19 Apr 2018 00:50:17 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: Metadata Upgrade Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10075

to look at the new patch set (#3).

Change subject: KUDU-2191: Metadata Upgrade Tool
..

KUDU-2191: Metadata Upgrade Tool

This commit introduces a upgrade tool to allows backward compatibility.
It provides support to upgrade the metadata format of existing Impala
managed/external tables in HMS entries, as well as rename the existing
tables in Kudu to adapt to the new naming rules. It adds test case
using external mini cluster to ensure the upgrade tool works as expect.

Change-Id: I1c8488e1c7b04b8bedc31c83b1496b53bae49cb3
---
M 
java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java
M 
java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_catalog.h
M src/kudu/hms/hms_client.cc
M src/kudu/hms/hms_client.h
M src/kudu/integration-tests/alter_table-randomized-test.cc
M src/kudu/integration-tests/master_failover-itest.cc
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/mini-cluster/external_mini_cluster.h
M src/kudu/tools/CMakeLists.txt
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool.proto
M src/kudu/tools/tool_action.h
A src/kudu/tools/tool_action_hms.cc
M src/kudu/tools/tool_action_test.cc
M src/kudu/tools/tool_main.cc
18 files changed, 578 insertions(+), 28 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/75/10075/3
--
To view, visit http://gerrit.cloudera.org:8080/10075
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1c8488e1c7b04b8bedc31c83b1496b53bae49cb3
Gerrit-Change-Number: 10075
Gerrit-PatchSet: 3
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191 (n/m): PoC: disallow Kudu alter column, create table operations from Hive

[Dan Burkert (Code Review)]

Hello Hao Hao,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/10111

to review the following change.


Change subject: KUDU-2191 (n/m): PoC: disallow Kudu alter column, create table 
operations from Hive
..

KUDU-2191 (n/m): PoC: disallow Kudu alter column, create table operations from 
Hive

This commit makes it more difficult for processes or users who are not
the Kudu Master to alter the columns of Kudu table HMS entries through
Hive APIs.

The bulk of this patch is introducing a new environment context key/pair
('kudu.master_event=true') which is sent by the Kudu Master with every
HMS create/drop/alter table call. The KuduMetastorePlugin checks that
this property is set appropriately when operations modify a Kudu table
HMS entry.

The result is any attempt to create a table through Hive APIs or alter
the columns of a Kudu table through Hive APIs will fail. These
operations can not be supported because Hive lacks the DDL syntax and
HMS APIs to support necessary Kudu-specific table and column options.

It's possible for a user to spoof this environment context entry, but
it's a big enough roadblock that it should prevent accidental DDL
operations.

Change-Id: Idc39b1df07952d37e9ce93f1673aad535767a8ac
---
M 
java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java
M 
java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_catalog.h
M src/kudu/hms/hms_client-test.cc
M src/kudu/hms/hms_client.cc
M src/kudu/hms/hms_client.h
M src/kudu/integration-tests/master_hms-itest.cc
8 files changed, 103 insertions(+), 29 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/11/10111/1
--
To view, visit http://gerrit.cloudera.org:8080/10111
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Idc39b1df07952d37e9ce93f1673aad535767a8ac
Gerrit-Change-Number: 10111
Gerrit-PatchSet: 1
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Hao Hao 

[kudu-CR](branch-1.7.x) KUDU-2378. Fix unaligned loads of int128 from rows

[Grant Henke (Code Review)]

Grant Henke has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10110


Change subject: KUDU-2378. Fix unaligned loads of int128 from rows
..

KUDU-2378. Fix unaligned loads of int128 from rows

Our row format doesn't ensure any alignment of data. This is usually fine on
x86 where unaligned loads of ints don't need any special instruction and
carry a relatively low performance penalty. However, loads of int128_t may
generate a 'movdqa' (aligned load) instruction which crashes if the
operand is not 16-byte aligned.

This fixes a bunch of spots where we are dereferencing a reinterpret_casted
int and replaces them with new UnalignedLoad<> and UnalignedStore<>
template functions. These use simple reinterpret_cast dereferences for
<=64bit types. For int128_t we use memcpy. The compiler should already
treat memcpy as an intrinsic and this will end up being a single
instruction.

Change-Id: Ic71149ed5c6881cb16dfe6cf8f7906c044a0b41a
Reviewed-on: http://gerrit.cloudera.org:8080/9848
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin 
Reviewed-by: Grant Henke 
(cherry picked from commit 35940a12aa38dedeff7e932f91448d2562163302)
---
M src/kudu/cfile/bshuf_block.cc
M src/kudu/cfile/bshuf_block.h
M src/kudu/cfile/plain_block.h
M src/kudu/cfile/rle_block.h
M src/kudu/common/partial_row.cc
M src/kudu/common/types.h
M src/kudu/gutil/port.h
M src/kudu/integration-tests/all_types-itest.cc
8 files changed, 93 insertions(+), 45 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/10/10110/1
--
To view, visit http://gerrit.cloudera.org:8080/10110
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.7.x
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic71149ed5c6881cb16dfe6cf8f7906c044a0b41a
Gerrit-Change-Number: 10110
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke 
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] tidy: run in parallel

[Todd Lipcon (Code Review)]

Todd Lipcon has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10107 )

Change subject: tidy: run in parallel
..

tidy: run in parallel

This makes the 'tidy' target run in parallel across all cores on the
machine, speeding it up substantially.

Change-Id: Idc611a32f55fa830d1ce7bed6c7cff6f6291d508
Reviewed-on: http://gerrit.cloudera.org:8080/10107
Reviewed-by: Dan Burkert 
Tested-by: Kudu Jenkins
---
M build-support/clang_tidy_gerrit.py
1 file changed, 35 insertions(+), 14 deletions(-)

Approvals:
  Dan Burkert: Looks good to me, approved
  Kudu Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/10107
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Idc611a32f55fa830d1ce7bed6c7cff6f6291d508
Gerrit-Change-Number: 10107
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] iwyu: run IWYU on multiple cores

[Todd Lipcon (Code Review)]

Todd Lipcon has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10104 )

Change subject: iwyu: run IWYU on multiple cores
..

iwyu: run IWYU on multiple cores

This changes iwyu_tool to use a number of parallel IWYU processes equal
to the number of cores, resulting in a huge speedup for common dev
workflows.

Change-Id: I97206318943b408c6fdd5637c6ccb79ee4d87d3f
Reviewed-on: http://gerrit.cloudera.org:8080/10104
Reviewed-by: Dan Burkert 
Tested-by: Kudu Jenkins
---
M build-support/iwyu/iwyu_tool.py
1 file changed, 22 insertions(+), 10 deletions(-)

Approvals:
  Dan Burkert: Looks good to me, approved
  Kudu Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/10104
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I97206318943b408c6fdd5637c6ccb79ee4d87d3f
Gerrit-Change-Number: 10104
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] Import original fix includes.py script from IWYU

[Todd Lipcon (Code Review)]

Todd Lipcon has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10105 )

Change subject: Import original fix_includes.py script from IWYU
..

Import original fix_includes.py script from IWYU

Imported from github at revision
5082fddccb3d5aabaace2208f1162029a27c0334 in order to support making a
few local modifications.

Change-Id: I98ebd8342cf7827d3204ff9063dbec23f16be4a6
Reviewed-on: http://gerrit.cloudera.org:8080/10105
Reviewed-by: Dan Burkert 
Tested-by: Kudu Jenkins
---
A build-support/iwyu/fix_includes.py
M build-support/release/rat_exclude_files.txt
2 files changed, 2,232 insertions(+), 0 deletions(-)

Approvals:
  Dan Burkert: Looks good to me, approved
  Kudu Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/10105
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I98ebd8342cf7827d3204ff9063dbec23f16be4a6
Gerrit-Change-Number: 10105
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull option

[Dan Burkert (Code Review)]

Dan Burkert has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9834 )

Change subject: [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull 
option
..

[spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull option

This patch adds the KuduWriteOptions class to allow configuration of
writes to the Kudu table when writing with Spark. This allows
extensibility via adding more fields in the future. The instance of
this class is passed to functions (insert/delete/upsert/update) in
KuduContext.

KuduWriteOptions is also supported in DefaultSource APIs. Clients can
set up write options in the parameters from SparkSQL.

This patch also adds the ignoreNull write option so that users can
upsert/update only non-Null columns and leave the rest of the columns
unchanged.

For example, this feature is useful when users use Spark streaming to
process JSON and upsert to Kudu, because missing column values from
JSON are set to NULL, resulting in some existing row values being
upserted to Null, which is not desired.

Change-Id: Ide908ea29f572849eca0ba850ee197c1b22a07c8
Reviewed-on: http://gerrit.cloudera.org:8080/9834
Reviewed-by: Dan Burkert 
Tested-by: Kudu Jenkins
---
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
A 
java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala
M 
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala
6 files changed, 237 insertions(+), 26 deletions(-)

Approvals:
  Dan Burkert: Looks good to me, approved
  Kudu Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9834
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ide908ea29f572849eca0ba850ee197c1b22a07c8
Gerrit-Change-Number: 9834
Gerrit-PatchSet: 11
Gerrit-Owner: Fengling Wang 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Fengling Wang 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 

[kudu-CR] KUDU-2351 Add IP/port for Recv() failure

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9818 )

Change subject: KUDU-2351 Add IP/port for Recv() failure
..


Patch Set 11:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/9818/11/src/kudu/security/tls_socket-test.cc
File src/kudu/security/tls_socket-test.cc:

http://gerrit.cloudera.org:8080/#/c/9818/11/src/kudu/security/tls_socket-test.cc@231
PS11, Line 231: threads.emplace_back(std::thread([&]{
Can you add some explanation for why this requires a background thread?  Seems 
like you should just be able to call server.Stop() near line 241 between the 
two Recv() calls and it would work OK.

If it does need to be done in the background then the way the cleanup is 
written is unsafe: 'server' only lives for the scope of this test.  The thread 
is holding onto a reference to 'server' until TearDown() is executed, which 
will be a dangling reference.  To make it safe you need to call join() before 
the end of the scope of this test.  That being said, I don't see why join() is 
necessary at all.


http://gerrit.cloudera.org:8080/#/c/9818/11/src/kudu/security/tls_socket.cc
File src/kudu/security/tls_socket.cc:

http://gerrit.cloudera.org:8080/#/c/9818/11/src/kudu/security/tls_socket.cc@111
PS11, Line 111:   std::string kErrString = strings::Substitute("failed to read 
from TLS socket (remote: $0)",
This is a really hot function, so the overhead of creating this message 
(allocation + the GetPeerAddress syscall) could end up showing up.  I suggest 
making it lazy, so that it's only created on demand:

auto err_string = [&] {
  Sockaddr remote;
  Socket::GetPeerAddress();
  std::string kErrString = strings::Substitute("failed to read from TLS socket 
(remote: $0)",
};

and then change use-sites from 'kErrString' to 'err_string()'.


http://gerrit.cloudera.org:8080/#/c/9818/11/src/kudu/security/tls_socket.cc@125
PS11, Line 125: "SSL_read error"
probably worth prefixing this with the message as well.


http://gerrit.cloudera.org:8080/#/c/9818/11/src/kudu/util/net/socket-test.cc
File src/kudu/util/net/socket-test.cc:

http://gerrit.cloudera.org:8080/#/c/9818/11/src/kudu/util/net/socket-test.cc@68
PS11, Line 68:   sockaddr_promise.Set(sockaddr);
Could this preamble  be done in the main thread?  Then you wouldn't need to use 
the promise.  Seems like the if/else blocks are the only thing that needs to be 
done in the background.



--
To view, visit http://gerrit.cloudera.org:8080/9818
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22436b13bb351b132e1c0b7159294dd0c980c2b3
Gerrit-Change-Number: 9818
Gerrit-PatchSet: 11
Gerrit-Owner: Attila Bukor 
Gerrit-Reviewer: Attila Bukor 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 18 Apr 2018 23:23:57 +
Gerrit-HasComments: Yes

[kudu-CR] iwyu: run IWYU on multiple cores

[Alexey Serbin (Code Review)]

Alexey Serbin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10104 )

Change subject: iwyu: run IWYU on multiple cores
..


Patch Set 1:

That looks great!  Now it should be much faster.


--
To view, visit http://gerrit.cloudera.org:8080/10104
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I97206318943b408c6fdd5637c6ccb79ee4d87d3f
Gerrit-Change-Number: 10104
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Wed, 18 Apr 2018 22:55:44 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2351 Add IP/port for Recv() failure

[Attila Bukor (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9818

to look at the new patch set (#11).

Change subject: KUDU-2351 Add IP/port for Recv() failure
..

KUDU-2351 Add IP/port for Recv() failure

Error messages like "Recv() got EOF from remote" and "recv error:"
don't contain the remote address, making it hard to locate the server
the connection failed to.

This change introduces adds the peer's IP address and port number to
these error messages.

Examples:

Recv() got EOF from remote 127.0.0.1:7050
recv error from 127.0.0.1:7050: Resource temporarily unavailable

Change-Id: I22436b13bb351b132e1c0b7159294dd0c980c2b3
---
M src/kudu/security/tls_socket-test.cc
M src/kudu/security/tls_socket.cc
M src/kudu/util/CMakeLists.txt
A src/kudu/util/net/socket-test.cc
M src/kudu/util/net/socket.cc
5 files changed, 158 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/18/9818/11
--
To view, visit http://gerrit.cloudera.org:8080/9818
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22436b13bb351b132e1c0b7159294dd0c980c2b3
Gerrit-Change-Number: 9818
Gerrit-PatchSet: 11
Gerrit-Owner: Attila Bukor 
Gerrit-Reviewer: Attila Bukor 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] tidy: run in parallel

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10107 )

Change subject: tidy: run in parallel
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10107
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc611a32f55fa830d1ce7bed6c7cff6f6291d508
Gerrit-Change-Number: 10107
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Wed, 18 Apr 2018 22:01:53 +
Gerrit-HasComments: No

[kudu-CR] Modify fix includes.py for Kudu usage, add a wrapper

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10106 )

Change subject: Modify fix_includes.py for Kudu usage, add a wrapper
..


Patch Set 1: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10106/1/build-support/iwyu/fix_includes.py
File build-support/iwyu/fix_includes.py:

http://gerrit.cloudera.org:8080/#/c/10106/1/build-support/iwyu/fix_includes.py@1220
PS1, Line 1220:   # We also never insert a blank line between C and C++-style 
#includes,
Update this comment.



--
To view, visit http://gerrit.cloudera.org:8080/10106
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3c286271a39a0d825fb11e5610d8eb7e5b0729b9
Gerrit-Change-Number: 10106
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Wed, 18 Apr 2018 21:56:57 +
Gerrit-HasComments: Yes

[kudu-CR] iwyu: run IWYU on multiple cores

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10104 )

Change subject: iwyu: run IWYU on multiple cores
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10104
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I97206318943b408c6fdd5637c6ccb79ee4d87d3f
Gerrit-Change-Number: 10104
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Wed, 18 Apr 2018 21:48:50 +
Gerrit-HasComments: No

[kudu-CR] iwyu: run IWYU on multiple cores

[Todd Lipcon (Code Review)]

Hello Dan Burkert,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/10104

to review the following change.


Change subject: iwyu: run IWYU on multiple cores
..

iwyu: run IWYU on multiple cores

This changes iwyu_tool to use a number of parallel IWYU processes equal
to the number of cores, resulting in a huge speedup for common dev
workflows.

Change-Id: I97206318943b408c6fdd5637c6ccb79ee4d87d3f
---
M build-support/iwyu/iwyu_tool.py
1 file changed, 22 insertions(+), 10 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/04/10104/1
--
To view, visit http://gerrit.cloudera.org:8080/10104
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I97206318943b408c6fdd5637c6ccb79ee4d87d3f
Gerrit-Change-Number: 10104
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 

[kudu-CR] tidy: run in parallel

[Todd Lipcon (Code Review)]

Hello Dan Burkert,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/10107

to review the following change.


Change subject: tidy: run in parallel
..

tidy: run in parallel

This makes the 'tidy' target run in parallel across all cores on the
machine, speeding it up substantially.

Change-Id: Idc611a32f55fa830d1ce7bed6c7cff6f6291d508
---
M build-support/clang_tidy_gerrit.py
1 file changed, 35 insertions(+), 14 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/07/10107/1
--
To view, visit http://gerrit.cloudera.org:8080/10107
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Idc611a32f55fa830d1ce7bed6c7cff6f6291d508
Gerrit-Change-Number: 10107
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 

[kudu-CR] Modify fix includes.py for Kudu usage, add a wrapper

[Todd Lipcon (Code Review)]

Hello Dan Burkert,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/10106

to review the following change.


Change subject: Modify fix_includes.py for Kudu usage, add a wrapper
..

Modify fix_includes.py for Kudu usage, add a wrapper

This makes a few small changes to the upstream fix_includes.py script:

- We name our tests 'foo-test.cc' instead of 'foo_test.cc'. Expands
  the regex appropriately.

- Add support for 'IWYU pragma: keep' to avoid removing duplicate
  headers. Without this, it was attempting to remove the duplicate
  include in bitshuffle_arch_wrapper.cc which includes the bitshuffle
  header twice with different #defines

- Adds a --blank_line_between_c_and_cxx_includes option which preserves
  the style that we use.

- Adds support for differentiating between thirdparty  includes
  versus system  includes by checking for file existence in a set
  of directories passed on the command line. Google apparently uses
  quoted includes like "protobuf/foo.h" for this purpose whereas we use
  <>.

This also adds a simple wrapper script build-support/fix_includes.py
which handles passing the appropriate options to fix_includes.py.
Currently the wrapper script only supports the "sort includes" mode of
operation but I plan to extend it to also support the "fix IWYU issues"
mode.

Change-Id: I3c286271a39a0d825fb11e5610d8eb7e5b0729b9
---
A build-support/fix_includes.py
M build-support/iwyu/fix_includes.py
M src/kudu/cfile/bitshuffle_arch_wrapper.cc
3 files changed, 125 insertions(+), 10 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/06/10106/1
--
To view, visit http://gerrit.cloudera.org:8080/10106
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3c286271a39a0d825fb11e5610d8eb7e5b0729b9
Gerrit-Change-Number: 10106
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 

[kudu-CR] Add krb5-server to SLES prerequisite libraries

[Todd Lipcon (Code Review)]

Todd Lipcon has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9727 )

Change subject: Add krb5-server to SLES prerequisite libraries
..

Add krb5-server to SLES prerequisite libraries

I was running a test with Kudu on SLES and realized that the MiniKDC
doesn't work due to 'kdb5_util' not being installed.

Running "zypper install krb5-server" fixes this.

Change-Id: I12f78066f9dabe60d63dcea03a1ba22da942ac0f
Reviewed-on: http://gerrit.cloudera.org:8080/9727
Reviewed-by: Alexey Serbin 
Tested-by: Kudu Jenkins
Reviewed-by: Sailesh Mukil 
---
M docs/installation.adoc
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Alexey Serbin: Looks good to me, approved
  Kudu Jenkins: Verified
  Sailesh Mukil: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/9727
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I12f78066f9dabe60d63dcea03a1ba22da942ac0f
Gerrit-Change-Number: 9727
Gerrit-PatchSet: 2
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] Import original fix includes.py script from IWYU

[Todd Lipcon (Code Review)]

Hello Dan Burkert,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/10105

to review the following change.


Change subject: Import original fix_includes.py script from IWYU
..

Import original fix_includes.py script from IWYU

Imported from github at revision
5082fddccb3d5aabaace2208f1162029a27c0334 in order to support making a
few local modifications.

Change-Id: I98ebd8342cf7827d3204ff9063dbec23f16be4a6
---
A build-support/iwyu/fix_includes.py
M build-support/release/rat_exclude_files.txt
2 files changed, 2,232 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/05/10105/1
--
To view, visit http://gerrit.cloudera.org:8080/10105
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I98ebd8342cf7827d3204ff9063dbec23f16be4a6
Gerrit-Change-Number: 10105
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Dan Burkert 

[kudu-CR] Add krb5-server to SLES prerequisite libraries

[Sailesh Mukil (Code Review)]

Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9727 )

Change subject: Add krb5-server to SLES prerequisite libraries
..


Patch Set 1: Code-Review+2

Carry +2.

I think we forgot to merge this.


--
To view, visit http://gerrit.cloudera.org:8080/9727
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I12f78066f9dabe60d63dcea03a1ba22da942ac0f
Gerrit-Change-Number: 9727
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Wed, 18 Apr 2018 21:31:17 +
Gerrit-HasComments: No

[kudu-CR] Re-enable IWYU checks on HMS module

[Dan Burkert (Code Review)]

Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10102

to look at the new patch set (#2).

Change subject: Re-enable IWYU checks on HMS module
..

Re-enable IWYU checks on HMS module

IWYU previously had some issues with the HMS module[1], probably due to
the Thrift generated classes. Now that IWYU has been upgraded[2] this is
an attempt to re-enable the IWYU checks.  This commit also includes some
associated fixups.

[1]: 
https://github.com/apache/kudu/commit/c09a6296181ecbb7d91687536f70fdbaac9fff5e
[2]: 
https://github.com/apache/kudu/commit/bba339b11795d84da9c9d1473366ca561730c141

Change-Id: I63e5f1ca6a480f22ede83acde6baea8e2f563e8a
---
M build-support/iwyu/iwyu-filter.awk
M src/kudu/hms/hms_catalog-test.cc
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_client-test.cc
M src/kudu/hms/hms_client.cc
M src/kudu/hms/mini_hms.cc
M src/kudu/hms/mini_hms.h
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
10 files changed, 43 insertions(+), 46 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/02/10102/2
--
To view, visit http://gerrit.cloudera.org:8080/10102
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I63e5f1ca6a480f22ede83acde6baea8e2f563e8a
Gerrit-Change-Number: 10102
Gerrit-PatchSet: 2
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] Re-enable IWYU checks on HMS module

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10102 )

Change subject: Re-enable IWYU checks on HMS module
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10102
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I63e5f1ca6a480f22ede83acde6baea8e2f563e8a
Gerrit-Change-Number: 10102
Gerrit-PatchSet: 1
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 18 Apr 2018 20:02:28 +
Gerrit-HasComments: No

[kudu-CR] Re-enable IWYU checks on HMS module

[Dan Burkert (Code Review)]

Hello Adar Dembo, Todd Lipcon,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/10102

to review the following change.


Change subject: Re-enable IWYU checks on HMS module
..

Re-enable IWYU checks on HMS module

IWYU previously had some issues with the HMS module[1], probably due to
the Thrift generated classes. Now that IWYU has been upgraded[2] this is
an attempt to re-enable the IWYU checks.  This commit also includes some
associated fixups.

[1]: 
https://github.com/apache/kudu/commit/c09a6296181ecbb7d91687536f70fdbaac9fff5e
[2]: 
https://github.com/apache/kudu/commit/bba339b11795d84da9c9d1473366ca561730c141

Change-Id: I63e5f1ca6a480f22ede83acde6baea8e2f563e8a
---
M build-support/iwyu/iwyu-filter.awk
M src/kudu/hms/hms_catalog-test.cc
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_client-test.cc
M src/kudu/hms/hms_client.cc
M src/kudu/hms/mini_hms.cc
M src/kudu/hms/mini_hms.h
M src/kudu/integration-tests/master_hms-itest.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
10 files changed, 43 insertions(+), 46 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/02/10102/1
--
To view, visit http://gerrit.cloudera.org:8080/10102
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I63e5f1ca6a480f22ede83acde6baea8e2f563e8a
Gerrit-Change-Number: 10102
Gerrit-PatchSet: 1
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull option

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9834 )

Change subject: [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull 
option
..


Patch Set 10: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/9834
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ide908ea29f572849eca0ba850ee197c1b22a07c8
Gerrit-Change-Number: 9834
Gerrit-PatchSet: 10
Gerrit-Owner: Fengling Wang 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Fengling Wang 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 18 Apr 2018 19:55:55 +
Gerrit-HasComments: No

[kudu-CR] Various fixes for clang-tidy errors

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9899 )

Change subject: Various fixes for clang-tidy errors
..


Patch Set 2:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/9899/2/src/kudu/cfile/index-test.cc
File src/kudu/cfile/index-test.cc:

http://gerrit.cloudera.org:8080/#/c/9899/2/src/kudu/cfile/index-test.cc@43
PS2, Line 43: const std::string _key,
Nit: const std::string&

Or were you trying to be consistent with the style of the other args?


http://gerrit.cloudera.org:8080/#/c/9899/2/src/kudu/common/partition.cc
File src/kudu/common/partition.cc:

http://gerrit.cloudera.org:8080/#/c/9899/2/src/kudu/common/partition.cc@119
PS2, Line 119: for (const PartitionSchemaPB_ColumnIdentifierPB& identifier 
: identifiers) {
Could use const auto here too.


http://gerrit.cloudera.org:8080/#/c/9899/2/src/kudu/common/partition_pruner.cc
File src/kudu/common/partition_pruner.cc:

http://gerrit.cloudera.org:8080/#/c/9899/2/src/kudu/common/partition_pruner.cc@163
PS2, Line 163:   for (const ColumnId& column : range_columns) {
Could use const auto


http://gerrit.cloudera.org:8080/#/c/9899/2/src/kudu/tablet/rowset_info.cc
File src/kudu/tablet/rowset_info.cc:

http://gerrit.cloudera.org:8080/#/c/9899/2/src/kudu/tablet/rowset_info.cc@207
PS2, Line 207:   for (const shared_ptr& rs : tree.all_rowsets()) {
Could use const auto


http://gerrit.cloudera.org:8080/#/c/9899/2/src/kudu/tablet/transactions/transaction_tracker.cc
File src/kudu/tablet/transactions/transaction_tracker.cc:

http://gerrit.cloudera.org:8080/#/c/9899/2/src/kudu/tablet/transactions/transaction_tracker.cc@230
PS2, Line 230:   for (const scoped_refptr& driver : 
txns) {
Const auto?



--
To view, visit http://gerrit.cloudera.org:8080/9899
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I69e2e1be80ec68537df101a4f508b4f1cf6e1342
Gerrit-Change-Number: 9899
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Wed, 18 Apr 2018 19:52:31 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2365: [Java] Fix Gradle Artifacts

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10097 )

Change subject: KUDU-2365: [Java] Fix Gradle Artifacts
..


Patch Set 1: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/10097
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaef485eca7eebe55286ad5670452839bebb0c47e
Gerrit-Change-Number: 10097
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Wed, 18 Apr 2018 19:28:11 +
Gerrit-HasComments: No

[kudu-CR] [Java] Fix Guava transitive dependecies

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10096 )

Change subject: [Java] Fix Guava transitive dependecies
..


Patch Set 2: Code-Review+1

Will defer to Dan.


--
To view, visit http://gerrit.cloudera.org:8080/10096
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8f1f6e762629c1dea30589a1ade1f3da6de6ad3d
Gerrit-Change-Number: 10096
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Wed, 18 Apr 2018 19:25:44 +
Gerrit-HasComments: No

[kudu-CR] [Java] Mark the Yetus annotations dependency optional

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10095 )

Change subject: [Java] Mark the Yetus annotations dependency optional
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10095/2/java/kudu-spark-tools/build.gradle
File java/kudu-spark-tools/build.gradle:

http://gerrit.cloudera.org:8080/#/c/10095/2/java/kudu-spark-tools/build.gradle@a26
PS2, Line 26:
Nit: an empty line here is actually nice, it helps emphasize the comment below 
it.



--
To view, visit http://gerrit.cloudera.org:8080/10095
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6daf4f21c6cdde08145f5467e84d68dbe40cec16
Gerrit-Change-Number: 10095
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Wed, 18 Apr 2018 19:24:09 +
Gerrit-HasComments: Yes

[kudu-CR] [Java] Use Yetus annotations in place of @VisibleForTesting

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10077 )

Change subject: [Java] Use Yetus annotations in place of @VisibleForTesting
..


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10077/6//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/10077/6//COMMIT_MSG@10
PS6, Line 10: This patch replaces Guava’s @VisibleForTesting with
: @InterfaceAudience.LimitedPrivate("Test”).
Should comment on the exception for kudu-hive.



--
To view, visit http://gerrit.cloudera.org:8080/10077
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic0d8845e8cbbf5ea81345c46c5487121d1098f91
Gerrit-Change-Number: 10077
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 18 Apr 2018 19:22:59 +
Gerrit-HasComments: Yes

[kudu-CR] [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull option

[Fengling Wang (Code Review)]

Fengling Wang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9834 )

Change subject: [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull 
option
..


Patch Set 10:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9834/8/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
File 
java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala:

http://gerrit.cloudera.org:8080/#/c/9834/8/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala@78
PS8, Line 78: ERATION) == "insert-ignore").getOrElse(false)
> The number of combinator/fallback method calls is confusing, and using Try
Done. Thanks.



--
To view, visit http://gerrit.cloudera.org:8080/9834
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ide908ea29f572849eca0ba850ee197c1b22a07c8
Gerrit-Change-Number: 9834
Gerrit-PatchSet: 10
Gerrit-Owner: Fengling Wang 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Fengling Wang 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 18 Apr 2018 18:23:31 +
Gerrit-HasComments: Yes

[kudu-CR] [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull option

[Fengling Wang (Code Review)]

Hello Will Berkeley, Dan Burkert, Kudu Jenkins, Hao Hao, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9834

to look at the new patch set (#10).

Change subject: [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull 
option
..

[spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull option

This patch adds the KuduWriteOptions class to allow configuration of
writes to the Kudu table when writing with Spark. This allows
extensibility via adding more fields in the future. The instance of
this class is passed to functions (insert/delete/upsert/update) in
KuduContext.

KuduWriteOptions is also supported in DefaultSource APIs. Clients can
set up write options in the parameters from SparkSQL.

This patch also adds the ignoreNull write option so that users can
upsert/update only non-Null columns and leave the rest of the columns
unchanged.

For example, this feature is useful when users use Spark streaming to
process JSON and upsert to Kudu, because missing column values from
JSON are set to NULL, resulting in some existing row values being
upserted to Null, which is not desired.

Change-Id: Ide908ea29f572849eca0ba850ee197c1b22a07c8
---
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
A 
java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduWriteOptions.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/OperationType.scala
M 
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala
6 files changed, 237 insertions(+), 26 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/34/9834/10
--
To view, visit http://gerrit.cloudera.org:8080/9834
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ide908ea29f572849eca0ba850ee197c1b22a07c8
Gerrit-Change-Number: 9834
Gerrit-PatchSet: 10
Gerrit-Owner: Fengling Wang 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Fengling Wang 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 

[kudu-CR] [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull option

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9834 )

Change subject: [spark] KUDU-2371: Add KuduWriteOptions class and ignoreNull 
option
..


Patch Set 9:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9834/8/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
File 
java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala:

http://gerrit.cloudera.org:8080/#/c/9834/8/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala@78
PS8, Line 78:  || Try(parameters(OPERATION) == "insert-ignore").getOrElse(false)
> Done. Could you tell me why writing like this?
The number of combinator/fallback method calls is confusing, and using Try 
without the getOrElse calls keeps it to a minimum.

I think you missed the fix to the first clause to make it consistent.



--
To view, visit http://gerrit.cloudera.org:8080/9834
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ide908ea29f572849eca0ba850ee197c1b22a07c8
Gerrit-Change-Number: 9834
Gerrit-PatchSet: 9
Gerrit-Owner: Fengling Wang 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Fengling Wang 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 18 Apr 2018 18:09:16 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2365: [Java] Fix Gradle Artifacts

[Grant Henke (Code Review)]

Grant Henke has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10097


Change subject: KUDU-2365: [Java] Fix Gradle Artifacts
..

KUDU-2365: [Java] Fix Gradle Artifacts

This resolves KUDU-2365 and adresses the remaining
Gradle artifact issues.

- Fixes the handling of compileUnshaded scope
- Adds NOTICE.txt and LICENSE.txt file to the jar manifest
- Moves SLF4J to provided scope in Flume, Spark, and
   Mapreduce integrations where we want to use the version
   on the existing classpath.
- Reduce the usage of large aggregate dependencies where
   smaller more pointed ones are available
- Enables the /build directory on verify_jars.pl

Change-Id: Iaef485eca7eebe55286ad5670452839bebb0c47e
---
M build-support/verify_jars.pl
M java/gradle/artifacts.gradle
M java/gradle/shadow.gradle
M java/kudu-client-tools/build.gradle
M java/kudu-client-tools/pom.xml
M java/kudu-client/build.gradle
M java/kudu-client/pom.xml
M java/kudu-client/src/test/java/org/apache/kudu/client/TestUtils.java
M java/kudu-flume-sink/build.gradle
M java/kudu-flume-sink/pom.xml
M java/kudu-mapreduce/build.gradle
M java/kudu-mapreduce/pom.xml
M java/kudu-spark-tools/build.gradle
M java/kudu-spark-tools/pom.xml
M java/kudu-spark/build.gradle
M java/kudu-spark/pom.xml
16 files changed, 119 insertions(+), 54 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/97/10097/1
--
To view, visit http://gerrit.cloudera.org:8080/10097
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iaef485eca7eebe55286ad5670452839bebb0c47e
Gerrit-Change-Number: 10097
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke 

[kudu-CR] [Java] Mark the Yetus annotations dependency optional

[Grant Henke (Code Review)]

Hello Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10095

to look at the new patch set (#2).

Change subject: [Java] Mark the Yetus annotations dependency optional
..

[Java] Mark the Yetus annotations dependency optional

Given the Yetus dependency is mainly used for documentation
generation and is not used at runtime, we can mark the
annotation as optional for users consuming Kudu.
This is similar to the jsr305 dependency which we mark
as optional.

Additionally we don’t need to include Yetus in our shaded
jars.

Change-Id: I6daf4f21c6cdde08145f5467e84d68dbe40cec16
---
M java/kudu-client-tools/build.gradle
M java/kudu-client-tools/pom.xml
M java/kudu-client/build.gradle
M java/kudu-client/pom.xml
M java/kudu-flume-sink/build.gradle
M java/kudu-flume-sink/pom.xml
M java/kudu-jepsen/build.gradle
M java/kudu-jepsen/pom.xml
M java/kudu-mapreduce/build.gradle
M java/kudu-mapreduce/pom.xml
M java/kudu-spark-tools/build.gradle
M java/kudu-spark-tools/pom.xml
M java/kudu-spark/build.gradle
M java/kudu-spark/pom.xml
14 files changed, 55 insertions(+), 42 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/95/10095/2
--
To view, visit http://gerrit.cloudera.org:8080/10095
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6daf4f21c6cdde08145f5467e84d68dbe40cec16
Gerrit-Change-Number: 10095
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] KUDU-2287 Expose election failures as a metric

[Attila Bukor (Code Review)]

Attila Bukor has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10076 )

Change subject: KUDU-2287 Expose election failures as a metric
..


Patch Set 5:

> Patch Set 5:
>
> Build Started http://jenkins.kudu.apache.org/job/kudu-gerrit/13010/

looking at the build, it's going to fail, but it doesn't seem related. someone 
pls rebuild it


--
To view, visit http://gerrit.cloudera.org:8080/10076
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1b25df258cdba7bdae7bb2d7b4eb3d73b53425c3
Gerrit-Change-Number: 10076
Gerrit-PatchSet: 5
Gerrit-Owner: Attila Bukor 
Gerrit-Reviewer: Attila Bukor 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 18 Apr 2018 17:01:44 +
Gerrit-HasComments: No

[kudu-CR] [Java] Mark the Yetus annotations dependency optional

[Grant Henke (Code Review)]

Grant Henke has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10095


Change subject: [Java] Mark the Yetus annotations dependency optional
..

[Java] Mark the Yetus annotations dependency optional

Given the Yetus dependency is mainly used for documentation
generation and is not used at runtime, we can mark the
annotation as optional for users consuming Kudu.
This is similar to the jsr305 dependency which we mark
as optional.

Additionally we don’t need to include Yetus in our shaded
jars.

Change-Id: I6daf4f21c6cdde08145f5467e84d68dbe40cec16
---
M java/kudu-client-tools/build.gradle
M java/kudu-client-tools/pom.xml
M java/kudu-client/build.gradle
M java/kudu-client/pom.xml
M java/kudu-flume-sink/build.gradle
M java/kudu-flume-sink/pom.xml
M java/kudu-jepsen/build.gradle
M java/kudu-jepsen/pom.xml
M java/kudu-mapreduce/build.gradle
M java/kudu-mapreduce/pom.xml
M java/kudu-spark-tools/build.gradle
M java/kudu-spark-tools/pom.xml
M java/kudu-spark/build.gradle
M java/kudu-spark/pom.xml
14 files changed, 55 insertions(+), 42 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/95/10095/1
--
To view, visit http://gerrit.cloudera.org:8080/10095
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6daf4f21c6cdde08145f5467e84d68dbe40cec16
Gerrit-Change-Number: 10095
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke 

[kudu-CR] [Java] Fix Guava transitive dependecies

[Grant Henke (Code Review)]

Grant Henke has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10096


Change subject: [Java] Fix Guava transitive dependecies
..

[Java] Fix Guava transitive dependecies

In Guava 22.0+ the annotations dependencies are no
longer marked as optional. However, if the annotations
are not used at runtime or compile time they are
actually not required.

- Excludes the transitive annotations libraries that are
   not marked as optional in version 22.0+
- Add’s Guava where it was used directly but not in the
   dependencies

Change-Id: I8f1f6e762629c1dea30589a1ade1f3da6de6ad3d
---
M java/kudu-client-tools/build.gradle
M java/kudu-client-tools/pom.xml
M java/kudu-client/build.gradle
M java/kudu-client/pom.xml
M java/kudu-flume-sink/pom.xml
M java/kudu-mapreduce/build.gradle
M java/kudu-mapreduce/pom.xml
7 files changed, 62 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/96/10096/1
--
To view, visit http://gerrit.cloudera.org:8080/10096
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8f1f6e762629c1dea30589a1ade1f3da6de6ad3d
Gerrit-Change-Number: 10096
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke 

[kudu-CR] KUDU-2287 Expose election failures as a metric

[Attila Bukor (Code Review)]

Hello Will Berkeley, Tidy Bot, Kudu Jenkins, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10076

to look at the new patch set (#5).

Change subject: KUDU-2287 Expose election failures as a metric
..

KUDU-2287 Expose election failures as a metric

This patch exposes as a metric the number of election failures since the
last successful election attempt

Change-Id: I1b25df258cdba7bdae7bb2d7b4eb3d73b53425c3
---
M src/kudu/consensus/raft_consensus.cc
M src/kudu/consensus/raft_consensus.h
2 files changed, 45 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/10076/5
--
To view, visit http://gerrit.cloudera.org:8080/10076
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1b25df258cdba7bdae7bb2d7b4eb3d73b53425c3
Gerrit-Change-Number: 10076
Gerrit-PatchSet: 5
Gerrit-Owner: Attila Bukor 
Gerrit-Reviewer: Attila Bukor 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 

[kudu-CR] KUDU-2287 Expose election failures as a metric

[Attila Bukor (Code Review)]

Hello Will Berkeley, Kudu Jenkins, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10076

to look at the new patch set (#4).

Change subject: KUDU-2287 Expose election failures as a metric
..

KUDU-2287 Expose election failures as a metric

This patch exposes as a metric the number of election failures since the
last successful election attempt

Change-Id: I1b25df258cdba7bdae7bb2d7b4eb3d73b53425c3
---
M src/kudu/consensus/raft_consensus.cc
M src/kudu/consensus/raft_consensus.h
2 files changed, 45 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/10076/4
--
To view, visit http://gerrit.cloudera.org:8080/10076
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1b25df258cdba7bdae7bb2d7b4eb3d73b53425c3
Gerrit-Change-Number: 10076
Gerrit-PatchSet: 4
Gerrit-Owner: Attila Bukor 
Gerrit-Reviewer: Attila Bukor 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 

[kudu-CR] KUDU-2287 Expose election failures as a metric

[Attila Bukor (Code Review)]

Attila Bukor has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10076 )

Change subject: KUDU-2287 Expose election failures as a metric
..


Patch Set 3:

(2 comments)

> Patch Set 2:
>
> (1 comment)

http://gerrit.cloudera.org:8080/#/c/10076/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/10076/1//COMMIT_MSG@7
PS1, Line 7: KUDU-2287 Expose election failures as a metric
> huh cool, didn't know about the function gauge. I'll look into this.
Done


http://gerrit.cloudera.org:8080/#/c/10076/3/src/kudu/consensus/raft_consensus.cc
File src/kudu/consensus/raft_consensus.cc:

http://gerrit.cloudera.org:8080/#/c/10076/3/src/kudu/consensus/raft_consensus.cc@161
PS3, Line 161:   kudu::MetricUnit::kNanoseconds,
unit might be a bit overkill, let me know if I should change it to something 
else.



--
To view, visit http://gerrit.cloudera.org:8080/10076
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1b25df258cdba7bdae7bb2d7b4eb3d73b53425c3
Gerrit-Change-Number: 10076
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Bukor 
Gerrit-Reviewer: Attila Bukor 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Wed, 18 Apr 2018 15:22:07 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2287 Expose election failures as a metric

[Attila Bukor (Code Review)]

Hello Will Berkeley, Kudu Jenkins, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10076

to look at the new patch set (#3).

Change subject: KUDU-2287 Expose election failures as a metric
..

KUDU-2287 Expose election failures as a metric

This patch exposes as a metric the number of election failures since the
last successful election attempt

Change-Id: I1b25df258cdba7bdae7bb2d7b4eb3d73b53425c3
---
M src/kudu/consensus/raft_consensus.cc
M src/kudu/consensus/raft_consensus.h
2 files changed, 44 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/10076/3
--
To view, visit http://gerrit.cloudera.org:8080/10076
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1b25df258cdba7bdae7bb2d7b4eb3d73b53425c3
Gerrit-Change-Number: 10076
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Bukor 
Gerrit-Reviewer: Attila Bukor 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley