[4/4] kudu git commit: KUDU-16 pt 2: add client-side limits on scanners

2018-04-03 Thread awong 
KUDU-16 pt 2: add client-side limits on scanners

This patch adds a public API to allow the specification of
per-client-side-scanner limits on the number of rows returned. Each
scanner will maintain a count of the number of rows already read, and
adjust the server-side limit upon sending the next scan request.

A couple of tests are included to verify that the limits act as
expected. I also verified that lowering the limit reduces the number of
bytes read on disk (at the granularity of a single scan batch at a
time).

Note: this patch does not implement the behavior for scan tokens.

Change-Id: Ib2d40e3d14e36f3bf1d09a4bfdb3e17a745d244d
Reviewed-on: http://gerrit.cloudera.org:8080/9790
Tested-by: Kudu Jenkins
Reviewed-by: Todd Lipcon 


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/725c6ce6
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/725c6ce6
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/725c6ce6

Branch: refs/heads/master
Commit: 725c6ce6391b42331e4ab4210ae4117779cba4a2
Parents: d720352
Author: Andrew Wong 
Authored: Fri Mar 23 12:22:23 2018 -0700
Committer: Todd Lipcon 
Committed: Wed Apr 4 00:22:05 2018 +

--
 src/kudu/client/client-test.cc| 188 +++--
 src/kudu/client/client.cc |   7 ++
 src/kudu/client/client.h  |   7 ++
 src/kudu/client/scan_configuration.cc |   8 ++
 src/kudu/client/scan_configuration.h  |   2 +
 src/kudu/client/scanner-internal.cc   |  12 +-
 src/kudu/client/scanner-internal.h|   3 +
 7 files changed, 186 insertions(+), 41 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/725c6ce6/src/kudu/client/client-test.cc
--
diff --git a/src/kudu/client/client-test.cc b/src/kudu/client/client-test.cc
index a227b15..3391f51 100644
--- a/src/kudu/client/client-test.cc
+++ b/src/kudu/client/client-test.cc
@@ -114,6 +114,8 @@ DECLARE_bool(fail_dns_resolution);
 DECLARE_bool(log_inject_latency);
 DECLARE_bool(master_support_connect_to_master_rpc);
 DECLARE_bool(rpc_trace_negotiation);
+DECLARE_int32(flush_threshold_mb);
+DECLARE_int32(flush_threshold_secs);
 DECLARE_int32(heartbeat_interval_ms);
 DECLARE_int32(leader_failure_exp_backoff_max_delta_ms);
 DECLARE_int32(log_inject_latency_ms_mean);
@@ -121,6 +123,7 @@ DECLARE_int32(log_inject_latency_ms_stddev);
 DECLARE_int32(master_inject_latency_on_tablet_lookups_ms);
 DECLARE_int32(max_create_tablets_per_ts);
 DECLARE_int32(raft_heartbeat_interval_ms);
+DECLARE_int32(scanner_batch_size_rows);
 DECLARE_int32(scanner_gc_check_interval_us);
 DECLARE_int32(scanner_inject_latency_on_each_batch_ms);
 DECLARE_int32(scanner_max_batch_size_bytes);
@@ -130,6 +133,7 @@ DECLARE_string(superuser_acl);
 DECLARE_string(user_acl);
 DEFINE_int32(test_scan_num_rows, 1000, "Number of rows to insert and scan");
 
+METRIC_DECLARE_counter(block_manager_total_bytes_read);
 METRIC_DECLARE_counter(rpcs_queue_overflow);
 
METRIC_DECLARE_histogram(handler_latency_kudu_master_MasterService_GetMasterRegistration);
 
METRIC_DECLARE_histogram(handler_latency_kudu_master_MasterService_GetTableLocations);
@@ -777,6 +781,98 @@ TEST_F(ClientTest, TestMasterDown) {
   ASSERT_TRUE(s.IsNetworkError()) << s.ToString();
 }
 
+// Test that we get errors when we try incorrectly configuring the scanner, and
+// that the scanner works well in a basic case.
+TEST_F(ClientTest, TestConfiguringScannerLimits) {
+  const int64_t kNumRows = 1234;
+  const int64_t kLimit = 123;
+  NO_FATALS(InsertTestRows(client_table_.get(), kNumRows));
+
+  // Ensure we can't set the limit to some negative number.
+  KuduScanner scanner(client_table_.get());
+  Status s = scanner.SetLimit(-1);
+  ASSERT_STR_CONTAINS(s.ToString(), "must be non-negative");
+  ASSERT_TRUE(s.IsInvalidArgument());
+
+  // Now actually set the limit and open the scanner.
+  ASSERT_OK(scanner.SetLimit(kLimit));
+  ASSERT_OK(scanner.Open());
+
+  // Ensure we can't set the limit once we've opened the scanner.
+  s = scanner.SetLimit(kLimit);
+  ASSERT_STR_CONTAINS(s.ToString(), "must be set before Open()");
+  ASSERT_TRUE(s.IsIllegalState());
+  int64_t count = 0;
+  KuduScanBatch batch;
+  while (scanner.HasMoreRows()) {
+ASSERT_OK(scanner.NextBatch());
+count += batch.NumRows();
+  }
+  ASSERT_EQ(kLimit, count);
+}
+
+// Test various scanner limits.
+TEST_F(ClientTest, TestRandomizedLimitScans) {
+  const string kTableName = "table";
+  unique_ptr table_creator(client_->NewTableCreator());
+  ASSERT_OK(table_creator->table_name(kTableName)
+  .schema(_)
+  .num_replicas(1)
+  .add_hash_partitions({ "key" }, 2)
+

[3/4] kudu git commit: KUDU-16 pt 1: add server-side limits on scanners

2018-04-03 Thread awong 
KUDU-16 pt 1: add server-side limits on scanners

This patch adds the functionality to specify limits to the number of
rows returned per tablet server scanner. Internally, each server-side
scanner now tracks the number of rows it has already returned and
adjusts the number of rows to serialize per batch based on this limit.
Setting a non-positive limit is grounds for short circuiting the scan.

Note: Use of this functionality by the client will come in a later
patch.

Change-Id: I3ca5ddff09f4910d12f80259bd11e4027f99aa7c
Reviewed-on: http://gerrit.cloudera.org:8080/9783
Tested-by: Kudu Jenkins
Reviewed-by: Todd Lipcon 


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/d7203522
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/d7203522
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/d7203522

Branch: refs/heads/master
Commit: d7203522445d337aa01a32b8ca23b5aa081a2dec
Parents: 52b50b7
Author: Andrew Wong 
Authored: Fri Mar 16 01:05:56 2018 -0700
Committer: Todd Lipcon 
Committed: Wed Apr 4 00:22:00 2018 +

--
 src/kudu/common/rowblock.cc| 24 ++
 src/kudu/common/rowblock.h |  6 +++
 src/kudu/common/scan_spec.cc   | 10 +++-
 src/kudu/common/scan_spec.h| 33 +
 src/kudu/tserver/scanners.cc   |  3 +-
 src/kudu/tserver/scanners.h| 25 +-
 src/kudu/tserver/tablet_server-test.cc | 74 +++--
 src/kudu/tserver/tablet_service.cc | 34 +
 src/kudu/tserver/tserver.proto |  7 +--
 9 files changed, 184 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/d7203522/src/kudu/common/rowblock.cc
--
diff --git a/src/kudu/common/rowblock.cc b/src/kudu/common/rowblock.cc
index f8dc532..2b8cd65 100644
--- a/src/kudu/common/rowblock.cc
+++ b/src/kudu/common/rowblock.cc
@@ -44,6 +44,30 @@ void SelectionVector::Resize(size_t n_rows) {
   }
 }
 
+void SelectionVector::ClearToSelectAtMost(size_t max_rows) {
+  if (max_rows < n_rows_) {
+BitmapIterator iter(_[0], n_rows_);
+bool selected;
+size_t run_size;
+size_t end_idx = 0;
+// Adjust the end index until we have selected 'max_rows' rows.
+while ((run_size = iter.Next()) && max_rows > 0) {
+  if (selected) {
+if (run_size >= max_rows) {
+  end_idx += max_rows;
+  break;
+}
+max_rows -= run_size;
+  }
+  end_idx += run_size;
+}
+// If the limit is reached, zero out the rest of the selection vector.
+if (n_rows_ > end_idx) {
+  BitmapChangeBits(_[0], end_idx, n_rows_ - end_idx, false);
+}
+  }
+}
+
 size_t SelectionVector::CountSelected() const {
   return Bits::Count(_[0], n_bytes_);
 }

http://git-wip-us.apache.org/repos/asf/kudu/blob/d7203522/src/kudu/common/rowblock.h
--
diff --git a/src/kudu/common/rowblock.h b/src/kudu/common/rowblock.h
index 2dc0c89..eaea3ec 100644
--- a/src/kudu/common/rowblock.h
+++ b/src/kudu/common/rowblock.h
@@ -63,6 +63,12 @@ class SelectionVector {
   // Ensures that all rows for indices < n_rows are unmodified.
   void Resize(size_t n_rows);
 
+  // Zeroes out the end of the selection vector such that it will be left with
+  // at most 'max_rows' selected.
+  //
+  // If 'max_rows' is greater than the allocated capacity, this does nothing.
+  void ClearToSelectAtMost(size_t max_rows);
+
   // Return the number of selected rows.
   size_t CountSelected() const;
 

http://git-wip-us.apache.org/repos/asf/kudu/blob/d7203522/src/kudu/common/scan_spec.cc
--
diff --git a/src/kudu/common/scan_spec.cc b/src/kudu/common/scan_spec.cc
index c232e79..18dff78 100644
--- a/src/kudu/common/scan_spec.cc
+++ b/src/kudu/common/scan_spec.cc
@@ -18,7 +18,6 @@
 #include "kudu/common/scan_spec.h"
 
 #include 
-#include 
 #include 
 #include 
 #include 
@@ -37,6 +36,7 @@
 #include "kudu/gutil/gscoped_ptr.h"
 #include "kudu/gutil/map-util.h"
 #include "kudu/gutil/strings/join.h"
+#include "kudu/gutil/strings/substitute.h"
 #include "kudu/util/auto_release_pool.h"
 #include "kudu/util/memory/arena.h"
 
@@ -46,6 +46,7 @@ using std::move;
 using std::pair;
 using std::string;
 using std::vector;
+using strings::Substitute;
 
 namespace kudu {
 
@@ -68,6 +69,10 @@ void ScanSpec::RemovePredicates() {
 }
 
 bool ScanSpec::CanShortCircuit() const {
+  if (has_limit() && *limit_ <= 0) {
+return true;
+  }
+
   if (lower_bound_key_ &&
   exclusive_upper_bound_key_ &&

kudu git commit: Followup to fe4d962c09012e to fix broken test

2018-04-03 Thread danburkert 
Repository: kudu
Updated Branches:
  refs/heads/master 9e918bf77 -> de067f792


Followup to fe4d962c09012e to fix broken test

The newly introduced tests weren't cleaning up the test directory
because they didn't inherit from KuduTest.

Change-Id: I0e323f5c9082013510cddc6873bd81675e66d644
Reviewed-on: http://gerrit.cloudera.org:8080/9917
Reviewed-by: Adar Dembo 
Tested-by: Kudu Jenkins


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/de067f79
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/de067f79
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/de067f79

Branch: refs/heads/master
Commit: de067f792afc051500a622b77d8c556511ae64f0
Parents: 9e918bf
Author: Dan Burkert 
Authored: Tue Apr 3 15:03:05 2018 -0700
Committer: Dan Burkert 
Committed: Tue Apr 3 23:00:47 2018 +

--
 src/kudu/hms/hms_catalog-test.cc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/de067f79/src/kudu/hms/hms_catalog-test.cc
--
diff --git a/src/kudu/hms/hms_catalog-test.cc b/src/kudu/hms/hms_catalog-test.cc
index 1510af0..d060320 100644
--- a/src/kudu/hms/hms_catalog-test.cc
+++ b/src/kudu/hms/hms_catalog-test.cc
@@ -37,6 +37,7 @@
 #include "kudu/util/net/net_util.h" // IWYU pragma: keep
 #include "kudu/util/status.h"
 #include "kudu/util/test_macros.h"
+#include "kudu/util/test_util.h"
 
 DECLARE_string(hive_metastore_uris);
 DECLARE_bool(hive_metastore_sasl_enabled);
@@ -113,7 +114,7 @@ TEST(HmsCatalogStaticTest, TestParseUris) {
 
 // Base class for HmsCatalog tests. Parameterized by whether
 // SASL/GSSAPI/Kerberos should be enabled.
-class HmsCatalogTest : public ::testing::Test {
+class HmsCatalogTest : public KuduTest {
  public:
 
   const char* const kMasterAddrs = "master-addrs";

kudu git commit: perf: avoid unnecessary floating point division and multiplication

2018-04-03 Thread mpercy 
Repository: kudu
Updated Branches:
  refs/heads/master 02560c188 -> 9e918bf77


perf: avoid unnecessary floating point division and multiplication

I noticed looking at a profile that Trace::SubstituteAndTrace was
spending a fair number of cycles in some floating point math, and
tracked it down to our use of constants like '1e3' and '1e6'. These are
floating point constants even though we would do just as well with
integer multiplication and division for this use case.

I verified that switching to the integer versions of the constants
changed the generated assembly to avoid costly division instructions[1].

This patch fixes that case as well as a few others with the same
pattern.

[1] https://godbolt.org/g/dmE7Av

Change-Id: I3985433056aab163cc65972e94bc2108f00546a3
Reviewed-on: http://gerrit.cloudera.org:8080/9894
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo 
Reviewed-by: Mike Percy 


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/9e918bf7
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/9e918bf7
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/9e918bf7

Branch: refs/heads/master
Commit: 9e918bf7766c2b4a4d36156891836b167a0563ff
Parents: 02560c1
Author: Todd Lipcon 
Authored: Mon Apr 2 16:05:15 2018 -0700
Committer: Mike Percy 
Committed: Tue Apr 3 19:45:49 2018 +

--
 src/kudu/clock/hybrid_clock.cc | 2 +-
 src/kudu/gutil/walltime.h  | 8 
 src/kudu/rpc/reactor.cc| 2 +-
 src/kudu/util/env_posix.cc | 4 ++--
 src/kudu/util/logging.h| 2 +-
 5 files changed, 9 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/9e918bf7/src/kudu/clock/hybrid_clock.cc
--
diff --git a/src/kudu/clock/hybrid_clock.cc b/src/kudu/clock/hybrid_clock.cc
index a9429e6..b97fd2d 100644
--- a/src/kudu/clock/hybrid_clock.cc
+++ b/src/kudu/clock/hybrid_clock.cc
@@ -387,7 +387,7 @@ Status HybridClock::WalltimeWithError(uint64_t* now_usec, 
uint64_t* error_usec)
 }
 MonoDelta time_since_last_read = read_time_after - last_clock_read_time_;
 int64_t micros_since_last_read = time_since_last_read.ToMicroseconds();
-int64_t accum_error_us = (micros_since_last_read * 
time_service_->skew_ppm()) / 1e6;
+int64_t accum_error_us = (micros_since_last_read * 
time_service_->skew_ppm()) / 100;
 *now_usec = last_clock_read_physical_ + micros_since_last_read;
 *error_usec = last_clock_read_error_ + accum_error_us;
 is_extrapolated = true;

http://git-wip-us.apache.org/repos/asf/kudu/blob/9e918bf7/src/kudu/gutil/walltime.h
--
diff --git a/src/kudu/gutil/walltime.h b/src/kudu/gutil/walltime.h
index 02338a8..e9cab67 100644
--- a/src/kudu/gutil/walltime.h
+++ b/src/kudu/gutil/walltime.h
@@ -81,7 +81,7 @@ inline void GetCurrentTime(mach_timespec_t* ts) {
 inline MicrosecondsInt64 GetCurrentTimeMicros() {
   mach_timespec_t ts;
   GetCurrentTime();
-  return ts.tv_sec * 1e6 + ts.tv_nsec / 1e3;
+  return ts.tv_sec * 100 + ts.tv_nsec / 1000;
 }
 
 inline int64_t GetMonoTimeNanos() {
@@ -96,7 +96,7 @@ inline int64_t GetMonoTimeNanos() {
 }
 
 inline MicrosecondsInt64 GetMonoTimeMicros() {
-  return GetMonoTimeNanos() / 1e3;
+  return GetMonoTimeNanos() / 1000;
 }
 
 inline MicrosecondsInt64 GetThreadCpuTimeMicros() {
@@ -122,7 +122,7 @@ inline MicrosecondsInt64 GetThreadCpuTimeMicros() {
 return 0;
   }
 
-  return thread_info_data.user_time.seconds * 1e6 + 
thread_info_data.user_time.microseconds;
+  return thread_info_data.user_time.seconds * 100 + 
thread_info_data.user_time.microseconds;
 }
 
 #else
@@ -130,7 +130,7 @@ inline MicrosecondsInt64 GetThreadCpuTimeMicros() {
 inline MicrosecondsInt64 GetClockTimeMicros(clockid_t clock) {
   timespec ts;
   clock_gettime(clock, );
-  return ts.tv_sec * 1e6 + ts.tv_nsec / 1e3;
+  return ts.tv_sec * 100 + ts.tv_nsec / 1000;
 }
 
 #endif // defined(__APPLE__)

http://git-wip-us.apache.org/repos/asf/kudu/blob/9e918bf7/src/kudu/rpc/reactor.cc
--
diff --git a/src/kudu/rpc/reactor.cc b/src/kudu/rpc/reactor.cc
index e841be6..d3052ea 100644
--- a/src/kudu/rpc/reactor.cc
+++ b/src/kudu/rpc/reactor.cc
@@ -179,7 +179,7 @@ void ReactorThread::InvokePendingCb(struct ev_loop* loop) {
   // Contribute this to our histogram.
   ReactorThread* thr = static_cast(ev_userdata(loop));
   if (thr->invoke_us_histogram_) {
-thr->invoke_us_histogram_->Increment(dur_cycles * 1e6 / 
base::CyclesPerSecond());
+thr->invoke_us_histogram_->Increment(dur_cycles * 100 / 
base::CyclesPerSecond());
   }
 }

kudu git commit: KUDU-2191 (8/n): Integrate HmsCatalog into CatalogManager

2018-04-03 Thread danburkert 
Repository: kudu
Updated Branches:
  refs/heads/master 5fc31db00 -> 02560c188


KUDU-2191 (8/n): Integrate HmsCatalog into CatalogManager

This commit connects the CatalogManager to the HMS via the HmsCatalog
abstraction. When the HMS integration is enabled (by setting the
--hive-metastore-uris flag) and Kudu tables are created, altered, or
dropped, the corresponding HMS entry is also modified appropriately.
Additionally, New table and column names are required to be valid
according to the Hive identifier rules, which are much stricter than
Kudu's existing identifier rules.

Testing: This commit adds a new integration test (master_hms-itest)
which tests that the integration works as expected with
create/alter/drop table operations. Additionally, some existing DDL
stress tests now have the HMS integration enabled in order to provide
more coverage.

Change-Id: Ie68e143c3c317c7690af097e6485934feb1010b4
Reviewed-on: http://gerrit.cloudera.org:8080/9863
Reviewed-by: Dan Burkert 
Tested-by: Dan Burkert 


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/02560c18
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/02560c18
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/02560c18

Branch: refs/heads/master
Commit: 02560c188f6b76714ee1908420e539f28bce99e8
Parents: 5fc31db
Author: Dan Burkert 
Authored: Wed Mar 28 16:45:35 2018 -0700
Committer: Dan Burkert 
Committed: Tue Apr 3 19:23:58 2018 +

--
 build-support/iwyu/iwyu-filter.awk  |   6 +
 src/kudu/hms/hms_catalog.cc |  62 +--
 src/kudu/hms/hms_catalog.h  |  11 +-
 src/kudu/hms/hms_client.h   |   6 +-
 src/kudu/hms/mini_hms.h |   2 +-
 src/kudu/integration-tests/CMakeLists.txt   |   1 +
 .../alter_table-randomized-test.cc  |   3 +-
 .../integration-tests/master-stress-test.cc |   7 +-
 .../integration-tests/master_failover-itest.cc  |  43 +-
 src/kudu/integration-tests/master_hms-itest.cc  | 394 +++
 src/kudu/master/CMakeLists.txt  |   3 +-
 src/kudu/master/catalog_manager.cc  | 184 +++--
 src/kudu/master/catalog_manager.h   |   9 +-
 src/kudu/master/master.proto|   3 +
 src/kudu/mini-cluster/external_mini_cluster.cc  |  10 +
 15 files changed, 655 insertions(+), 89 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/02560c18/build-support/iwyu/iwyu-filter.awk
--
diff --git a/build-support/iwyu/iwyu-filter.awk 
b/build-support/iwyu/iwyu-filter.awk
index c20f2d1..42f747f 100644
--- a/build-support/iwyu/iwyu-filter.awk
+++ b/build-support/iwyu/iwyu-filter.awk
@@ -90,7 +90,13 @@ BEGIN {
   muted["kudu/common/encoded_key-test.cc"]
   muted["kudu/common/schema.h"]
   muted["kudu/experiments/rwlock-perf.cc"]
+  muted["kudu/hms/hms_catalog.cc"]
+  muted["kudu/hms/hms_catalog.h"]
   muted["kudu/hms/hms_client.cc"]
+  muted["kudu/hms/hms_client.h"]
+  muted["kudu/hms/mini_hms.h"]
+  muted["kudu/master/catalog_manager.cc"]
+  muted["kudu/master/catalog_manager.h"]
   muted["kudu/rpc/reactor.cc"]
   muted["kudu/rpc/reactor.h"]
   muted["kudu/security/ca/cert_management.cc"]

http://git-wip-us.apache.org/repos/asf/kudu/blob/02560c18/src/kudu/hms/hms_catalog.cc
--
diff --git a/src/kudu/hms/hms_catalog.cc b/src/kudu/hms/hms_catalog.cc
index 3cfbd90..a6433b3 100644
--- a/src/kudu/hms/hms_catalog.cc
+++ b/src/kudu/hms/hms_catalog.cc
@@ -17,13 +17,13 @@
 
 #include "kudu/hms/hms_catalog.h"
 
-#include  // IWYU pragma: keep
-#include  // IWYU pragma: keep
+#include 
+#include 
 #include 
-#include  // IWYU pragma: keep
+#include 
 #include 
-#include  // IWYU pragma: keep
-#include  // IWYU pragma: keep
+#include 
+#include 
 #include 
 
 #include 
@@ -33,16 +33,16 @@
 #include "kudu/common/schema.h"
 #include "kudu/common/types.h"
 #include "kudu/gutil/macros.h"
-#include "kudu/gutil/strings/split.h" // IWYU pragma: keep
+#include "kudu/gutil/strings/split.h"
 #include "kudu/gutil/strings/substitute.h"
 #include "kudu/hms/hive_metastore_constants.h"
 #include "kudu/hms/hive_metastore_types.h"
 #include "kudu/hms/hms_client.h"
 #include "kudu/util/async_util.h"
 #include "kudu/util/flag_tags.h"
-#include "kudu/util/net/net_util.h" // IWYU pragma: keep
+#include "kudu/util/net/net_util.h"
 #include "kudu/util/slice.h"
-#include "kudu/util/threadpool.h" // IWYU pragma: keep
+#include "kudu/util/threadpool.h"
 
 using std::string;
 using std::vector;
@@ -273,8 +273,8 @@ Status HmsCatalog::AlterTable(const string& id,
 
 template
 Status

[3/4] kudu git commit: [docs] Update docs to point to new examples location

2018-04-03 Thread granthenke 
[docs] Update docs to point to new examples location

Change-Id: I685b2c9c8b24fc232c61bf809e248869c822530b
Reviewed-on: http://gerrit.cloudera.org:8080/9908
Reviewed-by: Adar Dembo 
Reviewed-by: Will Berkeley 
Tested-by: Grant Henke 


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/613fe93e
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/613fe93e
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/613fe93e

Branch: refs/heads/master
Commit: 613fe93e6048f34828660adf4d010c2e67398cf5
Parents: dbc943f
Author: Grant Henke 
Authored: Tue Apr 3 12:20:55 2018 -0500
Committer: Grant Henke 
Committed: Tue Apr 3 17:35:00 2018 +

--
 docs/developing.adoc | 14 ++
 1 file changed, 6 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/613fe93e/docs/developing.adoc
--
diff --git a/docs/developing.adoc b/docs/developing.adoc
index 09ffb82..4d1c659 100644
--- a/docs/developing.adoc
+++ b/docs/developing.adoc
@@ -40,13 +40,14 @@ include::installation.adoc[tags=view_api]
 == Working Examples
 
 Several example applications are provided in the
-link:https://github.com/cloudera/kudu-examples[kudu-examples] Github
-repository. Each example includes a `README` that shows how to compile and run
-it. These examples illustrate correct usage of the Kudu APIs, as well as how to
-set up a virtual machine to run Kudu. The following list includes some of the
+link:https://github.com/apache/kudu/tree/master/examples[examples directory]
+of the Apache Kudu git repository. Each example includes a `README` that shows
+how to compile and run it. The following list includes some of the
 examples that are available today. Check the repository itself in case this 
list goes
 out of date.
 
+`cpp/example.cc`::
+  A simple C++ application which connects to a Kudu instance, creates a table, 
writes data to it, then drops the table.
 `java/java-example`::
   A simple Java application which connects to a Kudu instance, creates a 
table, writes data to it, then drops the table.
 `java/collectl`::
@@ -58,10 +59,7 @@ out of date.
   An example program that shows how to use the Kudu Python API to load data 
into a new / existing Kudu table
   generated by an external program, `dstat` in this case.
 `python/graphite-kudu`::
-  An experimental plugin for using graphite-web with Kudu as a backend.
-`demo-vm-setup`::
-  Scripts to download and run a VirtualBox virtual machine with Kudu already 
installed.
-  See link:quickstart.html[Quickstart] for more information.
+  An example plugin for using graphite-web with Kudu as a backend.
 
 These examples should serve as helpful starting points for your own Kudu 
applications and integrations.

[1/4] kudu git commit: [build] Remove old trigger_gerrit.py script

2018-04-03 Thread granthenke 
Repository: kudu
Updated Branches:
  refs/heads/master c3684b6e1 -> 5fc31db00


[build] Remove old trigger_gerrit.py script

Change-Id: Ie1c844092f892391449e4cefd7243ae11fc0d52d
Reviewed-on: http://gerrit.cloudera.org:8080/9902
Tested-by: Grant Henke 
Reviewed-by: Adar Dembo 


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/1a24020e
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/1a24020e
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/1a24020e

Branch: refs/heads/master
Commit: 1a24020ecf60f4fa244679710bba80861a8b918b
Parents: c3684b6
Author: Grant Henke 
Authored: Tue Apr 3 11:15:43 2018 -0500
Committer: Grant Henke 
Committed: Tue Apr 3 17:32:48 2018 +

--
 build-support/trigger_gerrit.py | 114 ---
 1 file changed, 114 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/1a24020e/build-support/trigger_gerrit.py
--
diff --git a/build-support/trigger_gerrit.py b/build-support/trigger_gerrit.py
deleted file mode 100755
index 740af43..000
--- a/build-support/trigger_gerrit.py
+++ /dev/null
@@ -1,114 +0,0 @@
-#!/usr/bin/env python
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements.  See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership.  The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License.  You may obtain a copy of the License at
-#
-#   http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied.  See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-
-# This tool triggers a Jenkins build based on a particular gerrit URL.
-# The Jenkins build will post a +1 or -1 on the gerrit.
-#
-# NOTE: currently this is hard-coded to an internal Cloudera server.
-# We plan to move to upstream infrastructure at some later date.
-
-import logging
-import json
-import re
-import subprocess
-import sys
-import urllib
-import urllib2
-import urlparse
-
-from kudu_util import check_output
-
-GERRIT_HOST = "gerrit.cloudera.org"
-JENKINS_URL = "http://sandbox.jenkins.cloudera.com/;
-JENKINS_JOB = "kudu-public-gerrit"
-
-
-def get_gerrit_ssh_command():
-  url = check_output("git config --get remote.gerrit.url".split(" "))
-  m = re.match(r'ssh://(.+)@(.+):(\d+)/.+', url)
-  if not m:
-raise Exception("expected gerrit remote to be an ssh://user@host:port/ 
URL: %s" % url)
-  user, host, port = m.groups()
-  if host != GERRIT_HOST:
-raise Exception("unexpected gerrit host %s in remote 'gerrit'. Expected 
%s" % (
-host, GERRIT_HOST))
-  return ["ssh", "-p", port, "-l", user, host]
-
-
-def current_ref_for_gerrit_number(change_num):
-  j = check_output(get_gerrit_ssh_command() + [
-  "gerrit", "query", "--current-patch-set", "--format", "JSON",
-  "change:%d" % change_num])
-  j = json.loads(j.split("\n")[0])
-  return j['currentPatchSet']['ref']
-
-
-def url_to_ref(url):
-  u = urlparse.urlparse(url)
-  if not u.netloc.startswith(GERRIT_HOST):
-print >>sys.stderr, "unexpected gerrit host %s, expected %s\n" % (
-u.netloc, GERRIT_HOST)
-usage()
-sys.exit(1)
-  if u.path == '/':
-m = re.match(r'/c/(\d+)/', u.fragment)
-if m:
-  return current_ref_for_gerrit_number(int(m.group(1)))
-  print >>sys.stderr, "invalid gerrit URL: ", url
-  usage()
-  sys.exit(1)
-
-def usage():
-  print >>sys.stderr, "usage: %s \n" % sys.argv[0]
-  print >>sys.stderr, "The provided URL should look something like:"
-  print >>sys.stderr, "http://gerrit.cloudera.org:8080/#/c/963/\n;
-
-
-def determine_ref():
-  if len(sys.argv) != 2:
-usage()
-sys.exit(1)
-
-  arg = sys.argv[1]
-  if arg.startswith("http"):
-return url_to_ref(arg)
-  else:
-print >>sys.stderr, "Unable to parse argument: %s\n" % arg
-sys.exit(1)
-
-
-def trigger_jenkins(ref):
-  logging.info("Will trigger Jenkins for ref %s" % ref)
-  url = "%s/job/%s/buildWithParameters" % (JENKINS_URL, JENKINS_JOB)
-  params = dict(GERRIT_BRANCH=ref)
-  req = urllib2.Request(url,
-data=urllib.urlencode(params),
-headers={"Accept": "application/json"})
-  urllib2.urlopen(req).read()
-  logging.info("Successfuly triggered jenkins job!")
-
-
-def main():
-  gerrit_ref = determine_ref()
-

[2/4] kudu git commit: [docs] Update Impala links to the Apache docs/wiki

2018-04-03 Thread granthenke 
[docs] Update Impala links to the Apache docs/wiki

Change-Id: I77f02823f421ab801ac3b6a03cc69a4075c63c23
Reviewed-on: http://gerrit.cloudera.org:8080/9904
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo 


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/dbc943f7
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/dbc943f7
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/dbc943f7

Branch: refs/heads/master
Commit: dbc943f7bf374f86ea5785dca98d7a3dfde39bbe
Parents: 1a24020
Author: Grant Henke 
Authored: Tue Apr 3 11:26:40 2018 -0500
Committer: Grant Henke 
Committed: Tue Apr 3 17:34:45 2018 +

--
 docs/kudu_impala_integration.adoc | 17 -
 1 file changed, 8 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/dbc943f7/docs/kudu_impala_integration.adoc
--
diff --git a/docs/kudu_impala_integration.adoc 
b/docs/kudu_impala_integration.adoc
index a43b7ff..2b2f3ec 100755
--- a/docs/kudu_impala_integration.adoc
+++ b/docs/kudu_impala_integration.adoc
@@ -61,8 +61,7 @@ Although not strictly necessary, it is recommended to 
configure Impala with the
 locations of the Kudu Master servers:
 
 * Set the 
`--kudu_master_hosts=[:port],[:port],[:port]`
-  flag in the Impala service configuration. If you are using Cloudera Manager,
-  please refer to the appropriate Cloudera Manager documentation to do so.
+  flag in the Impala service configuration.
 
 If this flag is not set within the Impala service, it will be necessary to 
manually
 provide this configuration each time you create a table by specifying the
@@ -73,7 +72,7 @@ The rest of this guide assumes that the configuration has 
been set.
 == Using the Impala Shell
 
 NOTE: This is only a small sub-set of Impala Shell functionality. For more 
details, see the
-link:http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/impala_impala_shell.html[Impala
 Shell] documentation.
+link:https://impala.apache.org/docs/build/html/topics/impala_impala_shell.html[Impala
 Shell] documentation.
 
 - Start Impala Shell using the `impala-shell` command. By default, 
`impala-shell`
 attempts to connect to the Impala daemon on `localhost` on port 21000. To 
connect
@@ -97,7 +96,7 @@ Impala, and dropping such a table does not drop the table 
from its source locati
 the mode used in the syntax provided by Kudu for mapping an existing table to 
Impala.
 
 See the
-http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/impala_tables.html[Impala
 documentation]
+link:https://impala.apache.org/docs/build/html/topics/impala_tables.html[Impala
 documentation]
 for more information about internal and external tables.
 
 === Querying an Existing Kudu Table In Impala
@@ -554,7 +553,7 @@ both Impala and Kudu, is usually to import the data using a 
`SELECT FROM` statem
 in Impala.
 +
 . If your data is not already in Impala, one strategy is to
-link:http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/impala_txtfile.html[import
 it from a text file],
+link:https://impala.apache.org/docs/build/html/topics/impala_txtfile.html[import
 it from a text file],
 such as a TSV or CSV file.
 +
 . <>, being mindful that the 
columns
@@ -622,7 +621,7 @@ DELETE FROM my_first_table WHERE id < 3;
 
 You can also delete using more complex syntax. A comma in the `FROM` 
sub-clause is
 one way that Impala specifies a join query. For more information about Impala 
joins,
-see 
http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/impala_joins.html.
+see https://impala.apache.org/docs/build/html/topics/impala_joins.html.
 [source,sql]
 
 DELETE c FROM my_second_table c, stock_symbols s WHERE c.name = s.symbol;
@@ -721,9 +720,9 @@ DROP TABLE my_first_table;
 The examples above have only explored a fraction of what you can do with 
Impala Shell.
 
 - Learn about the link:http://impala.io[Impala project].
-- Read the 
link:http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/impala.html[Impala
 documentation].
-- View the 
link:http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/impala_langref.html[Impala
 SQL reference].
-- Read about Impala internals or learn how to contribute to Impala on the 
link:https://github.com/cloudera/Impala/wiki[Impala Wiki].
+- Read the link:https://impala.apache.org/impala-docs.html[Impala 
documentation].
+- View the 
link:https://impala.apache.org/docs/build/html/topics/impala_langref.html[Impala
 SQL reference].
+- Read about Impala internals or learn how to contribute to Impala on the

[4/4] kudu git commit: [docs] Update or remove outdated links

2018-04-03 Thread granthenke 
[docs] Update or remove outdated links

Change-Id: Ic26561e7601a63b16217f1ac0c5dc20d0f56ca53
Reviewed-on: http://gerrit.cloudera.org:8080/9903
Reviewed-by: Adar Dembo 
Tested-by: Kudu Jenkins


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/5fc31db0
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/5fc31db0
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/5fc31db0

Branch: refs/heads/master
Commit: 5fc31db007c805fc83db10220f5d051a7f200f8b
Parents: 613fe93
Author: Grant Henke 
Authored: Tue Apr 3 11:20:38 2018 -0500
Committer: Grant Henke 
Committed: Tue Apr 3 18:26:49 2018 +

--
 docs/installation.adoc |  2 +-
 docs/prior_release_notes.adoc  | 12 ++--
 src/kudu/client/schema.h   |  2 +-
 src/kudu/scripts/benchmarks.sh |  2 --
 src/kudu/scripts/tpch.sh   |  4 +---
 5 files changed, 9 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/5fc31db0/docs/installation.adoc
--
diff --git a/docs/installation.adoc b/docs/installation.adoc
index 5131fa0..b782c56 100644
--- a/docs/installation.adoc
+++ b/docs/installation.adoc
@@ -544,7 +544,7 @@ $ sudo xcode-select --install
 .macOS Known Issues
 
 Kudu support for macOS is experimental, and should only be used for 
development.
-See link:https://issues.cloudera.org/browse/KUDU-1219[macOS Limitations & 
Known Issues]
+See link:https://issues.apache.org/browse/KUDU-1219[macOS Limitations & Known 
Issues]
 for more information.
 
 

http://git-wip-us.apache.org/repos/asf/kudu/blob/5fc31db0/docs/prior_release_notes.adoc
--
diff --git a/docs/prior_release_notes.adoc b/docs/prior_release_notes.adoc
index 6616c14..9ab4e69 100644
--- a/docs/prior_release_notes.adoc
+++ b/docs/prior_release_notes.adoc
@@ -1617,15 +1617,15 @@ issue, upgrade all Kudu servers before upgrading 
clients.
 [[rn_0.8.0_fixed_issues]]
 === Fixed Issues
 
-- link:https://issues.cloudera.org/browse/KUDU-1337[KUDU-1337] Tablets from 
tables
+- link:https://issues.apache.org/jira/browse/KUDU-1337[KUDU-1337] Tablets from 
tables
   that were deleted might be unnecessarily re-bootstrapped when the leader 
gets the
   notification to delete itself after the replicas do.
 
-- link:https://issues.cloudera.org/browse/KUDU-969[KUDU-969] If a tablet server
+- link:https://issues.apache.org/jira/browse/KUDU-969[KUDU-969] If a tablet 
server
   shuts down while compacting a rowset and receiving updates for it, it might 
immediately
   crash upon restart while bootstrapping that rowset's tablet.
 
-- link:https://issues.cloudera.org/browse/KUDU-1354[KUDU-1354] Due to a bug in 
Kudu's
+- link:https://issues.apache.org/jira/browse/KUDU-1354[KUDU-1354] Due to a bug 
in Kudu's
   MVCC implementation where row locks were released before the MVCC commit 
happened,
   flushed data would include out-of-order transactions, triggering a crash on 
the
   next compaction.
@@ -1686,7 +1686,7 @@ which prevented reading rows with `NULL` values.
 Kudu 0.7.0 is the first release done as part of the Apache Incubator and 
includes a number
 of changes, new features, improvements, and fixes.
 
-See also +++https://issues.cloudera.org/issues/?jql=project%20%3D%20Kudu%20AND%20status%20in%20
+See also +++https://issues.apache.org/jira/issues/?jql=project%20%3D%20KUDU%20AND%20status%20in%20
 (Resolved)%20AND%20fixVersion%20%3D%200.7.0%20ORDER%20BY%20key%20ASC">JIRAs 
resolved
 for Kudu 0.7.0+++ and +++https://github.com/apache/kudu/compare/branch-0.6.0...branch-0.7.0;>Git
 changes between 0.6.0 and 0.7.0+++.
@@ -1735,10 +1735,10 @@ for SLES 12.
 [[rn_0.7.0_fixed_issues]]
 === Fixed Issues
 
-- https://issues.cloudera.org/browse/KUDU-1288[KUDU-1288] fixes a severe file 
descriptor
+- https://issues.apache.org/jira/browse/KUDU-1288[KUDU-1288] fixes a severe 
file descriptor
 leak, which could previously only be resolved by restarting the tablet server.
 
-- https://issues.cloudera.org/browse/KUDU-1250[KUDU-1250] fixes a hang in the 
Java
+- https://issues.apache.org/jira/browse/KUDU-1250[KUDU-1250] fixes a hang in 
the Java
 client when processing an in-flight batch and the previous batch encountered 
an error.
 
 [[rn_0.7.0_changes]]

http://git-wip-us.apache.org/repos/asf/kudu/blob/5fc31db0/src/kudu/client/schema.h
--
diff --git a/src/kudu/client/schema.h b/src/kudu/client/schema.h
index 79abbfc..c0abbf9 100644
--- a/src/kudu/client/schema.h
+++ b/src/kudu/client/schema.h
@@ -245,7 +245,7 @@ class KUDU_EXPORT KuduColumnSchema {
   friend class KuduSchema;
   friend class KuduSchemaBuilder;

[3/3] kudu git commit: [examples] Improve the basic Java example

2018-04-03 Thread granthenke 
[examples] Improve the basic Java example

This improves the Java example in a bunch of ways:

1. Renames java-sample -> java-example.
2. Renames class to org.apache.kudu.examples.Example.
3. Adds licenses everywhere.
4. Rewrites the README, using adoc format.
5. Updates the pom, including syncing some plugin versions with
   java/pom and updating the version of the client the example uses.
6. Expands and improves the Example class:
   - Adds range partitioning with split rows.
   - Adds an alter table example.
   - Scans with a predicate and checks the results.

Change-Id: I79ab5b0b2c30e8fd7c799857a0e3754b7618a580
Reviewed-on: http://gerrit.cloudera.org:8080/9852
Reviewed-by: Adar Dembo 
Tested-by: Will Berkeley 


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/c3684b6e
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/c3684b6e
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/c3684b6e

Branch: refs/heads/master
Commit: c3684b6e10ece50dd382d178e392a25e3130a86e
Parents: cc8c5e3
Author: Will Berkeley 
Authored: Wed Mar 28 21:27:10 2018 -0700
Committer: Will Berkeley 
Committed: Tue Apr 3 04:30:29 2018 +

--
 examples/java/java-example/README.adoc  |  46 
 examples/java/java-example/pom.xml  |  78 +++
 .../java/org/apache/kudu/examples/Example.java  | 215 +++
 examples/java/java-sample/README|  15 --
 examples/java/java-sample/pom.xml   |  72 ---
 .../java/org/kududb/examples/sample/Sample.java |  77 ---
 6 files changed, 339 insertions(+), 164 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/c3684b6e/examples/java/java-example/README.adoc
--
diff --git a/examples/java/java-example/README.adoc 
b/examples/java/java-example/README.adoc
new file mode 100644
index 000..950c4e5
--- /dev/null
+++ b/examples/java/java-example/README.adoc
@@ -0,0 +1,46 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+= Kudu Java client example README
+:author: Kudu Team
+:homepage: https://kudu.apache.org/
+
+This is an example program that uses the synchronous Kudu Java client APIs to
+
+- Create a table
+- Insert some rows
+- Alter the table
+- Scan some rows
+- Delete the table
+
+To build and run, ensure maven is installed and from the java-example 
directory run:
+
+[source,bash]
+
+$ mvn package
+$ java -jar target/kudu-java-example-1.0-SNAPSHOT.jar
+
+
+By default, the example assumes the Kudu cluster has a single master running on
+localhost with the default port 7051. To specify a different set of masters for
+Kudu cluster, set the property `kuduMasters` to a CSV of the master addresses 
in
+the form `host:port`, as shown:
+
+[source,bash]
+
+$ java -DkuduMasters=master-0:7051,master-1:7051,master-2:7051 -jar 
target/kudu-java-example-1.0-SNAPSHOT.jar
+

http://git-wip-us.apache.org/repos/asf/kudu/blob/c3684b6e/examples/java/java-example/pom.xml
--
diff --git a/examples/java/java-example/pom.xml 
b/examples/java/java-example/pom.xml
new file mode 100644
index 000..67b0754
--- /dev/null
+++ b/examples/java/java-example/pom.xml
@@ -0,0 +1,78 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd;>
+  4.0.0
+  org.apache.kudu
+  kudu-java-example
+  jar
+  1.0-SNAPSHOT
+  Kudu Java Client Examples
+
+  
+
+  
+org.apache.maven.plugins
+maven-compiler-plugin
+3.6.2
+
+  1.7
+  1.7
+
+  
+  
+org.apache.maven.plugins
+maven-shade-plugin
+3.0.0
+
+  
+package
+
+  shade
+
+
+  
+

[2/3] kudu git commit: [examples] Improve loadgen example

2018-04-03 Thread granthenke 
[examples] Improve loadgen example

This is a grab bag of improvements to the loadgen example:
- Updated pom
- README rewritten, in adoc format.
- Small improvements to the Java code.

Change-Id: Ibc0e655f56adcedec2c3dd8a8b3c900e3b2e7803
Reviewed-on: http://gerrit.cloudera.org:8080/9853
Reviewed-by: Adar Dembo 
Tested-by: Will Berkeley 


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/cc8c5e3e
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/cc8c5e3e
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/cc8c5e3e

Branch: refs/heads/master
Commit: cc8c5e3e8783c81f10233d531a361723fdceb3f9
Parents: 0804136
Author: Will Berkeley 
Authored: Wed Mar 28 21:56:08 2018 -0700
Committer: Will Berkeley 
Committed: Tue Apr 3 04:29:44 2018 +

--
 examples/java/insert-loadgen/README |  17 ---
 examples/java/insert-loadgen/README.adoc|  45 +++
 examples/java/insert-loadgen/pom.xml|  49 ---
 .../org/apache/kudu/examples/InsertLoadgen.java | 132 +++
 .../kududb/examples/loadgen/InsertLoadgen.java  | 110 
 5 files changed, 205 insertions(+), 148 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/kudu/blob/cc8c5e3e/examples/java/insert-loadgen/README
--
diff --git a/examples/java/insert-loadgen/README 
b/examples/java/insert-loadgen/README
deleted file mode 100644
index 7c0628e..000
--- a/examples/java/insert-loadgen/README
+++ /dev/null
@@ -1,17 +0,0 @@
-Random insert load generator. This will insert as fast as it can using
-AUTO_BACKGROUND_FLUSH with the specified number of threads. All fields are
-randomized, and insert failures (including errors from collisions) are ignored.
-
-To build and run, do the following:
-
-$ mvn package
-$ java -jar target/kudu-insert-loadgen-0.1-SNAPSHOT.jar kudu_master_host 
kudu_table_name num_threads
-
-For example, if you are running the Quickstart VM with the host name
-"quickstart.cloudera", then you can use:
-
-$ java -jar target/kudu-insert-loadgen-0.1-SNAPSHOT.jar quickstart.cloudera 
test_table 16
-
-Note: This program will not create the "test_table" table. You must do that
-via other means, such as through impala-shell or using the create-demo-table
-program included in the Kudu source tree.

http://git-wip-us.apache.org/repos/asf/kudu/blob/cc8c5e3e/examples/java/insert-loadgen/README.adoc
--
diff --git a/examples/java/insert-loadgen/README.adoc 
b/examples/java/insert-loadgen/README.adoc
new file mode 100644
index 000..e2e697f
--- /dev/null
+++ b/examples/java/insert-loadgen/README.adoc
@@ -0,0 +1,45 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+= Kudu Java insert loadgen README
+:author: Kudu Team
+:homepage: https://kudu.apache.org/
+
+== Summary
+Random insert load generator. This will insert as fast as it can into a
+pre-existing table using AUTO_BACKGROUND_FLUSH mode. All fields are randomized.
+The load generator will continue inserting until it is stopped or it encounters
+an error. This load generator is single-threaded.
+
+To build and run, do the following:
+
+[source,bash]
+
+$ mvn verify
+$ java -jar target/kudu-insert-loadgen-1.0-SNAPSHOT.jar master_addresses 
table_name
+
+
+where `master_addresses` is a CSV of the master addresses for your Kudu cluster
+and `table_name` is the name of a pre-existing table. For example,
+
+[source,bash]
+
+$ java -jar target/kudu-insert-loadgen-1.0-SNAPSHOT.jar localhost:7051 
test_table
+
+
+This program will not create the "test_table" table. You must do that
+via other means, such as through impala-shell.

http://git-wip-us.apache.org/repos/asf/kudu/blob/cc8c5e3e/examples/java/insert-loadgen/pom.xml
--
diff --git a/examples/java/insert-loadgen/pom.xml