[kudu-CR] WIP KUDU-2780: create thread for auto-rebalancing

2019-09-13 Thread Andrew Wong (Code Review)
Andrew Wong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14177 )

Change subject: WIP KUDU-2780: create thread for auto-rebalancing
..


Patch Set 3:

(35 comments)

Didn't look at auto_rebalancer-test yet

http://gerrit.cloudera.org:8080/#/c/14177/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14177/3//COMMIT_MSG@8
PS3, Line 8: 
   : Created auto-rebalancer thread in background tasks of catalog 
manager.
   : Set up framework for auto-rebalancing loop.
   : Loop retrieves information on tservers, tables, and tablets for 
rebalancing.
   : The number of replica moves per loop iteration is controlled by a 
flag.
   : If there are placement policy violations, the current loop 
iteration will only
   : perform replica moves to reinstate the policy.
   : Otherwise, the auto-rebalancer will perform moves to do 
inter-location(cross-location),
   : then intra-location(by table then by tserver) rebalancing.
   : If the cluster is balanced, the current rebalancing cycle 
completes, and the
   : thread will sleep for an interval, controlled by another flag.
nit: I found this a little difficult to read. Could you split it up into 
paragraphs?


http://gerrit.cloudera.org:8080/#/c/14177/3/src/kudu/master/auto_rebalancer.h
File src/kudu/master/auto_rebalancer.h:

http://gerrit.cloudera.org:8080/#/c/14177/3/src/kudu/master/auto_rebalancer.h@93
PS3, Line 93:   // The autorebalancing thread.
:   scoped_refptr thread_;
We don't expect there to be multiple variables containing `thread_` do we? 
Could this just be a unique_ptr?


http://gerrit.cloudera.org:8080/#/c/14177/3/src/kudu/master/auto_rebalancer.h@104
PS3, Line 104:   // The internal Rebalancer object.
 :   std::shared_ptr rebalancer_;
Same here; are there expected to be multiple threads holding onto `rebalancer_`?


http://gerrit.cloudera.org:8080/#/c/14177/3/src/kudu/master/auto_rebalancer.h@106
PS3, Line 106:
 :   // Functions for the Rebalancer object:
 :
 :   // Collects information about the cluster.
 :   Status BuildClusterRawInfo(const boost::optional& 
location,
 :  rebalance::ClusterRawInfo* 
raw_info) const;
 :
 :   Status GetMovesUsingRebalancingAlgo(
 :   const rebalance::ClusterRawInfo& raw_info,
 :   std::vector* 
replica_moves,
 :   rebalance::RebalancingAlgo* algo,
 :   bool cross_location = false) const;
 :
 :   Status GetMoves(const rebalance::ClusterRawInfo& raw_info,
 :   const rebalance::ClusterInfo& ci,
 :   
std::vector* replica_moves,
 :   const rebalance::TabletsPlacementInfo& 
placement_info,
 :   bool* policy_fixing) const;
 :
 :   Status GetTabletLeader(const std::string& tablet_id, 
std::string* leader_uuid, HostPort* leader_hp) const;
 :
 :   Status ExecuteMoves(const 
std::vector& replica_moves, const bool& 
policy_fixing);
 :
 :   // Returns whether or not the cluster needs rebalancing.
 :   bool IsClusterBalanced(const rebalance::TabletsPlacementInfo& 
placement_info,
 :  const rebalance::ClusterInfo& ci) const;
 :
 :   // Helper function to retrieve TSManager.
 :   TSManager* GetTSManager() const;
nit: Could you move these above the private member variables?


http://gerrit.cloudera.org:8080/#/c/14177/3/src/kudu/master/auto_rebalancer.cc
File src/kudu/master/auto_rebalancer.cc:

PS3:
nit: got a bunch of whitespace sprinkled throughout.


http://gerrit.cloudera.org:8080/#/c/14177/3/src/kudu/master/auto_rebalancer.cc@104
PS3, Line 104: DEFINE_string(auto_rebalancing_ignored_tservers, "",
 :   "UUIDs of tablet servers to ignore while 
auto-rebalancing the cluster "
 :   "(comma-separated list). If specified, allow the 
auto-rebalancing "
 :   "when some tablet servers in 'ignored_tservers' 
are unhealthy. "
 :   "If not specified, allow the rebalancing only when 
all tablet "
 :   "servers are healthy.");
 :
 : DEFINE_string(auto_rebalancing_tables, "",
 :   "Tables to rebalance (comma-separated list of 
table names)"
 :   "If not specified, includes all tables.");
 :
It seems like these are initialized once and cached, so it seems a bit odd to 
have these as a flag, seeing as we can't update them ever. Can you think of a 
use case in which these is actually 

[kudu-CR] KUDU-2914: Rebalance tool support moving replicas from some specific tablet servers

2019-09-13 Thread Yifan Zhang (Code Review)
Yifan Zhang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14154 )

Change subject: KUDU-2914: Rebalance tool support moving replicas from some 
specific tablet servers
..


Patch Set 3:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/14154/2/src/kudu/rebalance/rebalancer.h
File src/kudu/rebalance/rebalancer.h:

http://gerrit.cloudera.org:8080/#/c/14154/2/src/kudu/rebalance/rebalancer.h@205
PS2, Line 205: a specific tabl
> nit: document whether it's possible to specify 'nullptr' as an argument whe
Done


http://gerrit.cloudera.org:8080/#/c/14154/2/src/kudu/rebalance/rebalancer.cc
File src/kudu/rebalance/rebalancer.cc:

http://gerrit.cloudera.org:8080/#/c/14154/2/src/kudu/rebalance/rebalancer.cc@157
PS2, Line 157: // No healthy tablet of s
> This seems to be the only place where the output parameter is assigned.  Do
Done


http://gerrit.cloudera.org:8080/#/c/14154/2/src/kudu/tools/rebalancer_tool.cc
File src/kudu/tools/rebalancer_tool.cc:

http://gerrit.cloudera.org:8080/#/c/14154/2/src/kudu/tools/rebalancer_tool.cc@951
PS2, Line 951: for (const auto& s : raw_info.tablet_summaries) {
 :   int num_voters = 0;
 :   for (const auto& rs : s.replicas) {
 : if (rs.is_voter) {
 :   ++num_voters;
 : }
 :   }
 :   const auto rf = FindOrDie(replication_factors_by_table, 
s.table_id);
 :   EmplaceOrDie(_info_by_tablet_id,
 :s.id, TabletExtraInfo{rf, num_voters});
 : }
 :   }
> Moving the replicas from the healthy ignored tservers and moving the replic
Done



--
To view, visit http://gerrit.cloudera.org:8080/14154
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I86cfb740030946c13db1a9ca63d241f4907d6c89
Gerrit-Change-Number: 14154
Gerrit-PatchSet: 3
Gerrit-Owner: Yifan Zhang 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Comment-Date: Sat, 14 Sep 2019 00:55:08 +
Gerrit-HasComments: Yes


[kudu-CR] KUDU-2914: Rebalance tool support moving replicas from some specific tablet servers

2019-09-13 Thread Yifan Zhang (Code Review)
Hello Alexey Serbin, Kudu Jenkins, Andrew Wong, Adar Dembo,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14154

to look at the new patch set (#3).

Change subject: KUDU-2914: Rebalance tool support moving replicas from some 
specific tablet servers
..

KUDU-2914: Rebalance tool support moving replicas from some specific tablet 
servers

Aims to support moving replicas from specific tablet servers,
this patch re-uses the '--ignored_tservers' flag and adds a
'--move_replicas_from_ignored_tservers' flag to the
`kudu rebalance cluster` CLI tool.

Once the flag '--ignored_tservers' is specified, and the flag
'--move_replicas_from_ignored_tservers' is specified false,
the given tablet servers are not considered as a part of the
cluster, their health state and replicas on them are ignored
by the rebalancer tool.

If '--move_replicas_from_ignored_tservers' is specified true,
unhealthy tservers in the given tservers will also be ignored by
the rebalancer tool. But for healthy ignored tservers, replicas
on them would be moved to other tservers before starting the
rebalancing process, when running the rebalancing, these tservers
are not considered as a part of the cluster.

Change-Id: I86cfb740030946c13db1a9ca63d241f4907d6c89
---
M src/kudu/rebalance/rebalance-test.cc
M src/kudu/rebalance/rebalance_algo-test.cc
M src/kudu/rebalance/rebalance_algo.cc
M src/kudu/rebalance/rebalance_algo.h
M src/kudu/rebalance/rebalancer.cc
M src/kudu/rebalance/rebalancer.h
M src/kudu/tools/rebalancer_tool-test.cc
M src/kudu/tools/rebalancer_tool.cc
M src/kudu/tools/rebalancer_tool.h
M src/kudu/tools/tool_action_cluster.cc
10 files changed, 827 insertions(+), 67 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/54/14154/3
--
To view, visit http://gerrit.cloudera.org:8080/14154
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I86cfb740030946c13db1a9ca63d241f4907d6c89
Gerrit-Change-Number: 14154
Gerrit-PatchSet: 3
Gerrit-Owner: Yifan Zhang 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang 


[kudu-CR] [mini cluster] introduce 'builtin' clock source

2019-09-13 Thread Hao Hao (Code Review)
Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14227 )

Change subject: [mini_cluster] introduce 'builtin' clock source
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14227/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14227/2//COMMIT_MSG@13
PS2, Line 13: server-only
Can you explain a bit on this mode and why do we use it?



--
To view, visit http://gerrit.cloudera.org:8080/14227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c334ae6fa1fb12b033de7f8e8584b8dd3aa2d32
Gerrit-Change-Number: 14227
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Sat, 14 Sep 2019 00:40:55 +
Gerrit-HasComments: Yes


[kudu-CR] [mini cluster] introduce 'builtin' clock source

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14227 )

Change subject: [mini_cluster] introduce 'builtin' clock source
..


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/14227/2/src/kudu/mini-cluster/external_mini_cluster.h
File src/kudu/mini-cluster/external_mini_cluster.h:

http://gerrit.cloudera.org:8080/#/c/14227/2/src/kudu/mini-cluster/external_mini_cluster.h@190
PS2, Line 190:   // Number of NTP servers to start as part of the cluster, 
gated by the
 :   // 'use_builtin_ntp' property. The servers are used as 
reference NTP servers
 :   // for the built-in NTP client: it uses them to synchronize 
its internal
 :   // clock. This setting is effective (i.e. the specified number 
of NTP servers
 :   // is started) only if the 'use_builtin_ntp' property is set 
to 'true'.
 :   //
 :   // Default: 1
 :   int num_ntp_servers;
 :
 :   // Whether Kudu masters and tablet servers are using the 
built-in NTP client.
 :   //
 :   // Default: false
 :   int use_builtin_ntp;
How about you drop use_builtin_ntp and use num_ntp_servers=0 to mean "no 
built-in NTP"?

Or is it valuable to launch a bunch of NTP servers _without_ the built-in 
client?


http://gerrit.cloudera.org:8080/#/c/14227/2/src/kudu/mini-cluster/external_mini_cluster.cc
File src/kudu/mini-cluster/external_mini_cluster.cc:

http://gerrit.cloudera.org:8080/#/c/14227/2/src/kudu/mini-cluster/external_mini_cluster.cc@245
PS2, Line 245: // Collect and keep alive the set of sockets bound with 
SO_REUSEPORT option
As I mentioned in the other patch, I think it'd be better to patch chrony to 
support ephemeral port binding and then use the lsof approach to figure out 
where it landed. This is similar to how we manage the KDC and will be safer on 
non-UNIQUE_LOOPBACK environments.



--
To view, visit http://gerrit.cloudera.org:8080/14227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c334ae6fa1fb12b033de7f8e8584b8dd3aa2d32
Gerrit-Change-Number: 14227
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Sat, 14 Sep 2019 00:30:05 +
Gerrit-HasComments: Yes


[kudu-CR] [thirdparty] add SO REUSEPORT for chronyd NTP socket

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14228 )

Change subject: [thirdparty] add SO_REUSEPORT for chronyd NTP socket
..


Patch Set 1:

Could we get by with a patch to enable ephemeral port binding instead? That's 
less invasive than this, which seems unlikely to be accepted upstream.


--
To view, visit http://gerrit.cloudera.org:8080/14228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iee26fcf93976dd7affe77254751016bcbf398620
Gerrit-Change-Number: 14228
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Sat, 14 Sep 2019 00:25:14 +
Gerrit-HasComments: No


[kudu-CR] [thirdparty] add SO REUSEPORT for chronyd NTP socket

2019-09-13 Thread Hao Hao (Code Review)
Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14228 )

Change subject: [thirdparty] add SO_REUSEPORT for chronyd NTP socket
..


Patch Set 1: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/14228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iee26fcf93976dd7affe77254751016bcbf398620
Gerrit-Change-Number: 14228
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Sat, 14 Sep 2019 00:24:07 +
Gerrit-HasComments: No


[kudu-CR] Example shell scripts to start and stop Kudu cluster

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/14192 )

Change subject: Example shell scripts to start and stop Kudu cluster
..

Example shell scripts to start and stop Kudu cluster

You just have built Kudu executables and want quickly to run them before 
delving into documentation.

Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
Reviewed-on: http://gerrit.cloudera.org:8080/14192
Reviewed-by: Adar Dembo 
Reviewed-by: Alexey Serbin 
Tested-by: Adar Dembo 
---
A docs/quickstartdev.adoc
A src/kudu/scripts/start_kudu.sh
A src/kudu/scripts/stop_kudu.sh
3 files changed, 231 insertions(+), 0 deletions(-)

Approvals:
  Adar Dembo: Looks good to me, approved; Verified
  Alexey Serbin: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/14192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
Gerrit-Change-Number: 14192
Gerrit-PatchSet: 9
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 


[kudu-CR] Example shell scripts to start and stop Kudu cluster

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14192 )

Change subject: Example shell scripts to start and stop Kudu cluster
..


Patch Set 8: Verified+1

Overriding Jenkins, the failure had nothing to do with your change.


--
To view, visit http://gerrit.cloudera.org:8080/14192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
Gerrit-Change-Number: 14192
Gerrit-PatchSet: 8
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Sat, 14 Sep 2019 00:22:53 +
Gerrit-HasComments: No


[kudu-CR] Example shell scripts to start and stop Kudu cluster

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has removed a vote on this change.

Change subject: Example shell scripts to start and stop Kudu cluster
..


Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/14192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
Gerrit-Change-Number: 14192
Gerrit-PatchSet: 8
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 


[kudu-CR] [clock] add a built-in NTP client implementation

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has removed Dan Burkert from this change.  ( 
http://gerrit.cloudera.org:8080/7477 )

Change subject: [clock] add a built-in NTP client implementation
..


Removed reviewer Dan Burkert.
--
To view, visit http://gerrit.cloudera.org:8080/7477
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Ieb9eee9f0334236f39617492bd6f01304d1a0255
Gerrit-Change-Number: 7477
Gerrit-PatchSet: 16
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon 


[kudu-CR] [clock] add a built-in NTP client implementation

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has removed David Ribeiro Alves from this change.  ( 
http://gerrit.cloudera.org:8080/7477 )

Change subject: [clock] add a built-in NTP client implementation
..


Removed reviewer David Ribeiro Alves.
--
To view, visit http://gerrit.cloudera.org:8080/7477
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Ieb9eee9f0334236f39617492bd6f01304d1a0255
Gerrit-Change-Number: 7477
Gerrit-PatchSet: 16
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon 


[kudu-CR] [clock] add a built-in NTP client implementation

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/7477 )

Change subject: [clock] add a built-in NTP client implementation
..


Patch Set 16:

(50 comments)

I didn't read any RFCs so there's a fair amount of missing coverage there 
(would be great if Todd could take a look since he originally implemented this).

http://gerrit.cloudera.org:8080/#/c/7477/13/src/kudu/clock/builtin_ntp.h
File src/kudu/clock/builtin_ntp.h:

http://gerrit.cloudera.org:8080/#/c/7477/13/src/kudu/clock/builtin_ntp.h@45
PS13, Line 45: class BuiltInNtp : public TimeService {
Could use some more documentation, at least for the class (i.e. what are we 
trying to achieve with this?) and the private members.


http://gerrit.cloudera.org:8080/#/c/7477/13/src/kudu/clock/builtin_ntp.h@73
PS13, Line 73:   bool TryReceivePacket();
Should doc (amongst other things) what the return value means.


http://gerrit.cloudera.org:8080/#/c/7477/13/src/kudu/clock/builtin_ntp.h@92
PS13, Line 92:   int64_t last_compute_mono_ = 0;
 :   int64_t last_compute_wall_ = 0;
 :   int64_t last_compute_error_ = 0;
 :   bool is_synchronized_ = false;
Might want to encapsulate these in a struct so it's easy to make a local copy 
with the lock held.


http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc
File src/kudu/clock/builtin_ntp.cc:

http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc@78
PS9, Line 78: TAG_FLAG(builtin_ntp_request_timeout_ms, experimental);
> maybe this should be a flag?
Done


http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc@286
PS9, Line 286:
 :   // The monotonic timestamp wh
> I seem to recall I looked into how to find this info and determined that it
Do we need to convert this into a TODO, or a note?


http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc@385
PS9, Line 385:   }
> i seem to recall we now have some more general "wait for NTP at startup' th
Yeah, it looks like Todd's reimplementation of this logic remained and is now 
duplicated here and in system_ntp.cc. Could we consolidate them, perhaps in 
HybridClock::Init?


http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc@474
PS9, Line 474:
> maybe we should include the size of the packet, or the packet itself?
Alexey, what do you think?


http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc@509
PS9, Line 509:   {
 : MutexLock l(state_lock_);
> I guess we should read the RFC carefully and figure this out
Alexey, did you explore this comment?


http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc@523
PS9, Line 523: // will get EPIPE.
 : socket_.Shutdown(true, true);
> improve this error
Done


http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc@536
PS9, Line 536:   // unclear what clock those come from
> probably worth doing this
Alexey, what do you think of this?


http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc@583
PS9, Line 583:   // 3.  The Originate Timestamp in the server reply should 
match the
> should resolve this too
Alexey, did you work this out?


http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc@666
PS9, Line 666:   //Originate Timestamp T1   time request sent by client
> seems like we could probably factor this out and make it testable
+1, Alexey could you do this?


http://gerrit.cloudera.org:8080/#/c/7477/9/src/kudu/clock/builtin_ntp.cc@754
PS9, Line 754:
> yea todo
Done


http://gerrit.cloudera.org:8080/#/c/7477/13/src/kudu/clock/builtin_ntp.cc
File src/kudu/clock/builtin_ntp.cc:

http://gerrit.cloudera.org:8080/#/c/7477/13/src/kudu/clock/builtin_ntp.cc@58
PS13, Line 58:   "0.ubuntu.pool.ntp.org,"
 :   "1.ubuntu.pool.ntp.org,"
 :   "2.ubuntu.pool.ntp.org,"
 :   "3.ubuntu.pool.ntp.org",
Shouldn't we default to [0-3].pool.ntp.org? As per 
https://www.ntppool.org/en/use.html.


http://gerrit.cloudera.org:8080/#/c/7477/13/src/kudu/clock/builtin_ntp.cc@66
PS13, Line 66:
 : // As of 2019, recent version of ntpd allows setting minpoll as 
low as 3
 : // (2^3 = 8 seconds), and chronyd supports minpoll as low as to 
-6
 : // (2^-6 = 1/64 second), so 16 seconds looks like a reasonble 
default.
Do we also want to express this in terms of powers of 2? Is that an interface 
users are more comfortable with?


http://gerrit.cloudera.org:8080/#/c/7477/13/src/kudu/clock/builtin_ntp.cc@87
PS13, Line 87: DEFINE_int32(ntp_initial_sync_wait_secs, 60,
Why did this move from one time service impl to another?


http://gerrit.cloudera.org:8080/#/c/7477/13/src/kudu/clock/builtin_ntp.cc@167
PS13, Line 167:   // NOTE: this is used as a nonce, not the actual time.
Does this 

[kudu-CR] [clock] add a built-in NTP client implementation

2019-09-13 Thread Alexey Serbin (Code Review)
Alexey Serbin has uploaded a new patch set (#16) to the change originally 
created by Todd Lipcon. ( http://gerrit.cloudera.org:8080/7477 )

Change subject: [clock] add a built-in NTP client implementation
..

[clock] add a built-in NTP client implementation

This adds a stripped-down implementation of built-in NTP client
without any reliance on the kernel NTP discipline.  This initial
implementation doesn't have all the necessary sanity checks for true
time that RFC-compliant client would have and doesn't implement
everything from RFC5905.  However, this implementation is good enough
for running with well behaved and properly configured reference NTP
servers such as servers from pool.ntp.org, other public-domain NTP
services, and properly configured NTP servers in local network.

This is a first step on the road to eventually have robust and
RFC-compliant Kudu's own NTP client that is able to detect and
reject misbehaving and rogue reference NTP servers.

For Kudu, this should hopefully make it easier for users to configure
NTP even if they don't have root, and also can maintain better clock
error than the system implementation, since we can prioritize low error
bounds rather than low jitter.  This patch doesn't have the knobs
to control error vs jitter yet, though.

This patch also contains tests to verify the newly introduced
functionality in scenarios when the built-in NTP client is pointed
to various combinations of properly configured NTP servers or
unintentionally misconfigured/unavailable ones.

Change-Id: Ieb9eee9f0334236f39617492bd6f01304d1a0255
---
M src/kudu/clock/CMakeLists.txt
A src/kudu/clock/builtin_ntp.cc
A src/kudu/clock/builtin_ntp.h
M src/kudu/clock/hybrid_clock.cc
A src/kudu/clock/ntp-test.cc
M src/kudu/clock/system_ntp.cc
M src/kudu/clock/test/mini_chronyd-test.cc
7 files changed, 1,595 insertions(+), 49 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/77/7477/16
--
To view, visit http://gerrit.cloudera.org:8080/7477
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieb9eee9f0334236f39617492bd6f01304d1a0255
Gerrit-Change-Number: 7477
Gerrit-PatchSet: 16
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Brock Noland 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon 


[kudu-CR] [clock] add a built-in NTP client implementation

2019-09-13 Thread Alexey Serbin (Code Review)
Alexey Serbin has uploaded a new patch set (#15) to the change originally 
created by Todd Lipcon. ( http://gerrit.cloudera.org:8080/7477 )

Change subject: [clock] add a built-in NTP client implementation
..

[clock] add a built-in NTP client implementation

This adds a stripped-down implementation of built-in NTP client
without any reliance on the kernel NTP discipline.  This initial
implementation doesn't have all the necessary sanity checks for true
time that RFC-compliant client would have and doesn't implement
everything from RFC5905.  However, this implementation is good enough
for running with well behaved and properly configured reference NTP
servers such as servers from pool.ntp.org, other public-domain NTP
services, and properly configured NTP servers in local network.

This is a first step on the road to eventually have robust and
RFC-compliant Kudu's own NTP client that is able to detect and
reject misbehaving and rogue reference NTP servers.

For Kudu, this should hopefully make it easier for users to configure
NTP even if they don't have root, and also can maintain better clock
error than the system implementation, since we can prioritize low error
bounds rather than low jitter.  This patch doesn't have the knobs
to control error vs jitter yet, though.

This patch also contains tests to verify the newly introduced
functionality in scenarios when the built-in NTP client is pointed
to various combinations of properly configured NTP servers or
unintentionally misconfigured/unavailable ones.

Change-Id: Ieb9eee9f0334236f39617492bd6f01304d1a0255
---
M src/kudu/clock/CMakeLists.txt
A src/kudu/clock/builtin_ntp.cc
A src/kudu/clock/builtin_ntp.h
M src/kudu/clock/hybrid_clock.cc
A src/kudu/clock/ntp-test.cc
M src/kudu/clock/system_ntp.cc
M src/kudu/clock/test/mini_chronyd-test.cc
7 files changed, 1,595 insertions(+), 49 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/77/7477/15
--
To view, visit http://gerrit.cloudera.org:8080/7477
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ieb9eee9f0334236f39617492bd6f01304d1a0255
Gerrit-Change-Number: 7477
Gerrit-PatchSet: 15
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Brock Noland 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon 


[kudu-CR] Example shell scripts to start and stop Kudu cluster

2019-09-13 Thread Alexey Serbin (Code Review)
Alexey Serbin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14192 )

Change subject: Example shell scripts to start and stop Kudu cluster
..


Patch Set 8: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/14192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
Gerrit-Change-Number: 14192
Gerrit-PatchSet: 8
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Fri, 13 Sep 2019 23:13:36 +
Gerrit-HasComments: No


[kudu-CR] Example shell scripts to start and stop Kudu cluster

2019-09-13 Thread Volodymyr Verovkin (Code Review)
Volodymyr Verovkin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14192 )

Change subject: Example shell scripts to start and stop Kudu cluster
..


Patch Set 8:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/14192/7/src/kudu/scripts/start_kudu.sh
File src/kudu/scripts/start_kudu.sh:

PS7:
> You have some trailing whitespace on some lines of this file.
Still messing with Eclipse editor settings...
Done


http://gerrit.cloudera.org:8080/#/c/14192/7/src/kudu/scripts/start_kudu.sh@35
PS7, Line 35: b
> Maybe -b for this? -p connotes "port" in most database scripts.
Done



--
To view, visit http://gerrit.cloudera.org:8080/14192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
Gerrit-Change-Number: 14192
Gerrit-PatchSet: 8
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Fri, 13 Sep 2019 23:08:04 +
Gerrit-HasComments: Yes


[kudu-CR] Example shell scripts to start and stop Kudu cluster

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14192 )

Change subject: Example shell scripts to start and stop Kudu cluster
..


Patch Set 8: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/14192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
Gerrit-Change-Number: 14192
Gerrit-PatchSet: 8
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Fri, 13 Sep 2019 23:06:05 +
Gerrit-HasComments: No


[kudu-CR] Example shell scripts to start and stop Kudu cluster

2019-09-13 Thread Volodymyr Verovkin (Code Review)
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, Grant Henke, Greg Solovyev, 

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14192

to look at the new patch set (#8).

Change subject: Example shell scripts to start and stop Kudu cluster
..

Example shell scripts to start and stop Kudu cluster

You just have built Kudu executables and want quickly to run them before 
delving into documentation.

Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
---
A docs/quickstartdev.adoc
A src/kudu/scripts/start_kudu.sh
A src/kudu/scripts/stop_kudu.sh
3 files changed, 231 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/92/14192/8
--
To view, visit http://gerrit.cloudera.org:8080/14192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
Gerrit-Change-Number: 14192
Gerrit-PatchSet: 8
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 


[kudu-CR] KUDU-2920 Block cache capacity validator couldn't run on an NVM block cache

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/14212 )

Change subject: KUDU-2920 Block cache capacity validator couldn't run on an NVM 
block cache
..

KUDU-2920 Block cache capacity validator couldn't run on an NVM block cache

Validator should fail for options:
--block_cache_type=DRAM
--block_cache_capacity_mb=100
and succeed for options:
--block_cache_type=NVM
--block_cache_capacity_mb=100
--nvm_cache_path=/path/to/your/tmp/dir
where 100 means "size of cache bigger than RAM" (1000GB)

Change-Id: I9f50e9dd901280b5c32576e43165292299922995
Reviewed-on: http://gerrit.cloudera.org:8080/14212
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo 
---
M src/kudu/cfile/block_cache.cc
M src/kudu/cfile/block_cache.h
M src/kudu/cfile/cfile-test.cc
3 files changed, 27 insertions(+), 3 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Adar Dembo: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/14212
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I9f50e9dd901280b5c32576e43165292299922995
Gerrit-Change-Number: 14212
Gerrit-PatchSet: 5
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 


[kudu-CR] KUDU-2920 Block cache capacity validator couldn't run on an NVM block cache

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14212 )

Change subject: KUDU-2920 Block cache capacity validator couldn't run on an NVM 
block cache
..


Patch Set 4: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/14212
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9f50e9dd901280b5c32576e43165292299922995
Gerrit-Change-Number: 14212
Gerrit-PatchSet: 4
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Fri, 13 Sep 2019 23:04:20 +
Gerrit-HasComments: No


[kudu-CR] KUDU-1561 Implemented operator->() in KuduScanBatch::const iterator

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/14219 )

Change subject: KUDU-1561 Implemented operator->() in 
KuduScanBatch::const_iterator
..

KUDU-1561 Implemented operator->() in KuduScanBatch::const_iterator

operator->() requires returning pointer or object which overloads access
via pointer operator (operator->() again).
Since KuduScanBatch::const_iterator does to keep current
KuduScanBatch::RowPtr, we cannot return pointer to it in operator ->.
Instead, we return KuduScanBatch::RowPtr which implements pointer trait
(operator->).

Change-Id: Ib5ddd99073d6a93337c184bee8b930cabeeda145
Reviewed-on: http://gerrit.cloudera.org:8080/14219
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo 
---
M src/kudu/client/client-test.cc
M src/kudu/client/scan_batch.h
2 files changed, 28 insertions(+), 0 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Adar Dembo: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/14219
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib5ddd99073d6a93337c184bee8b930cabeeda145
Gerrit-Change-Number: 14219
Gerrit-PatchSet: 4
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Volodymyr Verovkin 


[kudu-CR] KUDU-1561 Implemented operator->() in KuduScanBatch::const iterator

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14219 )

Change subject: KUDU-1561 Implemented operator->() in 
KuduScanBatch::const_iterator
..


Patch Set 3: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/14219
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5ddd99073d6a93337c184bee8b930cabeeda145
Gerrit-Change-Number: 14219
Gerrit-PatchSet: 3
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Fri, 13 Sep 2019 23:03:01 +
Gerrit-HasComments: No


[kudu-CR] [docs] Kudu source code indexing in Eclipse

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/14193 )

Change subject: [docs] Kudu source code indexing in Eclipse
..

[docs] Kudu source code indexing in Eclipse

Added alternative recommendation to deal with Kudu source code indexing in 
Eclipse.

Change-Id: Ia214e536ff3a7ddafb3ab969b50ed5f97f4c725a
Reviewed-on: http://gerrit.cloudera.org:8080/14193
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo 
---
M README.adoc
1 file changed, 13 insertions(+), 0 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Adar Dembo: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/14193
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia214e536ff3a7ddafb3ab969b50ed5f97f4c725a
Gerrit-Change-Number: 14193
Gerrit-PatchSet: 7
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 


[kudu-CR] [docs] Kudu source code indexing in Eclipse

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14193 )

Change subject: [docs] Kudu source code indexing in Eclipse
..


Patch Set 6: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/14193
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia214e536ff3a7ddafb3ab969b50ed5f97f4c725a
Gerrit-Change-Number: 14193
Gerrit-PatchSet: 6
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Fri, 13 Sep 2019 22:56:49 +
Gerrit-HasComments: No


[kudu-CR] Example shell scripts to start and stop Kudu cluster

2019-09-13 Thread Adar Dembo (Code Review)
Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14192 )

Change subject: Example shell scripts to start and stop Kudu cluster
..


Patch Set 7:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/14192/7/src/kudu/scripts/start_kudu.sh
File src/kudu/scripts/start_kudu.sh:

PS7:
You have some trailing whitespace on some lines of this file.


http://gerrit.cloudera.org:8080/#/c/14192/7/src/kudu/scripts/start_kudu.sh@35
PS7, Line 35: p
Maybe -b for this? -p connotes "port" in most database scripts.



--
To view, visit http://gerrit.cloudera.org:8080/14192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
Gerrit-Change-Number: 14192
Gerrit-PatchSet: 7
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Fri, 13 Sep 2019 22:55:48 +
Gerrit-HasComments: Yes


[kudu-CR] Example shell scripts to start and stop Kudu cluster

2019-09-13 Thread Volodymyr Verovkin (Code Review)
Volodymyr Verovkin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14192 )

Change subject: Example shell scripts to start and stop Kudu cluster
..


Patch Set 7:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/14192/6/examples/quickstart/scripts/start_kudu.sh
File examples/quickstart/scripts/start_kudu.sh:

http://gerrit.cloudera.org:8080/#/c/14192/6/examples/quickstart/scripts/start_kudu.sh@27
PS6, Line 27:
> Does this work properly if your cwd is somewhere else and you're running th
I introduced --builddir parameter


http://gerrit.cloudera.org:8080/#/c/14192/6/examples/quickstart/scripts/start_kudu.sh@31
PS6, Line 31:
> May also want this to be configurable.
Done


http://gerrit.cloudera.org:8080/#/c/14192/6/examples/quickstart/scripts/start_kudu.sh@95
PS6, Line 95:
:
:
:
:
:
> Maybe you could allow the ports to be configurable via env variables? Seems
Done


http://gerrit.cloudera.org:8080/#/c/14192/6/examples/quickstart/scripts/start_kudu.sh@95
PS6, Line 95:
:
:
:
:
:
> I agree with Adar -- it's better have a way to configure, say, 'base port n
Done



--
To view, visit http://gerrit.cloudera.org:8080/14192
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e7bb66d4cce84d00d346bf50154f449d48e7a54
Gerrit-Change-Number: 14192
Gerrit-PatchSet: 7
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Fri, 13 Sep 2019 21:56:54 +
Gerrit-HasComments: Yes


[kudu-CR] KUDU-2920 Block cache capacity validator couldn't run on an NVM block cache

2019-09-13 Thread Volodymyr Verovkin (Code Review)
Volodymyr Verovkin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14212 )

Change subject: KUDU-2920 Block cache capacity validator couldn't run on an NVM 
block cache
..


Patch Set 4:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/14212/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14212/3//COMMIT_MSG@7
PS3, Line 7: couldn't
> Looks like you lost one letter too many. :)
Done


http://gerrit.cloudera.org:8080/#/c/14212/3/src/kudu/cfile/block_cache.h
File src/kudu/cfile/block_cache.h:

http://gerrit.cloudera.org:8080/#/c/14212/3/src/kudu/cfile/block_cache.h@225
PS3, Line 225: // enough to cause pernicious flushing behavior. See KUDU-2318.
 : bool ValidateBlockCacheCapacity();
> Nit: rather aggressive line wrapping, no? The Google style guide[1] (which
Done


http://gerrit.cloudera.org:8080/#/c/14212/3/src/kudu/cfile/cfile-test.cc
File src/kudu/cfile/cfile-test.cc:

http://gerrit.cloudera.org:8080/#/c/14212/3/src/kudu/cfile/cfile-test.cc@1064
PS3, Line 1064: "
> nit: seems to be an extra single quote
Done



--
To view, visit http://gerrit.cloudera.org:8080/14212
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9f50e9dd901280b5c32576e43165292299922995
Gerrit-Change-Number: 14212
Gerrit-PatchSet: 4
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Fri, 13 Sep 2019 21:52:53 +
Gerrit-HasComments: Yes


[kudu-CR] KUDU-2920 Block cache capacity validator couldn't run on an NVM block cache

2019-09-13 Thread Volodymyr Verovkin (Code Review)
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14212

to look at the new patch set (#4).

Change subject: KUDU-2920 Block cache capacity validator couldn't run on an NVM 
block cache
..

KUDU-2920 Block cache capacity validator couldn't run on an NVM block cache

Validator should fail for options:
--block_cache_type=DRAM
--block_cache_capacity_mb=100
and succeed for options:
--block_cache_type=NVM
--block_cache_capacity_mb=100
--nvm_cache_path=/path/to/your/tmp/dir
where 100 means "size of cache bigger than RAM" (1000GB)

Change-Id: I9f50e9dd901280b5c32576e43165292299922995
---
M src/kudu/cfile/block_cache.cc
M src/kudu/cfile/block_cache.h
M src/kudu/cfile/cfile-test.cc
3 files changed, 27 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/12/14212/4
--
To view, visit http://gerrit.cloudera.org:8080/14212
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9f50e9dd901280b5c32576e43165292299922995
Gerrit-Change-Number: 14212
Gerrit-PatchSet: 4
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)


[kudu-CR] KUDU-1561 Implemented operator->() in KuduScanBatch::const iterator

2019-09-13 Thread Volodymyr Verovkin (Code Review)
Volodymyr Verovkin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14219 )

Change subject: KUDU-1561 Implemented operator->() in 
KuduScanBatch::const_iterator
..


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/14219/2/src/kudu/client/client-test.cc
File src/kudu/client/client-test.cc:

http://gerrit.cloudera.org:8080/#/c/14219/2/src/kudu/client/client-test.cc@5626
PS2, Line 5626:   ASSERT_EQ(x, y);
> nit: whoops, I didn't notice it in prior revision, but a space is missing b
Done


http://gerrit.cloudera.org:8080/#/c/14219/2/src/kudu/client/scan_batch.h
File src/kudu/client/scan_batch.h:

http://gerrit.cloudera.org:8080/#/c/14219/2/src/kudu/client/scan_batch.h@165
PS2, Line 165:   /// Overloaded operator -> to support pointer trait
 :   /// for access via const_iterato
> I think it's worth keeping this for doxigen-generated doc (i.e. have triple
Done



--
To view, visit http://gerrit.cloudera.org:8080/14219
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5ddd99073d6a93337c184bee8b930cabeeda145
Gerrit-Change-Number: 14219
Gerrit-PatchSet: 3
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Volodymyr Verovkin 
Gerrit-Comment-Date: Fri, 13 Sep 2019 21:27:57 +
Gerrit-HasComments: Yes


[kudu-CR] KUDU-1561 Implemented operator->() in KuduScanBatch::const iterator

2019-09-13 Thread Volodymyr Verovkin (Code Review)
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Adar Dembo,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14219

to look at the new patch set (#3).

Change subject: KUDU-1561 Implemented operator->() in 
KuduScanBatch::const_iterator
..

KUDU-1561 Implemented operator->() in KuduScanBatch::const_iterator

operator->() requires returning pointer or object which overloads access
via pointer operator (operator->() again).
Since KuduScanBatch::const_iterator does to keep current
KuduScanBatch::RowPtr, we cannot return pointer to it in operator ->.
Instead, we return KuduScanBatch::RowPtr which implements pointer trait
(operator->).

Change-Id: Ib5ddd99073d6a93337c184bee8b930cabeeda145
---
M src/kudu/client/client-test.cc
M src/kudu/client/scan_batch.h
2 files changed, 28 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/19/14219/3
--
To view, visit http://gerrit.cloudera.org:8080/14219
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib5ddd99073d6a93337c184bee8b930cabeeda145
Gerrit-Change-Number: 14219
Gerrit-PatchSet: 3
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Volodymyr Verovkin 


[kudu-CR] WIP KUDU-2780: create thread for auto-rebalancing

2019-09-13 Thread Hannah Nguyen (Code Review)
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14177

to look at the new patch set (#3).

Change subject: WIP KUDU-2780: create thread for auto-rebalancing
..

WIP KUDU-2780: create thread for auto-rebalancing

Created auto-rebalancer thread in background tasks of catalog manager.
Set up framework for auto-rebalancing loop.
Loop retrieves information on tservers, tables, and tablets for rebalancing.
The number of replica moves per loop iteration is controlled by a flag.
If there are placement policy violations, the current loop iteration will only
perform replica moves to reinstate the policy.
Otherwise, the auto-rebalancer will perform moves to do 
inter-location(cross-location),
then intra-location(by table then by tserver) rebalancing.
If the cluster is balanced, the current rebalancing cycle completes, and the
thread will sleep for an interval, controlled by another flag.

Change-Id: Ifca25d1063c07047cf2123e6792b3c7395be20e4
TODO: write tests.
---
M src/kudu/master/CMakeLists.txt
A src/kudu/master/auto_rebalancer-test.cc
A src/kudu/master/auto_rebalancer.cc
A src/kudu/master/auto_rebalancer.h
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/ts_descriptor.cc
M src/kudu/master/ts_descriptor.h
M src/kudu/rebalance/rebalancer.cc
M src/kudu/rebalance/rebalancer.h
M src/kudu/tools/rebalancer_tool.cc
11 files changed, 1,297 insertions(+), 53 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/77/14177/3
--
To view, visit http://gerrit.cloudera.org:8080/14177
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifca25d1063c07047cf2123e6792b3c7395be20e4
Gerrit-Change-Number: 14177
Gerrit-PatchSet: 3
Gerrit-Owner: Hannah Nguyen 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Hannah Nguyen 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)


[kudu-CR] KUDU-2069 p5: recheck tablet health when exiting maintenance mode

2019-09-13 Thread Alexey Serbin (Code Review)
Alexey Serbin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14223 )

Change subject: KUDU-2069 p5: recheck tablet health when exiting maintenance 
mode
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14223/2//COMMIT_MSG
Commit Message:

PS2:
Not a question regarding this patch as is, but anyways.

In addition to changes in the catalog manager and tservers, do you think we 
need to update the kudu CLI tooling to report on the maintenance mode of tablet 
servers?  Also, would it make sense to mark/note a tablet which is 
under-replicated, but not being re-replicated due to the maintenance mode?



--
To view, visit http://gerrit.cloudera.org:8080/14223
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic0ab3d78cbc5b1228c01592a00118f11f76e43dd
Gerrit-Change-Number: 14223
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 13 Sep 2019 18:57:43 +
Gerrit-HasComments: Yes


[kudu-CR] KUDU-2069 p5: recheck tablet health when exiting maintenance mode

2019-09-13 Thread Hao Hao (Code Review)
Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14223 )

Change subject: KUDU-2069 p5: recheck tablet health when exiting maintenance 
mode
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14223/2/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:

http://gerrit.cloudera.org:8080/#/c/14223/2/src/kudu/master/catalog_manager.cc@5003
PS2, Line 5003: ts_manager->SetAllTServersRequireFullTabletReport();
> Wouldn't we have short-circuited out via L4997 then?
Ah, missed that, thanks!



--
To view, visit http://gerrit.cloudera.org:8080/14223
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic0ab3d78cbc5b1228c01592a00118f11f76e43dd
Gerrit-Change-Number: 14223
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 13 Sep 2019 17:43:27 +
Gerrit-HasComments: Yes


[kudu-CR] [thirdparty] add SO REUSEPORT for chronyd NTP socket

2019-09-13 Thread Greg Solovyev (Code Review)
Greg Solovyev has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14228 )

Change subject: [thirdparty] add SO_REUSEPORT for chronyd NTP socket
..


Patch Set 1: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/14228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iee26fcf93976dd7affe77254751016bcbf398620
Gerrit-Change-Number: 14228
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 13 Sep 2019 16:26:01 +
Gerrit-HasComments: No


[kudu-CR] [mini cluster] introduce 'builtin' clock source

2019-09-13 Thread Greg Solovyev (Code Review)
Greg Solovyev has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14227 )

Change subject: [mini_cluster] introduce 'builtin' clock source
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14227/2/src/kudu/integration-tests/master_migration-itest.cc
File src/kudu/integration-tests/master_migration-itest.cc:

http://gerrit.cloudera.org:8080/#/c/14227/2/src/kudu/integration-tests/master_migration-itest.cc@202
PS2, Line 202:   cluster.reset();
What is the reason for this change?



--
To view, visit http://gerrit.cloudera.org:8080/14227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c334ae6fa1fb12b033de7f8e8584b8dd3aa2d32
Gerrit-Change-Number: 14227
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 13 Sep 2019 16:22:29 +
Gerrit-HasComments: Yes


[kudu-CR] [mini cluster] introduce 'builtin' clock source

2019-09-13 Thread Alexey Serbin (Code Review)
Hello Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14227

to look at the new patch set (#2).

Change subject: [mini_cluster] introduce 'builtin' clock source
..

[mini_cluster] introduce 'builtin' clock source

This patch introduces the built-in NTP client into the external
minicluster test scaffolding.  With this patch, it's possible
to synchronize the built-in NTP client of the minicluster's masters
and tablet servers with chronyd NTP server which is run as a part
of the minicluster as well.  The latter runs in server-only mode,
using the machine's clock as a reference.

Change-Id: I5c334ae6fa1fb12b033de7f8e8584b8dd3aa2d32
---
M src/kudu/integration-tests/master_migration-itest.cc
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/mini-cluster/external_mini_cluster.h
3 files changed, 67 insertions(+), 23 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/14227/2
--
To view, visit http://gerrit.cloudera.org:8080/14227
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5c334ae6fa1fb12b033de7f8e8584b8dd3aa2d32
Gerrit-Change-Number: 14227
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)


[kudu-CR] [thirdparty] add SO REUSEPORT for chronyd NTP socket

2019-09-13 Thread Alexey Serbin (Code Review)
Alexey Serbin has removed a vote on this change.

Change subject: [thirdparty] add SO_REUSEPORT for chronyd NTP socket
..


Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/14228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Iee26fcf93976dd7affe77254751016bcbf398620
Gerrit-Change-Number: 14228
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)


[kudu-CR] [thirdparty] add SO REUSEPORT for chronyd NTP socket

2019-09-13 Thread Alexey Serbin (Code Review)
Alexey Serbin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14228 )

Change subject: [thirdparty] add SO_REUSEPORT for chronyd NTP socket
..


Patch Set 1: Verified+1

Unrelated test failures in:
  * MasterSentryTest.TestTableOwnership
  * HmsSentryConfigurations/MasterStressTest.Test/2


--
To view, visit http://gerrit.cloudera.org:8080/14228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iee26fcf93976dd7affe77254751016bcbf398620
Gerrit-Change-Number: 14228
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 13 Sep 2019 15:27:11 +
Gerrit-HasComments: No


[kudu-CR] [thirdparty] add SO REUSEPORT for chronyd NTP socket

2019-09-13 Thread Alexey Serbin (Code Review)
Alexey Serbin has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/14228


Change subject: [thirdparty] add SO_REUSEPORT for chronyd NTP socket
..

[thirdparty] add SO_REUSEPORT for chronyd NTP socket

This patch adds SO_REUSEPORT option for NTP server socket opened
by chronyd.  This is to allow for using the port reservation
technique from external minicluster when starting chronyd test
NTP servers.

Change-Id: Iee26fcf93976dd7affe77254751016bcbf398620
---
M thirdparty/download-thirdparty.sh
A thirdparty/patches/chrony-reuseport.patch
2 files changed, 36 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/28/14228/1
--
To view, visit http://gerrit.cloudera.org:8080/14228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iee26fcf93976dd7affe77254751016bcbf398620
Gerrit-Change-Number: 14228
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin 


[kudu-CR] KUDU-2069 p4: stop replication from failed servers in maintenance mode

2019-09-13 Thread Alexey Serbin (Code Review)
Alexey Serbin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/14222 )

Change subject: KUDU-2069 p4: stop replication from failed servers in 
maintenance mode
..


Patch Set 2:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/14222/2/src/kudu/consensus/quorum_util-test.cc
File src/kudu/consensus/quorum_util-test.cc:

http://gerrit.cloudera.org:8080/#/c/14222/2/src/kudu/consensus/quorum_util-test.cc@636
PS2, Line 636: TEST_P(QuorumUtilHealthPolicyParamTest, 
ShouldAddReplicaWhitelist) {
What about tests scenarios for ShouldRemoveReplica() when some of the tablet 
servers hosting replicas are put into the maintenance mode?


http://gerrit.cloudera.org:8080/#/c/14222/2/src/kudu/consensus/quorum_util-test.cc@648
PS2, Line 648: unknown server
server with unknown health


http://gerrit.cloudera.org:8080/#/c/14222/2/src/kudu/consensus/quorum_util-test.cc@689
PS2, Line 689: }
Does it make sense to add the following cases for replication factor 3:
  * A:? B:+ C:+, all voters, where A is in maintenance mode
  * A:- B:+ C:+, all voters, where A, B, and C in maintenance mode
  * A:- B:- C:+, all voters, where A and B in maintenance mode

Also, how does the maintenance mode affect configurations with non-voter 
replicas?  Imagine, some non-voter replica was in process of copying when a 
tablet server with the failed replica was marked/put into the maintenance.  
What will happen to the non-voter replica?  Will it be removed since the tablet 
server with the failed voter replica has just been put into the maintenance 
mode?


http://gerrit.cloudera.org:8080/#/c/14222/2/src/kudu/integration-tests/maintenance_mode-itest.cc
File src/kudu/integration-tests/maintenance_mode-itest.cc:

http://gerrit.cloudera.org:8080/#/c/14222/2/src/kudu/integration-tests/maintenance_mode-itest.cc@19
PS2, Line 19:
nit: remove the extra line


http://gerrit.cloudera.org:8080/#/c/14222/2/src/kudu/integration-tests/maintenance_mode-itest.cc@156
PS2, Line 156: TestMaintenanceModeDoesntRereplicate
> nit: TestMaintenanceModeWithoutRereplication?
Does it make sense to add a scenario to verify that it's possible to move 
replicas _from_ a tablet server put into the maintenance mode?


http://gerrit.cloudera.org:8080/#/c/14222/2/src/kudu/integration-tests/maintenance_mode-itest.cc@195
PS2, Line 195: // Bringing the tablet server down shouldn't lead to 
re-replication.
Another reasons for re-replication are:
  * follower replica falling behind leader's WAL GC threshold
  * replica failed to bootstrap due to IO error

Does it make sense to add test scenario for those or we assume that's already 
covered due to the way how the code is written in quorum_util.cc?  If the 
latter, then please add a comment to reflect those possible cases as well.


http://gerrit.cloudera.org:8080/#/c/14222/2/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:

http://gerrit.cloudera.org:8080/#/c/14222/2/src/kudu/master/catalog_manager.cc@4190
PS2, Line 4190: unordered_set whitelisted_uuids;
  : 
master_->ts_manager()->GetWhitelistedUuids(_uuids);
> It's wasteful to generate this for each tablet in the report.
+1



--
To view, visit http://gerrit.cloudera.org:8080/14222
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9a63b55011d16900c0d27eac0eb75880074204db
Gerrit-Change-Number: 14222
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 13 Sep 2019 07:47:45 +
Gerrit-HasComments: Yes


[kudu-CR] [docs] Kudu source code indexing in Eclipse

2019-09-13 Thread Volodymyr Verovkin (Code Review)
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, Greg Solovyev, Grant Henke, 

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14193

to look at the new patch set (#6).

Change subject: [docs] Kudu source code indexing in Eclipse
..

[docs] Kudu source code indexing in Eclipse

Added alternative recommendation to deal with Kudu source code indexing in 
Eclipse.

Change-Id: Ia214e536ff3a7ddafb3ab969b50ed5f97f4c725a
---
M README.adoc
1 file changed, 13 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/93/14193/6
--
To view, visit http://gerrit.cloudera.org:8080/14193
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia214e536ff3a7ddafb3ab969b50ed5f97f4c725a
Gerrit-Change-Number: 14193
Gerrit-PatchSet: 6
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin 


[kudu-CR] [docs] Kudu source code indexing in Eclipse

2019-09-13 Thread Volodymyr Verovkin (Code Review)
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, Greg Solovyev, Grant Henke, 

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/14193

to look at the new patch set (#5).

Change subject: [docs] Kudu source code indexing in Eclipse
..

[docs] Kudu source code indexing in Eclipse

Added alternative recommendation to deal with Kudu source code indexing in 
Eclipse.

Change-Id: Ia214e536ff3a7ddafb3ab969b50ed5f97f4c725a
---
M README.adoc
1 file changed, 13 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/93/14193/5
--
To view, visit http://gerrit.cloudera.org:8080/14193
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia214e536ff3a7ddafb3ab969b50ed5f97f4c725a
Gerrit-Change-Number: 14193
Gerrit-PatchSet: 5
Gerrit-Owner: Volodymyr Verovkin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Greg Solovyev 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Volodymyr Verovkin