[kudu-CR] Added steps to update HMS after migrating to multiple Kudu masters.

[Anonymous Coward (Code Review)]

arod...@cloudera.com has abandoned this change. ( 
http://gerrit.cloudera.org:8080/8946 )

Change subject: Added steps to update HMS after migrating to multiple Kudu 
masters.
..


Abandoned
--
To view, visit http://gerrit.cloudera.org:8080/8946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I920930f3e21c69effb8a5afc9c05ba9eb9e5285d
Gerrit-Change-Number: 8946
Gerrit-PatchSet: 1
Gerrit-Owner: arod...@cloudera.com
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Thomas Tauber-Marshall 

[kudu-CR] [docs] Added steps to update HMS after migrating to multiple Kudu masters.

[Anonymous Coward (Code Review)]

arod...@cloudera.com has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/8948


Change subject: [docs] Added steps to update HMS after migrating to multiple 
Kudu masters.
..

[docs] Added steps to update HMS after migrating to multiple Kudu masters.

Change-Id: Iab3999c9e581ed3591b220c08491cdae867c91db
---
M docs/administration.adoc
1 file changed, 2 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/48/8948/1
--
To view, visit http://gerrit.cloudera.org:8080/8948
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iab3999c9e581ed3591b220c08491cdae867c91db
Gerrit-Change-Number: 8948
Gerrit-PatchSet: 1
Gerrit-Owner: arod...@cloudera.com

[kudu-CR] [consensus] test-only option for replica replacement

[Alexey Serbin (Code Review)]

Alexey Serbin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8889 )

Change subject: [consensus] test-only option for replica replacement
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8889/2/src/kudu/consensus/quorum_util-test.cc
File src/kudu/consensus/quorum_util-test.cc:

http://gerrit.cloudera.org:8080/#/c/8889/2/src/kudu/consensus/quorum_util-test.cc@957
PS2, Line 957: const auto kPolicies = { MHP_H, MHP_I }
> so this is still repeated a bunch of times. can we put it at the top of the
I decided to make it a parameterized test instead.



--
To view, visit http://gerrit.cloudera.org:8080/8889
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4f89b6c584296e3da5047475c5c86c4cb1118ad0
Gerrit-Change-Number: 8889
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Comment-Date: Fri, 05 Jan 2018 00:53:48 +
Gerrit-HasComments: Yes

[kudu-CR] [consensus] test-only option for replica replacement

[Alexey Serbin (Code Review)]

Hello Mike Percy, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8889

to look at the new patch set (#3).

Change subject: [consensus] test-only option for replica replacement
..

[consensus] test-only option for replica replacement

Replica replacement: added an option to attempt Raft configuration
update even in absence of healthy majority of tablet replicas.

This option is for testing purposes: as we could see during latest load
testing, an attempt to change the configuration under such unusual
circumstances allowed for exposing some bugs.

In run-time, the option is controlled by the newly introduced flag
--raft_attempt_to_replace_replica_without_majority.  It affects both
3-4-3 and 3-2-3 replica replacement schemes.

Change-Id: I4f89b6c584296e3da5047475c5c86c4cb1118ad0
---
M src/kudu/consensus/consensus_queue.cc
M src/kudu/consensus/quorum_util-test.cc
M src/kudu/consensus/quorum_util.cc
M src/kudu/consensus/quorum_util.h
M src/kudu/consensus/raft_consensus.cc
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/master/catalog_manager.cc
7 files changed, 879 insertions(+), 753 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/8889/3
--
To view, visit http://gerrit.cloudera.org:8080/8889
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4f89b6c584296e3da5047475c5c86c4cb1118ad0
Gerrit-Change-Number: 8889
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 

[kudu-CR] Added steps to update HMS after migrating to multiple Kudu masters.

[Thomas Tauber-Marshall (Code Review)]

Thomas Tauber-Marshall has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8946 )

Change subject: Added steps to update HMS after migrating to multiple Kudu 
masters.
..


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/8946/1/docs/administration.adoc
File docs/administration.adoc:

http://gerrit.cloudera.org:8080/#/c/8946/1/docs/administration.adoc@382
PS1, Line 382: master-1
Feels a little weird to use the same values as the DNS aliases in the section 
specifically for if you didn't set the aliases up.

Maybe we can make it clear that these are host names by making them something 
like 'master-1.example.com' etc. or if that's too verbose then maybe just 
'host-1' etc.


http://gerrit.cloudera.org:8080/#/c/8946/1/docs/administration.adoc@383
PS1, Line 383: PARAM_VALUE = 'old-master';
How about we add "AND PARAM_KEY = 'kudu.master_addresses'" here.

In practice, I think this is unlikely to make a difference, but better to be 
safe.



--
To view, visit http://gerrit.cloudera.org:8080/8946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I920930f3e21c69effb8a5afc9c05ba9eb9e5285d
Gerrit-Change-Number: 8946
Gerrit-PatchSet: 1
Gerrit-Owner: arod...@cloudera.com
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Thomas Tauber-Marshall 
Gerrit-Comment-Date: Fri, 05 Jan 2018 00:41:10 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2115: avoid compacting already-compacted rowsets

[Andrew Wong (Code Review)]

Andrew Wong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8859 )

Change subject: KUDU-2115: avoid compacting already-compacted rowsets
..


Patch Set 6:

IWYU and clock sync issues


--
To view, visit http://gerrit.cloudera.org:8080/8859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
Gerrit-Change-Number: 8859
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Fri, 05 Jan 2018 00:38:42 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2115: avoid compacting already-compacted rowsets

[Andrew Wong (Code Review)]

Hello Tidy Bot, Mike Percy, Kudu Jenkins, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8859

to look at the new patch set (#7).

Change subject: KUDU-2115: avoid compacting already-compacted rowsets
..

KUDU-2115: avoid compacting already-compacted rowsets

Tablets will perform compaction selection on a copy of the currently
available rowsets in order to avoid holding the component_lock_ for the
duration of rowset selection.

It is then verified that nothing else compacted the selected rowsets by
iterating over the selected rowsets and checking that they still exist
to be compacted.

However, the initial selection of rowsets is performed on a snapshotted
copy of the available rowsets, and in between the snapshot of the
rowsets and the verification, the component_lock_ is dropped, allowing
the following race:

T1: runs PickRowSetsToCompact, picks {A, B}, begins compaction
T2: runs PickRowSetsToCompact:
T2:   snapshots the rowset tree (which still contains {A, B})
T2:   drops component_lock_ after taking the snapshot
T1: finishes compaction and removes {A, B} from the rowset tree,
unlocking each rowset's compact_flush_lock_
T2: iterates over the rowset tree and sees {A, B} as
IsAvailableForCompaction(), since they have been unlocked
T2: selects {A, B} as a part of its compaction
T2: grabs the component_lock_ to verify that the selected rowsets are
still available
T2: sees that some of the selected rowsets would not be returned because
they are missing from the currently available rowsets (DFATALs and
aborts the compaction)

I verified the diagnosis by placing a random sleep just after making the
copy in PickRowSetsToCompact() and seeing
TabletServerDiskFailureTest.TestRandomOpSequence, which schedules many
compactions, fail consistently with the failed verification. Upon making
the fix, this passes.

This can lead to scheduling a fairly useless compaction. This patch
adds an API to RowSets to mark itself as compacted, and ensure that when
selecting rowsets to compact, check that rowset candidates have not
already been compacted.

To verify the solution, I've added a small test that schedules several
concurrent compactions. Upon adding the aforementioned randomized
delay and removing a couple sanity D/CHECKs (including the above
verification), I saw the test fail, leading to an unexpected number of
rows left in the tablet.

Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
---
M src/kudu/tablet/compaction-test.cc
M src/kudu/tablet/diskrowset.cc
M src/kudu/tablet/diskrowset.h
M src/kudu/tablet/memrowset-test.cc
M src/kudu/tablet/memrowset.cc
M src/kudu/tablet/memrowset.h
M src/kudu/tablet/mock-rowsets.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/tablet.cc
9 files changed, 170 insertions(+), 77 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/8859/7
--
To view, visit http://gerrit.cloudera.org:8080/8859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
Gerrit-Change-Number: 8859
Gerrit-PatchSet: 7
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] Added steps to update HMS after migrating to multiple Kudu masters.

[Anonymous Coward (Code Review)]

arod...@cloudera.com has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/8946


Change subject: Added steps to update HMS after migrating to multiple Kudu 
masters.
..

Added steps to update HMS after migrating to multiple Kudu masters.

Change-Id: I920930f3e21c69effb8a5afc9c05ba9eb9e5285d
---
M docs/administration.adoc
1 file changed, 23 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/46/8946/1
--
To view, visit http://gerrit.cloudera.org:8080/8946
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I920930f3e21c69effb8a5afc9c05ba9eb9e5285d
Gerrit-Change-Number: 8946
Gerrit-PatchSet: 1
Gerrit-Owner: arod...@cloudera.com

[kudu-CR] KUDU-2115: avoid compacting already-compacted rowsets

[Andrew Wong (Code Review)]

Andrew Wong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8859 )

Change subject: KUDU-2115: avoid compacting already-compacted rowsets
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8859/5/src/kudu/tablet/tablet.cc
File src/kudu/tablet/tablet.cc:

http://gerrit.cloudera.org:8080/#/c/8859/5/src/kudu/tablet/tablet.cc@1626
PS5, Line 1626:   //  rs->set_has_been_compacted();
> oops
Done



--
To view, visit http://gerrit.cloudera.org:8080/8859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
Gerrit-Change-Number: 8859
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 23:15:37 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2115: avoid compacting already-compacted rowsets

[Andrew Wong (Code Review)]

Hello Tidy Bot, Mike Percy, Kudu Jenkins, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8859

to look at the new patch set (#6).

Change subject: KUDU-2115: avoid compacting already-compacted rowsets
..

KUDU-2115: avoid compacting already-compacted rowsets

Tablets will perform compaction selection on a copy of the currently
available rowsets in order to avoid holding the component_lock_ for the
duration of rowset selection.

It is then verified that nothing else compacted the selected rowsets by
iterating over the selected rowsets and checking that they still exist
to be compacted.

However, the initial selection of rowsets is performed on a snapshotted
copy of the available rowsets, and in between the snapshot of the
rowsets and the verification, the component_lock_ is dropped, allowing
the following race:

T1: runs PickRowSetsToCompact, picks {A, B}, begins compaction
T2: runs PickRowSetsToCompact:
T2:   snapshots the rowset tree (which still contains {A, B})
T2:   drops component_lock_ after taking the snapshot
T1: finishes compaction and removes {A, B} from the rowset tree,
unlocking each rowset's compact_flush_lock_
T2: iterates over the rowset tree and sees {A, B} as
IsAvailableForCompaction(), since they have been unlocked
T2: selects {A, B} as a part of its compaction
T2: grabs the component_lock_ to verify that the selected rowsets are
still available
T2: sees that some of the selected rowsets would not be returned because
they are missing from the currently available rowsets (DFATALs and
aborts the compaction)

I verified the diagnosis by placing a random sleep just after making the
copy in PickRowSetsToCompact() and seeing
TabletServerDiskFailureTest.TestRandomOpSequence, which schedules many
compactions, fail consistently with the failed verification. Upon making
the fix, this passes.

This can lead to scheduling a fairly useless compaction. This patch
adds an API to RowSets to mark itself as compacted, and ensure that when
selecting rowsets to compact, check that rowset candidates have not
already been compacted.

To verify the solution, I've added a small test that schedules several
concurrent compactions. Upon adding the aforementioned randomized
delay and removing a couple sanity D/CHECKs (including the above
verification), I saw the test fail, leading to an unexpected number of
rows left in the tablet.

Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
---
M src/kudu/tablet/compaction-test.cc
M src/kudu/tablet/diskrowset.cc
M src/kudu/tablet/diskrowset.h
M src/kudu/tablet/memrowset-test.cc
M src/kudu/tablet/memrowset.cc
M src/kudu/tablet/memrowset.h
M src/kudu/tablet/mock-rowsets.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/tablet.cc
9 files changed, 168 insertions(+), 75 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/8859/6
--
To view, visit http://gerrit.cloudera.org:8080/8859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
Gerrit-Change-Number: 8859
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] KUDU-2115: avoid compacting already-compacted rowsets

[Andrew Wong (Code Review)]

Andrew Wong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8859 )

Change subject: KUDU-2115: avoid compacting already-compacted rowsets
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8859/5/src/kudu/tablet/tablet.cc
File src/kudu/tablet/tablet.cc:

http://gerrit.cloudera.org:8080/#/c/8859/5/src/kudu/tablet/tablet.cc@1626
PS5, Line 1626:   //  rs->set_has_been_compacted();
oops



--
To view, visit http://gerrit.cloudera.org:8080/8859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
Gerrit-Change-Number: 8859
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 23:12:45 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2115: avoid compacting already-compacted rowsets

[Andrew Wong (Code Review)]

Hello Tidy Bot, Mike Percy, Kudu Jenkins, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8859

to look at the new patch set (#5).

Change subject: KUDU-2115: avoid compacting already-compacted rowsets
..

KUDU-2115: avoid compacting already-compacted rowsets

Tablets will perform compaction selection on a copy of the currently
available rowsets in order to avoid holding the component_lock_ for the
duration of rowset selection.

It is then verified that nothing else compacted the selected rowsets by
iterating over the selected rowsets and checking that they still exist
to be compacted.

However, the initial selection of rowsets is performed on a snapshotted
copy of the available rowsets, and in between the snapshot of the
rowsets and the verification, the component_lock_ is dropped, allowing
the following race:

T1: runs PickRowSetsToCompact, picks {A, B}, begins compaction
T2: runs PickRowSetsToCompact:
T2:   snapshots the rowset tree (which still contains {A, B})
T2:   drops component_lock_ after taking the snapshot
T1: finishes compaction and removes {A, B} from the rowset tree,
unlocking each rowset's compact_flush_lock_
T2: iterates over the rowset tree and sees {A, B} as
IsAvailableForCompaction(), since they have been unlocked
T2: selects {A, B} as a part of its compaction
T2: grabs the component_lock_ to verify that the selected rowsets are
still available
T2: sees that some of the selected rowsets would not be returned because
they are missing from the currently available rowsets (DFATALs and
aborts the compaction)

I verified the diagnosis by placing a random sleep just after making the
copy in PickRowSetsToCompact() and seeing
TabletServerDiskFailureTest.TestRandomOpSequence, which schedules many
compactions, fail consistently with the failed verification. Upon making
the fix, this passes.

This can lead to scheduling a fairly useless compaction. This patch
adds an API to RowSets to mark itself as compacted, and ensure that when
selecting rowsets to compact, check that rowset candidates have not
already been compacted.

To verify the solution, I've added a small test that schedules several
concurrent compactions. Upon adding the aforementioned randomized
delay and removing a couple sanity D/CHECKs (including the above
verification), I saw the test fail, leading to an unexpected number of
rows left in the tablet.

Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
---
M src/kudu/tablet/compaction-test.cc
M src/kudu/tablet/diskrowset.cc
M src/kudu/tablet/diskrowset.h
M src/kudu/tablet/memrowset-test.cc
M src/kudu/tablet/memrowset.cc
M src/kudu/tablet/memrowset.h
M src/kudu/tablet/mock-rowsets.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/tablet.cc
9 files changed, 168 insertions(+), 75 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/8859/5
--
To view, visit http://gerrit.cloudera.org:8080/8859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
Gerrit-Change-Number: 8859
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] KUDU-2115: avoid compacting already-compacted rowsets

[Andrew Wong (Code Review)]

Andrew Wong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8859 )

Change subject: KUDU-2115: avoid compacting already-compacted rowsets
..


Patch Set 4:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/diskrowset.h
File src/kudu/tablet/diskrowset.h:

http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/diskrowset.h@435
PS4, Line 435:   virtual Status DebugDump(std::vector *lines = 
NULL) override;
> warning: default arguments on virtual or override methods are prohibited [g
Done


http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/diskrowset.h@478
PS4, Line 478:   std::atomic has_been_compacted_;
> add doc
Done


http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/memrowset.h
File src/kudu/tablet/memrowset.h:

http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/memrowset.h@293
PS4, Line 293:   // TODO: unit test me
> warning: missing username/bug in TODO [google-readability-todo]
Done


http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/memrowset.h@350
PS4, Line 350:   virtual Status DebugDump(std::vector *lines = 
NULL) override;
> warning: default arguments on virtual or override methods are prohibited [g
Done


http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/memrowset.h@376
PS4, Line 376:   double 
DeltaStoresCompactionPerfImprovementScore(DeltaCompactionType type) const 
override {
> warning: parameter 'type' is unused [misc-unused-parameters]
Done


http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/memrowset.h@441
PS4, Line 441:   std::atomic has_been_compacted_;
> doc
Done


http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/rowset.h
File src/kudu/tablet/rowset.h:

http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/rowset.h@237
PS4, Line 237: rowset
> i think better to be explicit and say that it has been removed from the Row
Done



--
To view, visit http://gerrit.cloudera.org:8080/8859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
Gerrit-Change-Number: 8859
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 23:11:10 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2232. java: Remove deprecated call from KuduTable

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8749 )

Change subject: KUDU-2232. java: Remove deprecated call from KuduTable
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8749/2/java/kudu-client/src/main/java/org/apache/kudu/client/KuduTable.java
File java/kudu-client/src/main/java/org/apache/kudu/client/KuduTable.java:

http://gerrit.cloudera.org:8080/#/c/8749/2/java/kudu-client/src/main/java/org/apache/kudu/client/KuduTable.java@223
PS2, Line 223:   public List getFormattedRangePartitions(long deadline) 
throws Exception {
isn't this a breaking change, in that the deadline is now ignored?

Don't we need to somehow pass the deadline into the KuduScanTokenBuilder or 
somesuch so that this doesn't change semantics?



--
To view, visit http://gerrit.cloudera.org:8080/8749
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8c54d0483241166675c65a8002464d4bd9604a4f
Gerrit-Change-Number: 8749
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Bukor 
Gerrit-Reviewer: Attila Bukor 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 22:33:04 +
Gerrit-HasComments: Yes

[kudu-CR] [docs] Add scaling guide

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8842 )

Change subject: [docs] Add scaling guide
..


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/8842/2/docs/scaling_guide.adoc
File docs/scaling_guide.adoc:

http://gerrit.cloudera.org:8080/#/c/8842/2/docs/scaling_guide.adoc@22
PS2, Line 22: The recommendations and conclusions here are only approximations. 
Appropriate numbers depend on use
: case. There is no substitute for measurement and monitoring of 
resources used during a
: representative workload.
is there some adoc syntax we can use to make this show up with a big warning 
exclamation point?


http://gerrit.cloudera.org:8080/#/c/8842/2/docs/scaling_guide.adoc@88
PS2, Line 88: | Total | 38.5GB
maybe make two separate rows here -- one for "expected memory usage" and one 
for "recommmended hard limit" so that the expected memory is only 75% of the 
recommmendation, per the paragraph below?



--
To view, visit http://gerrit.cloudera.org:8080/8842
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I38d8999addc41fe0b726342a27dbba199ddf7dd2
Gerrit-Change-Number: 8842
Gerrit-PatchSet: 2
Gerrit-Owner: Will Berkeley 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Thu, 04 Jan 2018 21:53:44 +
Gerrit-HasComments: Yes

[kudu-CR](branch-1.5.x) KUDU-2251: rowset size can overflow int in RowSetInfo

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8941 )

Change subject: KUDU-2251: rowset size can overflow int in RowSetInfo
..


Patch Set 2:

Yep, I'll be adding the repro to this patch shortly.  Just giving it a few runs 
on dist-test.


--
To view, visit http://gerrit.cloudera.org:8080/8941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: comment
Gerrit-Change-Id: I598344e22fc1ecbb482bfe85ea3867ddf63963b4
Gerrit-Change-Number: 8941
Gerrit-PatchSet: 2
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 21:33:09 +
Gerrit-HasComments: No

[kudu-CR](branch-1.5.x) KUDU-2251: rowset size can overflow int in RowSetInfo

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8941 )

Change subject: KUDU-2251: rowset size can overflow int in RowSetInfo
..


Patch Set 2:

hm, thought you had a test that reproed this also. Worth backporting that as 
well, if so, unless it has a lot of test dependencies not available in this 
branch


--
To view, visit http://gerrit.cloudera.org:8080/8941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: comment
Gerrit-Change-Id: I598344e22fc1ecbb482bfe85ea3867ddf63963b4
Gerrit-Change-Number: 8941
Gerrit-PatchSet: 2
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 21:27:50 +
Gerrit-HasComments: No

[kudu-CR](branch-1.5.x) KUDU-2193 (part 1): switch to a waiting mutex in TSTabletManager

[Todd Lipcon (Code Review)]

Todd Lipcon has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/8525 )

Change subject: KUDU-2193 (part 1): switch to a waiting mutex in TSTabletManager
..

KUDU-2193 (part 1): switch to a waiting mutex in TSTabletManager

The TSTabletManager map is sometimes held for a long time. Given that, a
sleeping mutex is more appropriate than a spinlock.

Change-Id: I763abddd74d8b1dabb618318dc84256b533077e3
Reviewed-on: http://gerrit.cloudera.org:8080/8345
Reviewed-by: Adar Dembo 
Tested-by: Kudu Jenkins
(cherry picked from commit 94bf699645120bbe2a0e9f3d3a33ff65b8214dce)
Reviewed-on: http://gerrit.cloudera.org:8080/8525
---
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
2 files changed, 24 insertions(+), 23 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Adar Dembo: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/8525
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: merged
Gerrit-Change-Id: I763abddd74d8b1dabb618318dc84256b533077e3
Gerrit-Change-Number: 8525
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR](branch-1.4.x) KUDU-2193 (part 1): switch to a waiting mutex in TSTabletManager

[Todd Lipcon (Code Review)]

Todd Lipcon has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/8527 )

Change subject: KUDU-2193 (part 1): switch to a waiting mutex in TSTabletManager
..

KUDU-2193 (part 1): switch to a waiting mutex in TSTabletManager

The TSTabletManager map is sometimes held for a long time. Given that, a
sleeping mutex is more appropriate than a spinlock.

Change-Id: I763abddd74d8b1dabb618318dc84256b533077e3
Reviewed-on: http://gerrit.cloudera.org:8080/8345
Reviewed-by: Adar Dembo 
Tested-by: Kudu Jenkins
(cherry picked from commit 94bf699645120bbe2a0e9f3d3a33ff65b8214dce)
Reviewed-on: http://gerrit.cloudera.org:8080/8527
---
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
2 files changed, 24 insertions(+), 23 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Adar Dembo: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/8527
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.4.x
Gerrit-MessageType: merged
Gerrit-Change-Id: I763abddd74d8b1dabb618318dc84256b533077e3
Gerrit-Change-Number: 8527
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 

[kudu-CR] KUDU-2115: avoid compacting already-compacted rowsets

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8859 )

Change subject: KUDU-2115: avoid compacting already-compacted rowsets
..


Patch Set 4:

(3 comments)

looks like all the tests failed :)

http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/diskrowset.h
File src/kudu/tablet/diskrowset.h:

http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/diskrowset.h@478
PS4, Line 478:   std::atomic has_been_compacted_;
add doc


http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/memrowset.h
File src/kudu/tablet/memrowset.h:

http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/memrowset.h@441
PS4, Line 441:   std::atomic has_been_compacted_;
doc


http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/rowset.h
File src/kudu/tablet/rowset.h:

http://gerrit.cloudera.org:8080/#/c/8859/4/src/kudu/tablet/rowset.h@237
PS4, Line 237: rowset
i think better to be explicit and say that it has been removed from the 
RowSetTree



--
To view, visit http://gerrit.cloudera.org:8080/8859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
Gerrit-Change-Number: 8859
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 21:23:18 +
Gerrit-HasComments: Yes

[kudu-CR](branch-1.5.x) Add support for clang from XCode 9 in macOS

[Dan Burkert (Code Review)]

Dan Burkert has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/8940 )

Change subject: Add support for clang from XCode 9 in macOS
..

Add support for clang from XCode 9 in macOS

Change-Id: Ib636deee77471e60c02e90028a0e5cc0f303b45a
Reviewed-on: http://gerrit.cloudera.org:8080/8119
Reviewed-by: Alexey Serbin 
Tested-by: Kudu Jenkins
Reviewed-by: David Ribeiro Alves 
Reviewed-on: http://gerrit.cloudera.org:8080/8940
---
M cmake_modules/CompilerInfo.cmake
1 file changed, 13 insertions(+), 0 deletions(-)

Approvals:
  David Ribeiro Alves: Looks good to me, approved
  Alexey Serbin: Looks good to me, approved
  Kudu Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/8940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib636deee77471e60c02e90028a0e5cc0f303b45a
Gerrit-Change-Number: 8940
Gerrit-PatchSet: 2
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] [thirdparty] fix lib link in PREFIX DIR

[Alexey Serbin (Code Review)]

Alexey Serbin has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/8943 )

Change subject: [thirdparty] fix lib link in PREFIX_DIR
..

[thirdparty] fix lib link in PREFIX_DIR

Avoid creating the unintended 'lib' soft link in already soft-linked
directory.

Change-Id: I9ff2d57414eb048a97b5494bebab5826a9d83fdf
Reviewed-on: http://gerrit.cloudera.org:8080/8943
Reviewed-by: Todd Lipcon 
Tested-by: Kudu Jenkins
---
M thirdparty/build-thirdparty.sh
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Todd Lipcon: Looks good to me, approved
  Kudu Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/8943
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I9ff2d57414eb048a97b5494bebab5826a9d83fdf
Gerrit-Change-Number: 8943
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] [thirdparty] fix lib link in PREFIX DIR

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8943 )

Change subject: [thirdparty] fix lib link in PREFIX_DIR
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/8943
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9ff2d57414eb048a97b5494bebab5826a9d83fdf
Gerrit-Change-Number: 8943
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 20:04:02 +
Gerrit-HasComments: No

[kudu-CR] [thirdparty] fix lib link in PREFIX DIR

[Alexey Serbin (Code Review)]

Alexey Serbin has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/8943


Change subject: [thirdparty] fix lib link in PREFIX_DIR
..

[thirdparty] fix lib link in PREFIX_DIR

Avoid creating the unintended 'lib' soft link in already soft-linked
directory.

Change-Id: I9ff2d57414eb048a97b5494bebab5826a9d83fdf
---
M thirdparty/build-thirdparty.sh
1 file changed, 1 insertion(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/43/8943/1
--
To view, visit http://gerrit.cloudera.org:8080/8943
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9ff2d57414eb048a97b5494bebab5826a9d83fdf
Gerrit-Change-Number: 8943
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin 

[kudu-CR](branch-1.4.x) KUDU-2193 (part 1): switch to a waiting mutex in TSTabletManager

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8527 )

Change subject: KUDU-2193 (part 1): switch to a waiting mutex in TSTabletManager
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/8527
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.4.x
Gerrit-MessageType: comment
Gerrit-Change-Id: I763abddd74d8b1dabb618318dc84256b533077e3
Gerrit-Change-Number: 8527
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Thu, 04 Jan 2018 19:43:50 +
Gerrit-HasComments: No

[kudu-CR](branch-1.5.x) KUDU-2193 (part 1): switch to a waiting mutex in TSTabletManager

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8525 )

Change subject: KUDU-2193 (part 1): switch to a waiting mutex in TSTabletManager
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/8525
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: comment
Gerrit-Change-Id: I763abddd74d8b1dabb618318dc84256b533077e3
Gerrit-Change-Number: 8525
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Thu, 04 Jan 2018 19:43:47 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2115: avoid compacting already-compacted rowsets

[Andrew Wong (Code Review)]

Andrew Wong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8859 )

Change subject: KUDU-2115: avoid compacting already-compacted rowsets
..


Patch Set 4:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/8859/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/8859/3//COMMIT_MSG@19
PS3, Line 19: rowsets and the verification, the component_lock_ is dropped, 
allowing
: the following race:
:
> The race wasn't immediately clear to me from the commit message but I looke
Yep, the biggest piece of this is that the component_lock_ is dropped while 
running compaction selection policy.

T1: makes copy {A,B} and drops component_lock_
T2: makes copy {A,B} and drops component_lock_

T1: considers {A,B}, and selects them for compaction, taking their 
compact_flush_lock_
T1: finishes compaction, removes rowsets from tree, drops {A,B}'s 
compact_flush_lock_

T2: considers {A,B}, and selects them for compaction, taking their 
compact_flush_lock_
T2: can't proceed because it can't find them in {A,B} in the rowset tree


http://gerrit.cloudera.org:8080/#/c/8859/3//COMMIT_MSG@24
PS3, Line 24: T2:   snapshots the rowset tree (which still contains {A, B})
> Ignoring and proceeding could have the side effect of running a fairly usel
I'm not super convinced (a) would be able to avoid the race without holding the 
component lock while selecting, so I'm going with (b).



--
To view, visit http://gerrit.cloudera.org:8080/8859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
Gerrit-Change-Number: 8859
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 19:00:21 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2115: avoid compacting already-compacted rowsets

[Andrew Wong (Code Review)]

Hello Mike Percy, Kudu Jenkins, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8859

to look at the new patch set (#4).

Change subject: KUDU-2115: avoid compacting already-compacted rowsets
..

KUDU-2115: avoid compacting already-compacted rowsets

Tablets will perform compaction selection on a copy of the currently
available rowsets in order to avoid holding the component_lock_ for the
duration of rowset selection.

It is then verified that nothing else compacted the selected rowsets by
iterating over the selected rowsets and checking that they still exist
to be compacted.

However, the initial selection of rowsets is performed on a snapshotted
copy of the available rowsets, and in between the snapshot of the
rowsets and the verification, the component_lock_ is dropped, allowing
the following race:

T1: runs PickRowSetsToCompact, picks {A, B}, begins compaction
T2: runs PickRowSetsToCompact:
T2:   snapshots the rowset tree (which still contains {A, B})
T2:   drops component_lock_ after taking the snapshot
T1: finishes compaction and removes {A, B} from the rowset tree,
unlocking each rowset's compact_flush_lock_
T2: iterates over the rowset tree and sees {A, B} as
IsAvailableForCompaction(), since they have been unlocked
T2: selects {A, B} as a part of its compaction
T2: grabs the component_lock_ to verify that the selected rowsets are
still available
T2: sees that some of the selected rowsets would not be returned because
they are missing from the currently available rowsets (DFATALs and
aborts the compaction)

I verified the diagnosis by placing a random sleep just after making the
copy in PickRowSetsToCompact() and seeing
TabletServerDiskFailureTest.TestRandomOpSequence, which schedules many
compactions, fail consistently with the failed verification. Upon making
the fix, this passes.

This can lead to scheduling a fairly useless compaction. This patch
adds an API to RowSets to mark itself as compacted, and ensure that when
selecting rowsets to compact, check that rowset candidates have not
already been compacted.

To verify the solution, I've added a small test that schedules several
concurrent compactions. Upon adding the aforementioned randomized
delay and removing a couple sanity D/CHECKs (including the above
verification), I saw the test fail, leading to an unexpected number of
rows left in the tablet.

Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
---
M src/kudu/tablet/compaction-test.cc
M src/kudu/tablet/diskrowset.h
M src/kudu/tablet/memrowset.h
M src/kudu/tablet/mock-rowsets.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/tablet.cc
6 files changed, 155 insertions(+), 69 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/59/8859/4
--
To view, visit http://gerrit.cloudera.org:8080/8859
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4dab330d61facb18717f6faf179f9b94a9e55236
Gerrit-Change-Number: 8859
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong 
Gerrit-Reviewer: Andrew Wong 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] KUDU-2249 Prevent race condition between getSplits() method and TableRecordReader

[David Ribeiro Alves (Code Review)]

David Ribeiro Alves has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8921 )

Change subject: KUDU-2249 Prevent race condition between getSplits() method and 
TableRecordReader
..


Patch Set 3:

(3 comments)

thanks for addressing the comments. couple of nits and should be gtg

http://gerrit.cloudera.org:8080/#/c/8921/3//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/8921/3//COMMIT_MSG@7
PS3, Line 7: KUDU-2249 Prevent race condition between getSplits() method and 
TableRecordReader
nits:
- try to keep column widths under 80 lines (not mandatory but possible here for 
the most part)
- maybe a better title for this patch would be: "Avoid sharing the client 
between the InputFormat and RecordReader" (or something close to this) and the 
explain in the commit message that this is meant to avoid a race condition.


http://gerrit.cloudera.org:8080/#/c/8921/3//COMMIT_MSG@12
PS3, Line 12: Kud
typo


http://gerrit.cloudera.org:8080/#/c/8921/3/java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java
File 
java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java:

http://gerrit.cloudera.org:8080/#/c/8921/3/java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java@268
PS3, Line 268: getKuduClient
rename to newKuduClient() or buildKuduClient() so that this doesn't read like a 
plain getter



--
To view, visit http://gerrit.cloudera.org:8080/8921
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I24f45ee9253790c5348cabd0afe6c6a4b6d3f3d4
Gerrit-Change-Number: 8921
Gerrit-PatchSet: 3
Gerrit-Owner: Clemens Valiente 
Gerrit-Reviewer: Clemens Valiente 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Thu, 04 Jan 2018 18:52:17 +
Gerrit-HasComments: Yes

[kudu-CR](branch-1.5.x) Add support for clang from XCode 9 in macOS

[Alexey Serbin (Code Review)]

Alexey Serbin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8940 )

Change subject: Add support for clang from XCode 9 in macOS
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/8940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib636deee77471e60c02e90028a0e5cc0f303b45a
Gerrit-Change-Number: 8940
Gerrit-PatchSet: 1
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Thu, 04 Jan 2018 18:48:38 +
Gerrit-HasComments: No

[kudu-CR](branch-1.5.x) KUDU-2251: rowset size can overflow int in RowSetInfo

[David Ribeiro Alves (Code Review)]

David Ribeiro Alves has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8941 )

Change subject: KUDU-2251: rowset size can overflow int in RowSetInfo
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/8941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: comment
Gerrit-Change-Id: I598344e22fc1ecbb482bfe85ea3867ddf63963b4
Gerrit-Change-Number: 8941
Gerrit-PatchSet: 2
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 18:42:51 +
Gerrit-HasComments: No

[kudu-CR](branch-1.5.x) KUDU-2251: rowset size can overflow int in RowSetInfo

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8941 )

Change subject: KUDU-2251: rowset size can overflow int in RowSetInfo
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8941/2/src/kudu/tablet/rowset_info.h
File src/kudu/tablet/rowset_info.h:

http://gerrit.cloudera.org:8080/#/c/8941/2/src/kudu/tablet/rowset_info.h@86
PS2, Line 86:   // Cached version of rowset_->OnDiskDataSizeNoUndos().
> this is caching rs->OnDiskBaseDataSizeWithRedos() right?
No, on the 1.5.x branch it's called OnDiskDataSizeNoUndos.  It's been renamed 
on master.



--
To view, visit http://gerrit.cloudera.org:8080/8941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: comment
Gerrit-Change-Id: I598344e22fc1ecbb482bfe85ea3867ddf63963b4
Gerrit-Change-Number: 8941
Gerrit-PatchSet: 2
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 18:42:32 +
Gerrit-HasComments: Yes

[kudu-CR](branch-1.5.x) Add support for clang from XCode 9 in macOS

[David Ribeiro Alves (Code Review)]

David Ribeiro Alves has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8940 )

Change subject: Add support for clang from XCode 9 in macOS
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/8940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib636deee77471e60c02e90028a0e5cc0f303b45a
Gerrit-Change-Number: 8940
Gerrit-PatchSet: 1
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Thu, 04 Jan 2018 18:40:16 +
Gerrit-HasComments: No

[kudu-CR] [docs] Document how to recover from a majority failed tablet

[Will Berkeley (Code Review)]

Will Berkeley has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8402 )

Change subject: [docs] Document how to recover from a majority failed tablet
..


Patch Set 4:

Rendering at 
https://github.com/wdberkeley/kudu/blob/majorityrecoverydocs/docs/administration.adoc#tablet_majority_down_recovery


--
To view, visit http://gerrit.cloudera.org:8080/8402
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic6326f65d029a1cd75e487b16ce5be4baea2f215
Gerrit-Change-Number: 8402
Gerrit-PatchSet: 4
Gerrit-Owner: Will Berkeley 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Thu, 04 Jan 2018 18:37:09 +
Gerrit-HasComments: No

[kudu-CR] [docs] Document how to recover from a majority failed tablet

[Will Berkeley (Code Review)]

Hello Mike Percy, Jean-Daniel Cryans, Kudu Jenkins, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8402

to look at the new patch set (#5).

Change subject: [docs] Document how to recover from a majority failed tablet
..

[docs] Document how to recover from a majority failed tablet

This adds some docs on how to recover when a tablet can no longer find
a majority due to the permanent failure of replicas.

I tested this procedure by failing tablets in various ways:
- deleting important bits like cmeta or tablet metadata
- deleting entire data dirs
- tombstoning 2/3 replicas (and disabling tombstoned voting)
and I was always able to recover using these instructions.

Change-Id: Ic6326f65d029a1cd75e487b16ce5be4baea2f215
---
M docs/administration.adoc
1 file changed, 59 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/02/8402/5
--
To view, visit http://gerrit.cloudera.org:8080/8402
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic6326f65d029a1cd75e487b16ce5be4baea2f215
Gerrit-Change-Number: 8402
Gerrit-PatchSet: 5
Gerrit-Owner: Will Berkeley 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 

[kudu-CR](branch-1.5.x) KUDU-2251: rowset size can overflow int in RowSetInfo

[David Ribeiro Alves (Code Review)]

David Ribeiro Alves has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8941 )

Change subject: KUDU-2251: rowset size can overflow int in RowSetInfo
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8941/2/src/kudu/tablet/rowset_info.h
File src/kudu/tablet/rowset_info.h:

http://gerrit.cloudera.org:8080/#/c/8941/2/src/kudu/tablet/rowset_info.h@86
PS2, Line 86:   // Cached version of rowset_->OnDiskDataSizeNoUndos().
this is caching rs->OnDiskBaseDataSizeWithRedos() right?



--
To view, visit http://gerrit.cloudera.org:8080/8941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: comment
Gerrit-Change-Id: I598344e22fc1ecbb482bfe85ea3867ddf63963b4
Gerrit-Change-Number: 8941
Gerrit-PatchSet: 2
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Thu, 04 Jan 2018 18:31:49 +
Gerrit-HasComments: Yes

[kudu-CR] [docs] Document how to recover from a majority failed tablet

[Will Berkeley (Code Review)]

Will Berkeley has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8402 )

Change subject: [docs] Document how to recover from a majority failed tablet
..


Patch Set 4:

(7 comments)

http://gerrit.cloudera.org:8080/#/c/8402/4/docs/administration.adoc
File docs/administration.adoc:

http://gerrit.cloudera.org:8080/#/c/8402/4/docs/administration.adoc@709
PS4, Line 709: Reviving a tablet that's lost a majority of replicas
> how about: Bringing a tablet that's lost a majority of replicas back online
Done


http://gerrit.cloudera.org:8080/#/c/8402/4/docs/administration.adoc@711
PS4, Line 711: If a tablet has permanently lost a majority of its replicas, it 
cannot recover
> It is critical to emphasize that in a majority-lost scenario, permanent dat
Done


http://gerrit.cloudera.org:8080/#/c/8402/4/docs/administration.adoc@723
PS4, Line 723:   638a20403e3e4ae3b55d4d07d920e6de (tserver-00:7150): RUNNING 
[LEADER]
> This is kind of a cool scenario but this whole thing only works if the lead
The procedure works if the leader doesn't survive, but yes the chance of data 
loss is much higher then.


http://gerrit.cloudera.org:8080/#/c/8402/4/docs/administration.adoc@760
PS4, Line 760: $ kudu remote_replica delete tserver-01:7150 
e822cab6c0584bc0858219d1539a17e6 "delete failed replica"
> this is not actually required; the master should do it automatically once t
OK, if you're sure about this. I had a couple of situations in my testing where 
I had to do the deletion manually, but they were mock situations that probably 
should be treated as disk failure if they actually happened, e.g. deleting 
consensus metadata files.


http://gerrit.cloudera.org:8080/#/c/8402/4/docs/administration.adoc@767
PS4, Line 767: [source,bash]
 : 
 : $ kudu remote_replica unsafe_change_config  
   ...
 : 
> I found this confusing. It seems like a command, I was trying to figure out
Done


http://gerrit.cloudera.org:8080/#/c/8402/4/docs/administration.adoc@775
PS4, Line 775: [source,bash]
> If you are going to put this in, at least mark it with a label like "Exampl
Removed


http://gerrit.cloudera.org:8080/#/c/8402/4/docs/administration.adoc@777
PS4, Line 777: $ kudu remote_replica unsafe_change_config tserver-00:7150 
e822cab6c0584bc0858219d1539a17e6 638a20403e3e4ae3b55d4d07d920e6de
> Because having a long UUID for tablet_id and UUID for tablet server id can
Done



--
To view, visit http://gerrit.cloudera.org:8080/8402
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic6326f65d029a1cd75e487b16ce5be4baea2f215
Gerrit-Change-Number: 8402
Gerrit-PatchSet: 4
Gerrit-Owner: Will Berkeley 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Reviewer: Will Berkeley 
Gerrit-Comment-Date: Thu, 04 Jan 2018 18:31:26 +
Gerrit-HasComments: Yes

[kudu-CR](branch-1.5.x) KUDU-2251: rowset size can overflow int in RowSetInfo

[Dan Burkert (Code Review)]

Hello Jean-Daniel Cryans, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8941

to look at the new patch set (#2).

Change subject: KUDU-2251: rowset size can overflow int in RowSetInfo
..

KUDU-2251: rowset size can overflow int in RowSetInfo

This overflow causes a CHECK failure from rowset compaction planning in
tablets with rowsets with more than 2GiB of REDO deltafiles:

*** SIGABRT (@0x3ce7614) received by PID 30228 (TID 0x7fbb52a5e700) from 
PID 30228; stack trace: ***
@ 0x7fbb977cb100 (unknown)
@ 0x7fbb95a985f7 __GI_raise
@ 0x7fbb95a99ce8 __GI_abort
@  0x1af56d9 (unknown)
@   0x8baf3d google::LogMessage::Fail()
@   0x8bce93 google::LogMessage::SendToLog()
@   0x8baa99 google::LogMessage::Flush()
@   0x8bd81f google::LogMessageFatal::~LogMessageFatal()
@   0x9f71d6 kudu::tablet::RowSetInfo::CollectOrdered()
@   0x9d42d9 
kudu::tablet::BudgetedCompactionPolicy::SetupKnapsackInput()
@   0x9d5a3a kudu::tablet::BudgetedCompactionPolicy::PickRowSets()
@   0x98e28f kudu::tablet::Tablet::UpdateCompactionStats()
@   0x9aff08 kudu::tablet::CompactRowSetsOp::UpdateStats()
@  0x1ae02b5 kudu::MaintenanceManager::FindBestOp()
@  0x1ae2bce kudu::MaintenanceManager::RunSchedulerThread()
@  0x1b27eda kudu::Thread::SuperviseThread()
@ 0x7fbb977c3dc5 start_thread
@ 0x7fbb95b5921d __clone
@0x0 (unknown)

Change-Id: I598344e22fc1ecbb482bfe85ea3867ddf63963b4
---
M src/kudu/tablet/rowset_info.h
1 file changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/8941/2
--
To view, visit http://gerrit.cloudera.org:8080/8941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I598344e22fc1ecbb482bfe85ea3867ddf63963b4
Gerrit-Change-Number: 8941
Gerrit-PatchSet: 2
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR](branch-1.5.x) Add support for clang from XCode 9 in macOS

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8940 )

Change subject: Add support for clang from XCode 9 in macOS
..


Patch Set 1:

This isn't a strictly necessary backport, but it does make it a bit easier for 
those of us on macOS to work on branch-1.5


--
To view, visit http://gerrit.cloudera.org:8080/8940
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib636deee77471e60c02e90028a0e5cc0f303b45a
Gerrit-Change-Number: 8940
Gerrit-PatchSet: 1
Gerrit-Owner: Dan Burkert 
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Thu, 04 Jan 2018 18:20:03 +
Gerrit-HasComments: No

[kudu-CR](branch-1.5.x) KUDU-2251: rowset size can overflow int in RowSetInfo

[Dan Burkert (Code Review)]

Dan Burkert has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/8941


Change subject: KUDU-2251: rowset size can overflow int in RowSetInfo
..

KUDU-2251: rowset size can overflow int in RowSetInfo

This overflow causes a CHECK failure from rowset compaction planning in
tablets with rowsets with more than 2GiB of REDO deltafiles:

Change-Id: I598344e22fc1ecbb482bfe85ea3867ddf63963b4
---
M src/kudu/tablet/rowset_info.h
1 file changed, 3 insertions(+), 3 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/8941/1
--
To view, visit http://gerrit.cloudera.org:8080/8941
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: branch-1.5.x
Gerrit-MessageType: newchange
Gerrit-Change-Id: I598344e22fc1ecbb482bfe85ea3867ddf63963b4
Gerrit-Change-Number: 8941
Gerrit-PatchSet: 1
Gerrit-Owner: Dan Burkert 

[kudu-CR] docs: fix tablet moving in 1.5.0 release notes

[Jean-Daniel Cryans (Code Review)]

Jean-Daniel Cryans has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/8939 )

Change subject: docs: fix tablet moving in 1.5.0 release notes
..

docs: fix tablet moving in 1.5.0 release notes

Release notes said the new command to move replicas between
TServers was "kudu tablet move" which is not correct. The actual
command is "kudu tablet change_config move_replica"

Change-Id: I26dbefeb5f8b19b6eef823caeb5d255a3c6523f5
Reviewed-on: http://gerrit.cloudera.org:8080/8939
Tested-by: Kudu Jenkins
Reviewed-by: Jean-Daniel Cryans 
---
M docs/prior_release_notes.adoc
1 file changed, 7 insertions(+), 6 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Jean-Daniel Cryans: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/8939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I26dbefeb5f8b19b6eef823caeb5d255a3c6523f5
Gerrit-Change-Number: 8939
Gerrit-PatchSet: 2
Gerrit-Owner: Attila Bukor 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] docs: fix tablet moving in 1.5.0 release notes

[Jean-Daniel Cryans (Code Review)]

Jean-Daniel Cryans has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8939 )

Change subject: docs: fix tablet moving in 1.5.0 release notes
..


Patch Set 1: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/8939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I26dbefeb5f8b19b6eef823caeb5d255a3c6523f5
Gerrit-Change-Number: 8939
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Bukor 
Gerrit-Reviewer: Jean-Daniel Cryans 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Thu, 04 Jan 2018 16:57:49 +
Gerrit-HasComments: No

[kudu-CR] docs: fix tablet moving in 1.5.0 release notes

[Attila Bukor (Code Review)]

Attila Bukor has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/8939


Change subject: docs: fix tablet moving in 1.5.0 release notes
..

docs: fix tablet moving in 1.5.0 release notes

Release notes said the new command to move replicas between
TServers was "kudu tablet move" which is not correct. The actual
command is "kudu tablet change_config move_replica"

Change-Id: I26dbefeb5f8b19b6eef823caeb5d255a3c6523f5
---
M docs/prior_release_notes.adoc
1 file changed, 7 insertions(+), 6 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/39/8939/1
--
To view, visit http://gerrit.cloudera.org:8080/8939
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I26dbefeb5f8b19b6eef823caeb5d255a3c6523f5
Gerrit-Change-Number: 8939
Gerrit-PatchSet: 1
Gerrit-Owner: Attila Bukor 

[kudu-CR] KUDU-2249 Prevent race condition between getSplits() method and TableRecordReader

[Clemens Valiente (Code Review)]

Clemens Valiente has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/8921 )

Change subject: KUDU-2249 Prevent race condition between getSplits() method and 
TableRecordReader
..


Patch Set 2:

> Patch Set 1:
>
> (1 comment)

Hi David, I clarified the behaviour of MR applications and number of open 
clients in the class documentation and motivation for the changes in the commit 
message.


--
To view, visit http://gerrit.cloudera.org:8080/8921
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I24f45ee9253790c5348cabd0afe6c6a4b6d3f3d4
Gerrit-Change-Number: 8921
Gerrit-PatchSet: 2
Gerrit-Owner: Clemens Valiente 
Gerrit-Reviewer: Clemens Valiente 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Thu, 04 Jan 2018 09:41:51 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2249 Prevent race condition between getSplits() method and TableRecordReader

[Clemens Valiente (Code Review)]

Hello David Ribeiro Alves, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8921

to look at the new patch set (#3).

Change subject: KUDU-2249 Prevent race condition between getSplits() method and 
TableRecordReader
..

KUDU-2249 Prevent race condition between getSplits() method and 
TableRecordReader

Both are sharing the same client and shut it down after use. In some scenarios 
the client might
still be accessed after that and throwing an error.
So the TableRecordReader gets its own client with this commit. This increases 
the number of
opened Kud clients by a MR application at most by one (The one that was shared 
by getSplits()
with a TableRecordReader)
Also clarified the behaviour of MR applications and how many open Kudu clients
one might have to expect in total.

Change-Id: I24f45ee9253790c5348cabd0afe6c6a4b6d3f3d4
---
M 
java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java
1 file changed, 45 insertions(+), 21 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/21/8921/3
--
To view, visit http://gerrit.cloudera.org:8080/8921
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I24f45ee9253790c5348cabd0afe6c6a4b6d3f3d4
Gerrit-Change-Number: 8921
Gerrit-PatchSet: 3
Gerrit-Owner: Clemens Valiente 
Gerrit-Reviewer: Clemens Valiente 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] KUDU-2249 Prevent a possible race conditions between the getSplits() method and TableRecordReader

[Clemens Valiente (Code Review)]

Clemens Valiente has abandoned this change. ( 
http://gerrit.cloudera.org:8080/8938 )

Change subject: KUDU-2249 Prevent a possible race conditions between the 
getSplits() method and TableRecordReader
..


Abandoned

duplicate
--
To view, visit http://gerrit.cloudera.org:8080/8938
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I833902b7ae813c1b4ae0505e20f206edc9875719
Gerrit-Change-Number: 8938
Gerrit-PatchSet: 1
Gerrit-Owner: Clemens Valiente 
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] KUDU-2249 Prevent race condition between getSplits() method and TableRecordReader

[Clemens Valiente (Code Review)]

Hello David Ribeiro Alves, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/8921

to look at the new patch set (#2).

Change subject: KUDU-2249 Prevent race condition between getSplits() method and 
TableRecordReader
..

KUDU-2249 Prevent race condition between getSplits() method and 
TableRecordReader

Both are sharing the same client and shut it down after use. In some scenarios 
the client might
still be accessed after that and throwing an error.
So the TableRecordReader gets its own client with this commit.
Also clarified the behaviour of MR applications and how many open Kudu clients
one might have to expect.

Change-Id: I24f45ee9253790c5348cabd0afe6c6a4b6d3f3d4
---
M 
java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java
1 file changed, 45 insertions(+), 21 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/21/8921/2
--
To view, visit http://gerrit.cloudera.org:8080/8921
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I24f45ee9253790c5348cabd0afe6c6a4b6d3f3d4
Gerrit-Change-Number: 8921
Gerrit-PatchSet: 2
Gerrit-Owner: Clemens Valiente 
Gerrit-Reviewer: Clemens Valiente 
Gerrit-Reviewer: David Ribeiro Alves 
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] KUDU-2249 Prevent a possible race conditions between the getSplits() method and TableRecordReader class

[Clemens Valiente (Code Review)]

Clemens Valiente has abandoned this change. ( 
http://gerrit.cloudera.org:8080/8937 )

Change subject: KUDU-2249 Prevent a possible race conditions between the 
getSplits() method and TableRecordReader class
..


Abandoned
--
To view, visit http://gerrit.cloudera.org:8080/8937
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I6508a02edaba76f3234e64e2a44b4846dc7e818b
Gerrit-Change-Number: 8937
Gerrit-PatchSet: 1
Gerrit-Owner: Clemens Valiente 
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] KUDU-2249 Prevent a possible race conditions between the getSplits() method and TableRecordReader

[Clemens Valiente (Code Review)]

Clemens Valiente has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/8938


Change subject: KUDU-2249 Prevent a possible race conditions between the 
getSplits() method and TableRecordReader
..

KUDU-2249 Prevent a possible race conditions between the getSplits() method and 
TableRecordReader

Both are sharing the same client and shut it down after use. In some scenarios 
the client might
still be accessed after that and throwing an error. Also clarified the 
behaviour of MR
applications and how many open Kudu clients one might have to expect.

Change-Id: I833902b7ae813c1b4ae0505e20f206edc9875719
---
M 
java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java
1 file changed, 18 insertions(+), 3 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/38/8938/1
--
To view, visit http://gerrit.cloudera.org:8080/8938
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I833902b7ae813c1b4ae0505e20f206edc9875719
Gerrit-Change-Number: 8938
Gerrit-PatchSet: 1
Gerrit-Owner: Clemens Valiente 

[kudu-CR] KUDU-2249 Prevent a possible race conditions between the getSplits() method and TableRecordReader class

[Clemens Valiente (Code Review)]

Clemens Valiente has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/8937


Change subject: KUDU-2249 Prevent a possible race conditions between the 
getSplits() method and TableRecordReader class
..

KUDU-2249 Prevent a possible race conditions between the getSplits() method and 
TableRecordReader class

Both are sharing the same client and shut it down after use. In some scenarios 
the client might still be
accessed after that and throwing an error. Also clarified the behaviour of MR 
applications and how many
open Kudu clients one might have to expect.

Change-Id: I6508a02edaba76f3234e64e2a44b4846dc7e818b
---
M 
java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java
1 file changed, 18 insertions(+), 3 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/37/8937/1
--
To view, visit http://gerrit.cloudera.org:8080/8937
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6508a02edaba76f3234e64e2a44b4846dc7e818b
Gerrit-Change-Number: 8937
Gerrit-PatchSet: 1
Gerrit-Owner: Clemens Valiente