[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and column name and type) between Kudu and HMS, and
reports inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Reviewed-on: http://gerrit.cloudera.org:8080/10217
Reviewed-by: Dan Burkert 
Tested-by: Kudu Jenkins
---
M src/kudu/client/client.h
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
4 files changed, 406 insertions(+), 46 deletions(-)

Approvals:
  Dan Burkert: Looks good to me, approved
  Kudu Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 15
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 14: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 14
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Tue, 26 Jun 2018 00:12:35 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 14:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/10217/13/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/13/src/kudu/tools/tool_action_hms.cc@343
PS13, Line 343: "KuduMasterAddresses"});
> These don't appear to be printed to the console.
Done


http://gerrit.cloudera.org:8080/#/c/10217/13/src/kudu/tools/tool_action_hms.cc@372
PS13, Line 372:   shared_ptr kudu_table;
> Perhaps the tool could output a table for legacy tables, and if there are a
Done


http://gerrit.cloudera.org:8080/#/c/10217/13/src/kudu/tools/tool_action_hms.cc@375
PS13, Line 375: unsynced_tables_map[kudu_table->id()].first = 
kudu_table;
> Is it meant that this exact same status will get added multiple times?
Done



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 14
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Tue, 26 Jun 2018 00:04:59 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#14).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and column name and type) between Kudu and HMS, and
reports inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/client/client.h
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
4 files changed, 406 insertions(+), 46 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/14
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 14
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 13:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/10217/13/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/13/src/kudu/tools/tool_action_hms.cc@343
PS13, Line 343:   std::vector error_messages;
These don't appear to be printed to the console.


http://gerrit.cloudera.org:8080/#/c/10217/13/src/kudu/tools/tool_action_hms.cc@372
PS13, Line 372:   LOG(WARNING) << Substitute("Found legacy table $0.$1 in 
the Hive Metastore, "
Perhaps the tool could output a table for legacy tables, and if there are any 
then you could additionally print this hint to use the upgrade tool once.


http://gerrit.cloudera.org:8080/#/c/10217/13/src/kudu/tools/tool_action_hms.cc@375
PS13, Line 375:   error_messages.push_back(Status::IllegalState("Legacy 
table found"));
Is it meant that this exact same status will get added multiple times?



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 13
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Mon, 25 Jun 2018 21:43:36 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 13: Verified+1

Unrelated flaky test.


--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 13
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Mon, 25 Jun 2018 04:03:07 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has removed a vote on this change.

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 13
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 13:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc@394
PS10, Line 394:
> Ah, I missed that.  However I think this should also cause the tool to fail
Done


http://gerrit.cloudera.org:8080/#/c/10217/12/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/12/src/kudu/tools/tool_action_hms.cc@372
PS12, Line 372: 0.$1 in
> upgrade
Done


http://gerrit.cloudera.org:8080/#/c/10217/12/src/kudu/tools/tool_action_hms.cc@372
PS12, Line 372:   LOG(WARNING) << Substitute("Found legacy table $0.$1 in 
the Hive Metastore, "
> add to the end the exact kudu tool invocation command
Done


http://gerrit.cloudera.org:8080/#/c/10217/12/src/kudu/tools/tool_action_hms.cc@404
PS12, Line 404:   unsynced_tables_map, cout),
> This failure should cause the tool to return a non-0 exit status.  I imagin
Done



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 13
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Sat, 23 Jun 2018 00:50:40 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#13).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and column name and type) between Kudu and HMS, and
reports inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/client/client.h
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
4 files changed, 382 insertions(+), 46 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/13
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 13
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 12:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc@394
PS10, Line 394:   if (unsynced_tables_map.empty()) {
> Hmm, right. L392 is logging a warning. You think it is not sufficient?
Ah, I missed that.  However I think this should also cause the tool to fail 
(return non-0 exist status), but perhaps not immediately if avoidable.  Check 
out how ksck aggregates error messages.


http://gerrit.cloudera.org:8080/#/c/10217/12/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/12/src/kudu/tools/tool_action_hms.cc@372
PS12, Line 372:  "try to use metadata updrade 
tool first",
add to the end the exact kudu tool invocation command


http://gerrit.cloudera.org:8080/#/c/10217/12/src/kudu/tools/tool_action_hms.cc@372
PS12, Line 372: updrade
upgrade


http://gerrit.cloudera.org:8080/#/c/10217/12/src/kudu/tools/tool_action_hms.cc@404
PS12, Line 404:   return Status::OK();
This failure should cause the tool to return a non-0 exit status.  I imagine 
that requires returning a non-OK exist status, right?



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 12
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Fri, 22 Jun 2018 18:57:21 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#12).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and column name and type) between Kudu and HMS, and
reports inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/client/client.h
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
4 files changed, 368 insertions(+), 45 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/12
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 12
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 12:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/kudu-tool-test.cc@2188
PS11, Line 2188:   // 2. Create tables in Kudu and the HMS with inconsistent 
metadata. Then validate the tool
> Should this cover the case of master address not matching?
Done


http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/kudu-tool-test.cc@2195
PS11, Line 2195: // but different table ID.
> It's pretty difficult to see what each of the a/b/c/d/e tables are testing
Done


http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc@394
PS10, Line 394:   if (unsynced_tables_map.empty()) {
> I think it's still worth warning about legacy tables, otherwise the admin m
Hmm, right. L392 is logging a warning. You think it is not sufficient?


http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/tool_action_hms.cc@368
PS11, Line 368:   }
> Might be worth seeing if you can remove some of the nested if/else levels h
Done


http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/tool_action_hms.cc@396
PS11, Line 396: return Status::OK();
> Sorry my suggestion before on R10 was bad.  This should be either
Done



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 12
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Fri, 22 Jun 2018 01:29:29 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 11:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/kudu-tool-test.cc@2188
PS11, Line 2188:   // 2. Create a table in Kudu and in the HMS with the same 
table name
Should this cover the case of master address not matching?


http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/kudu-tool-test.cc@2195
PS11, Line 2195: ASSERT_OK(CreateKuduTable(kudu_client, "default.b"));
It's pretty difficult to see what each of the a/b/c/d/e tables are testing 
specifically.  I'd suggest moving the comment from the block to right before 
the initialization of each, or perhaps name them to indicate the case.


http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc@394
PS10, Line 394:   }
> No, it doesn't. I was thinking add it here so later the fix tool can take c
I think it's still worth warning about legacy tables, otherwise the admin might 
not know to run the upgrade tool?  Perhaps you could just output a list of 
legacy tables, and hint that the upgrade tool should be used.


http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/tool_action_hms.cc@368
PS11, Line 368:   if (!s.ok() && !s.IsNotFound()) {
Might be worth seeing if you can remove some of the nested if/else levels here. 
 Perhaps:

if (s.ok() && !isSynced(master_addresses_str, kudu_table, hms_table)) {
...
} else if (s.IsNotFound()) {
   ...
} else {
// All errors other than IsNotFound are fatal.
RETURN_NOT_OK(s);
}


http://gerrit.cloudera.org:8080/#/c/10217/11/src/kudu/tools/tool_action_hms.cc@396
PS11, Line 396:   // 3. If any Kudu tables are not present in the HMS, consider 
it as an out of sync
Sorry my suggestion before on R10 was bad.  This should be either

If any Kudu tables are not present in the HMS, consider _them as out of sync 
tables_.

or

If any Kudu _table is not present_ in the HMS, consider it as an out of sync 
table.



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 11
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Wed, 20 Jun 2018 17:40:59 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 11:

(10 comments)

http://gerrit.cloudera.org:8080/#/c/10217/8//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/10217/8//COMMIT_MSG@11
PS8, Line 11: column
> column
Done


http://gerrit.cloudera.org:8080/#/c/10217/8//COMMIT_MSG@12
PS8, Line 12: report
> reports
Done


http://gerrit.cloudera.org:8080/#/c/10217/10//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/10217/10//COMMIT_MSG@11
PS10, Line 11: column
> column
Done


http://gerrit.cloudera.org:8080/#/c/10217/7/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/10217/7/src/kudu/tools/kudu-tool-test.cc@1957
PS7, Line 1957:
> warning: local copy 'kudu_table_name' of the variable 'table_name' is never
Done


http://gerrit.cloudera.org:8080/#/c/10217/8/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/10217/8/src/kudu/tools/kudu-tool-test.cc@2177
PS8, Line 2177:
> schema
Done


http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc@130
PS10, Line 130: Status AlterLegacyKuduTables(KuduClient* kudu_client,
> It's kind of confusing to pass a const ref to a mutable pointer, perhaps ju
Done


http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc@376
PS10, Line 376:   } else {
> Consider just doing an OpenTable instead of TableExists + OpenTable, and ch
Done


http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc@394
PS10, Line 394:   }
> Do legacy tables have a table ID param?
No, it doesn't. I was thinking add it here so later the fix tool can take care 
of it. But after more thoughts, I think the best way to fix this is to run 
upgrade tool. So removed the logic here.


http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc@399
PS10, Line 399: ables(
> 'are not present'
Done


http://gerrit.cloudera.org:8080/#/c/10217/10/src/kudu/tools/tool_action_hms.cc@403
PS10, Line 403:   shared_ptr kudu_table;
> move hms_tables, if this is the last use of it.
Done



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 11
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Tue, 19 Jun 2018 22:35:35 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#11).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and column name and type) between Kudu and HMS, and
reports inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/client/client.h
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
4 files changed, 370 insertions(+), 45 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/11
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 11
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#9).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and colnum name and type) between Kudu and HMS, and
report inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
3 files changed, 367 insertions(+), 40 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/9
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 9
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#8).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and colnum name and type) between Kudu and HMS, and
report inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
3 files changed, 355 insertions(+), 40 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/8
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 8
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#7).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and colnum name and type) between Kudu and HMS, and
report inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
3 files changed, 276 insertions(+), 40 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/7
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 7
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 7:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/kudu-tool-test.cc@1919
PS6, Line 1919: Status CreateLegacyHmsTable(HmsClient* client,
> Might be more straightforward to split this into two methods, since there i
Done


http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/tool_action_hms.cc@271
PS6, Line 271:
> Might be easier to make a copy of 'hive_table', then call PopulateTable wit
Done


http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/tool_action_hms.cc@293
PS6, Line 293: kudu_table
> should this be 'unsynchronized tables:'
Done


http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/tool_action_hms.cc@348
PS6, Line 348:
> remove period to be consistent
Done



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 7
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Sat, 19 May 2018 00:55:39 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Dan Burkert (Code Review)]

Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 6:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/kudu-tool-test.cc@1919
PS6, Line 1919: Status CreateHmsTable(bool legacy,
Might be more straightforward to split this into two methods, since there isn't 
a lot of overlap and the arguments don't apply to all cases.


http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/tool_action_hms.cc@271
PS6, Line 271:   bool schema_synced = true;
Might be easier to make a copy of 'hive_table', then call PopulateTable with 
it, and compare it back to itself.  If the tables are synced then they should 
be equal.  Makes it a bit more resilient to changes to PopulateTable (like if 
we add additional fields or metadata to the entry).


http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/tool_action_hms.cc@293
PS6, Line 293: sync table
should this be 'unsynchronized tables:'


http://gerrit.cloudera.org:8080/#/c/10217/6/src/kudu/tools/tool_action_hms.cc@348
PS6, Line 348: .
remove period to be consistent



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 6
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Fri, 18 May 2018 23:10:13 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#6).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and colnum name and type) between Kudu and HMS, and
report inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
4 files changed, 286 insertions(+), 49 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/6
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 6
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 5:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/10217/4/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/10217/4/src/kudu/tools/kudu-tool-test.cc@2090
PS4, Line 2090:   hive::EnvironmentContext env_ctx;
> error: no member named 'DropTableWithContext' in 'kudu::hms::HmsClient' [cl
Done


http://gerrit.cloudera.org:8080/#/c/10217/4/src/kudu/tools/kudu-tool-test.cc@2091
PS4, Line 2091:   env_ctx.__set_properties({ 
make_pair(HmsClient::kKuduTableIdKey, hms_table_id) });
> error: no member named 'DropTableWithContext' in 'kudu::hms::HmsClient' [cl
Done


http://gerrit.cloudera.org:8080/#/c/10217/4/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/4/src/kudu/tools/tool_action_hms.cc@60
PS4, Line 60: using std::ostream;
> warning: using decl 'make_pair' is unused [misc-unused-using-decls]
Done



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 5
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Fri, 04 May 2018 00:32:58 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#5).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and colnum name and type) between Kudu and HMS, and
report inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/hms/hms_catalog-test.cc
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
5 files changed, 304 insertions(+), 36 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/5
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 5
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#4).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and colnum name and type) between Kudu and HMS, and
report inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/hms/hms_catalog-test.cc
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
5 files changed, 302 insertions(+), 35 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/4
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 4
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 3: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 3
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Thu, 26 Apr 2018 20:16:46 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 3:

Flaky tests due to clock synchronization issue.


--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 3
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Thu, 26 Apr 2018 20:16:41 +
Gerrit-HasComments: No

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#3).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and colnum name and type) between Kudu and HMS, and
report inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/hms/hms_catalog-test.cc
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
5 files changed, 302 insertions(+), 35 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/3
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 3
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/10217/2/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/2/src/kudu/tools/tool_action_hms.cc@59
PS2, Line 59: using std::cerr;
> warning: using decl 'cerr' is unused [misc-unused-using-decls]
Done


http://gerrit.cloudera.org:8080/#/c/10217/2/src/kudu/tools/tool_action_hms.cc@168
PS2, Line 168:   for (const pair& entry : hms_tables_map) {
> warning: the type of the loop variable 'entry' is different from the one re
Done



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 2
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Thu, 26 Apr 2018 18:35:16 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hello Tidy Bot, Dan Burkert, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10217

to look at the new patch set (#2).

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and colnum name and type) between Kudu and HMS, and
report inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/hms/hms_catalog-test.cc
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
5 files changed, 302 insertions(+), 35 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/2
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 2
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10217 )

Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/10217/1/src/kudu/hms/hms_catalog.h
File src/kudu/hms/hms_catalog.h:

http://gerrit.cloudera.org:8080/#/c/10217/1/src/kudu/hms/hms_catalog.h@35
PS1, Line 35: class ColumnSchema;
> warning: no definition found for 'ColumnSchema', but a definition with the
Done


http://gerrit.cloudera.org:8080/#/c/10217/1/src/kudu/tools/tool_action_hms.cc
File src/kudu/tools/tool_action_hms.cc:

http://gerrit.cloudera.org:8080/#/c/10217/1/src/kudu/tools/tool_action_hms.cc@271
PS1, Line 271:   kudu_client->OpenTable(table_name, _table);
> error: cannot initialize a parameter of type 'sp::shared_ptr *'
Done


http://gerrit.cloudera.org:8080/#/c/10217/1/src/kudu/tools/tool_action_hms.cc@290
PS1, Line 290:   RETURN_NOT_OK(kudu_client->OpenTable(table_name, 
_table));
> error: cannot initialize a parameter of type 'sp::shared_ptr *'
Done



--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 1
Gerrit-Owner: Hao Hao 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Comment-Date: Thu, 26 Apr 2018 06:13:22 +
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2191: HMS Metadata Consistency Check Tool

[Hao Hao (Code Review)]

Hao Hao has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/10217


Change subject: KUDU-2191: HMS Metadata Consistency Check Tool
..

KUDU-2191: HMS Metadata Consistency Check Tool

This commit introduces a metadata consistency check CLI tool for Hive
Metastore integration. It checks metadata (table ID, table name, Kudu
master addresses, and colnum name and type) between Kudu and HMS, and
report inconsistent metadata to the user if any. It adds test case
using external mini cluster to ensure the tool works as expected.

Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
---
M src/kudu/hms/hms_catalog-test.cc
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_catalog.h
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_hms.cc
5 files changed, 299 insertions(+), 31 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/10217/1
--
To view, visit http://gerrit.cloudera.org:8080/10217
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3495e9c755571b85beaecca849f83457b592ee90
Gerrit-Change-Number: 10217
Gerrit-PatchSet: 1
Gerrit-Owner: Hao Hao