[kudu-CR] KUDU-2849 Docker image for python kudu client
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14000 ) Change subject: KUDU-2849 Docker image for python kudu client .. KUDU-2849 Docker image for python kudu client A base image that has all the development libraries required to run Python Kudu Client. Building Python Kudu Client Image: Set Env export TARGETS=kudu-python ./docker/docker-build.sh Run docker kudu python client: docker run --name kudu-python -it apache/kudu:kudu-python-latest Change-Id: Ia7cab8ffe114c73752c261f27ebc7a58e4b57a6e Reviewed-on: http://gerrit.cloudera.org:8080/14000 Tested-by: Kudu Jenkins Reviewed-by: Grant Henke --- M docker/Dockerfile M docker/README.adoc 2 files changed, 40 insertions(+), 0 deletions(-) Approvals: Kudu Jenkins: Verified Grant Henke: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/14000 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia7cab8ffe114c73752c261f27ebc7a58e4b57a6e Gerrit-Change-Number: 14000 Gerrit-PatchSet: 7 Gerrit-Owner: Sandish Kumar HN Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Sandish Kumar HN
[kudu-CR] KUDU-2849 Docker image for python kudu client
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/14000 )
Change subject: KUDU-2849 Docker image for python kudu client
..
Patch Set 6: Code-Review+2
I verified this by building and running the following test script in a
quickstart environment:
docker run -it --rm --network="docker_default" apache/kudu:kudu-python-latest
import kudu
from kudu.client import Partitioning
from datetime import datetime
client =
kudu.connect(['kudu-master-1','kudu-master-2','kudu-master-3'],['7051','7151','7251'])
client.list_tables()
table = client.table('random_user')
scanner = table.scanner()
result = scanner.open().read_all_tuples()
print(result[:1])
exit()
--
To view, visit http://gerrit.cloudera.org:8080/14000
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia7cab8ffe114c73752c261f27ebc7a58e4b57a6e
Gerrit-Change-Number: 14000
Gerrit-PatchSet: 6
Gerrit-Owner: Sandish Kumar HN
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Sandish Kumar HN
Gerrit-Comment-Date: Wed, 07 Aug 2019 23:36:01 +
Gerrit-HasComments: No
[kudu-CR] POC: Disable Sentry related tests
Grant Henke has removed a vote on this change. Change subject: POC: Disable Sentry related tests .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/14020 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I7b99b0de221f6d4acfb427f830cd3344c8d0a10b Gerrit-Change-Number: 14020 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] Prepare for upgrading to Hive 3
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/14018 ) Change subject: Prepare for upgrading to Hive 3 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/14018 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If43ae2330b3d99374c68bae313a3f8bc070f9c69 Gerrit-Change-Number: 14018 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 07 Aug 2019 21:46:19 + Gerrit-HasComments: No
[kudu-CR] Prepare for upgrading to Hive 3
Grant Henke has removed a vote on this change. Change subject: Prepare for upgrading to Hive 3 .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/14018 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: If43ae2330b3d99374c68bae313a3f8bc070f9c69 Gerrit-Change-Number: 14018 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] Prepare for upgrading to Hive 3
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/14018 )
Change subject: Prepare for upgrading to Hive 3
..
Patch Set 1:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/14018/1//COMMIT_MSG
Commit Message:
PS1:
> Just curious: why are you wrapping your commit messages so aggressively? Th
No specific reason, the edit window was recently adjusted. Switched back.
http://gerrit.cloudera.org:8080/#/c/14018/1//COMMIT_MSG@13
PS1, Line 13: initialized
> initialize
Done
http://gerrit.cloudera.org:8080/#/c/14018/1//COMMIT_MSG@17
PS1, Line 17: MiniHMS
> Nit: MiniHms (or change MiniHms above to MiniHMS).
Done
http://gerrit.cloudera.org:8080/#/c/14018/1/src/kudu/hms/hms_catalog.cc
File src/kudu/hms/hms_catalog.cc:
http://gerrit.cloudera.org:8080/#/c/14018/1/src/kudu/hms/hms_catalog.cc@233
PS1, Line 233: // NOTE: LIKE filters are used instead of = filters due to
HIVE-21614
> Does this have a tangible impact on performance (on a non-Derby HMS)? Any w
I did a bit of research and it seems that the effect of LIKE vs = should be
near negligible when special characters like % are not used. In the case of
Kudu, this is always the case.
http://gerrit.cloudera.org:8080/#/c/14018/1/src/kudu/hms/mini_hms.cc
File src/kudu/hms/mini_hms.cc:
http://gerrit.cloudera.org:8080/#/c/14018/1/src/kudu/hms/mini_hms.cc@158
PS1, Line 158:
: // Set HADOOP_OS_TYPE=Linux due to HADOOP-8719.
: // Remove after HADOOP-15966 is available (Hadoop 3.1.3+)
: env_vars.insert({ "HADOOP_OS_TYPE", "Linux" });
> Can this be combined into the env_vars declaration above?
Done
http://gerrit.cloudera.org:8080/#/c/14018/1/src/kudu/hms/mini_hms.cc@310
PS1, Line 310:
> Please rationalize these new parameters above.
Done
http://gerrit.cloudera.org:8080/#/c/14018/1/src/kudu/hms/mini_hms.cc@320
PS1, Line 320:
:
: hive.async.log.enabled
: false
:
> I experimented with this but didn't find that it made any difference w.r.t.
I could remove it, it was precautionary more than anything.
http://gerrit.cloudera.org:8080/#/c/14018/1/src/kudu/util/subprocess.cc
File src/kudu/util/subprocess.cc:
http://gerrit.cloudera.org:8080/#/c/14018/1/src/kudu/util/subprocess.cc@726
PS1, Line 726: p.SetEnvVars(env_vars);
> Could you pass env_vars by value and std::move() it here?
Done
--
To view, visit http://gerrit.cloudera.org:8080/14018
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If43ae2330b3d99374c68bae313a3f8bc070f9c69
Gerrit-Change-Number: 14018
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Hao Hao
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 07 Aug 2019 18:52:54 +
Gerrit-HasComments: Yes
[kudu-CR] POC: Disable Sentry related tests
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/14020 ) Change subject: POC: Disable Sentry related tests .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/14020/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/14020/1//COMMIT_MSG@7 PS1, Line 7: POC: Disable Sentry related tests > May also want to disable the mini cluster options that allow Sentry to be c I was on the fence about this. I wanted to keep this patch as small as possible while still allowing all tests to pass. If some test comes along that tries to use Sentry and it's broken, I think that is okay and we should be aware of the breakage. http://gerrit.cloudera.org:8080/#/c/14020/1/src/kudu/hms/mini_hms.cc File src/kudu/hms/mini_hms.cc: http://gerrit.cloudera.org:8080/#/c/14020/1/src/kudu/hms/mini_hms.cc@127 PS1, Line 127: reoved > removed Done http://gerrit.cloudera.org:8080/#/c/14020/1/src/kudu/integration-tests/CMakeLists.txt File src/kudu/integration-tests/CMakeLists.txt: http://gerrit.cloudera.org:8080/#/c/14020/1/src/kudu/integration-tests/CMakeLists.txt@93 PS1, Line 93: # NOTE: Sentry tests are disabled to allow upgrading to Hive 3. : #ADD_KUDU_TEST(master_sentry-itest RUN_SERIAL true NUM_SHARDS 8 PROCESSORS 4) > This means we won't even compile the binary though. Could we disable these What's the value in compiling the binary if we never intend for it to be used/run? -- To view, visit http://gerrit.cloudera.org:8080/14020 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7b99b0de221f6d4acfb427f830cd3344c8d0a10b Gerrit-Change-Number: 14020 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 07 Aug 2019 18:52:50 + Gerrit-HasComments: Yes
[kudu-CR] POC: Disable Sentry related tests
Hello Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14020 to look at the new patch set (#2). Change subject: POC: Disable Sentry related tests .. POC: Disable Sentry related tests This is done as an experiment to enable upgrading to Hive 3. Change-Id: I7b99b0de221f6d4acfb427f830cd3344c8d0a10b --- M src/kudu/hms/mini_hms.cc M src/kudu/integration-tests/CMakeLists.txt M src/kudu/integration-tests/alter_table-randomized-test.cc M src/kudu/integration-tests/master-stress-test.cc M src/kudu/integration-tests/master_failover-itest.cc M src/kudu/master/CMakeLists.txt M src/kudu/sentry/CMakeLists.txt 7 files changed, 28 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/14020/2 -- To view, visit http://gerrit.cloudera.org:8080/14020 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7b99b0de221f6d4acfb427f830cd3344c8d0a10b Gerrit-Change-Number: 14020 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] Prepare for upgrading to Hive 3
Hello Kudu Jenkins, Andrew Wong, Adar Dembo, Hao Hao, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/14018 to look at the new patch set (#2). Change subject: Prepare for upgrading to Hive 3 .. Prepare for upgrading to Hive 3 This patch contains build and test changes that work on both Hive 2 and Hive 3 to minimize thechanges required when upgrading to Hive 3. - Uses the Hive `schematool` to initialize the derby database in the MiniHms. This fixes issues with autoCreate and is more representative of a production environment. - Adjust logging configuration in the MiniHms to be more explicit. - Workaround HADOOP-8719 by hardcoding the HADOOP_OS_TYPE. - Workaround HIVE-21614 by using `LIKE` instead of `=` when filtering tables. Change-Id: If43ae2330b3d99374c68bae313a3f8bc070f9c69 --- M src/kudu/hms/CMakeLists.txt M src/kudu/hms/hms_catalog.cc M src/kudu/hms/hms_client-test.cc M src/kudu/hms/mini_hms.cc M src/kudu/hms/mini_hms.h M src/kudu/util/subprocess.cc M src/kudu/util/subprocess.h M thirdparty/package-hadoop.sh 8 files changed, 55 insertions(+), 23 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/18/14018/2 -- To view, visit http://gerrit.cloudera.org:8080/14018 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: If43ae2330b3d99374c68bae313a3f8bc070f9c69 Gerrit-Change-Number: 14018 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] docs: correct invalid link
XiaokaiWang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14028 Change subject: docs: correct invalid link .. docs: correct invalid link Change-Id: I41e8671f518bb898dd357dc5024a7d8ab335b401 --- M docs/known_issues.adoc 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/28/14028/1 -- To view, visit http://gerrit.cloudera.org:8080/14028 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I41e8671f518bb898dd357dc5024a7d8ab335b401 Gerrit-Change-Number: 14028 Gerrit-PatchSet: 1 Gerrit-Owner: XiaokaiWang
[kudu-CR] KUDU-2797 p2: the master aggregates tablet statistics
Hello Mike Percy, Kudu Jenkins, Andrew Wong, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13426 to look at the new patch set (#31). Change subject: KUDU-2797 p2: the master aggregates tablet statistics .. KUDU-2797 p2: the master aggregates tablet statistics There are some jiras are talking about metrics: KUDU-1067, KUDU-1373, KUDU-2019, KUDU-2797. In this patch, it makes the following improvements: 1) disk size and live row count of the replicas that are the leadership roles are aggregated on the master server; 2) disk size and live row count of the table are exposed as metrics on the master server; 3) disk size and live row count of the table are exposed on the master server's Web-UI; Change-Id: I74406ab7cca7c22fda455c328b8ee9989a6b2d99 --- M src/kudu/integration-tests/cluster_itest_util.cc M src/kudu/integration-tests/cluster_itest_util.h M src/kudu/integration-tests/tablet_replacement-itest.cc M src/kudu/integration-tests/ts_tablet_manager-itest.cc M src/kudu/master/CMakeLists.txt M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_path_handlers.cc A src/kudu/master/table_metrics.cc A src/kudu/master/table_metrics.h M src/kudu/tablet/local_tablet_writer.h M src/kudu/tablet/metadata.proto M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tserver/heartbeater.cc M src/kudu/tserver/heartbeater.h M src/kudu/tserver/ts_tablet_manager-test.cc M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h M src/kudu/util/metrics.cc M www/table.mustache 22 files changed, 765 insertions(+), 35 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/26/13426/31 -- To view, visit http://gerrit.cloudera.org:8080/13426 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I74406ab7cca7c22fda455c328b8ee9989a6b2d99 Gerrit-Change-Number: 13426 Gerrit-PatchSet: 31 Gerrit-Owner: helifu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: helifu
[kudu-CR] KUDU-2797 p2: the master aggregates tablet statistics
helifu has posted comments on this change. ( http://gerrit.cloudera.org:8080/13426 ) Change subject: KUDU-2797 p2: the master aggregates tablet statistics .. Patch Set 30: > Build Failed > > http://jenkins.kudu.apache.org/job/kudu-gerrit/18465/ : FAILURE I ran the test case "mt-tablet-test" in my local environment in debug/release/tsan/asan mode many times, but never failed. And also I read the relevant code, but found nothing. :( -- To view, visit http://gerrit.cloudera.org:8080/13426 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I74406ab7cca7c22fda455c328b8ee9989a6b2d99 Gerrit-Change-Number: 13426 Gerrit-PatchSet: 30 Gerrit-Owner: helifu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: helifu Gerrit-Comment-Date: Wed, 07 Aug 2019 12:25:32 + Gerrit-HasComments: No
[kudu-CR] [tools] Add table tools to delete column and alter column
Yifan Zhang has posted comments on this change. (
http://gerrit.cloudera.org:8080/13976 )
Change subject: [tools] Add table tools to delete column and alter column
..
Patch Set 5:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/13976/4/src/kudu/client/schema.cc
File src/kudu/client/schema.cc:
http://gerrit.cloudera.org:8080/#/c/13976/4/src/kudu/client/schema.cc@720
PS4, Line 720:
: bool KuduSchema::Column(const std::string& col_name,
KuduColumnSchema* col_schema) const {
: int idx = schema_->find_column(col_name);
: if (idx == Schema::kColumnNotFound) {
: return false;
:
> Hitting a CHECK or DCHECK as a user is pretty unpleasant. It's a better use
Done
http://gerrit.cloudera.org:8080/#/c/13976/4/src/kudu/tools/tool_action_table.cc
File src/kudu/tools/tool_action_table.cc:
http://gerrit.cloudera.org:8080/#/c/13976/4/src/kudu/tools/tool_action_table.cc@183
PS4, Line 183: enum PartitionAction {
: ADD,
> nit: not from this patch, but could you reduce the indent here by two space
Done
http://gerrit.cloudera.org:8080/#/c/13976/4/src/kudu/tools/tool_action_table.cc@710
PS4, Line 710: *value = KuduValue::CopyString(str_value);
> Why aren't other types (e.g. bool, binary, decimal) represented here?
I agreed that using Status as return value is more friendly to users, and I
removed all CHECKs and DCHECKs in this patch.
I added bool, binary and unixtime_micros types, the decimal type column is not
supported now for it's quiet difficult to parse input value.
http://gerrit.cloudera.org:8080/#/c/13976/4/src/kudu/tools/tool_action_table.cc@732
PS4, Line 732: *value = KuduValue::FromDouble(strtod(str_value.c_str(),
nullptr));
: }
: break;
: default:
: return Status::NotSupported(Substitute(
: "Unsupported column type $0 for setting default value.",
type));
: }
: return Status::OK();
: }
:
: Status ParseSetEncodingArgs(const std::string& args,
:
KuduColumnStorageAttributes::EncodingType* encod
> What do you think about storing these in a map of { string => KuduColumnSt
Done
http://gerrit.cloudera.org:8080/#/c/13976/4/src/kudu/tools/tool_action_table.cc@810
PS4, Line 810:
> Should we check that the input here is valid?
Done
--
To view, visit http://gerrit.cloudera.org:8080/13976
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I228340e46fe48ffc782c4c7346f890444b8c550f
Gerrit-Change-Number: 13976
Gerrit-PatchSet: 5
Gerrit-Owner: Yifan Zhang
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai <405403...@qq.com>
Gerrit-Comment-Date: Wed, 07 Aug 2019 11:39:46 +
Gerrit-HasComments: Yes
[kudu-CR] [tools] Add table tools to delete column and alter column
Hello Tidy Bot, Yingchun Lai, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13976 to look at the new patch set (#5). Change subject: [tools] Add table tools to delete column and alter column .. [tools] Add table tools to delete column and alter column This patch supports to delete column and alter column write_default value, encoding/compression type and block_size by command line tools. The command can be used as: 1. kudu table delete_column 2. kudu table alter_column [-alter_args=] could be 'set_default', 'remove_default', 'set_compression', 'set_encoding' or 'set_block_size'. Change-Id: I228340e46fe48ffc782c4c7346f890444b8c550f --- M src/kudu/client/schema.cc M src/kudu/client/schema.h M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_table.cc 4 files changed, 356 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/13976/5 -- To view, visit http://gerrit.cloudera.org:8080/13976 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I228340e46fe48ffc782c4c7346f890444b8c550f Gerrit-Change-Number: 13976 Gerrit-PatchSet: 5 Gerrit-Owner: Yifan Zhang Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai <405403...@qq.com>
[kudu-CR] [ksck] Filter tables and tablets in KsckCluster
Yifan Zhang has posted comments on this change. (
http://gerrit.cloudera.org:8080/13937 )
Change subject: [ksck] Filter tables and tablets in KsckCluster
..
Patch Set 10:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/13937/9/src/kudu/tools/ksck.h
File src/kudu/tools/ksck.h:
http://gerrit.cloudera.org:8080/#/c/13937/9/src/kudu/tools/ksck.h@471
PS9, Line 471: int filtered_tables_count() const {
: return filtered_tables_count_;
: }
:
: int filtered_tablets_count() const {
: return filtered_tablets_count_;
: }
> Value types (i.e. int, double, etc.) should be returned by value rather tha
Done
http://gerrit.cloudera.org:8080/#/c/13937/9/src/kudu/tools/ksck.h@489
PS9, Line 489: int filtered_tables_count_;
: int filtered_tablets_count_;
> Nit: 'int' is fine too.
Done
http://gerrit.cloudera.org:8080/#/c/13937/9/src/kudu/tools/ksck_checksum.cc
File src/kudu/tools/ksck_checksum.cc:
http://gerrit.cloudera.org:8080/#/c/13937/9/src/kudu/tools/ksck_checksum.cc@491
PS9, Line 491: &&
> && n
Done
http://gerrit.cloudera.org:8080/#/c/13937/9/src/kudu/tools/ksck_remote.cc
File src/kudu/tools/ksck_remote.cc:
http://gerrit.cloudera.org:8080/#/c/13937/9/src/kudu/tools/ksck_remote.cc@526
PS9, Line 526: filtered_tables_count_ = 0;
> Should probably also initialize these new variables to 0 in the constructor
Done
--
To view, visit http://gerrit.cloudera.org:8080/13937
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I23b6e6ef258d3498a42af7f92b63392a59c99761
Gerrit-Change-Number: 13937
Gerrit-PatchSet: 10
Gerrit-Owner: Yifan Zhang
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai <405403...@qq.com>
Gerrit-Comment-Date: Wed, 07 Aug 2019 08:09:26 +
Gerrit-HasComments: Yes
[kudu-CR] [ksck] Filter tables and tablets in KsckCluster
Hello Alexey Serbin, Yingchun Lai, Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13937 to look at the new patch set (#10). Change subject: [ksck] Filter tables and tablets in KsckCluster .. [ksck] Filter tables and tablets in KsckCluster The ksck tool executes slowly if there are too many tables in a cluster even if we just want to check some specific tables and tablets. Filtering tables and tablets in KsckCluster and only tracking specified tables and tablets would speed up the execution of ksck tool with '--tables=' and '--tablets=' args. Change-Id: I23b6e6ef258d3498a42af7f92b63392a59c99761 --- M src/kudu/tools/ksck-test.cc M src/kudu/tools/ksck.cc M src/kudu/tools/ksck.h M src/kudu/tools/ksck_checksum.cc M src/kudu/tools/ksck_remote.cc M src/kudu/tools/rebalancer.cc M src/kudu/tools/tool_action_cluster.cc M src/kudu/tools/tool_replica_util.cc 8 files changed, 120 insertions(+), 40 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/37/13937/10 -- To view, visit http://gerrit.cloudera.org:8080/13937 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I23b6e6ef258d3498a42af7f92b63392a59c99761 Gerrit-Change-Number: 13937 Gerrit-PatchSet: 10 Gerrit-Owner: Yifan Zhang Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai <405403...@qq.com>
[kudu-CR] KUDU-2797 p2: the master aggregates tablet statistics
Hello Mike Percy, Kudu Jenkins, Andrew Wong, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13426 to look at the new patch set (#30). Change subject: KUDU-2797 p2: the master aggregates tablet statistics .. KUDU-2797 p2: the master aggregates tablet statistics There are some jiras are talking about metrics: KUDU-1067, KUDU-1373, KUDU-2019, KUDU-2797. In this patch, it makes the following improvements: 1) disk size and live row count of the replicas that are the leadership roles are aggregated on the master server; 2) disk size and live row count of the table are exposed as metrics on the master server; 3) disk size and live row count of the table are exposed on the master server's Web-UI; Change-Id: I74406ab7cca7c22fda455c328b8ee9989a6b2d99 --- M src/kudu/integration-tests/cluster_itest_util.cc M src/kudu/integration-tests/cluster_itest_util.h M src/kudu/integration-tests/tablet_replacement-itest.cc M src/kudu/integration-tests/ts_tablet_manager-itest.cc M src/kudu/master/CMakeLists.txt M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_path_handlers.cc A src/kudu/master/table_metrics.cc A src/kudu/master/table_metrics.h M src/kudu/tablet/local_tablet_writer.h M src/kudu/tablet/metadata.proto M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tserver/heartbeater.cc M src/kudu/tserver/heartbeater.h M src/kudu/tserver/ts_tablet_manager-test.cc M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h M src/kudu/util/metrics.cc M www/table.mustache 22 files changed, 765 insertions(+), 35 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/26/13426/30 -- To view, visit http://gerrit.cloudera.org:8080/13426 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I74406ab7cca7c22fda455c328b8ee9989a6b2d99 Gerrit-Change-Number: 13426 Gerrit-PatchSet: 30 Gerrit-Owner: helifu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: helifu
