[kudu-CR] KUDU-2823 [java client] Support setting dimension for the newly created tablet
Yao Xu has posted comments on this change. ( http://gerrit.cloudera.org:8080/13875 ) Change subject: KUDU-2823 [java client] Support setting dimension for the newly created tablet .. Patch Set 4: (2 comments) http://gerrit.cloudera.org:8080/#/c/13875/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java: http://gerrit.cloudera.org:8080/#/c/13875/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java@754 PS2, Line 754: a t > nit: a Done http://gerrit.cloudera.org:8080/#/c/13875/2/src/kudu/master/catalog_manager.cc File src/kudu/master/catalog_manager.cc: http://gerrit.cloudera.org:8080/#/c/13875/2/src/kudu/master/catalog_manager.cc@4790 PS2, Line 4790: } : } : > remove: this is the piece of now deprecated structure Done -- To view, visit http://gerrit.cloudera.org:8080/13875 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 Gerrit-Change-Number: 13875 Gerrit-PatchSet: 4 Gerrit-Owner: Yao Xu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yao Xu Gerrit-Comment-Date: Thu, 18 Jul 2019 05:49:41 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-2823 [java client] Support setting dimension for the newly created tablet
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13875 to look at the new patch set (#4). Change subject: KUDU-2823 [java client] Support setting dimension for the newly created tablet .. KUDU-2823 [java client] Support setting dimension for the newly created tablet Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/LocatedTablet.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto 7 files changed, 108 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/75/13875/4 -- To view, visit http://gerrit.cloudera.org:8080/13875 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 Gerrit-Change-Number: 13875 Gerrit-PatchSet: 4 Gerrit-Owner: Yao Xu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yao Xu
[kudu-CR] KUDU-2823 [java client] Support setting dimension for the newly created tablet
Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13875 to look at the new patch set (#3). Change subject: KUDU-2823 [java client] Support setting dimension for the newly created tablet .. KUDU-2823 [java client] Support setting dimension for the newly created tablet Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/LocatedTablet.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto 7 files changed, 111 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/75/13875/3 -- To view, visit http://gerrit.cloudera.org:8080/13875 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 Gerrit-Change-Number: 13875 Gerrit-PatchSet: 3 Gerrit-Owner: Yao Xu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yao Xu
[kudu-CR] KUDU-2823 [java client] Support setting dimension for the newly created tablet
Yao Xu has posted comments on this change. ( http://gerrit.cloudera.org:8080/13875 ) Change subject: KUDU-2823 [java client] Support setting dimension for the newly created tablet .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/13875/2/src/kudu/master/catalog_manager.cc File src/kudu/master/catalog_manager.cc: PS2: > +1 Thanks for your comments. :) I do this for two main reasons: 1. The cluster rebalancer tool needs to know the dimension label of tablet replica. 2. I don't think it's necessary to verify where the tablet replica are placed(It has been done in C++ API). In Java API, I think we just need to make sure the dimension label are set correctly. Emmm, maybe in this patch I should implement get dimension label of tablet replica in C++ too. http://gerrit.cloudera.org:8080/#/c/13875/2/src/kudu/master/master.proto File src/kudu/master/master.proto: http://gerrit.cloudera.org:8080/#/c/13875/2/src/kudu/master/master.proto@375 PS2, Line 375: optional string dimension_label = 3; > This structure/message has been deprecated a long time ago > (v0.5.0), so I don't think anything that's related to a newly added > features should touch this. Basically, no dimension label should > be sent in response to those ancient clients. This is because the InternedReplicaPB is converted into ReplicaPB in java client. https://github.com/apache/kudu/blob/99e94efeae33271910348a388531f58e460174a6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java#L2289 -- To view, visit http://gerrit.cloudera.org:8080/13875 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 Gerrit-Change-Number: 13875 Gerrit-PatchSet: 2 Gerrit-Owner: Yao Xu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yao Xu Gerrit-Comment-Date: Thu, 18 Jul 2019 05:18:44 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-2881 Support create/drop range partition by command line
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Yao Xu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13833 to look at the new patch set (#4). Change subject: KUDU-2881 Support create/drop range partition by command line .. KUDU-2881 Support create/drop range partition by command line Sometimes we need to drop the range partition and then recreate it in order to rewrite in case that the partition was written wrongly. This change supports to drop/create range partition by command line. The usage is: 1. kudu table create_range_partition [-lower_bound_type] [-upper_bound_type] 2. kudu table drop_range_partition [-lower_bound_type] [-upper_bound_type] The parameters of lower_bound_type and upper_bound_type is optional. If these two parameters are not specified, the default value are INCLUSIVE_BOUND and EXCLUSIVE_BOUND respectively. Change-Id: I647d9c3cf01807bd8ae9f8cf4091568ea6f1161c --- M src/kudu/common/partition.cc M src/kudu/common/partition.h M src/kudu/integration-tests/ts_itest-base.cc M src/kudu/integration-tests/ts_itest-base.h M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool_action_table.cc 6 files changed, 444 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/33/13833/4 -- To view, visit http://gerrit.cloudera.org:8080/13833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I647d9c3cf01807bd8ae9f8cf4091568ea6f1161c Gerrit-Change-Number: 13833 Gerrit-PatchSet: 4 Gerrit-Owner: honeyhexin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yao Xu
[kudu-CR] [examples] Add a complete Nifi quickstart example
Andrew Wong has posted comments on this change. (
http://gerrit.cloudera.org:8080/13878 )
Change subject: [examples] Add a complete Nifi quickstart example
..
Patch Set 3:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/13878/3/examples/quickstart/nifi/README.adoc
File examples/quickstart/nifi/README.adoc:
http://gerrit.cloudera.org:8080/#/c/13878/3/examples/quickstart/nifi/README.adoc@5
PS3, Line 5: rt
Kudu Quickstart Container? Environment?
http://gerrit.cloudera.org:8080/#/c/13878/3/examples/quickstart/nifi/README.adoc@33
PS3, Line 33: docker
nit: capitalize
http://gerrit.cloudera.org:8080/#/c/13878/3/examples/quickstart/nifi/README.adoc@62
PS3, Line 62: uild()
missing semicolon
http://gerrit.cloudera.org:8080/#/c/13878/3/examples/quickstart/nifi/README.adoc@61
PS3, Line 61: KuduClient client =
: new
KuduClientBuilder("kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251").build()
:
: if(client.tableExists("random_user")) {
: client.deleteTable("random_user");
: }
:
: Schema schema = new Schema(Arrays.asList(
: new ColumnSchemaBuilder("ssn", Type.STRING).key(true).build(),
: new ColumnSchemaBuilder("firstName", Type.STRING).build(),
: new ColumnSchemaBuilder("lastName", Type.STRING).build(),
: new ColumnSchemaBuilder("email", Type.STRING).build())
: );
: CreateTableOptions tableOptions =
: new
CreateTableOptions().setNumReplicas(3).addHashPartitions(Arrays.asList("ssn"),
4);
: client.createTable("random_user", schema, tableOptions);
Maybe wrap this in {}s so we can copy-paste it? per
https://stackoverflow.com/a/53923372
http://gerrit.cloudera.org:8080/#/c/13878/3/examples/quickstart/nifi/README.adoc@91
PS3, Line 91: ssn, firstName, lastName, email
nit: maybe put quotes around these?
http://gerrit.cloudera.org:8080/#/c/13878/3/examples/quickstart/nifi/README.adoc@98
PS3, Line 98: Them
Then
http://gerrit.cloudera.org:8080/#/c/13878/3/examples/quickstart/nifi/README.adoc@102
PS3, Line 102: Once the template is added to the canvas you can start each
processor by
: right-clicking each processor and selecting `Start`. You can
also explore
: the configuration, queue contents, and more by right-clicking on
each element.
:
This is where I got stuck running through these steps. Was there another
processor I was supposed to add? The included template only has a PutKudu
processor; seems it needs a source.
http://gerrit.cloudera.org:8080/#/c/13878/3/examples/quickstart/nifi/README.adoc@137
PS3, Line 137: le:
nit: ... from within the Spark quickstart container
--
To view, visit http://gerrit.cloudera.org:8080/13878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I71f3bc5898c15d7bc19cffb3a91b9efac3f6928b
Gerrit-Change-Number: 13878
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 18 Jul 2019 04:40:43 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2797 p1: expose the tablet's live row count
Hello Kudu Jenkins, Adar Dembo, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13877 to look at the new patch set (#3). Change subject: KUDU-2797 p1: expose the tablet's live row count .. KUDU-2797 p1: expose the tablet's live row count The tablet's live row count is exposed in below ways: 1) be exposed as metrics on the tablet server; 2) be exposed on the tablet server's Web-UI; Change-Id: I2b15d7d2a5c84a3716215608b2444395ca94e64c --- M src/kudu/tablet/tablet_replica-test.cc M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tserver/tserver_path_handlers.cc M www/tablet.mustache 5 files changed, 61 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/77/13877/3 -- To view, visit http://gerrit.cloudera.org:8080/13877 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2b15d7d2a5c84a3716215608b2444395ca94e64c Gerrit-Change-Number: 13877 Gerrit-PatchSet: 3 Gerrit-Owner: helifu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2881 Support create/drop range partition by command line
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Yao Xu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13833 to look at the new patch set (#3). Change subject: KUDU-2881 Support create/drop range partition by command line .. KUDU-2881 Support create/drop range partition by command line Sometimes we need to drop the range partition and then recreate it in order to rewrite in case that the partition was written wrongly. This change supports to drop/create range partition by command line. The usage is: 1. kudu table create_range_partition [-lower_bound_type] [-upper_bound_type] 2. kudu table drop_range_partition [-lower_bound_type] [-upper_bound_type] The parameters of lower_bound_type and upper_bound_type is optional. If these two parameters are not specified, the default value are INCLUSIVE_BOUND and EXCLUSIVE_BOUND respectively. Change-Id: I647d9c3cf01807bd8ae9f8cf4091568ea6f1161c --- M src/kudu/common/partition.cc M src/kudu/common/partition.h M src/kudu/integration-tests/ts_itest-base.cc M src/kudu/integration-tests/ts_itest-base.h M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool_action_table.cc 6 files changed, 438 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/33/13833/3 -- To view, visit http://gerrit.cloudera.org:8080/13833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I647d9c3cf01807bd8ae9f8cf4091568ea6f1161c Gerrit-Change-Number: 13833 Gerrit-PatchSet: 3 Gerrit-Owner: honeyhexin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yao Xu
[kudu-CR] KUDU-2881 Support create/drop range partition by command line
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, Yao Xu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13833 to look at the new patch set (#2). Change subject: KUDU-2881 Support create/drop range partition by command line .. KUDU-2881 Support create/drop range partition by command line Sometimes we need to drop the range partition and then recreate it in order to rewrite in case that the partition was written wrongly. This change supports to drop/create range partition by command line. The usage is: 1. kudu table create_range_partition [-lower_bound_type] [-upper_bound_type] 2. kudu table drop_range_partition [-lower_bound_type] [-upper_bound_type] The parameters of lower_bound_type and upper_bound_type is optional. If these two parameters are not specified, the default value are INCLUSIVE_BOUND and EXCLUSIVE_BOUND respectively. Change-Id: I647d9c3cf01807bd8ae9f8cf4091568ea6f1161c --- M src/kudu/common/partition.cc M src/kudu/common/partition.h M src/kudu/integration-tests/ts_itest-base.cc M src/kudu/integration-tests/ts_itest-base.h M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool_action_table.cc 6 files changed, 439 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/33/13833/2 -- To view, visit http://gerrit.cloudera.org:8080/13833 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I647d9c3cf01807bd8ae9f8cf4091568ea6f1161c Gerrit-Change-Number: 13833 Gerrit-PatchSet: 2 Gerrit-Owner: honeyhexin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yao Xu
[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 ) Change subject: WIP: Upgrade Hive dependency to 3.1.1 .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/13256/2/java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java File java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java: http://gerrit.cloudera.org:8080/#/c/13256/2/java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java@338 PS2, Line 338: client.dropTable > What happens if you use the same dropTable API existed in Hive 2? Will it b They sort of broke the APIs that use envContext. They kept the others though. If a catalog isn't specified, the default catalog is used. http://gerrit.cloudera.org:8080/#/c/13256/3/thirdparty/vars.sh File thirdparty/vars.sh: http://gerrit.cloudera.org:8080/#/c/13256/3/thirdparty/vars.sh@220 PS3, Line 220: HIVE_NAME=hive-$HIVE_VERSION : HIVE_SOURCE=$TP_SOURCE_DIR/$HIVE_NAME > Need to figure out whether this needs to be amended, or removed altogether. All of these are included in Hive 3. I will remove. -- To view, visit http://gerrit.cloudera.org:8080/13256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c Gerrit-Change-Number: 13256 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 17 Jul 2019 19:27:14 + Gerrit-HasComments: Yes
[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1
Hello Kudu Jenkins, Adar Dembo, Hao Hao, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13256 to look at the new patch set (#4). Change subject: WIP: Upgrade Hive dependency to 3.1.1 .. WIP: Upgrade Hive dependency to 3.1.1 Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c --- M java/gradle/dependencies.gradle M java/kudu-hive/build.gradle M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java M src/kudu/hms/CMakeLists.txt M src/kudu/hms/hive_metastore.thrift M src/kudu/hms/mini_hms.cc M thirdparty/package-hadoop.sh M thirdparty/vars.sh 8 files changed, 823 insertions(+), 103 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/4 -- To view, visit http://gerrit.cloudera.org:8080/13256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c Gerrit-Change-Number: 13256 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 ) Change subject: WIP: Upgrade Hive dependency to 3.1.1 .. Patch Set 3: (1 comment) Did a pass through the new code. Interesting to see some backwards incompatibilities in hive_metastore.thrift; guess the Hive devs didn't maintain that scrupulously. http://gerrit.cloudera.org:8080/#/c/13256/3/thirdparty/vars.sh File thirdparty/vars.sh: http://gerrit.cloudera.org:8080/#/c/13256/3/thirdparty/vars.sh@220 PS3, Line 220: # TODO(dan): bump to a release version once HIVE-17747 and HIVE-16886/HIVE-18526 : # are published. The SHA below is the current head of branch-2. Need to figure out whether this needs to be amended, or removed altogether. -- To view, visit http://gerrit.cloudera.org:8080/13256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c Gerrit-Change-Number: 13256 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 17 Jul 2019 19:16:17 + Gerrit-HasComments: Yes
[kudu-CR] [examples] Add a complete Nifi quickstart example
Hello Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13878 to look at the new patch set (#3). Change subject: [examples] Add a complete Nifi quickstart example .. [examples] Add a complete Nifi quickstart example This patchs adds a brief example using Apache Nifi to ingest data into Apache Kudu. Change-Id: I71f3bc5898c15d7bc19cffb3a91b9efac3f6928b --- M docs/quickstart.adoc A examples/quickstart/nifi/README.adoc A examples/quickstart/nifi/Random_User_Kudu.xml 3 files changed, 423 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/78/13878/3 -- To view, visit http://gerrit.cloudera.org:8080/13878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I71f3bc5898c15d7bc19cffb3a91b9efac3f6928b Gerrit-Change-Number: 13878 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1
Hello Kudu Jenkins, Adar Dembo, Hao Hao, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13256 to look at the new patch set (#3). Change subject: WIP: Upgrade Hive dependency to 3.1.1 .. WIP: Upgrade Hive dependency to 3.1.1 Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c --- M java/gradle/dependencies.gradle M java/kudu-hive/build.gradle M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java M src/kudu/hms/CMakeLists.txt M src/kudu/hms/hive_metastore.thrift M src/kudu/hms/mini_hms.cc M thirdparty/package-hadoop.sh M thirdparty/vars.sh 8 files changed, 823 insertions(+), 101 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/3 -- To view, visit http://gerrit.cloudera.org:8080/13256 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c Gerrit-Change-Number: 13256 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [examples] Add a complete Nifi quickstart example
Hello Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13878 to look at the new patch set (#2). Change subject: [examples] Add a complete Nifi quickstart example .. [examples] Add a complete Nifi quickstart example This patchs adds a brief example using Apache Nifi to ingest data into Apache Kudu. Change-Id: I71f3bc5898c15d7bc19cffb3a91b9efac3f6928b --- M docs/quickstart.adoc A examples/quickstart/nifi/README.adoc A examples/quickstart/nifi/Random_User_Kudu.xml 3 files changed, 423 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/78/13878/2 -- To view, visit http://gerrit.cloudera.org:8080/13878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I71f3bc5898c15d7bc19cffb3a91b9efac3f6928b Gerrit-Change-Number: 13878 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [examples] Add a complete Nifi quickstart example
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13878 Change subject: [examples] Add a complete Nifi quickstart example .. [examples] Add a complete Nifi quickstart example This patchs adds a brief example using Apache Nifi to ingest data into Apache Kudu. Change-Id: I71f3bc5898c15d7bc19cffb3a91b9efac3f6928b --- M docs/quickstart.adoc A examples/quickstart/nifi/README.adoc A examples/quickstart/nifi/Random_User_Kudu.xml 3 files changed, 423 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/78/13878/1 -- To view, visit http://gerrit.cloudera.org:8080/13878 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I71f3bc5898c15d7bc19cffb3a91b9efac3f6928b Gerrit-Change-Number: 13878 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] KUDU-1938 Add CHAR/VARCHAR to C++ client pt 2
Adar Dembo has posted comments on this change. (
http://gerrit.cloudera.org:8080/13869 )
Change subject: KUDU-1938 Add CHAR/VARCHAR to C++ client pt 2
..
Patch Set 3:
(5 comments)
Would be nice to see an end-to-end test that proves the expected
padding/truncation semantics for the new types.
http://gerrit.cloudera.org:8080/#/c/13869/3/src/kudu/client/predicate-test.cc
File src/kudu/client/predicate-test.cc:
http://gerrit.cloudera.org:8080/#/c/13869/3/src/kudu/client/predicate-test.cc@1200
PS3, Line 1200: TEST_F(PredicateTest, TestCharPredicates) {
Would it be possible to type-parameterize these new tests, perhaps along with
TestStringPredicates and TestBinaryPredicates?
http://gerrit.cloudera.org:8080/#/c/13869/3/src/kudu/client/scan_batch.h
File src/kudu/client/scan_batch.h:
PS3:
Do we want to doc that the char getter does _not_ pad with whitespace?
http://gerrit.cloudera.org:8080/#/c/13869/3/src/kudu/client/schema.h
File src/kudu/client/schema.h:
http://gerrit.cloudera.org:8080/#/c/13869/3/src/kudu/client/schema.h@84
PS3, Line 84: /// The length of a CHAR or VARCHAR column.
"Maximum length". Should also doc whether we're talking about bytes or
characters.
http://gerrit.cloudera.org:8080/#/c/13869/3/src/kudu/client/schema.h@398
PS3, Line 398: fixed length of a CHAR column
In light of the fact that we truncate excess whitespace when writing and don't
pad when reading, what does "fixed length" mean?
http://gerrit.cloudera.org:8080/#/c/13869/3/src/kudu/client/schema.cc
File src/kudu/client/schema.cc:
http://gerrit.cloudera.org:8080/#/c/13869/3/src/kudu/client/schema.cc@320
PS3, Line 320: if (data_->type.value() == KuduColumnSchema::DECIMAL) {
Maybe convert this into a switch on data_->type.value()?
--
To view, visit http://gerrit.cloudera.org:8080/13869
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifaf609565b0a0a87d6e645cd3ac14c0965af5ba8
Gerrit-Change-Number: 13869
Gerrit-PatchSet: 3
Gerrit-Owner: Attila Bukor
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Will Berkeley
Gerrit-Comment-Date: Wed, 17 Jul 2019 17:21:39 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1938 Add support for CHAR/VARCHAR pt 1
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13760 ) Change subject: KUDU-1938 Add support for CHAR/VARCHAR pt 1 .. Patch Set 20: (1 comment) http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/common/partial_row.h File src/kudu/common/partial_row.h: http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/common/partial_row.h@413 PS20, Line 413: /// @name Getters for string/binary/char/varchar column by column name. > Do we want to doc that the CHAR getters do _not_ pad with whitespace? I think the method should be GetUnpadedChar so we can add a GetChar that does padding if we want later. This matches GetUnscaledDecimal. -- To view, visit http://gerrit.cloudera.org:8080/13760 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I998982dba93831db91c43a97ce30d3e68c2a4a54 Gerrit-Change-Number: 13760 Gerrit-PatchSet: 20 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Wed, 17 Jul 2019 17:17:51 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-1938 Add support for CHAR/VARCHAR pt 1
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13760 ) Change subject: KUDU-1938 Add support for CHAR/VARCHAR pt 1 .. Patch Set 20: (1 comment) http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/common/partial_row.h File src/kudu/common/partial_row.h: http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/common/partial_row.h@413 PS20, Line 413: /// @name Getters for string/binary/char/varchar column by column name. Do we want to doc that the CHAR getters do _not_ pad with whitespace? -- To view, visit http://gerrit.cloudera.org:8080/13760 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I998982dba93831db91c43a97ce30d3e68c2a4a54 Gerrit-Change-Number: 13760 Gerrit-PatchSet: 20 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley Gerrit-Comment-Date: Wed, 17 Jul 2019 17:15:53 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-1938 Add support for CHAR/VARCHAR pt 1
Adar Dembo has posted comments on this change. (
http://gerrit.cloudera.org:8080/13760 )
Change subject: KUDU-1938 Add support for CHAR/VARCHAR pt 1
..
Patch Set 20:
(7 comments)
http://gerrit.cloudera.org:8080/#/c/13760/20//COMMIT_MSG
Commit Message:
PS20:
I'd use the commit message to document the expected padding/truncation
semantics, as well as a discussion of trade-offs (i.e. why are we truncating
before we store the data server-side?)
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/common/partial_row-test.cc
File src/kudu/common/partial_row-test.cc:
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/common/partial_row-test.cc@290
PS20, Line 290: s = row.SetVarchar("varchar_val", "shortval");
Also add a VARCHAR test that proves that it doesn't truncate excess whitespace.
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/common/partial_row.cc
File src/kudu/common/partial_row.cc:
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/common/partial_row.cc@423
PS20, Line 423: case CHAR:
: relocated_val = UTF8Truncate(val,
col.type_attributes().length,
:TRUNCATE_WHITESPACE);
: break;
: case VARCHAR:
: relocated_val = UTF8Truncate(val,
col.type_attributes().length,
:NO_TRUNCATE_WHITESPACE);
: break;
Maybe rewrite like this:
case CHAR:
case VARCHAR:
relocated_val = UTF8Truncate(val, col.type_attributes().length,
T::type == CHAR ? TRUNCATE_WHITESPACE :
NO_TRUNCATE_WHITESPACE);
break;
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/common/schema.h
File src/kudu/common/schema.h:
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/common/schema.h@124
PS20, Line 124: // Maximum value of the length is 65,535 for compatibility
reasons as it's
: // used by VARCHAR type which can be set to a maximum of
65,535 in case of
: // MySQL and less for other major RDMBMS implementations.
Did you consider limiting it to 255 in order to be exactly compatible with
Impala? Looks like Impala will truncate excess characters if the underlying
storage stores more than 255. Do we care?
Also, I'm trying to figure out whether the length should refer to a _byte
quantity_ or a _char quantity_ (the two aren't the same in UTF-8). The Impala
docs are a little ambiguous here.
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/util/char_util.h
File src/kudu/util/char_util.h:
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/util/char_util.h@30
PS20, Line 30: enum TruncateType {
: NO_TRUNCATE_WHITESPACE,
: TRUNCATE_WHITESPACE,
: };
I might rename this:
enum class TrailingWhitespace {
IGNORE,
TRUNCATE
};
By making it an enum class, references to it have to be fully qualified (i.e.
TrailingWhitespace::IGNORE). This makes it perfectly clear what the effect will
be.
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/util/char_util.h@43
PS20, Line 43: // Copy and truncate a slice
Should also say that the returned Slice "owns" its memory (which is somewhat
unusual for a Slice, hence the need to doc it).
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/util/char_util.cc
File src/kudu/util/char_util.cc:
http://gerrit.cloudera.org:8080/#/c/13760/20/src/kudu/util/char_util.cc@45
PS20, Line 45: / sizeof(uint8_t)
Not necessary: isn't this always "divide by 1"?
--
To view, visit http://gerrit.cloudera.org:8080/13760
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I998982dba93831db91c43a97ce30d3e68c2a4a54
Gerrit-Change-Number: 13760
Gerrit-PatchSet: 20
Gerrit-Owner: Attila Bukor
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Attila Bukor
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Will Berkeley
Gerrit-Comment-Date: Wed, 17 Jul 2019 17:14:18 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2823 [java client] Support setting dimension for the newly created tablet
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/13875 )
Change subject: KUDU-2823 [java client] Support setting dimension for the newly
created tablet
..
Patch Set 2:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/13875/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java:
http://gerrit.cloudera.org:8080/#/c/13875/2/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java@754
PS2, Line 754: the
nit: a
http://gerrit.cloudera.org:8080/#/c/13875/2/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
PS2:
> It looks like the dimension label is only exposed in the locations code pat
+1
>From the other side, if making the cluster rebalancer tool to be aware of
>dimension labels, what might be the best place to add this information?
http://gerrit.cloudera.org:8080/#/c/13875/2/src/kudu/master/catalog_manager.cc@4790
PS2, Line 4790: if (dimension) {
: replica_pb->set_dimension_label(*dimension);
: }
remove: this is the piece of now deprecated structure
http://gerrit.cloudera.org:8080/#/c/13875/2/src/kudu/master/master.proto
File src/kudu/master/master.proto:
http://gerrit.cloudera.org:8080/#/c/13875/2/src/kudu/master/master.proto@375
PS2, Line 375: optional string dimension_label = 3;
This structure/message has been deprecated a long time ago (v0.5.0), so I don't
think anything that's related to a newly added features should touch this.
Basically, no dimension label should be sent in response to those ancient
clients.
--
To view, visit http://gerrit.cloudera.org:8080/13875
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5fc7797011df578000544147938bd15a6027f4c8
Gerrit-Change-Number: 13875
Gerrit-PatchSet: 2
Gerrit-Owner: Yao Xu
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 17 Jul 2019 16:43:27 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1938 Add support for CHAR/VARCHAR pt 1
Hello Will Berkeley, Tidy Bot, Alexey Serbin, Kudu Jenkins, Andrew Wong, Adar Dembo, Grant Henke, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13760 to look at the new patch set (#20). Change subject: KUDU-1938 Add support for CHAR/VARCHAR pt 1 .. KUDU-1938 Add support for CHAR/VARCHAR pt 1 Introduces the CHAR and VARCHAR data types to the server. Follow up commits will add integration to the clients. The CHAR and VARCHAR types are parameterized with a length column type attribute similar to DECIMAL's scale and precision. Internally both of them are stored as slices. Change-Id: I998982dba93831db91c43a97ce30d3e68c2a4a54 --- M src/kudu/common/column_predicate-test.cc M src/kudu/common/common.proto M src/kudu/common/partial_row-test.cc M src/kudu/common/partial_row.cc M src/kudu/common/partial_row.h M src/kudu/common/schema.cc M src/kudu/common/schema.h M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/common/wire_protocol.cc M src/kudu/util/CMakeLists.txt A src/kudu/util/char_util.cc A src/kudu/util/char_util.h 13 files changed, 326 insertions(+), 30 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/60/13760/20 -- To view, visit http://gerrit.cloudera.org:8080/13760 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I998982dba93831db91c43a97ce30d3e68c2a4a54 Gerrit-Change-Number: 13760 Gerrit-PatchSet: 20 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley
[kudu-CR] KUDU-1938 Add support for CHAR/VARCHAR pt 1
Attila Bukor has posted comments on this change. (
http://gerrit.cloudera.org:8080/13760 )
Change subject: KUDU-1938 Add support for CHAR/VARCHAR pt 1
..
Patch Set 19:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/13760/19/src/kudu/util/char_util.h
File src/kudu/util/char_util.h:
http://gerrit.cloudera.org:8080/#/c/13760/19/src/kudu/util/char_util.h@44
PS19, Line 44: <==>
> What's this mean?
tried to denote "if and only if" with this. Deleting this method anyway for now
as it's easier to do this whole thing as part of UTF8Truncate and it's not used
anywhere else now anyway without UTF8Resize.
http://gerrit.cloudera.org:8080/#/c/13760/19/src/kudu/util/char_util.h@54
PS19, Line 54: std::string UTF8Resize(Slice val, size_t utf8_length);
> For simplicity, could UTF8Truncate be folded into UTF8Resize? Meaning, coul
yeah I think I originally I split them because I needed an std::string in one
case and Slice in the other. I think we don't even need this as we're not
planning to pad, only to truncate, so I'll go ahead and remove this method for
now.
http://gerrit.cloudera.org:8080/#/c/13760/19/src/kudu/util/char_util.cc
File src/kudu/util/char_util.cc:
http://gerrit.cloudera.org:8080/#/c/13760/19/src/kudu/util/char_util.cc@44
PS19, Line 44: Slice UTF8Truncate(Slice val, size_t max_utf8_length) {
> Should DCHECK that max_utf8_length < val.size()?
I don't think we need to, calling this method with max_utf8_length >=
val.size() should be valid, in which case it's not truncated, only copied. At
least that's how I use it right now as we need to memcpy val.data() either way.
--
To view, visit http://gerrit.cloudera.org:8080/13760
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I998982dba93831db91c43a97ce30d3e68c2a4a54
Gerrit-Change-Number: 13760
Gerrit-PatchSet: 19
Gerrit-Owner: Attila Bukor
Gerrit-Reviewer: Adar Dembo
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Attila Bukor
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon
Gerrit-Reviewer: Will Berkeley
Gerrit-Comment-Date: Wed, 17 Jul 2019 16:35:48 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1938 Add CHAR/VARCHAR to C++ client pt 2
Hello Will Berkeley, Alexey Serbin, Kudu Jenkins, Andrew Wong, Adar Dembo, Grant Henke, Todd Lipcon, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13869 to look at the new patch set (#3). Change subject: KUDU-1938 Add CHAR/VARCHAR to C++ client pt 2 .. KUDU-1938 Add CHAR/VARCHAR to C++ client pt 2 Adds support for CHAR and VARCHAR types to the C++ client. Change-Id: Ifaf609565b0a0a87d6e645cd3ac14c0965af5ba8 --- M src/kudu/client/predicate-test.cc M src/kudu/client/scan_batch.cc M src/kudu/client/scan_batch.h M src/kudu/client/schema-internal.h M src/kudu/client/schema.cc M src/kudu/client/schema.h M src/kudu/client/value.cc M src/kudu/integration-tests/all_types-itest.cc 8 files changed, 226 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/69/13869/3 -- To view, visit http://gerrit.cloudera.org:8080/13869 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifaf609565b0a0a87d6e645cd3ac14c0965af5ba8 Gerrit-Change-Number: 13869 Gerrit-PatchSet: 3 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Will Berkeley
[kudu-CR] KUDU-2797 p1: expose the tablet's live row count
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13877 ) Change subject: KUDU-2797 p1: expose the tablet's live row count .. Patch Set 2: -Code-Review Actually, could you add a unit test that verifies the metric values? -- To view, visit http://gerrit.cloudera.org:8080/13877 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2b15d7d2a5c84a3716215608b2444395ca94e64c Gerrit-Change-Number: 13877 Gerrit-PatchSet: 2 Gerrit-Owner: helifu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 17 Jul 2019 16:23:52 + Gerrit-HasComments: No
[kudu-CR] KUDU-2797 p1: expose the tablet's live row count
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13877 ) Change subject: KUDU-2797 p1: expose the tablet's live row count .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13877 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2b15d7d2a5c84a3716215608b2444395ca94e64c Gerrit-Change-Number: 13877 Gerrit-PatchSet: 2 Gerrit-Owner: helifu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 17 Jul 2019 16:23:37 + Gerrit-HasComments: No
[kudu-CR] KUDU-2823 [java client] Support setting dimension for the newly created tablet
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13875 ) Change subject: KUDU-2823 [java client] Support setting dimension for the newly created tablet .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/13875/2/src/kudu/master/catalog_manager.cc File src/kudu/master/catalog_manager.cc: PS2: It looks like the dimension label is only exposed in the locations code path for the purpose of the new Java test. Could we avoid doing that and have the test verify correctness some other way? Perhaps by looking at where the master placed the created table's replicas? -- To view, visit http://gerrit.cloudera.org:8080/13875 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 Gerrit-Change-Number: 13875 Gerrit-PatchSet: 2 Gerrit-Owner: Yao Xu Gerrit-Reviewer: Adar Dembo Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 17 Jul 2019 16:22:12 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-2797 p1: expose the tablet's live row count
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13877 to look at the new patch set (#2). Change subject: KUDU-2797 p1: expose the tablet's live row count .. KUDU-2797 p1: expose the tablet's live row count The tablet's live row count is exposed in below ways: 1) be exposed as metrics on the tablet server; 2) be exposed on the tablet server's Web-UI; Change-Id: I2b15d7d2a5c84a3716215608b2444395ca94e64c --- M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tserver/tserver_path_handlers.cc M www/tablet.mustache 4 files changed, 36 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/77/13877/2 -- To view, visit http://gerrit.cloudera.org:8080/13877 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2b15d7d2a5c84a3716215608b2444395ca94e64c Gerrit-Change-Number: 13877 Gerrit-PatchSet: 2 Gerrit-Owner: helifu Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2797 p1: expose the tablet's live row count
helifu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13877 Change subject: KUDU-2797 p1: expose the tablet's live row count .. KUDU-2797 p1: expose the tablet's live row count The tablet's live row count is exposed in below ways: 1) be exposed as metrics on the tablet server; 2) be exposed on the tablet server's Web-UI; Change-Id: I2b15d7d2a5c84a3716215608b2444395ca94e64c --- M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tserver/tserver_path_handlers.cc M www/tablet.mustache 4 files changed, 34 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/77/13877/1 -- To view, visit http://gerrit.cloudera.org:8080/13877 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2b15d7d2a5c84a3716215608b2444395ca94e64c Gerrit-Change-Number: 13877 Gerrit-PatchSet: 1 Gerrit-Owner: helifu
[kudu-CR] KUDU-2823 [java client] Support setting dimension for the newly created tablet
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13875 to look at the new patch set (#2). Change subject: KUDU-2823 [java client] Support setting dimension for the newly created tablet .. KUDU-2823 [java client] Support setting dimension for the newly created tablet Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/LocatedTablet.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto 7 files changed, 111 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/75/13875/2 -- To view, visit http://gerrit.cloudera.org:8080/13875 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 Gerrit-Change-Number: 13875 Gerrit-PatchSet: 2 Gerrit-Owner: Yao Xu Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2828 [java client] Support setting dimension for the newly created tablet
Yao Xu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13875 Change subject: KUDU-2828 [java client] Support setting dimension for the newly created tablet .. KUDU-2828 [java client] Support setting dimension for the newly created tablet Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AlterTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/CreateTableOptions.java M java/kudu-client/src/main/java/org/apache/kudu/client/LocatedTablet.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto 7 files changed, 109 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/75/13875/1 -- To view, visit http://gerrit.cloudera.org:8080/13875 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5fc7797011df578000544147938bd15a6027f4c8 Gerrit-Change-Number: 13875 Gerrit-PatchSet: 1 Gerrit-Owner: Yao Xu
[kudu-CR] KUDU-2882: Increase the timeout value for TestSentryClientMetrics.Basic
Hello Yingchun Lai, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13770 to look at the new patch set (#2). Change subject: KUDU-2882: Increase the timeout value for TestSentryClientMetrics.Basic .. KUDU-2882: Increase the timeout value for TestSentryClientMetrics.Basic It will be better to increase the timeout value to avoid the failures in some test environments. Change-Id: I3f7f1dc0887b31c4e1a56071be2f0223aeb292e9 --- M src/kudu/master/sentry_authz_provider-test.cc 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/70/13770/2 -- To view, visit http://gerrit.cloudera.org:8080/13770 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3f7f1dc0887b31c4e1a56071be2f0223aeb292e9 Gerrit-Change-Number: 13770 Gerrit-PatchSet: 2 Gerrit-Owner: helifu Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai <405403...@qq.com> Gerrit-Reviewer: helifu
[kudu-CR] KUDU-2882: Increase the timeout value for TestSentryClientMetrics.Basic
helifu has posted comments on this change. ( http://gerrit.cloudera.org:8080/13770 ) Change subject: KUDU-2882: Increase the timeout value for TestSentryClientMetrics.Basic .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13770/1/src/kudu/master/sentry_authz_provider-test.cc File src/kudu/master/sentry_authz_provider-test.cc: http://gerrit.cloudera.org:8080/#/c/13770/1/src/kudu/master/sentry_authz_provider-test.cc@1192 PS1, Line 1192: FLAGS_sentry_service_send_timeout_seconds = AllowSlowTests() ? 5 : 2; > Perhaps do something like this, so the effect on runtime isn't as great? Done -- To view, visit http://gerrit.cloudera.org:8080/13770 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3f7f1dc0887b31c4e1a56071be2f0223aeb292e9 Gerrit-Change-Number: 13770 Gerrit-PatchSet: 1 Gerrit-Owner: helifu Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai <405403...@qq.com> Gerrit-Reviewer: helifu Gerrit-Comment-Date: Wed, 17 Jul 2019 09:19:15 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-2797: the master aggregates tablet statistics
Adar Dembo has posted comments on this change. (
http://gerrit.cloudera.org:8080/13426 )
Change subject: KUDU-2797: the master aggregates tablet statistics
..
Patch Set 20:
(35 comments)
http://gerrit.cloudera.org:8080/#/c/13426/20//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/13426/20//COMMIT_MSG@13
PS20, Line 13: 1) live row count of the tablet is exposed as metrics on
:the tablet server;
: 2) live row count of the tablet is exposed on the tablet
:server's Web-UI;
Could you split this off into a separate patch? It's non-controversial and can
be merged while the rest remains under review.
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.h
File src/kudu/master/catalog_manager.h:
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.h@220
PS20, Line 220: std::unordered_map replica_stats_;
Given the low replication factors that we often use (i.e. 3 or 5), do you think
it'd be more performant to make this a vector of pairs and iterate through it
to find the desired replica?
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.h@318
PS20, Line 318: void RegisterMetrics(const std::string& table_name,
MetricRegistry* metric_registry);
I'd pass metric_registry first. It's a style preference, but generally we pass
"boring" singletons and such earlier in the list of args so it's easier to skip
them while reading.
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.h@365
PS20, Line 365: gscoped_ptr metrics_;
unique_ptr for new code.
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.cc@370
PS20, Line 370: table->RegisterMetrics(metadata.name(),
catalog_manager_->master_->metric_registry());
Should this use the normalized table name?
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.cc@1785
PS20, Line 1785: table->RegisterMetrics(metadata->name(),
master_->metric_registry());
Likewise, normalized table name?
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.cc@3938
PS20, Line 3938: tablet->unset_replica_stats(ts_desc->permanent_uuid());
This is one area of the patch that still gives me pause: how do we know that we
got the replica metrics lifecycle right? Bear in mind that:
1. A master may be offline for a while, or restarted.
2. A tserver may be offline for a while, or restarted.
3. Replicas may be deleted when a table or range partition is deleted, and
(maybe?) they never show up in a report after that.
So how do we know that we've got full coverage of all the "unset" events?
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.cc@4171
PS20, Line 4171: if (ts_desc->permanent_uuid() ==
report.consensus_state().leader_uuid()) {
If the table-level metrics kept track of which replica UUID was used to
populate the metrics for each tablet, we could be more robust about a few
things:
- Shifting reporting from one leader to another, or from a follower to a leader
(if there's no leader).
- Not publishing the metric until all tablets have sent at least one report.
In general I'd like to better understand how this value fluctuates. At the very
least, it seems like when the master restarts there's going to be fluctuation
from 0 up to the correct value as more and more tservers report in.
Fluctuations are bad as they make the metric less usable for policy decisions.
What can we do to avoid them?
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.cc@4172
PS20, Line 4172: tablet->table()->UpdateMetrics(
The function signature will change with each new aggregated metric. Perhaps we
can pass report.stats() and tablet->replica_stats(prev_cstate.leader_uuid())
directly into it, and let it handle the specifics of which metrics exist?
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.cc@4174
PS20, Line 4174:
tablet->replica_stats(prev_cstate.leader_uuid()).on_disk_size(),
Call replica_stats() once and reuse it for both updates here.
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.cc@5470
PS20, Line 5470: void TableInfo::RegisterMetrics(const string& table_name,
MetricRegistry* metric_registry) {
Seems like we should unpublish the entity when the table is deleted, no?
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.cc@5473
PS20, Line 5473: attrs["table_name"] = table_name;
How does this get updated when a table is renamed?
http://gerrit.cloudera.org:8080/#/c/13426/20/src/kudu/master/catalog_manager.cc@5486
PS20, Line 5486: TableMetrics* TableInfo::metrics() {
Where is this
