[kudu-CR] [rpc] clean up JWT-related client-side negotiation code
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20076 Change subject: [rpc] clean up JWT-related client-side negotiation code .. [rpc] clean up JWT-related client-side negotiation code Since now there is an API to add a trusted TLS certificate into the chain of trusted certificates of a Kudu C++ client application, the test-only flag --jwt_client_require_trusted_tls_cert isn't longer needed. This patch removes the flag along with corresponding test scenario. Correspondingly, the client now verifies the server's TLS certificate during TLS handshake since there isn't a case when a client would send out its JWT to a server it doesn't trust once the --jwt_client_require_trusted_tls_cert test-only flag is removed. This patch also adds an extra logging about a connection negotiation condition when the client has a JWT, but it doesn't trust the server's TLS certificate. In addition, I took the liberty of removing a few TODOs related to KUDU-1921 since the referred functionality has already been implemented. Change-Id: I85574ed05396fcf3740d9d068afa524cf125f5ff --- M src/kudu/integration-tests/security-itest.cc M src/kudu/rpc/client_negotiation.cc 2 files changed, 27 insertions(+), 49 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/76/20076/1 -- To view, visit http://gerrit.cloudera.org:8080/20076 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I85574ed05396fcf3740d9d068afa524cf125f5ff Gerrit-Change-Number: 20076 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19949 )
Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed
..
Patch Set 8:
(7 comments)
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
File
java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@346
PS7, Line 346: batches by tablet
> nit: The comments have to be updated.
Done
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@372
PS7, Line 372: Extra batches for operations which have different table schemas.
: // 'extraBatches' is used to st
> How about: 'extraBatches' is used to store batches of operations which have
Done
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@415
PS7, Line 415: se if
> nit: !batch
Done
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@423
PS7, Line 423:
> Should be:
Done
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java:
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java@139
PS7, Line 139:
> I see you flush manually below, why set the mode as AUTO_FLUSH_BACKGROUND h
Every mode will cause the problem. It no needs to set the mode
AUTO_FLUSH_BACKGROUND.
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java@151
PS7, Line 151:* @param end the exclusive end key
:*/
:
> Is it too special that upsert on the same row? It's enough to cover the cas
Done
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java@190
PS7, Line 190: assertEquals(String.format("row errors: %s",
> Also check we can read the rows correctly.
Done
--
To view, visit http://gerrit.cloudera.org:8080/19949
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db
Gerrit-Change-Number: 19949
Gerrit-PatchSet: 8
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Thu, 15 Jun 2023 04:21:16 +
Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Hello Yuqi Du, Alexey Serbin, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19949 to look at the new patch set (#8). Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed .. [KUDU-3483] Fixbug of auto flush data when table schema changed In auto_flush_background mode, applying an operation firstly inserts the row into the buffer. When the buffer is full or function flush() is called, it tries to flush multiple data into Kudu server. Firstly, it groups these data according to the tablet id as a batch. A batch may contains multiple rows which belong to the same tablet. Then a batch will encode into bytes. At this time, it reads the table schema of the first row and decides the format of the data. If two rows has different schema but belongs to the same tablet, which maybe because of altering the table between inserting two rows, it causes array index outbound exception. This patch will validate the schema of multiple rows which belong to the same tablet. If the schema is different, it puts them into the different groups as different batches. Change-Id: Ie6501962b32814d121f180b2942999c402d927db --- M java/kudu-client/src/main/java/org/apache/kudu/Schema.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java 3 files changed, 270 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/49/19949/8 -- To view, visit http://gerrit.cloudera.org:8080/19949 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db Gerrit-Change-Number: 19949 Gerrit-PatchSet: 8 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [java] add buffer space limit for KuduSession
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19918 ) Change subject: [java] add buffer space limit for KuduSession .. Patch Set 8: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I312d47c98566f9405361d969a4b68b326bb3c4d9 Gerrit-Change-Number: 19918 Gerrit-PatchSet: 8 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 15 Jun 2023 03:48:35 + Gerrit-HasComments: No
[kudu-CR] [java] add buffer space limit for KuduSession
KeDeng has posted comments on this change. ( http://gerrit.cloudera.org:8080/19918 ) Change subject: [java] add buffer space limit for KuduSession .. Patch Set 7: (7 comments) Thanks for your reviews. http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG@11 PS6, Line 11: buffer space > buffer space Done http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG@11 PS6, Line 11: configuring > nit: configuring Same spelling? http://gerrit.cloudera.org:8080/#/c/19918/5/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java: http://gerrit.cloudera.org:8080/#/c/19918/5/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@751 PS5, Line 751: : > Sorry I didn't get the point of how can it be more complex. If add a functi Sorry for misunderstood your meaning at the beginning. I have re implemented this part of the logic according to your suggestion. http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java: http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@649 PS6, Line 649: isExcessMaxSiz > what about isExcessMaxSize ? Done http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@894 PS6, Line 894: flu > nit: from Done http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java File java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java: http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java@410 PS6, Line 410: assertFalse(resp.hasRowError()); > How about use assertEventuallyTrue? Done http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java@410 PS6, Line 410: assertFalse(resp.hasRowError()); > If session.hasPendingOperations() is always true, it will be an infinity lo Done -- To view, visit http://gerrit.cloudera.org:8080/19918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I312d47c98566f9405361d969a4b68b326bb3c4d9 Gerrit-Change-Number: 19918 Gerrit-PatchSet: 7 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 15 Jun 2023 03:44:15 + Gerrit-HasComments: Yes
[kudu-CR] [java] add buffer space limit for KuduSession
Hello Yuqi Du, Yingchun Lai, Kudu Jenkins, Wang Xixu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19918 to look at the new patch set (#8). Change subject: [java] add buffer space limit for KuduSession .. [java] add buffer space limit for KuduSession Currently, KuduSession only supports limiting the number of operations. In this patch, I added the functionality of configuring the buffer space according to the data size for KuduSession. To verify that the new feature works effectively, I also added corresponding unit tests. Change-Id: I312d47c98566f9405361d969a4b68b326bb3c4d9 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduSession.java M java/kudu-client/src/main/java/org/apache/kudu/client/PartialRow.java M java/kudu-client/src/main/java/org/apache/kudu/client/SessionConfiguration.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java 5 files changed, 142 insertions(+), 27 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/18/19918/8 -- To view, visit http://gerrit.cloudera.org:8080/19918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I312d47c98566f9405361d969a4b68b326bb3c4d9 Gerrit-Change-Number: 19918 Gerrit-PatchSet: 8 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [java] add buffer space limit for KuduSession
Hello Yuqi Du, Yingchun Lai, Kudu Jenkins, Wang Xixu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19918 to look at the new patch set (#7). Change subject: [java] add buffer space limit for KuduSession .. [java] add buffer space limit for KuduSession Currently, KuduSession only supports limiting the number of operations. In this patch, I added the functionality of configuring the buffer space according to the data size for KuduSession. To verify that the new feature works effectively, I also added corresponding unit tests. Change-Id: I312d47c98566f9405361d969a4b68b326bb3c4d9 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduSession.java M java/kudu-client/src/main/java/org/apache/kudu/client/PartialRow.java M java/kudu-client/src/main/java/org/apache/kudu/client/SessionConfiguration.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java 5 files changed, 142 insertions(+), 27 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/18/19918/7 -- To view, visit http://gerrit.cloudera.org:8080/19918 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I312d47c98566f9405361d969a4b68b326bb3c4d9 Gerrit-Change-Number: 19918 Gerrit-PatchSet: 7 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [duplication] KUDU-3290 Support write ops to kafka with kafka client
Hello Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19909 to look at the new patch set (#3). Change subject: [duplication] KUDU-3290 Support write ops to kafka with kafka client .. [duplication] KUDU-3290 Support write ops to kafka with kafka client This patch is the first patch for duplication, it includes: 1. adding 2 thirdparty libraries: cppkafka, librdkafka. librdkfka supports c language apis to operate kafka, and cppkafka is a C++ wrapper for librdkakfa, its an api sugger. 2. Duplicator: write records to kafka As my plans, I will commit about 9 commits for duplication. 1. Support write ops to kafka with kafka client 2. Add a new raft role DUPLICATOR and raft processing 3. registe kafka uri and validate and unregiste 4. recovering duplication when a leader switches 5. creating/altering in server-side for duplication 6. cpp client apis and CLI tool 7. java client apis 8. suppports kerberos 9. alter a duplication During this progressing, plans maybe adjust according to advices. Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 --- M CMakeLists.txt A cmake_modules/FindCppKafka.cmake A cmake_modules/FindRdKafka.cmake M src/kudu/common/types-test.cc M src/kudu/common/types.cc M src/kudu/common/types.h M src/kudu/consensus/consensus.proto M src/kudu/consensus/metadata.proto A src/kudu/duplicator/CMakeLists.txt A src/kudu/duplicator/connector.cc A src/kudu/duplicator/connector.h A src/kudu/duplicator/connector_manager.h A src/kudu/duplicator/duplicator.cc A src/kudu/duplicator/duplicator.h A src/kudu/duplicator/kafka/kafka.proto A src/kudu/duplicator/kafka/kafka_connector-test.cc A src/kudu/duplicator/kafka/kafka_connector.cc A src/kudu/duplicator/kafka/kafka_connector.h A src/kudu/integration-tests/single_broker_kafka.h A src/kudu/scripts/kafka-simple-control.sh M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/ops/write_op.h M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_replica.cc M src/kudu/tablet/tablet_replica.h M src/kudu/tserver/tablet_server.cc M src/kudu/tserver/tablet_server.h M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 33 files changed, 2,126 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/19909/3 -- To view, visit http://gerrit.cloudera.org:8080/19909 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icbd5738a61d2521b363c628f88e3699879920a49 Gerrit-Change-Number: 19909 Gerrit-PatchSet: 3 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du
[kudu-CR] [util] KUDU-3392: add validator for --trusted certificate file
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20075 Change subject: [util] KUDU-3392: add validator for --trusted_certificate_file .. [util] KUDU-3392: add validator for --trusted_certificate_file While testing/troubleshooting the newly introduced JWT-based client authentication, I found that setting the flag to a non-existing file hadn't been handled as I expected. This patch addresses the issue, adding a validator for the flag. I didn't add any test to cover the newly added validator, but I have verified that it worked as expected in various scenarios w.r.t. the current value of the --trusted_certificate_file flag. This is a follow-up to 152211658ef9d33e0ad727ccba46f8af24cd45b0. Change-Id: I0d9d816a821a93037293d3985a2f577711d64ef2 --- M src/kudu/util/curl_util.cc 1 file changed, 44 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/75/20075/1 -- To view, visit http://gerrit.cloudera.org:8080/20075 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I0d9d816a821a93037293d3985a2f577711d64ef2 Gerrit-Change-Number: 20075 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin
[kudu-CR] WIP KUDU-3458 Startup tserver even if some metadata is corrupted
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20067 to look at the new patch set (#3). Change subject: WIP KUDU-3458 Startup tserver even if some metadata is corrupted .. WIP KUDU-3458 Startup tserver even if some metadata is corrupted kudu-tserver should not stop loading if some tablet metadatas are corrupt. If a power failure/ kerner panic etc. whould cause it, tserver could just bootstrap the other tablets and run. The system catalog will eventually detect that some tablets are underreplicated and would re-replicate them either on this server or elsewhere. This also allows for a 3 tserver setup with RF=3 to automatically restore itself if no other error than a metadata corruption is present. Change-Id: Id44abaef5238b1de99350d8db43494a7120de608 --- M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h 2 files changed, 27 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/67/20067/3 -- To view, visit http://gerrit.cloudera.org:8080/20067 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id44abaef5238b1de99350d8db43494a7120de608 Gerrit-Change-Number: 20067 Gerrit-PatchSet: 3 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19949 ) Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/19949/7//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19949/7//COMMIT_MSG@20 PS7, Line 20: This patch will validate the schema of multiple rows which belong : to the same tablet. If the schema is different, it puts them into : the different groups as different batches. Another idea is to trigger a flush() in apply(op) when found op's schema is different (add/drop columns) with the buffered ops' schema in AUTO_FLUSH_BACKGROUND mode. In this way, the logic in TabletLookupCB can be kept as before. -- To view, visit http://gerrit.cloudera.org:8080/19949 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db Gerrit-Change-Number: 19949 Gerrit-PatchSet: 7 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 14 Jun 2023 16:17:47 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19949 ) Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/19949/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java: http://gerrit.cloudera.org:8080/#/c/19949/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@417 PS6, Line 417: reAllNotF > It is hard to handle this case. For example, if there are 5 operations, op1 Only "add columns" will cause this issue, other alter ops like renaming column, deleting column are safe? If they are, compare with the Schema objects is too heavy IMO. -- To view, visit http://gerrit.cloudera.org:8080/19949 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db Gerrit-Change-Number: 19949 Gerrit-PatchSet: 7 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 14 Jun 2023 15:58:51 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Yingchun Lai has posted comments on this change. (
http://gerrit.cloudera.org:8080/19949 )
Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed
..
Patch Set 7:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
File
java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@346
PS7, Line 346: batches by tablet
nit: The comments have to be updated.
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@423
PS7, Line 423: batches.size() + 1
Should be:
batches.size() + extraBatches.size() + 1 ?
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java:
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java@139
PS7, Line 139: AUTO_FLUSH_BACKGROUND
I see you flush manually below, why set the mode as AUTO_FLUSH_BACKGROUND here?
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java@151
PS7, Line 151: row2.addInt("c0", 101);
: row2.addInt("c1", 101);
:
Is it too special that upsert on the same row? It's enough to cover the case
you fix to make sure they are on the same tablet, right? (You can create a
table with a single partiton to make sure the 2 rows are on the same tablet)
http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java@190
PS7, Line 190:
Also check we can read the rows correctly.
--
To view, visit http://gerrit.cloudera.org:8080/19949
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db
Gerrit-Change-Number: 19949
Gerrit-PatchSet: 7
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Wed, 14 Jun 2023 15:36:11 +
Gerrit-HasComments: Yes
[kudu-CR] WIP KUDU-3458 Startup tserver even if some metadata is corrupted
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/20067 to look at the new patch set (#2). Change subject: WIP KUDU-3458 Startup tserver even if some metadata is corrupted .. WIP KUDU-3458 Startup tserver even if some metadata is corrupted kudu-tserver should not stop loading if some tablet metadatas are corrupt. If a power failure/ kerner panic etc. whould cause it, tserver could just bootstrap the other tablets and run. The system catalog will eventually detect that some tablets are underreplicated and would re-replicate them either on this server or elsewhere. This also allows for a 3 tserver setup with RF=3 to automatically restore itself if no other error than a metadata corruption is present. Change-Id: Id44abaef5238b1de99350d8db43494a7120de608 --- M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h 2 files changed, 26 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/67/20067/2 -- To view, visit http://gerrit.cloudera.org:8080/20067 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id44abaef5238b1de99350d8db43494a7120de608 Gerrit-Change-Number: 20067 Gerrit-PatchSet: 2 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] WIP KUDU-3458 Startup tserver even if some metadata is corrupted
zmarto...@cloudera.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20067 Change subject: WIP KUDU-3458 Startup tserver even if some metadata is corrupted .. WIP KUDU-3458 Startup tserver even if some metadata is corrupted kudu-tserver should not stop loading if some tablet metadatas are corrupt. If a power failure/ kerner etc. whould cause it, tserver could just bootstrap the other tablets and run. The system catalog will eventually detect that some tablets are underreplicated and would re-replicate them either on this server or elsewhere. This also allows for a 3 tserver setup with RF=3 to automatically restore itself if no other error than the metadata corruption is present. Change-Id: Id44abaef5238b1de99350d8db43494a7120de608 --- M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h 2 files changed, 26 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/67/20067/1 -- To view, visit http://gerrit.cloudera.org:8080/20067 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id44abaef5238b1de99350d8db43494a7120de608 Gerrit-Change-Number: 20067 Gerrit-PatchSet: 1 Gerrit-Owner: Anonymous Coward
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19949 ) Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed .. Patch Set 7: (2 comments) http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java: http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@372 PS7, Line 372: extraBatches is used to store operations when found table has changed : // compared with previous rows. How about: 'extraBatches' is used to store batches of operations which have different schemas to the previous ones. http://gerrit.cloudera.org:8080/#/c/19949/7/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@415 PS7, Line 415: batch nit: !batch -- To view, visit http://gerrit.cloudera.org:8080/19949 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db Gerrit-Change-Number: 19949 Gerrit-PatchSet: 7 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 14 Jun 2023 15:09:02 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19949 ) Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed .. Patch Set 7: (7 comments) http://gerrit.cloudera.org:8080/#/c/19949/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java: http://gerrit.cloudera.org:8080/#/c/19949/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@371 PS6, Line 371: // Extra batches for operations which have different table schemas. > nit: Done http://gerrit.cloudera.org:8080/#/c/19949/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@372 PS6, Line 372: hes is used > Could you please add some comments to clearify the relationship between 'ba Done http://gerrit.cloudera.org:8080/#/c/19949/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@413 PS6, Line 413: > nit: schemas Done http://gerrit.cloudera.org:8080/#/c/19949/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@414 PS6, Line 414: se if (!batch.operations.is > nit: !batch.operations.isEmpty() Done http://gerrit.cloudera.org:8080/#/c/19949/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@414 PS6, Line 414: } > It would be better to use 'else if', becaue it not possible that the 'batch Done http://gerrit.cloudera.org:8080/#/c/19949/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@415 PS6, Line 415: eq > nit: equals ? Done http://gerrit.cloudera.org:8080/#/c/19949/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@417 PS6, Line 417: reAllNotF > Is it possible that there are ops to be sent in schema version 1, 2, 1 ? Wh It is hard to handle this case. For example, if there are 5 operations, op1, op2, op3, op4, op5. op1 and op2 have the same schema, op2 and op4 have the same schema, op5 has a schema different from other operations. There will be 3 groups. It is high-price cost to compare the schema with others operations. Therefore here just simplify it, if found an operation with different schema, put it in extra_batch -- To view, visit http://gerrit.cloudera.org:8080/19949 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db Gerrit-Change-Number: 19949 Gerrit-PatchSet: 7 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 14 Jun 2023 13:40:06 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3483] Fixbug of auto flush data when table schema changed
Hello Yuqi Du, Alexey Serbin, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19949 to look at the new patch set (#7). Change subject: [KUDU-3483] Fixbug of auto flush data when table schema changed .. [KUDU-3483] Fixbug of auto flush data when table schema changed In auto_flush_background mode, applying an operation firstly inserts the row into the buffer. When the buffer is full or function flush() is called, it tries to flush multiple data into Kudu server. Firstly, it groups these data according to the tablet id as a batch. A batch may contains multiple rows which belong to the same tablet. Then a batch will encode into bytes. At this time, it reads the table schema of the first row and decides the format of the data. If two rows has different schema but belongs to the same tablet, which maybe because of altering the table between inserting two rows, it causes array index outbound exception. This patch will validate the schema of multiple rows which belong to the same tablet. If the schema is different, it puts them into the different groups as different batches. Change-Id: Ie6501962b32814d121f180b2942999c402d927db --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAlterTable.java 2 files changed, 157 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/49/19949/7 -- To view, visit http://gerrit.cloudera.org:8080/19949 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie6501962b32814d121f180b2942999c402d927db Gerrit-Change-Number: 19949 Gerrit-PatchSet: 7 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3483] Log requestor info and request id for write interface
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19950 ) Change subject: [KUDU-3483] Log requestor info and request id for write interface .. Patch Set 2: Code-Review+1 (2 comments) http://gerrit.cloudera.org:8080/#/c/19950/2/src/kudu/tserver/tablet_service.cc File src/kudu/tserver/tablet_service.cc: http://gerrit.cloudera.org:8080/#/c/19950/2/src/kudu/tserver/tablet_service.cc@1544 PS2, Line 1544: request_id() Deal with the case when request_id() is nullptr. http://gerrit.cloudera.org:8080/#/c/19950/2/src/kudu/tserver/tablet_service.cc@1544 PS2, Line 1544: SecureDebugString Is it necessary to use SecureDebugString on request_id()? -- To view, visit http://gerrit.cloudera.org:8080/19950 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0a3bacd947146fb52f94ab09dda94c96ed6d8ab8 Gerrit-Change-Number: 19950 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Wed, 14 Jun 2023 12:00:24 + Gerrit-HasComments: Yes
[kudu-CR] [tool] Fix outputs 'kudu pbc dump instance' when using -json/-json pretty
Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,
Abhishek Chennaka, KeDeng, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18330
to look at the new patch set (#17).
Change subject: [tool] Fix outputs 'kudu pbc dump instance' when using
-json/-json_pretty
..
[tool] Fix outputs 'kudu pbc dump instance' when using -json/-json_pretty
The output of two commands below about the 'uuid' field are not the same
1. kudu pbc dump xxx/instance -debug/default
2. kudu pbc dump xxx/instance -json/-json_pretty
For example:
1. kudu pbc dump instance, which is a plain output format.
Message 0
---
uuid: "ac6b3e392cb34b4681e54a33c95fdf9c"
format_stamp: "Formatted at 2023-03-10 09:16:22 on 2a43db713d56"
2. kudu pbc dump instance --json, which is a json output format
{"uuid":"YWM2YjNlMzkyY2IzNGI0NjgxZTU0YTMzYzk1ZmRmOWM=","formatStamp":"Formatted
at 2023-03-10 09:16:22 on 2a43db713d56"}
The reason is the type of 'uuid' is bytes and protobuf's bytes type is
base64-encoded.
'--debug/default' mode shows uuid's plain text while '--debug/default'
mode uses protobuf's API and shows base64-encoded text.
This patch fixes this problem.
Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
---
M src/kudu/fs/fs.proto
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_pbc.cc
M src/kudu/util/easy_json.cc
M src/kudu/util/easy_json.h
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
M src/kudu/util/pb_util.proto
8 files changed, 103 insertions(+), 24 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/18330/17
--
To view, visit http://gerrit.cloudera.org:8080/18330
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia21afb03d9b7b4b2d4ea5aaa642701451282bebf
Gerrit-Change-Number: 18330
Gerrit-PatchSet: 17
Gerrit-Owner: Yuqi Du
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
[kudu-CR] KUDU-3477 Function to get flags of minicluster daemons.
Hello Zoltan Chovan, Yingchun Lai, Attila Bukor, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19938 to look at the new patch set (#4). Change subject: KUDU-3477 Function to get flags of minicluster daemons. .. KUDU-3477 Function to get flags of minicluster daemons. Having a call which returns flags set for a given daemon in a minicluster will be useful for some additional testing when upgrading the python client. This functionality is already present in server_base.proto (GetFlagsRequestPB), only the layer for the minicluster is missing. Change-Id: I3480bfccce18c4605ed9b4ed1dab367fa1fd525a --- M src/kudu/mini-cluster/external_mini_cluster.cc M src/kudu/mini-cluster/external_mini_cluster.h M src/kudu/tools/CMakeLists.txt M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_test.cc 6 files changed, 72 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/38/19938/4 -- To view, visit http://gerrit.cloudera.org:8080/19938 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3480bfccce18c4605ed9b4ed1dab367fa1fd525a Gerrit-Change-Number: 19938 Gerrit-PatchSet: 4 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Zoltan Chovan
[kudu-CR](branch-1.17.x) [docs] Add release notes for 1.17.0
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19512 ) Change subject: [docs] Add release notes for 1.17.0 .. Patch Set 9: (9 comments) http://gerrit.cloudera.org:8080/#/c/19512/6/docs/release_notes.adoc File docs/release_notes.adoc: http://gerrit.cloudera.org:8080/#/c/19512/6/docs/release_notes.adoc@73 PS6, Line 73: table-wide > nit: table-wide Done http://gerrit.cloudera.org:8080/#/c/19512/6/docs/release_notes.adoc@78 PS6, Line 78: soft-deleted table > nit: 'soft deletion of tables' or 'soft-deleted tables' Done http://gerrit.cloudera.org:8080/#/c/19512/6/docs/release_notes.adoc@80 PS6, Line 80: DeleteTable > DeleteTable Done http://gerrit.cloudera.org:8080/#/c/19512/6/docs/release_notes.adoc@90 PS6, Line 90: is adde > is Done http://gerrit.cloudera.org:8080/#/c/19512/6/docs/release_notes.adoc@103 PS6, Line 103: e newly introduced `--enable_jwt_token_auth` is set > Introduced an experimental feature: authentication using JWT tokens ... Done http://gerrit.cloudera.org:8080/#/c/19512/6/docs/release_notes.adoc@104 PS6, Line 104: be sent by the client which will be verifi > `--enable_jwt_token_auth` is set to `true` Done http://gerrit.cloudera.org:8080/#/c/19512/6/docs/release_notes.adoc@108 PS6, Line 108: y bytes of data each token > use the newly introduced `SetSplitSizeBytes()` method of the Kudu client AP Done http://gerrit.cloudera.org:8080/#/c/19512/6/docs/release_notes.adoc@115 PS6, Line 115: all its > all its metrics except for string gauges Done http://gerrit.cloudera.org:8080/#/c/19512/6/docs/release_notes.adoc@115 PS6, Line 115: ring gauges in Prometheus format via the embedd > via the embedded webserver's `/metrics_prometheus` endpoint Done -- To view, visit http://gerrit.cloudera.org:8080/19512 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: I4381e7824dcf7f500165aa3af9dca09a86ac1dd9 Gerrit-Change-Number: 19512 Gerrit-PatchSet: 9 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Hongjiang Zhang Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Khazar Mammadli Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: Zoltan Chovan Gerrit-Reviewer: bsglz <18031...@qq.com> Gerrit-Reviewer: harinreddy Gerrit-Reviewer: yejiabao Gerrit-Reviewer: zhanghongbin <1079039...@qq.com> Gerrit-Reviewer: Ádám Bakai Gerrit-Comment-Date: Wed, 14 Jun 2023 09:35:56 + Gerrit-HasComments: Yes
[kudu-CR](branch-1.17.x) [docs] Add release notes for 1.17.0
Hello Marton Greber, Alexey Serbin, Kurt Deschler, Yuqi Du, Andrew Wong, Yifan Zhang, Khazar Mammadli, Joe McDonnell, Zoltan Chovan, Ashwani Raina, Attila Bukor, KeDeng, Ádám Bakai, Mahesh Reddy, zhanghongbin, Wenzhe Zhou, bsglz, harinreddy, Riza Suminto, Hongjiang Zhang, Kudu Jenkins, Abhishek Chennaka, yejiabao, Wang Xixu, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19512 to look at the new patch set (#9). Change subject: [docs] Add release notes for 1.17.0 .. [docs] Add release notes for 1.17.0 Change-Id: I4381e7824dcf7f500165aa3af9dca09a86ac1dd9 --- M docs/release_notes.adoc 1 file changed, 253 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/12/19512/9 -- To view, visit http://gerrit.cloudera.org:8080/19512 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4381e7824dcf7f500165aa3af9dca09a86ac1dd9 Gerrit-Change-Number: 19512 Gerrit-PatchSet: 9 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Hongjiang Zhang Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Khazar Mammadli Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: Zoltan Chovan Gerrit-Reviewer: bsglz <18031...@qq.com> Gerrit-Reviewer: harinreddy Gerrit-Reviewer: yejiabao Gerrit-Reviewer: zhanghongbin <1079039...@qq.com> Gerrit-Reviewer: Ádám Bakai
[kudu-CR] [java] add buffer space limit for KuduSession
Yuqi Du has posted comments on this change. (
http://gerrit.cloudera.org:8080/19918 )
Change subject: [java] add buffer space limit for KuduSession
..
Patch Set 6:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG@11
PS6, Line 11: configuring
nit: configuring
http://gerrit.cloudera.org:8080/#/c/19918/6//COMMIT_MSG@11
PS6, Line 11: Buffer Space
buffer space
http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
File
java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@649
PS6, Line 649: isAboveMaxSize
what about isExcessMaxSize ?
http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@894
PS6, Line 894: with
nit: from
http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java
File
java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/19918/6/java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java@410
PS6, Line 410: while (session.hasPendingOperations()) {
If session.hasPendingOperations() is always true, it will be an infinity loop.
Our expecting is session.hasPendingOperations() = false very soon. So should
give it a limit time, assertEventuallyTrue is ok.
I think you can comments why sleep to wait at this
--
To view, visit http://gerrit.cloudera.org:8080/19918
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I312d47c98566f9405361d969a4b68b326bb3c4d9
Gerrit-Change-Number: 19918
Gerrit-PatchSet: 6
Gerrit-Owner: KeDeng
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Wed, 14 Jun 2023 08:39:26 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3413 add tenant info in metadata for multi-tenancy
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19622 ) Change subject: KUDU-3413 add tenant info in metadata for multi-tenancy .. Patch Set 17: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19622 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9e450d73940eb1dbaac6f905a46d6ccd084f15cf Gerrit-Change-Number: 19622 Gerrit-PatchSet: 17 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 14 Jun 2023 08:16:50 + Gerrit-HasComments: No
[kudu-CR] [java] add buffer space limit for KuduSession
Yingchun Lai has posted comments on this change. (
http://gerrit.cloudera.org:8080/19918 )
Change subject: [java] add buffer space limit for KuduSession
..
Patch Set 6: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19918/5/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
File
java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/19918/5/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@751
PS5, Line 751: (activeBufferOps + 1 >= mutationBufferMaxOps ||
: isAboveMaxSize(activeBufferSize +
operation.getRow().size()
> I'm afraid doing so will make the code more complex. At present, the logic
Sorry I didn't get the point of how can it be more complex. If add a function
like:
boolean needFlush(int activeBufferOps, long activeBufferSize) {
...
}
the caller here is:
if (needFlush(activeBufferOps + 1, activeBufferSize +
operation.getRow().size()) &&
inactiveBufferAvailable()) {
...
}
--
To view, visit http://gerrit.cloudera.org:8080/19918
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I312d47c98566f9405361d969a4b68b326bb3c4d9
Gerrit-Change-Number: 19918
Gerrit-PatchSet: 6
Gerrit-Owner: KeDeng
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Wed, 14 Jun 2023 08:10:38 +
Gerrit-HasComments: Yes
[kudu-CR] [server] KUDU-1945 Auto incrementing column Upsert support
Yuqi Du has posted comments on this change. ( http://gerrit.cloudera.org:8080/20061 ) Change subject: [server] KUDU-1945 Auto_incrementing column Upsert support .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/20061/1/src/kudu/common/row_operations.cc File src/kudu/common/row_operations.cc: http://gerrit.cloudera.org:8080/#/c/20061/1/src/kudu/common/row_operations.cc@478 PS1, Line 478: op->type == RowOperationsPB_Type_UPSERT || : op->type == RowOperationsPB_Type_UPSERT_IGNORE RowOperationsPB_Type_UPSERT means RowOperationsPB_Type_INSERT or RowOperationsPB_Type_UPDATE, according to context, I guess it is for RowOperationsPB_Type_UPDATE. So should we add RowOperationsPB_Type_UPDATE at this? http://gerrit.cloudera.org:8080/#/c/20061/1/src/kudu/common/row_operations.cc@491 PS1, Line 491: "column value"); nit: alignment http://gerrit.cloudera.org:8080/#/c/20061/1/src/kudu/common/row_operations.cc@495 PS1, Line 495: // Check if the provided counter value is less than what is in memory : // and update the counter in memory. This auto-incrementing counter is set by the user at client? If a user set a smaller value, server side will ignore this set and change it to a bigger one using an obscure way for users. In kudu, this is correct, but for users it seems not good enough. Whether it is necessary to provide more response messages or return not ok? -- To view, visit http://gerrit.cloudera.org:8080/20061 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib5cc4d80f77c165452572948f68c76fc70394d47 Gerrit-Change-Number: 20061 Gerrit-PatchSet: 1 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 14 Jun 2023 08:07:02 + Gerrit-HasComments: Yes
