[kudu-CR] [tool] Add a '-list statistics' flag to the 'kudu table list'.
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/17912 ) Change subject: [tool] Add a '-list_statistics' flag to the 'kudu table list'. .. Patch Set 10: (2 comments) http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/master/master.proto File src/kudu/master/master.proto: http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/master/master.proto@597 PS10, Line 597: record_number nit: Maybe use x_count instead of x_number for better clarity? http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/tools/tool_action_table.cc File src/kudu/tools/tool_action_table.cc: http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/tools/tool_action_table.cc@107 PS10, Line 107: like tablet number nit: maybe replace it with "such as number of tablets"? Same with replica number and record number -- To view, visit http://gerrit.cloudera.org:8080/17912 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04e9fba724648cd2b774d031e884c5a25cc4084f Gerrit-Change-Number: 17912 Gerrit-PatchSet: 10 Gerrit-Owner: yejiabao Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: yejiabao Gerrit-Comment-Date: Thu, 21 Oct 2021 04:26:51 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-3326 : Add soft delete table supports
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17917 to look at the new patch set (#13). Change subject: KUDU-3326 : Add soft delete table supports .. KUDU-3326 : Add soft delete table supports Kudu system will not delete the table immediately after receiving the command to delete the table. Instead, it will mark the table and set a validity period. After the validity period, will try again to determine whether the table really needs to be deleted. Change-Id: I3d1dddfbca55a5c4bcac4028157325ad618ea665 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/DeleteTableRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java A java/kudu-client/src/main/java/org/apache/kudu/client/RecallDeletedTableRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/RecallDeletedTableResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthTokenReacquire.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestClientFailoverSupport.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client-test.cc M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/master_proxy_rpc.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/common/common.proto M src/kudu/common/wire_protocol.cc M src/kudu/common/wire_protocol.h M src/kudu/integration-tests/alter_table-test.cc M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.cc M src/kudu/master/master.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/master_service.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_table.cc 36 files changed, 1,174 insertions(+), 96 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/17917/13 -- To view, visit http://gerrit.cloudera.org:8080/17917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3d1dddfbca55a5c4bcac4028157325ad618ea665 Gerrit-Change-Number: 17917 Gerrit-PatchSet: 13 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-3297 fix Thrift client used for HMS integration
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17958 Change subject: KUDU-3297 fix Thrift client used for HMS integration .. KUDU-3297 fix Thrift client used for HMS integration As it turns out, in the context of KUDU-3297, the SASL negotiation code needs to be updated in one more place: src/kudu/thrift/sasl_client_transport.cc I also thought about unifying the code between the Thrift client and the RPC code to have a single place to have the correct ordering between the calls EnableProtection() and sasl_client_start(), but after some consideration I realized it's not worth it. As for the testing, I verified that before this patch the following scenarios in hms_client-test were failing every time when running on RedHat/CentOS 8.4: * ProtectionTypes/HmsClientTest.TestHmsOperations/1 * ProtectionTypes/HmsClientTest.TestHmsOperations/3 * ProtectionTypes/HmsClientTest.TestLargeObjects/1 * ProtectionTypes/HmsClientTest.TestLargeObjects/3 The output of the failed test scenarios always contained the following: Bad status: Runtime error: failed to open Hive Metastore connection: SASL(-15): mechanism too weak for this user: With this patch, all scenarios of the hms_client-test pass when running on RedHat/CentOS 8.4: This is a follow-up to fff48ea4e5eadd365a85a05a82f66b3eb76d0b0b. Change-Id: Ic6af12932647eda7092f9f42a57eb211fe31f062 --- M src/kudu/rpc/client_negotiation.cc M src/kudu/thrift/sasl_client_transport.cc 2 files changed, 14 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/17958/1 -- To view, visit http://gerrit.cloudera.org:8080/17958 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic6af12932647eda7092f9f42a57eb211fe31f062 Gerrit-Change-Number: 17958 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin
[kudu-CR] [docs] suggest installing lsof if running tests
Hello Attila Bukor, Kudu Jenkins, Bankim Bhavsar, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17957 to look at the new patch set (#2). Change subject: [docs] suggest installing lsof if running tests .. [docs] suggest installing lsof if running tests Some of the C++ tests (e.g., tests related to HMS integration) use lsof to determine whether a third-party component has successfully bound to a TCP port, so lsof is a prerequisite if running such tests. Change-Id: I2281d29d5c6399840c3d6405339873bd14d90b01 --- M docs/installation.adoc 1 file changed, 24 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/17957/2 -- To view, visit http://gerrit.cloudera.org:8080/17957 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2281d29d5c6399840c3d6405339873bd14d90b01 Gerrit-Change-Number: 17957 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [docs] suggest installing lsof if running tests
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17957 Change subject: [docs] suggest installing lsof if running tests .. [docs] suggest installing lsof if running tests Some of the C++ tests (e.g., tests related to HMS integration) use lsof to determine whether a third-party component has successfully bound to a TCP port, so lsof is a prerequisite if running such tests. Change-Id: I2281d29d5c6399840c3d6405339873bd14d90b01 --- M docs/installation.adoc 1 file changed, 6 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/57/17957/1 -- To view, visit http://gerrit.cloudera.org:8080/17957 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2281d29d5c6399840c3d6405339873bd14d90b01 Gerrit-Change-Number: 17957 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin
[kudu-CR] KUDU-3326 : Add soft delete table supports
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17917 to look at the new patch set (#12). Change subject: KUDU-3326 : Add soft delete table supports .. KUDU-3326 : Add soft delete table supports Kudu system will not delete the table immediately after receiving the command to delete the table. Instead, it will mark the table and set a validity period. After the validity period, will try again to determine whether the table really needs to be deleted. Change-Id: I3d1dddfbca55a5c4bcac4028157325ad618ea665 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/DeleteTableRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java A java/kudu-client/src/main/java/org/apache/kudu/client/RecallDeletedTableRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/RecallDeletedTableResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthTokenReacquire.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestClientFailoverSupport.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client-test.cc M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/master_proxy_rpc.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/common/common.proto M src/kudu/common/wire_protocol.cc M src/kudu/common/wire_protocol.h M src/kudu/integration-tests/alter_table-test.cc M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.cc M src/kudu/master/master.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/master_service.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_table.cc 36 files changed, 1,174 insertions(+), 96 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/17917/12 -- To view, visit http://gerrit.cloudera.org:8080/17917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3d1dddfbca55a5c4bcac4028157325ad618ea665 Gerrit-Change-Number: 17917 Gerrit-PatchSet: 12 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-1959 - Implement server startup progress page for tablet and master servers
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17730 to look at the new patch set (#24). Change subject: KUDU-1959 - Implement server startup progress page for tablet and master servers .. KUDU-1959 - Implement server startup progress page for tablet and master servers This patch implements adjusting the server startup sequence to start the web server and load the startup page which shows the progress of the startup. This progress is broken down into the below steps: Initializing Reading Filesystem Reading instance metadata files Opening container files Bootstrapping and opening the tablets Starting RPC server Sample screenshot of the page can be found here: https://i.imgur.com/J4NJD5i.png Each of the steps above has a progress status of either 0 or 100 except for the step “Bootstrapping and opening the tablets” which tracks the processing of each of the tablet and the step “Opening container files”. For log block manager this step shows the progress of opening the log block containers. For file block manager, this is renamed to “Reporting Filesystem” and has only 0 or 100 progress status. All of the steps have time elapsed presented in seconds. For the master startup page, the step “Bootstrapping and opening the tablets” is replaced by “Initializing master catalog”. Along with the above startup page the web server will also display the pages - /home, /config, /logs, /mem-trackers, /memz, /threadz, /varz. During the initialization since we expect the user's primary reason to open the web UI to be to check the startup progress, /home will be redirected to /startup until the startup is completed. The usual homepage contents and the other remaining web pages will be available once the rpc server is started due to dependency. Manually validated the startup when security is enabled and filesystem is not present. The footer of the WebUI has the UUID by default. In case of new instances, since we are starting the Webserver before the UUID is generated the footer doesn’t contain UUID but it is set later when the rpc server is started. The next steps for this patch include writing tests for the startup page. So far all the testing has been manual. Change-Id: I1db1fcf16261d4ced1b3657a697766a5335271b4 --- M src/kudu/fs/block_manager.h M src/kudu/fs/file_block_manager.cc M src/kudu/fs/file_block_manager.h M src/kudu/fs/fs_manager.cc M src/kudu/fs/fs_manager.h M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h M src/kudu/master/master.cc M src/kudu/server/CMakeLists.txt M src/kudu/server/default_path_handlers.cc M src/kudu/server/default_path_handlers.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h A src/kudu/server/startup_path_handler.cc A src/kudu/server/startup_path_handler.h M src/kudu/server/tracing_path_handlers.h M src/kudu/server/webserver-test.cc M src/kudu/server/webserver.cc M src/kudu/server/webserver.h M src/kudu/tserver/tablet_server.cc M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h A src/kudu/util/timer.h M src/kudu/util/web_callback_registry.h A www/startup.mustache 25 files changed, 641 insertions(+), 71 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/17730/24 -- To view, visit http://gerrit.cloudera.org:8080/17730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1db1fcf16261d4ced1b3657a697766a5335271b4 Gerrit-Change-Number: 17730 Gerrit-PatchSet: 24 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-1959 - Implement aggregate startup progress metrics
Hello Tidy Bot, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17903 to look at the new patch set (#5). Change subject: KUDU-1959 - Implement aggregate startup progress metrics .. KUDU-1959 - Implement aggregate startup progress metrics We expose the below metrics as a part of this commit: * startup_progress_steps_remaining : count of server startup steps which are yet to be completed. This value is in the range [0,4]. * startup_progress_time_elapsed : the time elapsed so far for the server to startup. If the startup is completed, this is the total time taken for the startup. This is in milliseconds. These metrics are primarily expected to be used by third party monitoring tools to see how long has the server taken to startup historically for any sort of trend analysis. The startup_progress_time_elapsed metric can also be used to check the previous startup time as an alternative to the startup page in the WebUI. Change-Id: I0a508c3baf0a0d77baf75f36f7bb305a6ad821e1 --- M src/kudu/server/server_base.cc M src/kudu/server/startup_path_handler.cc M src/kudu/server/startup_path_handler.h 3 files changed, 58 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/03/17903/5 -- To view, visit http://gerrit.cloudera.org:8080/17903 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0a508c3baf0a0d77baf75f36f7bb305a6ad821e1 Gerrit-Change-Number: 17903 Gerrit-PatchSet: 5 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-1959 - Implement server startup progress page for tablet and master servers
Abhishek Chennaka has posted comments on this change. (
http://gerrit.cloudera.org:8080/17730 )
Change subject: KUDU-1959 - Implement server startup progress page for tablet
and master servers
..
Patch Set 22:
(10 comments)
Thanks for the review Alexey
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/fs/block_manager.h
File src/kudu/fs/block_manager.h:
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/fs/block_manager.h@226
PS22, Line 226: std::atomic* containers_processed = nullptr,
: std::atomic* containers_total =
nullptr) = 0
> Since these are cumulative counters, would it be cleaner to expose these vi
As discussed, we need a startup page needs a handle on these value to update
them as and when needed. We can use the metrics which are implemented in later
patches as an alternative way of achieving the goal in a cleaner manner. Will
raise a followup patch for that. We can do the same approach for tablets
processed/total count too probably.
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/fs/file_block_manager.h
File src/kudu/fs/file_block_manager.h:
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/fs/file_block_manager.h@83
PS22, Line 83: std::atomic* containers_processed = nullptr,
: std::atomic* containers_total = nullptr
> Passing atomics like this looks is a bit strange to me -- are these new par
Since the declaration contains these parameters, included them here. If we ever
go on a route of opening the file block containers during startup we can use
these I guess. But the cleaner approach would be the one discussed here
https://gerrit.cloudera.org/#/c/17730/22/src/kudu/fs/block_manager.h@227
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/fs/fs_manager.h
File src/kudu/fs/fs_manager.h:
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/fs/fs_manager.h@188
PS22, Line 188: spend
> nit: spent
Done
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/fs/fs_manager.h@191
PS22, Line 191: // If 'containers_processed' and 'containers_total' are not
nullptr, they will
: // be populated with total containers attempted to be
opened/processed and
: // total containers present respectively in the subsequent
calls made to
: // the block manager.
> If those are cumulative counters, why not to expose them via accessors to F
Discussed this below
https://gerrit.cloudera.org/#/c/17730/22/src/kudu/fs/block_manager.h@227
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/fs/log_block_manager.cc
File src/kudu/fs/log_block_manager.cc:
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/fs/log_block_manager.cc@2515
PS22, Line 2515: Status* result_status, std::atomic*
containers_processed,
: std::atomic* containers_total) {
> style nit: why not to keep one parameter per line, as in the original code?
Done
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/server/server_base.cc
File src/kudu/server/server_base.cc:
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/server/server_base.cc@498
PS22, Line 498: ()
> nit: you omit drop the parentheses
Done and tested too. Works.
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/server/webserver.cc
File src/kudu/server/webserver.cc:
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/server/webserver.cc@734
PS22, Line 734: // As the home page is redirected to startup until the
server's initialization is complete,
: // do not use the template for home page if the server's
initialization is not complete.
: if (!is_started_ && (path == "/home")) {
: return false;
: }
> I saw you addressed Andrew's comment from PS16, but I'm not sure this is t
Agree, moved this up and now it doesn't render the home page is the startup is
not complete.
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/util/timer.h
File src/kudu/util/timer.h:
PS22:
> I guess I missed some discussion in prior versions, but why to introduce t
Yes, there was a discussion about it but we thought stopwatch might provide
more than needed functionality for this purpose i.e. we just need total time
between two points and hence implemented as a simpler version of stopwatch
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/util/timer.h@57
PS22, Line 57: std::lock_guard l(lock_);
: return (end_time_ != MonoTime::Min());
> What if the caller called Start() second time?
Done
http://gerrit.cloudera.org:8080/#/c/17730/22/src/kudu/util/timer.h@62
PS22, Line 62: MonoTime start_time_;
: MonoTime end_time_;
: mutable simple_spinlock lock_;
> Since atomics are already used in src/kudu/server/startup_path_handler.h, w
Done
--
To view, visit http://gerrit.cloudera.org:8080/17730
To unsubscribe, visit
[kudu-CR] KUDU-1959 - Implement server startup progress page for tablet and master servers
Hello Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17730 to look at the new patch set (#23). Change subject: KUDU-1959 - Implement server startup progress page for tablet and master servers .. KUDU-1959 - Implement server startup progress page for tablet and master servers This patch implements adjusting the server startup sequence to start the web server and load the startup page which shows the progress of the startup. This progress is broken down into the below steps: Initializing Reading Filesystem Reading instance metadata files Opening container files Bootstrapping and opening the tablets Starting RPC server Sample screenshot of the page can be found here: https://i.imgur.com/J4NJD5i.png Each of the steps above has a progress status of either 0 or 100 except for the step “Bootstrapping and opening the tablets” which tracks the processing of each of the tablet and the step “Opening container files”. For log block manager this step shows the progress of opening the log block containers. For file block manager, this is renamed to “Reporting Filesystem” and has only 0 or 100 progress status. All of the steps have time elapsed presented in seconds. For the master startup page, the step “Bootstrapping and opening the tablets” is replaced by “Initializing master catalog”. Along with the above startup page the web server will also display the pages - /home, /config, /logs, /mem-trackers, /memz, /threadz, /varz. During the initialization since we expect the user's primary reason to open the web UI to be to check the startup progress, /home will be redirected to /startup until the startup is completed. The usual homepage contents and the other remaining web pages will be available once the rpc server is started due to dependency. Manually validated the startup when security is enabled and filesystem is not present. The footer of the WebUI has the UUID by default. In case of new instances, since we are starting the Webserver before the UUID is generated the footer doesn’t contain UUID but it is set later when the rpc server is started. The next steps for this patch include writing tests for the startup page. So far all the testing has been manual. Change-Id: I1db1fcf16261d4ced1b3657a697766a5335271b4 --- M src/kudu/fs/block_manager.h M src/kudu/fs/file_block_manager.cc M src/kudu/fs/file_block_manager.h M src/kudu/fs/fs_manager.cc M src/kudu/fs/fs_manager.h M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h M src/kudu/master/master.cc M src/kudu/server/CMakeLists.txt M src/kudu/server/default_path_handlers.cc M src/kudu/server/default_path_handlers.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h A src/kudu/server/startup_path_handler.cc A src/kudu/server/startup_path_handler.h M src/kudu/server/tracing_path_handlers.h M src/kudu/server/webserver-test.cc M src/kudu/server/webserver.cc M src/kudu/server/webserver.h M src/kudu/tserver/tablet_server.cc M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h A src/kudu/util/timer.h M src/kudu/util/web_callback_registry.h A www/startup.mustache 25 files changed, 641 insertions(+), 71 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/17730/23 -- To view, visit http://gerrit.cloudera.org:8080/17730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1db1fcf16261d4ced1b3657a697766a5335271b4 Gerrit-Change-Number: 17730 Gerrit-PatchSet: 23 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [tool] Add a '-list statistics' flag to the 'kudu table list'.
Bankim Bhavsar has posted comments on this change. (
http://gerrit.cloudera.org:8080/17912 )
Change subject: [tool] Add a '-list_statistics' flag to the 'kudu table list'.
..
Patch Set 10:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/client/client.h
File src/kudu/client/client.h:
http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/client/client.h@758
PS10, Line 758: list_table_info
The parameter name must match.
http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/client/client.cc
File src/kudu/client/client.cc:
http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/client/client.cc@614
PS10, Line 614: for (int i = 0; i < resp.tables_size(); i++) {
: const auto& table = resp.tables(i);
Better to use for range loop with resp.tables instead.
http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/client/client.cc@621
PS10, Line 621: list_table_infos->push_back(list_table_info);
emplace_back(std::move(list_table_info))
http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/17912/10/src/kudu/tools/kudu-tool-test.cc@4093
PS10, Line 4093: int
const int kReplicaNum
--
To view, visit http://gerrit.cloudera.org:8080/17912
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I04e9fba724648cd2b774d031e884c5a25cc4084f
Gerrit-Change-Number: 17912
Gerrit-PatchSet: 10
Gerrit-Owner: yejiabao
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: yejiabao
Gerrit-Comment-Date: Wed, 20 Oct 2021 18:14:41 +
Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [docs] Add info about Informatica-Kudu integration
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17941 ) Change subject: [docs] Add info about Informatica-Kudu integration .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/17941/4/ecosystem.md File ecosystem.md: http://gerrit.cloudera.org:8080/#/c/17941/4/ecosystem.md@145 PS4, Line 145: nit for here and below at line 146: remove the trailing spaces -- To view, visit http://gerrit.cloudera.org:8080/17941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: If81e5530b324f951a98f132ac4cc9121db5f8501 Gerrit-Change-Number: 17941 Gerrit-PatchSet: 4 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Attila Bukor Gerrit-Comment-Date: Wed, 20 Oct 2021 16:34:00 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-3326 : Add soft delete table supports
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17917 to look at the new patch set (#11). Change subject: KUDU-3326 : Add soft delete table supports .. KUDU-3326 : Add soft delete table supports Kudu system will not delete the table immediately after receiving the command to delete the table. Instead, it will mark the table and set a validity period. After the validity period, will try again to determine whether the table really needs to be deleted. Change-Id: I3d1dddfbca55a5c4bcac4028157325ad618ea665 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/DeleteTableRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java A java/kudu-client/src/main/java/org/apache/kudu/client/RecallDeletedTableRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/RecallDeletedTableResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthTokenReacquire.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestClientFailoverSupport.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client-test.cc M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/master_proxy_rpc.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/common/common.proto M src/kudu/common/wire_protocol.cc M src/kudu/common/wire_protocol.h M src/kudu/integration-tests/alter_table-test.cc M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.cc M src/kudu/master/master.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/master_service.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_table.cc 36 files changed, 1,174 insertions(+), 96 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/17917/11 -- To view, visit http://gerrit.cloudera.org:8080/17917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3d1dddfbca55a5c4bcac4028157325ad618ea665 Gerrit-Change-Number: 17917 Gerrit-PatchSet: 11 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR](gh-pages) [docs] Add info about Informatica-Kudu integration
zcho...@cloudera.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/17941 ) Change subject: [docs] Add info about Informatica-Kudu integration .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/17941/3/ecosystem.md File ecosystem.md: http://gerrit.cloudera.org:8080/#/c/17941/3/ecosystem.md@145 PS3, Line 145: Informatica® PowerExchange® is a family of products that enables retrieval of a variety of data > nit: avoid lines longer than 100 chars where possible Ack -- To view, visit http://gerrit.cloudera.org:8080/17941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: If81e5530b324f951a98f132ac4cc9121db5f8501 Gerrit-Change-Number: 17941 Gerrit-PatchSet: 4 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Attila Bukor Gerrit-Comment-Date: Wed, 20 Oct 2021 09:51:49 + Gerrit-HasComments: Yes
[kudu-CR](gh-pages) [docs] Add info about Informatica-Kudu integration
Hello Attila Bukor, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17941 to look at the new patch set (#4). Change subject: [docs] Add info about Informatica-Kudu integration .. [docs] Add info about Informatica-Kudu integration Change-Id: If81e5530b324f951a98f132ac4cc9121db5f8501 --- M ecosystem.md 1 file changed, 5 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/41/17941/4 -- To view, visit http://gerrit.cloudera.org:8080/17941 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: newpatchset Gerrit-Change-Id: If81e5530b324f951a98f132ac4cc9121db5f8501 Gerrit-Change-Number: 17941 Gerrit-PatchSet: 4 Gerrit-Owner: Anonymous Coward Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Anonymous Coward Gerrit-Reviewer: Attila Bukor
[kudu-CR] KUDU-3326 : Add soft delete table supports
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17917 to look at the new patch set (#10). Change subject: KUDU-3326 : Add soft delete table supports .. KUDU-3326 : Add soft delete table supports Kudu system will not delete the table immediately after receiving the command to delete the table. Instead, it will mark the table and set a validity period. After the validity period, will try again to determine whether the table really needs to be deleted. Change-Id: I3d1dddfbca55a5c4bcac4028157325ad618ea665 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/DeleteTableRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java A java/kudu-client/src/main/java/org/apache/kudu/client/RecallDeletedTableRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/RecallDeletedTableResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthTokenReacquire.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestClientFailoverSupport.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client-test.cc M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/master_proxy_rpc.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/common/common.proto M src/kudu/common/wire_protocol.cc M src/kudu/common/wire_protocol.h M src/kudu/integration-tests/alter_table-test.cc M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.cc M src/kudu/master/master.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/master_service.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_table.cc 36 files changed, 1,174 insertions(+), 96 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/17917/10 -- To view, visit http://gerrit.cloudera.org:8080/17917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3d1dddfbca55a5c4bcac4028157325ad618ea665 Gerrit-Change-Number: 17917 Gerrit-PatchSet: 10 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-3326 : Add soft delete table supports
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17917 to look at the new patch set (#9). Change subject: KUDU-3326 : Add soft delete table supports .. KUDU-3326 : Add soft delete table supports Kudu system will not delete the table immediately after receiving the command to delete the table. Instead, it will mark the table and set a validity period. After the validity period, will try again to determine whether the table really needs to be deleted. Change-Id: I3d1dddfbca55a5c4bcac4028157325ad618ea665 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/DeleteTableRequest.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java A java/kudu-client/src/main/java/org/apache/kudu/client/RecallDeletedTableRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/RecallDeletedTableResponse.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduSession.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthTokenReacquire.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAuthnTokenReacquireOpen.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestClientFailoverSupport.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTable.java M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client-test.cc M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/master_proxy_rpc.cc M src/kudu/client/table_alterer-internal.cc M src/kudu/client/table_alterer-internal.h M src/kudu/common/common.proto M src/kudu/common/wire_protocol.cc M src/kudu/common/wire_protocol.h M src/kudu/integration-tests/alter_table-test.cc M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.cc M src/kudu/master/master.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/master_service.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_table.cc 36 files changed, 1,174 insertions(+), 95 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/17917/9 -- To view, visit http://gerrit.cloudera.org:8080/17917 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3d1dddfbca55a5c4bcac4028157325ad618ea665 Gerrit-Change-Number: 17917 Gerrit-PatchSet: 9 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
