[kudu-CR] [common] more generic API for IN list predicate pruning
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17764 ) Change subject: [common] more generic API for IN list predicate pruning .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/17764/4/src/kudu/common/scan_spec.cc File src/kudu/common/scan_spec.cc: http://gerrit.cloudera.org:8080/#/c/17764/4/src/kudu/common/scan_spec.cc@200 PS4, Line 200: auto s = row.Set(idx, reinterpret_cast(value)); : !s.ok() > Woohoo C++17! Yep, this construct comes handy, and if I'm not mistaken some compilers supported that even before it appeared in the C++17 standard :) -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Wed, 11 Aug 2021 02:14:29 + Gerrit-HasComments: Yes
[kudu-CR] [common] more generic API for IN list predicate pruning
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17764 ) Change subject: [common] more generic API for IN list predicate pruning .. [common] more generic API for IN list predicate pruning While working on KUDU-2671, I found that the exposed internals of the PartitionSchema class doesn't allow for updating the implementation of the partition-related code to include per-range custom hash bucket schemas in a consistent manner. This patch introduces a bit more generic interface for pruning values of IN list predicates by adding a new PartitionMayContainRow() method and removes the following methods from the public API of the PartitionSchema class: * HashPartitionContainsRow() * RangePartitionContainsRow() * IsColumnSingleRangeSchema() * TryGetSingleColumnHashPartitionIndex() I also added one extra test scenario and updated existing ones to increase readability of the assertion messages if they are triggered. This is a follow-up to 6a7cadc7e and 83b8caf4f. Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Reviewed-on: http://gerrit.cloudera.org:8080/17764 Tested-by: Kudu Jenkins Reviewed-by: Mahesh Reddy Reviewed-by: Andrew Wong --- M src/kudu/common/partition.cc M src/kudu/common/partition.h M src/kudu/common/scan_spec-test.cc M src/kudu/common/scan_spec.cc M src/kudu/common/scan_spec.h 5 files changed, 417 insertions(+), 557 deletions(-) Approvals: Kudu Jenkins: Verified Mahesh Reddy: Looks good to me, but someone else must approve Andrew Wong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [common] more generic API for IN list predicate pruning
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17764 ) Change subject: [common] more generic API for IN list predicate pruning .. Patch Set 4: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/17764/4/src/kudu/common/scan_spec.cc File src/kudu/common/scan_spec.cc: http://gerrit.cloudera.org:8080/#/c/17764/4/src/kudu/common/scan_spec.cc@200 PS4, Line 200: auto s = row.Set(idx, reinterpret_cast(value)); : !s.ok() Woohoo C++17! -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Wed, 11 Aug 2021 01:25:09 + Gerrit-HasComments: Yes
[kudu-CR] [common] more generic API for IN list predicate pruning
Mahesh Reddy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17764 ) Change subject: [common] more generic API for IN list predicate pruning .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Wed, 11 Aug 2021 00:13:16 + Gerrit-HasComments: No
[kudu-CR] KUDU-1954 [maintenance] Add an optional flush-only thread pool
Andrew Wong has posted comments on this change. (
http://gerrit.cloudera.org:8080/17743 )
Change subject: KUDU-1954 [maintenance] Add an optional flush-only thread pool
..
Patch Set 3:
(5 comments)
Quick first pass so far, and questions regarding the approach.
http://gerrit.cloudera.org:8080/#/c/17743/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17743/3//COMMIT_MSG@7
PS3, Line 7: KUDU-1954
Reading through the RocksDB link
https://github.com/facebook/rocksdb/blob/4361d6d16380f619833d58225183cbfbb2c7a1dd/include/rocksdb/options.h#L599-L658
it looks like 'max_background_flushes', as well as guidance around the high
priority and low priority pool, is deprecated, in favor of a single
'max_background_jobs'. Or were you referring to a different configuration?
In general, I wonder if, for such deployments, we should lower the
--memory_pressure_percentage flag for slower deployments. I'm curious if you've
tried that, now that Kudu no longer starves compactions when under memory
pressure.
http://gerrit.cloudera.org:8080/#/c/17743/3//COMMIT_MSG@9
PS3, Line 9: In some special environments,
I'm curious whether these environments can be categorized as clusters with slow
devices (e.g. HDDs). If so, one question I have is how much this actually
improves things, especially if the bottleneck is IO.
I also suspect this patch would improve the stability of memory usage, if the
compaction threadpool is otherwise blocked on IO. It may not be, if all disks
are near saturated, but the fact that Kudu spreads data across disks might help
our case here.
http://gerrit.cloudera.org:8080/#/c/17743/3/src/kudu/util/maintenance_manager.cc
File src/kudu/util/maintenance_manager.cc:
http://gerrit.cloudera.org:8080/#/c/17743/3/src/kudu/util/maintenance_manager.cc@68
PS3, Line 68: specially
nit: "specifically"
http://gerrit.cloudera.org:8080/#/c/17743/3/src/kudu/util/maintenance_manager.cc@71
PS3, Line 71: maintenance_manager_num_flush_threads
maintenance_manager_num_threads?
http://gerrit.cloudera.org:8080/#/c/17743/3/src/kudu/util/maintenance_manager.cc@535
PS3, Line 535: return {low_io_most_logs_retained_bytes_op,
: std::move(notes),
:
LAUNCH_POOL(low_io_most_logs_retained_bytes_op->type(), flush_pool,
general_pool)};
Why bother evaluating which pool to pick here, rather than pushing that burden
onto the caller of FindBestOp()?
--
To view, visit http://gerrit.cloudera.org:8080/17743
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6dca9d87e7287cc04482a1c18b1e0eefc5f9bcb1
Gerrit-Change-Number: 17743
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai
Gerrit-Comment-Date: Tue, 10 Aug 2021 23:52:19 +
Gerrit-HasComments: Yes
[kudu-CR] [common] more generic API for IN list predicate pruning
Hello Mahesh Reddy, Tidy Bot, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17764 to look at the new patch set (#4). Change subject: [common] more generic API for IN list predicate pruning .. [common] more generic API for IN list predicate pruning While working on KUDU-2671, I found that the exposed internals of the PartitionSchema class doesn't allow for updating the implementation of the partition-related code to include per-range custom hash bucket schemas in a consistent manner. This patch introduces a bit more generic interface for pruning values of IN list predicates by adding a new PartitionMayContainRow() method and removes the following methods from the public API of the PartitionSchema class: * HashPartitionContainsRow() * RangePartitionContainsRow() * IsColumnSingleRangeSchema() * TryGetSingleColumnHashPartitionIndex() I also added one extra test scenario and updated existing ones to increase readability of the assertion messages if they are triggered. This is a follow-up to 6a7cadc7e and 83b8caf4f. Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d --- M src/kudu/common/partition.cc M src/kudu/common/partition.h M src/kudu/common/scan_spec-test.cc M src/kudu/common/scan_spec.cc M src/kudu/common/scan_spec.h 5 files changed, 417 insertions(+), 557 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/17764/4 -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [common] more generic API for IN list predicate pruning
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/17764 )
Change subject: [common] more generic API for IN list predicate pruning
..
Patch Set 3:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/17764/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17764/3//COMMIT_MSG@7
PS3, Line 7: IN(
> nit: IN list?
Done
http://gerrit.cloudera.org:8080/#/c/17764/3//COMMIT_MSG@15
PS3, Line 15: IN
> same as above
Done
http://gerrit.cloudera.org:8080/#/c/17764/3//COMMIT_MSG@23
PS3, Line 23: allow
: for easier reading the assertion messages in case if those
assertions
: ever trigger.
> nit: maybe replace with "increase readability of the assertion messages if
Done
http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/partition.h
File src/kudu/common/partition.h:
http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/partition.h@239
PS3, Line 239: IN
> nit: IN list?
Done
http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc
File src/kudu/common/scan_spec-test.cc:
http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc@540
PS3, Line 540: Schema
> nit: change to auto, more consistent with other callsites.
Done
http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc@839
PS3, Line 839: }
> nit: add space before brace
Done
http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc@914
PS3, Line 914: IN(
> nit: IN list
Done
http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc@917
PS3, Line 917: ,
> nit: remove comma
Done
http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc@965
PS3, Line 965: TEST_F(CompositeIntKeysTest, TestLiftPrimaryKeyBounds_NoBounds) {
> warning: avoid using "_" in test name "TestLiftPrimaryKeyBounds_NoBounds" a
I'm going to ignore this warning since it's not related to the changes I did
here :)
http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec.h
File src/kudu/common/scan_spec.h:
http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec.h@79
PS3, Line 79: IN
> nit: IN list
Done
--
To view, visit http://gerrit.cloudera.org:8080/17764
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d
Gerrit-Change-Number: 17764
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 10 Aug 2021 23:34:37 +
Gerrit-HasComments: Yes
[kudu-CR] [common] more generic API for IN() predicate's pruning
Mahesh Reddy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17764 ) Change subject: [common] more generic API for IN() predicate's pruning .. Patch Set 3: (9 comments) Overall lgtm, just a few nits. http://gerrit.cloudera.org:8080/#/c/17764/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17764/3//COMMIT_MSG@7 PS3, Line 7: IN( nit: IN list? http://gerrit.cloudera.org:8080/#/c/17764/3//COMMIT_MSG@15 PS3, Line 15: IN same as above http://gerrit.cloudera.org:8080/#/c/17764/3//COMMIT_MSG@23 PS3, Line 23: allow : for easier reading the assertion messages in case if those assertions : ever trigger. nit: maybe replace with "increase readability of the assertion messages if they are triggered"? http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/partition.h File src/kudu/common/partition.h: http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/partition.h@239 PS3, Line 239: IN nit: IN list? http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc File src/kudu/common/scan_spec-test.cc: http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc@540 PS3, Line 540: Schema nit: change to auto, more consistent with other callsites. http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc@839 PS3, Line 839: } nit: add space before brace http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc@914 PS3, Line 914: IN( nit: IN list http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec-test.cc@917 PS3, Line 917: , nit: remove comma http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec.h File src/kudu/common/scan_spec.h: http://gerrit.cloudera.org:8080/#/c/17764/3/src/kudu/common/scan_spec.h@79 PS3, Line 79: IN nit: IN list -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 10 Aug 2021 23:13:10 + Gerrit-HasComments: Yes
[kudu-CR] [tools] Add a tool to recover master data from tablet servers
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/9490 ) Change subject: [tools] Add a tool to recover master data from tablet servers .. Patch Set 9: (7 comments) Just a quick initial glance. http://gerrit.cloudera.org:8080/#/c/9490/9/src/kudu/tools/kudu-admin-test.cc File src/kudu/tools/kudu-admin-test.cc: http://gerrit.cloudera.org:8080/#/c/9490/9/src/kudu/tools/kudu-admin-test.cc@3020 PS9, Line 3020: shouldnt nit: shouldn't or should not http://gerrit.cloudera.org:8080/#/c/9490/9/src/kudu/tools/kudu-admin-test.cc@3133 PS9, Line 3133: from nit: drop http://gerrit.cloudera.org:8080/#/c/9490/9/src/kudu/tools/master_rebuilder.h File src/kudu/tools/master_rebuilder.h: http://gerrit.cloudera.org:8080/#/c/9490/9/src/kudu/tools/master_rebuilder.h@35 PS9, Line 35: POD object f nit: this is not a POD type since its non-static members are not PODs. If in doubt, you can check that using the template struct std::is_pod trait: https://en.cppreference.com/w/cpp/types/is_pod http://gerrit.cloudera.org:8080/#/c/9490/9/src/kudu/tools/master_rebuilder.h@81 PS9, Line 81: const RebuildReport& GetRebuildReport(); nit: make this method 'const'? http://gerrit.cloudera.org:8080/#/c/9490/9/src/kudu/tools/master_rebuilder.cc File src/kudu/tools/master_rebuilder.cc: http://gerrit.cloudera.org:8080/#/c/9490/9/src/kudu/tools/master_rebuilder.cc@63 PS9, Line 63: using master::Master; : using master::MasterOptions; : using master::SysCatalogTable; : using master::SysTablesEntryPB; : using master::SysTabletsEntryPB; : using master::TableInfo; : using master::TableMetadataLock; : using master::TabletInfo; : using master::TabletMetadataGroupLock; : using master::TabletMetadataLock; : using std::string; : using std::vector; : using strings::Substitute; : using tserver::ListTabletsResponsePB; nit: move these using declarations out from the kudu namespace http://gerrit.cloudera.org:8080/#/c/9490/9/src/kudu/tools/master_rebuilder.cc@155 PS9, Line 155: InvalidArgument Would IllegalState/Incomplete/ServiceUnavailable be better choices here? http://gerrit.cloudera.org:8080/#/c/9490/9/src/kudu/tools/master_rebuilder.cc@321 PS9, Line 321: Status s = sys_catalog.CreateNew(master.fs_manager()); As an option, would it be a safer approach to create the whole master's filesystem data directory structure in some temporary directory, perform all the necessary steps to populate the catalog with the metadata, and only in the very end move the directory into the specified destination? That way the critical interval of placing the data into the destination might be shortened. -- To view, visit http://gerrit.cloudera.org:8080/9490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If29e421d466a531ebad72e281ae27e74e458f8c6 Gerrit-Change-Number: 9490 Gerrit-PatchSet: 9 Gerrit-Owner: Will Berkeley Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Will Berkeley Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Tue, 10 Aug 2021 21:28:48 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-3204: Add a metric for amount of available space
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/17725 ) Change subject: KUDU-3204: Add a metric for amount of available space .. Patch Set 5: (10 comments) http://gerrit.cloudera.org:8080/#/c/17725/2/src/kudu/server/server_base.cc File src/kudu/server/server_base.cc: http://gerrit.cloudera.org:8080/#/c/17725/2/src/kudu/server/server_base.cc@446 PS2, Line 446: return std::max(static_cast (0), space_info.free_bytes - reserved_bytes); > Not exactly, I meant wrapping the calculation too, to make sure they're cal Done http://gerrit.cloudera.org:8080/#/c/17725/4/src/kudu/server/server_base.cc File src/kudu/server/server_base.cc: http://gerrit.cloudera.org:8080/#/c/17725/4/src/kudu/server/server_base.cc@259 PS4, Line 259: erv > nit: WAL. Same elsewhere Done http://gerrit.cloudera.org:8080/#/c/17725/4/src/kudu/server/server_base.cc@266 PS4, Line 266: "data directories space free", > We should also mention the error behavior that this returns -1 Done http://gerrit.cloudera.org:8080/#/c/17725/4/src/kudu/server/server_base.cc@431 PS4, Line 431: return shared_ptr(MemTracker::C > nit: could you also document the error semantics? Same with the other metho Done http://gerrit.cloudera.org:8080/#/c/17725/4/src/kudu/server/server_base.cc@432 PS4, Line 432: } > nit: this doesn't need to have a return value if values are being returned Done http://gerrit.cloudera.org:8080/#/c/17725/4/src/kudu/server/server_base.cc@434 PS4, Line 434: WAL/data directory's di > nit: we should keep the naming of our variables standardized as separating Done http://gerrit.cloudera.org:8080/#/c/17725/4/src/kudu/server/server_base.cc@434 PS4, Line 434: the given WAL/data directory's disk. Returns -1 in case of disk : // failure. > nit: parameters that are output values (even if they're also input values) Done http://gerrit.cloudera.org:8080/#/c/17725/4/src/kudu/server/server_base.cc@437 PS4, Line 437:int64 flag_reserved_b > As discussed with Andrew offline will add a new flag. Done http://gerrit.cloudera.org:8080/#/c/17725/4/src/kudu/server/server_base.cc@443 PS4, Line 443: should_reserve_one_percent = flag_reserved_bytes == -1; : reserved_bytes = should_reserve_one > Yes, that makes sense. Will try to ensure: As disucssed it is designed to: 1. For WAL dir failure initialize it to -1 and return -1 in case of disk failure after the TS started 2. For data dir initialize it to -1 and return -1 if any of the calls to statvfs fails. So in case of complete initial failure of WAL or all data disks, -1 will be returned. http://gerrit.cloudera.org:8080/#/c/17725/4/src/kudu/server/server_base.cc@447 PS4, Line 447: } > should_preserve_one_percent is already a boolean. You don't need further co Done -- To view, visit http://gerrit.cloudera.org:8080/17725 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I58a7419847d080498aaf431d1dab098e1af71ad0 Gerrit-Change-Number: 17725 Gerrit-PatchSet: 5 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 10 Aug 2021 18:58:38 + Gerrit-HasComments: Yes
[kudu-CR] [common] more generic API for IN() predicate's pruning
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17764 ) Change subject: [common] more generic API for IN() predicate's pruning .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/17764/2/src/kudu/common/scan_spec.cc File src/kudu/common/scan_spec.cc: http://gerrit.cloudera.org:8080/#/c/17764/2/src/kudu/common/scan_spec.cc@184 PS2, Line 184: // TODO(aserbin): is it possible that FindColumn() returns non-OK here? > From its usage in tablet_service, it seems like the answer is no. If the pr Yep, indeed. Thank you for confirmation. http://gerrit.cloudera.org:8080/#/c/17764/2/src/kudu/common/scan_spec.cc@185 PS2, Line 185: DCHECK(false); > nit: Maybe LOG(DFATAL)? Same below? Good point -- done. -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 10 Aug 2021 18:43:55 + Gerrit-HasComments: Yes
[kudu-CR] [common] more generic API for IN() predicate's pruning
Hello Mahesh Reddy, Tidy Bot, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17764 to look at the new patch set (#3). Change subject: [common] more generic API for IN() predicate's pruning .. [common] more generic API for IN() predicate's pruning While working on KUDU-2671, I found that the exposed internals of the PartitionSchema class doesn't allow for updating the implementation of the partition-related code to include per-range custom hash bucket schemas in a consistent manner. This patch introduces a bit more generic interface for pruning values of IN predicates by adding a new PartitionMayContainRow() method and removes the following methods from the public API of the PartitionSchema class: * HashPartitionContainsRow() * RangePartitionContainsRow() * IsColumnSingleRangeSchema() * TryGetSingleColumnHashPartitionIndex() I also added one extra test scenario and updated existing ones to allow for easier reading the assertion messages in case if those assertions ever trigger. This is a follow-up to 6a7cadc7e and 83b8caf4f. Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d --- M src/kudu/common/partition.cc M src/kudu/common/partition.h M src/kudu/common/scan_spec-test.cc M src/kudu/common/scan_spec.cc M src/kudu/common/scan_spec.h 5 files changed, 415 insertions(+), 556 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/17764/3 -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [java] KUDU-1921 Add ability to require authn/encryption to Java client
Attila Bukor has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17732 ) Change subject: [java] KUDU-1921 Add ability to require authn/encryption to Java client .. [java] KUDU-1921 Add ability to require authn/encryption to Java client Change-Id: Ic951b2090a4933eca70dc53b6f93cdcff5a74929 Reviewed-on: http://gerrit.cloudera.org:8080/17732 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestSecurity.java 7 files changed, 201 insertions(+), 10 deletions(-) Approvals: Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17732 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic951b2090a4933eca70dc53b6f93cdcff5a74929 Gerrit-Change-Number: 17732 Gerrit-PatchSet: 10 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1921 Add ability to require auth/encryption to C++ client
Attila Bukor has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17731 ) Change subject: KUDU-1921 Add ability to require auth/encryption to C++ client .. KUDU-1921 Add ability to require auth/encryption to C++ client Kudu servers support requiring authentication and encryption to be enabled, and clients prefer connecting in a secure way, but if a server doesn't support authentication and/or encryption, the client will silently connect insecurely, which can lead to a downgrade attack. With this patch, clients can require authentication and encryption to be set using the client API, where if such an attack is attempted, the client will fail to connect to the cluster. Change-Id: Ia3e800eb7c4e2f8787f0adf1f040d47358d29320 Reviewed-on: http://gerrit.cloudera.org:8080/17731 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin --- M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/client_builder-internal.cc M src/kudu/client/client_builder-internal.h M src/kudu/integration-tests/security-itest.cc M src/kudu/rpc/client_negotiation.cc M src/kudu/rpc/client_negotiation.h M src/kudu/rpc/messenger.cc M src/kudu/rpc/messenger.h M src/kudu/rpc/negotiation-test.cc M src/kudu/rpc/negotiation.cc M src/kudu/rpc/negotiation.h M src/kudu/rpc/reactor.cc M src/kudu/rpc/server_negotiation.cc M src/kudu/rpc/server_negotiation.h 15 files changed, 192 insertions(+), 19 deletions(-) Approvals: Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia3e800eb7c4e2f8787f0adf1f040d47358d29320 Gerrit-Change-Number: 17731 Gerrit-PatchSet: 10 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [common] more generic API for IN() predicate's pruning
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17764 ) Change subject: [common] more generic API for IN() predicate's pruning .. Patch Set 2: Code-Review+1 (2 comments) Thanks for this cleanup! http://gerrit.cloudera.org:8080/#/c/17764/2/src/kudu/common/scan_spec.cc File src/kudu/common/scan_spec.cc: http://gerrit.cloudera.org:8080/#/c/17764/2/src/kudu/common/scan_spec.cc@184 PS2, Line 184: // TODO(aserbin): is it possible that FindColumn() returns non-OK here? >From its usage in tablet_service, it seems like the answer is no. If the >predicate isn't in the tablet schema, the SetupScanSpec() call would return an >error when building the spec. http://gerrit.cloudera.org:8080/#/c/17764/2/src/kudu/common/scan_spec.cc@185 PS2, Line 185: DCHECK(false); nit: Maybe LOG(DFATAL)? Same below? -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 10 Aug 2021 17:37:46 + Gerrit-HasComments: Yes
[kudu-CR] [python] KUDU-1921 Add ability to require auth/encryption
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17733 ) Change subject: [python] KUDU-1921 Add ability to require auth/encryption .. Patch Set 8: Code-Review+1 Looks good to me overall. Is it possible to add a small test scenario to cover the newly introduced flags, maybe just for non-kerberized case? It seems there is mini-cluster test scaffolding in python/kudu/tests/common.py. -- To view, visit http://gerrit.cloudera.org:8080/17733 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I10173145611ad2991c0a1b173ecadc7141ae6f5e Gerrit-Change-Number: 17733 Gerrit-PatchSet: 8 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Tue, 10 Aug 2021 17:26:25 + Gerrit-HasComments: No
[kudu-CR] [python] KUDU-1921 Add ability to require auth/encryption
Hello Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17733 to look at the new patch set (#8). Change subject: [python] KUDU-1921 Add ability to require auth/encryption .. [python] KUDU-1921 Add ability to require auth/encryption Change-Id: I10173145611ad2991c0a1b173ecadc7141ae6f5e --- M python/kudu/client.pyx M python/kudu/libkudu_client.pxd 2 files changed, 18 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/33/17733/8 -- To view, visit http://gerrit.cloudera.org:8080/17733 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I10173145611ad2991c0a1b173ecadc7141ae6f5e Gerrit-Change-Number: 17733 Gerrit-PatchSet: 8 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [consensus] remove ForTests suffix from WaitUntilLeader
Andrew Wong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17763 ) Change subject: [consensus] remove ForTests suffix from WaitUntilLeader .. [consensus] remove ForTests suffix from WaitUntilLeader I found WaitUntilLeaderForTests useful in making the master rebuilder tool more robust to slow startup. Since I intend on using it outside of tests, I removed the ForTests suffix. I also left a note in the header mentioning it should be used sparingly, given the implementation is admittedly simple and not suited for heavy usage. This patch contains no functional changes. Change-Id: I4f5d61a59651c3a5f11e317a77c0344f5dd3e707 Reviewed-on: http://gerrit.cloudera.org:8080/17763 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin --- M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h M src/kudu/consensus/raft_consensus_quorum-test.cc M src/kudu/integration-tests/alter_table-test.cc M src/kudu/integration-tests/fuzz-itest.cc M src/kudu/integration-tests/ts_tablet_manager-itest.cc M src/kudu/integration-tests/txn_participant-itest.cc M src/kudu/tablet/tablet_replica-test-base.cc M src/kudu/tserver/tablet_copy_source_session-test.cc M src/kudu/tserver/tablet_server-test-base.cc M src/kudu/tserver/ts_tablet_manager-test.cc 11 files changed, 36 insertions(+), 45 deletions(-) Approvals: Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17763 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4f5d61a59651c3a5f11e317a77c0344f5dd3e707 Gerrit-Change-Number: 17763 Gerrit-PatchSet: 3 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [consensus] remove ForTests suffix from WaitUntilLeader
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17763 ) Change subject: [consensus] remove ForTests suffix from WaitUntilLeader .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/17763/1/src/kudu/consensus/raft_consensus.h File src/kudu/consensus/raft_consensus.h: http://gerrit.cloudera.org:8080/#/c/17763/1/src/kudu/consensus/raft_consensus.h@188 PS1, Line 188: Status WaitUntilLeader(const MonoDelta& timeout) > nit: does it make sense to add WARN_UNUSED_RESULT attribute? Done http://gerrit.cloudera.org:8080/#/c/17763/1/src/kudu/consensus/raft_consensus_quorum-test.cc File src/kudu/consensus/raft_consensus_quorum-test.cc: http://gerrit.cloudera.org:8080/#/c/17763/1/src/kudu/consensus/raft_consensus_quorum-test.cc@905 PS1, Line 905: // Make sure the last operation is > Is it possible to replace this with Done -- To view, visit http://gerrit.cloudera.org:8080/17763 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f5d61a59651c3a5f11e317a77c0344f5dd3e707 Gerrit-Change-Number: 17763 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 10 Aug 2021 16:44:20 + Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-1921 Add ability to require authn/encryption to Java client
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17732 ) Change subject: [java] KUDU-1921 Add ability to require authn/encryption to Java client .. Patch Set 9: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/17732/8/java/kudu-client/src/test/java/org/apache/kudu/client/TestSecurity.java File java/kudu-client/src/test/java/org/apache/kudu/client/TestSecurity.java: http://gerrit.cloudera.org:8080/#/c/17732/8/java/kudu-client/src/test/java/org/apache/kudu/client/TestSecurity.java@574 PS8, Line 574: > Sure, added it. Thank you! -- To view, visit http://gerrit.cloudera.org:8080/17732 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic951b2090a4933eca70dc53b6f93cdcff5a74929 Gerrit-Change-Number: 17732 Gerrit-PatchSet: 9 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Tue, 10 Aug 2021 16:32:20 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-1921 Add ability to require auth/encryption to C++ client
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17731 ) Change subject: KUDU-1921 Add ability to require auth/encryption to C++ client .. Patch Set 9: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3e800eb7c4e2f8787f0adf1f040d47358d29320 Gerrit-Change-Number: 17731 Gerrit-PatchSet: 9 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 10 Aug 2021 16:31:43 + Gerrit-HasComments: No
[kudu-CR] [java] KUDU-1921 Add ability to require authn/encryption to Java client
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/17732 ) Change subject: [java] KUDU-1921 Add ability to require authn/encryption to Java client .. Patch Set 9: (2 comments) http://gerrit.cloudera.org:8080/#/c/17732/8/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java File java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java: http://gerrit.cloudera.org:8080/#/c/17732/8/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java@2735 PS8, Line 2735: // Only connects to remote servers that support encryption, fails : // otherwise. It can connect to insecure servers only locally. : REQUIRED_REMOTE, : // Only connects to any server, including on the loopback interface, : // that support encryption, fails otherwise. : REQUIRED, > nit: consider renaming these as well per suggestion at https://gerrit.cloud Done http://gerrit.cloudera.org:8080/#/c/17732/8/java/kudu-client/src/test/java/org/apache/kudu/client/TestSecurity.java File java/kudu-client/src/test/java/org/apache/kudu/client/TestSecurity.java: http://gerrit.cloudera.org:8080/#/c/17732/8/java/kudu-client/src/test/java/org/apache/kudu/client/TestSecurity.java@574 PS8, Line 574: > While you are working on this, do you mind adding one extra test scenario t Sure, added it. -- To view, visit http://gerrit.cloudera.org:8080/17732 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic951b2090a4933eca70dc53b6f93cdcff5a74929 Gerrit-Change-Number: 17732 Gerrit-PatchSet: 9 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Tue, 10 Aug 2021 14:18:05 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-1921 Add ability to require auth/encryption to C++ client
Attila Bukor has posted comments on this change. ( http://gerrit.cloudera.org:8080/17731 ) Change subject: KUDU-1921 Add ability to require auth/encryption to C++ client .. Patch Set 9: (3 comments) http://gerrit.cloudera.org:8080/#/c/17731/8/src/kudu/client/client.h File src/kudu/client/client.h: http://gerrit.cloudera.org:8080/#/c/17731/8/src/kudu/client/client.h@233 PS8, Line 233: on-the-wir > nit: on-the-wire encryption Done http://gerrit.cloudera.org:8080/#/c/17731/8/src/kudu/client/client.h@238 PS8, Line 238: REQUIRED > nit: maybe, REQUIRED_EXCLUDING_LOOPBACK ? Changed it to REQUIRED_REMOTE, EXCLUDING_LOOPBACK seemed too long to me. http://gerrit.cloudera.org:8080/#/c/17731/8/src/kudu/client/client.h@241 PS8, Line 241: REQUIRED > nit: maybe, REQUIRED ? Done -- To view, visit http://gerrit.cloudera.org:8080/17731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia3e800eb7c4e2f8787f0adf1f040d47358d29320 Gerrit-Change-Number: 17731 Gerrit-PatchSet: 9 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 10 Aug 2021 14:18:02 + Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-1921 Add ability to require authn/encryption to Java client
Hello Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17732 to look at the new patch set (#9). Change subject: [java] KUDU-1921 Add ability to require authn/encryption to Java client .. [java] KUDU-1921 Add ability to require authn/encryption to Java client Change-Id: Ic951b2090a4933eca70dc53b6f93cdcff5a74929 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/Connection.java M java/kudu-client/src/main/java/org/apache/kudu/client/ConnectionCache.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/Negotiator.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestNegotiator.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestSecurity.java 7 files changed, 201 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/32/17732/9 -- To view, visit http://gerrit.cloudera.org:8080/17732 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic951b2090a4933eca70dc53b6f93cdcff5a74929 Gerrit-Change-Number: 17732 Gerrit-PatchSet: 9 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-1921 Add ability to require auth/encryption to C++ client
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17731 to look at the new patch set (#9). Change subject: KUDU-1921 Add ability to require auth/encryption to C++ client .. KUDU-1921 Add ability to require auth/encryption to C++ client Kudu servers support requiring authentication and encryption to be enabled, and clients prefer connecting in a secure way, but if a server doesn't support authentication and/or encryption, the client will silently connect insecurely, which can lead to a downgrade attack. With this patch, clients can require authentication and encryption to be set using the client API, where if such an attack is attempted, the client will fail to connect to the cluster. Change-Id: Ia3e800eb7c4e2f8787f0adf1f040d47358d29320 --- M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/client_builder-internal.cc M src/kudu/client/client_builder-internal.h M src/kudu/integration-tests/security-itest.cc M src/kudu/rpc/client_negotiation.cc M src/kudu/rpc/client_negotiation.h M src/kudu/rpc/messenger.cc M src/kudu/rpc/messenger.h M src/kudu/rpc/negotiation-test.cc M src/kudu/rpc/negotiation.cc M src/kudu/rpc/negotiation.h M src/kudu/rpc/reactor.cc M src/kudu/rpc/server_negotiation.cc M src/kudu/rpc/server_negotiation.h 15 files changed, 192 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/31/17731/9 -- To view, visit http://gerrit.cloudera.org:8080/17731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia3e800eb7c4e2f8787f0adf1f040d47358d29320 Gerrit-Change-Number: 17731 Gerrit-PatchSet: 9 Gerrit-Owner: Attila Bukor Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [common] more generic API for IN() predicate's pruning
Hello Mahesh Reddy, Tidy Bot, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17764 to look at the new patch set (#2). Change subject: [common] more generic API for IN() predicate's pruning .. [common] more generic API for IN() predicate's pruning While working on KUDU-2671, I found that the exposed internals of the PartitionSchema class doesn't allow for updating the implementation of the partition-related code to include per-range custom hash bucket schemas in a consistent manner. This patch introduces a bit more generic interface for pruning values of IN predicates by adding a new PartitionMayContainRow() method and removes the following methods from the public API of the PartitionSchema class: * HashPartitionContainsRow() * RangePartitionContainsRow() * IsColumnSingleRangeSchema() * TryGetSingleColumnHashPartitionIndex() I also added one extra test scenario and updated existing ones to allow for easier reading the assertion messages in case if those assertions ever trigger. This is a follow-up to 6a7cadc7e and 83b8caf4f. Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d --- M src/kudu/common/partition.cc M src/kudu/common/partition.h M src/kudu/common/scan_spec-test.cc M src/kudu/common/scan_spec.cc M src/kudu/common/scan_spec.h 5 files changed, 416 insertions(+), 556 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/17764/2 -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [common] more generic API for IN() predicate's pruning
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17764 Change subject: [common] more generic API for IN() predicate's pruning .. [common] more generic API for IN() predicate's pruning While working on KUDU-2671, I found that the exposed internals of the PartitionSchema class doesn't allow for updating the implementation of the partition-related code to include per-range custom hash bucket schemas in a consistent manner. This patch introduces a bit more generic interface for pruning values of IN predicates by adding a new PartitionMayContainRow() method and removes the following methods from the public API of the PartitionSchema class: * HashPartitionContainsRow() * RangePartitionContainsRow() * IsColumnSingleRangeSchema() * TryGetSingleColumnHashPartitionIndex() I also added one extra test scenario and updated existing ones to allow for easier reading the assertion messages in case if those assertions ever trigger. This is a follow-up to 6a7cadc7e and 83b8caf4f. Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d --- M src/kudu/common/partition.cc M src/kudu/common/partition.h M src/kudu/common/scan_spec-test.cc M src/kudu/common/scan_spec.cc M src/kudu/common/scan_spec.h 5 files changed, 417 insertions(+), 550 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/17764/1 -- To view, visit http://gerrit.cloudera.org:8080/17764 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2e2390cc4747864fdac71656dd7125ac3b15bf9d Gerrit-Change-Number: 17764 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin
