Todd Lipcon has submitted this change and it was merged. Change subject: KUDU-1775 (part 3): enforce max cell size and max PK size ......................................................................
KUDU-1775 (part 3): enforce max cell size and max PK size This adds limits on the size of any individual cell as well as a limit on the maximum size of an encoded key. * Large cells are known to cause problems such as KUDU-1524 (crash during flush). * Large PKs have been seen to cause problems where the key column's cfile footer increases beyond the hard-coded maximum size of 64KB. If a user attempts to insert a non-conforming row, or update a cell to be larger than the maximum, they will receive an InvalidArgument error for that row. This patch takes the approach of doing the validation at the tablet layer rather than somewhere higher up the stack. The reasoning is that we don't want to reject an entire batch of operations due to one bad row, and we are not well equipped to set per-row errors anywhere except at the tablet layer. Because this is an incompatible change (it's possible that users have been successfully using larger cells) the limits are added as configurable flags, but tagged as unsafe. A new test verifies that, if the user has insertions in the WAL that were previously allowed but now are disallowed, the bootstrap fails without any crashes. It also tests that by raising the flag to a higher value, bootstraps will succeed. Making this test pass involved a small fix in bootstrap to properly handle the case where an Apply failed on restart the previously succeeded. Change-Id: Ib6b94cffd9c3efbe80a4b31e9272b376a4e41d15 Reviewed-on: http://gerrit.cloudera.org:8080/5475 Tested-by: Kudu Jenkins Reviewed-by: Adar Dembo <[email protected]> --- M docs/schema_design.adoc M src/kudu/client/client-test.cc M src/kudu/integration-tests/raft_consensus-itest.cc M src/kudu/integration-tests/test_workload.cc M src/kudu/integration-tests/ts_recovery-itest.cc M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_bootstrap.cc M src/kudu/tserver/tablet_server-test.cc 9 files changed, 276 insertions(+), 41 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5475 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib6b94cffd9c3efbe80a4b31e9272b376a4e41d15 Gerrit-PatchSet: 6 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy <[email protected]> Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon <[email protected]>
