Andrew Wong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12595 )
Change subject: KUDU-2690: don't roll log schema on failed alter ...................................................................... KUDU-2690: don't roll log schema on failed alter It is possible to update the log segment header schema version based on an AlterSchema operation that failed. This is because an alter operation that didn't succeed (e.g. because of a schema version mismatch) is treated as a successful transaction (similar to how we treat a duplicated insert transaction as a successful transaction), and can thus mistakenly lead to updating the log segment schema version, even on failure. This can lead to a mismatch of schemas between the log segment headers and the write ops in those log segments, which can lead to a failure to bootstrap. This patch addresses this by: 1. making the tablet no-op if it sees that an alter didn't go through, 2. storing the error in the commit message, so further bootstraps will skip over the op (this is KUDU-860). Only the former is necessary to prevent the issue, and though the latter uses extra space, it may be helpful for added visibility and debugging. This patch adds a unit test that reproduced the scenario at the TabletReplica level, as well as a less targeted test that uses the same principal to test that we update the log segment schema upon replaying an alter. A more end-to-end test that arrives at the reported state by working around master-level table locking can be found here[1]. [1] https://gist.github.com/andrwng/3a049bb038680cc0254c5ba52b9a7507 Change-Id: Id761851741297e29a4666bec0c34fc4f7285f715 Reviewed-on: http://gerrit.cloudera.org:8080/12462 Tested-by: Kudu Jenkins Reviewed-by: Adar Dembo <[email protected]> Reviewed-by: Todd Lipcon <[email protected]> (cherry picked from commit 16d9a863d0f31069965aee1a71912a65339a4160) Reviewed-on: http://gerrit.cloudera.org:8080/12595 Tested-by: Andrew Wong <[email protected]> --- M src/kudu/consensus/log.cc M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet_bootstrap.cc M src/kudu/tablet/tablet_replica-test.cc M src/kudu/tablet/transactions/alter_schema_transaction.cc M src/kudu/tablet/transactions/alter_schema_transaction.h 6 files changed, 296 insertions(+), 82 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Andrew Wong: Verified -- To view, visit http://gerrit.cloudera.org:8080/12595 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.9.x Gerrit-MessageType: merged Gerrit-Change-Id: Id761851741297e29a4666bec0c34fc4f7285f715 Gerrit-Change-Number: 12595 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Wong <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120)
