Andrew Wong has uploaded this change for review. (
http://gerrit.cloudera.org:8080/12561
Change subject: no-review: tool to rewrite log segments
......................................................................
no-review: tool to rewrite log segments
Point the tool at a tablet metadata file and the tool will rewrite the
input segments, putting the tablet metadata's schema into the log
segment headers.
This may be useful for rewriting WALs that would fail to replay due to columns
unknown to the log segment header schema, e.g. doing a dump on the schemas
would result in:
...
columns {
id: 10
name: "<COLUMN10>"
type: DOUBLE
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: SNAPPY
cfile_block_size: 0
}
columns {
id: 11
name: "<COLUMN11>"
type: DOUBLE
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: SNAPPY
cfile_block_size: 0
}
}
schema_version: 2
compression_codec: LZ4
27.3012750@6343411509778300928▷⋅REPLICATE WRITE_OP
Invalid argument: Client provided column <COLUMN11> DOUBLE NULLABLE not
present in tablet
even though <COLUMN11> existed in the tablet's metadata.
./bin/kudu wal rewrite_segment_schema
/data/8/awong/tserver/wals/d71208c8e39c4eba8ccc03dcae2b2256/wal-000000001
/data/8/awong/tserver/tablet-meta/d71208c8e39c4eba8ccc03dcae2b2256
I loosely tested on some WALs and tmeta I had and confirmed that I could at
least do a `kudu wal dump` on the resulting segment and it yielded the tmeta's
schema that contained <COLUMN11>.
This might be useful to fix scenarios like KUDU-2690, but may not work
if the tablet metadata's schema is the result of DROP COLUMN operations.
Change-Id: Iea5944e1be4113acafced39359a4ede58b43b3ce
---
M src/kudu/consensus/log.h
M src/kudu/consensus/log_util.cc
M src/kudu/consensus/log_util.h
M src/kudu/tools/tool_action_wal.cc
4 files changed, 127 insertions(+), 11 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/12561/1
--
To view, visit http://gerrit.cloudera.org:8080/12561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iea5944e1be4113acafced39359a4ede58b43b3ce
Gerrit-Change-Number: 12561
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <[email protected]>
