HDFS-11433. Document missing usages of OfflineEditsViewer processors. Contributed by Yiqun Lin.
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/4a58870a Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/4a58870a Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/4a58870a Branch: refs/heads/HDFS-7240 Commit: 4a58870a0427ecdf4c5c895088b96a0b4d1c0c19 Parents: e24ed47 Author: Yiqun Lin <yq...@apache.org> Authored: Sat Feb 25 10:00:05 2017 +0800 Committer: Yiqun Lin <yq...@apache.org> Committed: Sat Feb 25 10:00:05 2017 +0800 ---------------------------------------------------------------------- .../src/site/markdown/HdfsEditsViewer.md | 73 +++++++++++++++++++- 1 file changed, 72 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/4a58870a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md index 7a46eb0..5e069bb 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md @@ -40,9 +40,80 @@ The Offline Edits Viewer provides several output processors (unless stated other Usage ----- +### XML Processor + +XML processor can create an XML file that contains the edits log information. Users can specify input and output file via -i and -o command-line. + + bash$ bin/hdfs oev -p xml -i edits -o edits.xml + +XML processor is the default processor in Offline Edits Viewer, users can also use the following command: + bash$ bin/hdfs oev -i edits -o edits.xml -| Flag | Description | +This would result in the following output: + + <?xml version="1.0" encoding="UTF-8"?> + <EDITS> + <EDITS_VERSION>-64</EDITS_VERSION> + <RECORD> + <OPCODE>OP_START_LOG_SEGMENT</OPCODE> + <DATA> + <TXID>1</TXID> + </DATA> + </RECORD> + <RECORD> + <OPCODE>OP_UPDATE_MASTER_KEY</OPCODE> + <DATA> + <TXID>2</TXID> + <DELEGATION_KEY> + <KEY_ID>1</KEY_ID> + <EXPIRY_DATE>1487921580728</EXPIRY_DATE> + <KEY>2e127ca41c7de215</KEY> + </DELEGATION_KEY> + </DATA> + </RECORD> + <RECORD> + ...remaining output omitted... + +### Binary Processor + +Binary processor is the opposite of the XML processor. Users can specify input XML file and output file via -i and -o command-line. + + bash$ bin/hdfs oev -p binary -i edits.xml -o edits + +This will reconstruct an edits log file from an XML file. + +### Stats Processor + +Stats processor is used to aggregate counts of op codes contained in the edits log file. Users can specify this processor by -p option. + + bash$ bin/hdfs oev -p stats -i edits -o edits.stats + +The output result of this processor should be like the following output: + + VERSION : -64 + OP_ADD ( 0): 8 + OP_RENAME_OLD ( 1): 1 + OP_DELETE ( 2): 1 + OP_MKDIR ( 3): 1 + OP_SET_REPLICATION ( 4): 1 + OP_DATANODE_ADD ( 5): 0 + OP_DATANODE_REMOVE ( 6): 0 + OP_SET_PERMISSIONS ( 7): 1 + OP_SET_OWNER ( 8): 1 + OP_CLOSE ( 9): 9 + OP_SET_GENSTAMP_V1 ( 10): 0 + ...some output omitted... + OP_APPEND ( 47): 1 + OP_SET_QUOTA_BY_STORAGETYPE ( 48): 1 + OP_INVALID ( -1): 0 + +The output is formatted as a colon separated two column table: OpCode and OpCodeCount. Each OpCode corresponding to the specific operation(s) in NameNode. + +Options +------- + +| Flag | Description | |:---- |:---- | | [`-i` ; `--inputFile`] *input file* | Specify the input edits log file to process. Xml (case insensitive) extension means XML format otherwise binary format is assumed. Required. | | [`-o` ; `--outputFile`] *output file* | Specify the output filename, if the specified output processor generates one. If the specified file already exists, it is silently overwritten. Required. | --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
