[
https://issues.apache.org/jira/browse/HBASE-26464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17449187#comment-17449187
]
Nihal Jain commented on HBASE-26464:
------------------------------------
So we should skip printing writer and codec classes by default and keep schema
same as existing output (i.e. proposal #1) just adding editHeapSize and
position fields when some one say "-j" flag. For eg:
{code:java}
[
{
"sequence": "3",
"editHeapSize": "328",
"position": "389",
"region": "f0153c5d890d96f06ec304eebc4aacf9",
"actions": [
{
"qualifier": "HBASE::REGION_EVENT::REGION_OPEN",
"vlen": 176,
"row": "\\x00",
"type": "Put",
"family": "METAFAMILY",
"timestamp": "1637170532553",
"total_size_sum": "288"
}
],
"table": {
"name": [
104,
98,
97,
115,
101,
58,
110,
97,
109,
101,
115,
112,
97,
99,
101
],
"nameAsString": "hbase:namespace",
"namespace": [
104,
98,
97,
115,
101
],
"namespaceAsString": "hbase",
"qualifier": [
110,
97,
109,
101,
115,
112,
97,
99,
101
],
"qualifierAsString": "namespace",
"systemTable": true,
"hashCode": 771659354
}
}
] {code}
Next, we introduce a new flag say which allows user to print a detailed json,
but we are saying we want to have schema as #2.
Would not that be causing non-uniformity in output? Would not it make sense to
stick to one schema for both json modes and may be just keep writer and codec
classes for each edit, i.e #1?
Also should we even skip editHeapSize and position in normal json mode and make
it part of detailed json mode that we plan to add?
What do you think [~wchevreuil] ?
> WALPrettyPrinter: JSON output is broken/malformed
> -------------------------------------------------
>
> Key: HBASE-26464
> URL: https://issues.apache.org/jira/browse/HBASE-26464
> Project: HBase
> Issue Type: Bug
> Components: tooling
> Affects Versions: 3.0.0-alpha-1, 1.7.1, 2.4.8
> Reporter: Nihal Jain
> Assignee: Nihal Jain
> Priority: Major
>
> The JSON output of wal pretty printer is malformed and cannot be parsed.
> There are 2 major issue:
> * The writer classes and cell codec classes are added (as normal text)
> whenever a new WAL file printing starts (NOTE: WALPrettyPrinter can take
> multiple WAL files as input)
> * The edit heap size and position are outside json formatting
> These are side effects of some patches done in past which missed to cover
> json output scenarios.
> Current sample output, which is malformed JSON:
> {code:java}
> [Writer Classes: ProtobufLogWriter AsyncProtobufLogWriter
> Cell Codec Class: org.apache.hadoop.hbase.regionserver.wal.WALCellCodec
> {"sequence":"3","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":176,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170532553","total_size_sum":"288"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 328
> position: 389
> ,{"sequence":"26","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":218,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170533215","total_size_sum":"336"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit
> heap size: 376
> position: 726
> ,{"sequence":"4","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":9,"row":"default","type":"Put","family":"info","timestamp":"1637170533278","total_size_sum":"96"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 136
> position: 834
> ,{"sequence":"5","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":7,"row":"hbase","type":"Put","family":"info","timestamp":"1637170533340","total_size_sum":"88"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 128
> position: 938
> ,{"sequence":"27","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"","vlen":2,"row":"r6","type":"Put","family":"cf","timestamp":"1637170714545","total_size_sum":"80"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit
> heap size: 120
> position: 1020
> Writer Classes: ProtobufLogWriter AsyncProtobufLogWriter
> Cell Codec Class: org.apache.hadoop.hbase.regionserver.wal.WALCellCodec
> ,{"sequence":"3","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":176,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170532553","total_size_sum":"288"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 328
> position: 389
> ,{"sequence":"26","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":218,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170533215","total_size_sum":"336"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit
> heap size: 376
> position: 726
> ,{"sequence":"4","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":9,"row":"default","type":"Put","family":"info","timestamp":"1637170533278","total_size_sum":"96"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 136
> position: 834
> ,{"sequence":"5","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":7,"row":"hbase","type":"Put","family":"info","timestamp":"1637170533340","total_size_sum":"88"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 128
> position: 938
> ,{"sequence":"27","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"","vlen":2,"row":"r6","type":"Put","family":"cf","timestamp":"1637170714545","total_size_sum":"80"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit
> heap size: 120
> position: 1020
> ]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
