[
https://issues.apache.org/jira/browse/HBASE-26464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17449187#comment-17449187
]
Nihal Jain edited comment on HBASE-26464 at 11/25/21, 1:09 PM:
---------------------------------------------------------------
So we should skip printing writer and codec classes by default and keep schema
same as existing output (i.e. proposal #1) just adding editHeapSize and
position fields when some one say "-j" flag. For eg:
{code:java}
[
{
"sequence": "3",
"editHeapSize": "328",
"position": "389",
"region": "f0153c5d890d96f06ec304eebc4aacf9",
"actions": [
{
"qualifier": "HBASE::REGION_EVENT::REGION_OPEN",
"vlen": 176,
"row": "\\x00",
"type": "Put",
"family": "METAFAMILY",
"timestamp": "1637170532553",
"total_size_sum": "288"
}
],
"table": {
"name": [
104,
98,
97,
115,
101,
58,
110,
97,
109,
101,
115,
112,
97,
99,
101
],
"nameAsString": "hbase:namespace",
"namespace": [
104,
98,
97,
115,
101
],
"namespaceAsString": "hbase",
"qualifier": [
110,
97,
109,
101,
115,
112,
97,
99,
101
],
"qualifierAsString": "namespace",
"systemTable": true,
"hashCode": 771659354
}
}
] {code}
Next, we introduce a new flag say which allows user to print a detailed json,
but we are saying we want to have schema as #2.
Would not that be causing non-uniformity in output? Would not it make sense to
stick to one schema for both json modes and may be just keep writer and codec
classes for each edit, i.e #1? Although i do agree, we would be adding
redundant info to each edit in the list.
Also should we even skip editHeapSize and position in normal json mode and make
it part of detailed json mode that we plan to add?
What do you think [~wchevreuil] ?
was (Author: nihaljain.cs):
So we should skip printing writer and codec classes by default and keep schema
same as existing output (i.e. proposal #1) just adding editHeapSize and
position fields when some one say "-j" flag. For eg:
{code:java}
[
{
"sequence": "3",
"editHeapSize": "328",
"position": "389",
"region": "f0153c5d890d96f06ec304eebc4aacf9",
"actions": [
{
"qualifier": "HBASE::REGION_EVENT::REGION_OPEN",
"vlen": 176,
"row": "\\x00",
"type": "Put",
"family": "METAFAMILY",
"timestamp": "1637170532553",
"total_size_sum": "288"
}
],
"table": {
"name": [
104,
98,
97,
115,
101,
58,
110,
97,
109,
101,
115,
112,
97,
99,
101
],
"nameAsString": "hbase:namespace",
"namespace": [
104,
98,
97,
115,
101
],
"namespaceAsString": "hbase",
"qualifier": [
110,
97,
109,
101,
115,
112,
97,
99,
101
],
"qualifierAsString": "namespace",
"systemTable": true,
"hashCode": 771659354
}
}
] {code}
Next, we introduce a new flag say which allows user to print a detailed json,
but we are saying we want to have schema as #2.
Would not that be causing non-uniformity in output? Would not it make sense to
stick to one schema for both json modes and may be just keep writer and codec
classes for each edit, i.e #1?
Also should we even skip editHeapSize and position in normal json mode and make
it part of detailed json mode that we plan to add?
What do you think [~wchevreuil] ?
> WALPrettyPrinter: JSON output is broken/malformed
> -------------------------------------------------
>
> Key: HBASE-26464
> URL: https://issues.apache.org/jira/browse/HBASE-26464
> Project: HBase
> Issue Type: Bug
> Components: tooling
> Affects Versions: 3.0.0-alpha-1, 1.7.1, 2.4.8
> Reporter: Nihal Jain
> Assignee: Nihal Jain
> Priority: Major
>
> The JSON output of wal pretty printer is malformed and cannot be parsed.
> There are 2 major issue:
> * The writer classes and cell codec classes are added (as normal text)
> whenever a new WAL file printing starts (NOTE: WALPrettyPrinter can take
> multiple WAL files as input)
> * The edit heap size and position are outside json formatting
> These are side effects of some patches done in past which missed to cover
> json output scenarios.
> Current sample output, which is malformed JSON:
> {code:java}
> [Writer Classes: ProtobufLogWriter AsyncProtobufLogWriter
> Cell Codec Class: org.apache.hadoop.hbase.regionserver.wal.WALCellCodec
> {"sequence":"3","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":176,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170532553","total_size_sum":"288"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 328
> position: 389
> ,{"sequence":"26","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":218,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170533215","total_size_sum":"336"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit
> heap size: 376
> position: 726
> ,{"sequence":"4","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":9,"row":"default","type":"Put","family":"info","timestamp":"1637170533278","total_size_sum":"96"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 136
> position: 834
> ,{"sequence":"5","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":7,"row":"hbase","type":"Put","family":"info","timestamp":"1637170533340","total_size_sum":"88"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 128
> position: 938
> ,{"sequence":"27","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"","vlen":2,"row":"r6","type":"Put","family":"cf","timestamp":"1637170714545","total_size_sum":"80"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit
> heap size: 120
> position: 1020
> Writer Classes: ProtobufLogWriter AsyncProtobufLogWriter
> Cell Codec Class: org.apache.hadoop.hbase.regionserver.wal.WALCellCodec
> ,{"sequence":"3","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":176,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170532553","total_size_sum":"288"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 328
> position: 389
> ,{"sequence":"26","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":218,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170533215","total_size_sum":"336"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit
> heap size: 376
> position: 726
> ,{"sequence":"4","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":9,"row":"default","type":"Put","family":"info","timestamp":"1637170533278","total_size_sum":"96"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 136
> position: 834
> ,{"sequence":"5","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":7,"row":"hbase","type":"Put","family":"info","timestamp":"1637170533340","total_size_sum":"88"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit
> heap size: 128
> position: 938
> ,{"sequence":"27","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"","vlen":2,"row":"r6","type":"Put","family":"cf","timestamp":"1637170714545","total_size_sum":"80"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit
> heap size: 120
> position: 1020
> ]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
