[jira] [Commented] (HDFS-12389) Ozone: oz commandline list calls should return valid JSON format output
[
https://issues.apache.org/jira/browse/HDFS-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455223#comment-16455223
]
Hudson commented on HDFS-12389:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14070 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/14070/])
HDFS-12389. Ozone: oz commandline list calls should return valid JSON (omalley:
rev 0c177766461b68559e4ce2f0c11f33bf1253021b)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/ListBucketHandler.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/ListVolumeHandler.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/ozShell/TestOzoneShell.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/client/OzoneVolume.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/utils/JsonUtils.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/ListKeyHandler.java
> Ozone: oz commandline list calls should return valid JSON format output
> ---
>
> Key: HDFS-12389
> URL: https://issues.apache.org/jira/browse/HDFS-12389
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Labels: ozoneMerge
> Fix For: HDFS-7240
>
> Attachments: HDFS-12389-HDFS-7240.001.patch,
> HDFS-12389-HDFS-7240.002.patch, json_output_test.log
>
>
> At present the outputs of {{listVolume}}, {{listBucket}} and {{listKey}} are
> hard to parse, for example following call
> {code}
> ./bin/hdfs oz -listVolume http://localhost:9864 -user wwei
> {code}
> lists all volumes in my cluster and it returns
> {noformat}
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> ...
> {noformat}
> this is not a valid JSON format output hence it is hard to parse in clients'
> script for further interactions. Propose to reformat them to valid JSON data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12389) Ozone: oz commandline list calls should return valid JSON format output
[
https://issues.apache.org/jira/browse/HDFS-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450988#comment-16450988
]
Hudson commented on HDFS-12389:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14057 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/14057/])
HDFS-12389. Ozone: oz commandline list calls should return valid JSON (wwei:
rev d05d0788c2786b42261210a1852c7ef8b9955e01)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/utils/JsonUtils.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/ozShell/TestOzoneShell.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/ozShell/keys/ListKeyHandler.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/ozShell/volume/ListVolumeHandler.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/client/OzoneVolume.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/web/ozShell/bucket/ListBucketHandler.java
> Ozone: oz commandline list calls should return valid JSON format output
> ---
>
> Key: HDFS-12389
> URL: https://issues.apache.org/jira/browse/HDFS-12389
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
> Labels: ozoneMerge
> Fix For: HDFS-7240
>
> Attachments: HDFS-12389-HDFS-7240.001.patch,
> HDFS-12389-HDFS-7240.002.patch, json_output_test.log
>
>
> At present the outputs of {{listVolume}}, {{listBucket}} and {{listKey}} are
> hard to parse, for example following call
> {code}
> ./bin/hdfs oz -listVolume http://localhost:9864 -user wwei
> {code}
> lists all volumes in my cluster and it returns
> {noformat}
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> ...
> {noformat}
> this is not a valid JSON format output hence it is hard to parse in clients'
> script for further interactions. Propose to reformat them to valid JSON data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12389) Ozone: oz commandline list calls should return valid JSON format output
[
https://issues.apache.org/jira/browse/HDFS-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162435#comment-16162435
]
Weiwei Yang commented on HDFS-12389:
Thanks [~anu] for the quick review, I have committed this to the feature branch.
> Ozone: oz commandline list calls should return valid JSON format output
> ---
>
> Key: HDFS-12389
> URL: https://issues.apache.org/jira/browse/HDFS-12389
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Labels: ozoneMerge
> Fix For: HDFS-7240
>
> Attachments: HDFS-12389-HDFS-7240.001.patch,
> HDFS-12389-HDFS-7240.002.patch, json_output_test.log
>
>
> At present the outputs of {{listVolume}}, {{listBucket}} and {{listKey}} are
> hard to parse, for example following call
> {code}
> ./bin/hdfs oz -listVolume http://localhost:9864 -user wwei
> {code}
> lists all volumes in my cluster and it returns
> {noformat}
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> ...
> {noformat}
> this is not a valid JSON format output hence it is hard to parse in clients'
> script for further interactions. Propose to reformat them to valid JSON data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12389) Ozone: oz commandline list calls should return valid JSON format output
[
https://issues.apache.org/jira/browse/HDFS-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162432#comment-16162432
]
Weiwei Yang commented on HDFS-12389:
Thanks for the +1 [~anu], I am going to commit this shortly. Thank you.
> Ozone: oz commandline list calls should return valid JSON format output
> ---
>
> Key: HDFS-12389
> URL: https://issues.apache.org/jira/browse/HDFS-12389
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Labels: ozoneMerge
> Attachments: HDFS-12389-HDFS-7240.001.patch,
> HDFS-12389-HDFS-7240.002.patch, json_output_test.log
>
>
> At present the outputs of {{listVolume}}, {{listBucket}} and {{listKey}} are
> hard to parse, for example following call
> {code}
> ./bin/hdfs oz -listVolume http://localhost:9864 -user wwei
> {code}
> lists all volumes in my cluster and it returns
> {noformat}
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> ...
> {noformat}
> this is not a valid JSON format output hence it is hard to parse in clients'
> script for further interactions. Propose to reformat them to valid JSON data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12389) Ozone: oz commandline list calls should return valid JSON format output
[
https://issues.apache.org/jira/browse/HDFS-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162422#comment-16162422
]
Anu Engineer commented on HDFS-12389:
-
+1, Thanks for fixing this. That was a dumb mistake. Really appreciate you
fixing it. Thanks for flagging to get the code review attention too.
> Ozone: oz commandline list calls should return valid JSON format output
> ---
>
> Key: HDFS-12389
> URL: https://issues.apache.org/jira/browse/HDFS-12389
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Labels: ozoneMerge
> Attachments: HDFS-12389-HDFS-7240.001.patch,
> HDFS-12389-HDFS-7240.002.patch, json_output_test.log
>
>
> At present the outputs of {{listVolume}}, {{listBucket}} and {{listKey}} are
> hard to parse, for example following call
> {code}
> ./bin/hdfs oz -listVolume http://localhost:9864 -user wwei
> {code}
> lists all volumes in my cluster and it returns
> {noformat}
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> ...
> {noformat}
> this is not a valid JSON format output hence it is hard to parse in clients'
> script for further interactions. Propose to reformat them to valid JSON data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12389) Ozone: oz commandline list calls should return valid JSON format output
[
https://issues.apache.org/jira/browse/HDFS-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162413#comment-16162413
]
Weiwei Yang commented on HDFS-12389:
Hi [~anu], [~linyiqun], [~vagarychen]
Excuse me for interrupting, could you please help to review this patch when you
have time? This is pretty straightforward but would be helpful to folks who
wants to play with CLI outputs. Thanks a lot.
> Ozone: oz commandline list calls should return valid JSON format output
> ---
>
> Key: HDFS-12389
> URL: https://issues.apache.org/jira/browse/HDFS-12389
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Labels: ozoneMerge
> Attachments: HDFS-12389-HDFS-7240.001.patch,
> HDFS-12389-HDFS-7240.002.patch, json_output_test.log
>
>
> At present the outputs of {{listVolume}}, {{listBucket}} and {{listKey}} are
> hard to parse, for example following call
> {code}
> ./bin/hdfs oz -listVolume http://localhost:9864 -user wwei
> {code}
> lists all volumes in my cluster and it returns
> {noformat}
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> ...
> {noformat}
> this is not a valid JSON format output hence it is hard to parse in clients'
> script for further interactions. Propose to reformat them to valid JSON data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12389) Ozone: oz commandline list calls should return valid JSON format output
[
https://issues.apache.org/jira/browse/HDFS-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154841#comment-16154841
]
Hadoop QA commented on HDFS-12389:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-7240 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
46s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} HDFS-7240 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
48s{color} | {color:red} hadoop-hdfs-project_hadoop-hdfs generated 1 new + 9
unchanged - 0 fixed = 10 total (was 9) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 47s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}119m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestClientProtocolForPipelineRecovery |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 |
| | hadoop.hdfs.TestLeaseRecoveryStriped |
| | hadoop.hdfs.crypto.TestHdfsCryptoStreams |
| | hadoop.hdfs.TestReplaceDatanodeOnFailure |
| | hadoop.hdfs.TestReadStripedFileWithDecoding |
| | hadoop.hdfs.client.impl.TestBlockReaderLocal |
| | hadoop.hdfs.TestGetFileChecksum |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050 |
| Timed out junit tests | org.apache.hadoop.hdfs.TestWriteReadStripedFile |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12389 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12885513/HDFS-12389-HDFS-7240.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 421ac1c70c9a 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / d23e1ed |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| javadoc |
https://builds.apache.org/job/PreCommit-HDFS-Build/21012/artifact/patchprocess/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/21012/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
[jira] [Commented] (HDFS-12389) Ozone: oz commandline list calls should return valid JSON format output
[
https://issues.apache.org/jira/browse/HDFS-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154771#comment-16154771
]
Weiwei Yang commented on HDFS-12389:
Attach the same patch again to trigger the jenkins report...
> Ozone: oz commandline list calls should return valid JSON format output
> ---
>
> Key: HDFS-12389
> URL: https://issues.apache.org/jira/browse/HDFS-12389
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: HDFS-12389-HDFS-7240.001.patch,
> HDFS-12389-HDFS-7240.002.patch, json_output_test.log
>
>
> At present the outputs of {{listVolume}}, {{listBucket}} and {{listKey}} are
> hard to parse, for example following call
> {code}
> ./bin/hdfs oz -listVolume http://localhost:9864 -user wwei
> {code}
> lists all volumes in my cluster and it returns
> {noformat}
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> ...
> {noformat}
> this is not a valid JSON format output hence it is hard to parse in clients'
> script for further interactions. Propose to reformat them to valid JSON data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12389) Ozone: oz commandline list calls should return valid JSON format output
[
https://issues.apache.org/jira/browse/HDFS-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152428#comment-16152428
]
Weiwei Yang commented on HDFS-12389:
Patch attached, please see UT in {{TestOzoneShell}} and functional test in
[^json_output_test.log]. Please kindly review, thanks.
> Ozone: oz commandline list calls should return valid JSON format output
> ---
>
> Key: HDFS-12389
> URL: https://issues.apache.org/jira/browse/HDFS-12389
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: HDFS-12389-HDFS-7240.001.patch, json_output_test.log
>
>
> At present the outputs of {{listVolume}}, {{listBucket}} and {{listKey}} are
> hard to parse, for example following call
> {code}
> ./bin/hdfs oz -listVolume http://localhost:9864 -user wwei
> {code}
> lists all volumes in my cluster and it returns
> {noformat}
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> ...
> {noformat}
> this is not a valid JSON format output hence it is hard to parse in clients'
> script for further interactions. Propose to reformat them to valid JSON data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12389) Ozone: oz commandline list calls should return valid JSON format output
[
https://issues.apache.org/jira/browse/HDFS-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152110#comment-16152110
]
Weiwei Yang commented on HDFS-12389:
Following code to parse output
{code}
with open('keys.json') as data_file:
data = json.load(data_file)
for key in data:
for attribute, value in key.iteritems():
print attribute, value
{code}
it fails with error
{noformat}
Traceback (most recent call last):
File "testkeys.py", line 4, in
data = json.load(data_file)
File "/usr/lib64/python2.7/json/__init__.py", line 290, in load
**kw)
File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python2.7/json/decoder.py", line 369, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 9 column 1 - line 17 column 1 (char 186 - 372)
{noformat}
> Ozone: oz commandline list calls should return valid JSON format output
> ---
>
> Key: HDFS-12389
> URL: https://issues.apache.org/jira/browse/HDFS-12389
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>
> At present the outputs of {{listVolume}}, {{listBucket}} and {{listKey}} are
> hard to parse, for example following call
> {code}
> ./bin/hdfs oz -listVolume http://localhost:9864 -user wwei
> {code}
> lists all volumes in my cluster and it returns
> {noformat}
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> {
> "version" : 0,
> "md5hash" : null,
> "createdOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "modifiedOn" : "Mon, 04 Sep 2017 03:25:22 GMT",
> "size" : 10240,
> "keyName" : "key-0-22381",
> "dataFileName" : null
> }
> ...
> {noformat}
> this is not a valid JSON format output hence it is hard to parse in clients'
> script for further interactions. Propose to reformat them to valid JSON data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
