[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103861#comment-14103861 ] Kihwal Lee commented on HDFS-6293: -- [~fnie] Please see the release note. This is availabl

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103647#comment-14103647 ] Nie Gus commented on HDFS-6293: --- Here is one user case to process OIV output, we export the f

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992501#comment-13992501 ] Haohui Mai commented on HDFS-6293: -- bq. In the results above, the amount of memory on the

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13999012#comment-13999012 ] Hudson commented on HDFS-6293: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5605 (See [https:

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13998220#comment-13998220 ] Hudson commented on HDFS-6293: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1753 (See [https://

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996839#comment-13996839 ] Hadoop QA commented on HDFS-6293: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996528#comment-13996528 ] Kihwal Lee commented on HDFS-6293: -- I have filed HDFS-6384 for further discussion and desi

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13997584#comment-13997584 ] Akira AJISAKA commented on HDFS-6293: - I found "hdfs oiv_legacy" cannot be executed. {c

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13998182#comment-13998182 ] Hudson commented on HDFS-6293: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1779 (See [htt

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995711#comment-13995711 ] Haohui Mai commented on HDFS-6293: -- bq. If we only think about this, we can deprecate OIV

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993944#comment-13993944 ] Haohui Mai commented on HDFS-6293: -- Thanks [~kihwal], please feel free to modify the patch

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993859#comment-13993859 ] Kihwal Lee commented on HDFS-6293: -- Thanks for the patch, Haohui. I tested the output gen

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996968#comment-13996968 ] Kihwal Lee commented on HDFS-6293: -- Thanks for the review, Suresh. I will commit this afte

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996873#comment-13996873 ] Kihwal Lee commented on HDFS-6293: -- I will fix all warnings except the two coming from FSI

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13997142#comment-13997142 ] Kihwal Lee commented on HDFS-6293: -- The failed test is unrelated to this JIRA. It will be

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13997051#comment-13997051 ] Hadoop QA commented on HDFS-6293: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996582#comment-13996582 ] Hadoop QA commented on HDFS-6293: - {color:red}-1 overall{color}. Here are the results of t

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996922#comment-13996922 ] Suresh Srinivas commented on HDFS-6293: --- +1 for the patch. Thanks [~hmai] and [~kihwa

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996649#comment-13996649 ] Kihwal Lee commented on HDFS-6293: -- Ouch. The patch applies to trunk but it introduces a d

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996442#comment-13996442 ] Kihwal Lee commented on HDFS-6293: -- Here is recap. * oiv_legacy for debugging and inspecti

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995444#comment-13995444 ] Kihwal Lee commented on HDFS-6293: -- bq. can you please mark the class that are brought

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995668#comment-13995668 ] Suresh Srinivas commented on HDFS-6293: --- I agree with [~kihwal] on deprecation. Is th

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995480#comment-13995480 ] Akira AJISAKA commented on HDFS-6293: - The patch looks good to me. Two comments about o

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995335#comment-13995335 ] Haohui Mai commented on HDFS-6293: -- The patch looks good to me. One nit is that can you pl

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992502#comment-13992502 ] Haohui Mai commented on HDFS-6293: -- The v2 patch simply brings back the old saver of the f

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13990204#comment-13990204 ] Haohui Mai commented on HDFS-6293: -- bq. There is existing apps that use a custom Visitor s

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989540#comment-13989540 ] Kihwal Lee commented on HDFS-6293: -- bq. To demonstrate my points, I'm attaching a patch wh

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989427#comment-13989427 ] Hadoop QA commented on HDFS-6293: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989348#comment-13989348 ] Haohui Mai commented on HDFS-6293: -- Just to recap: # The requirement is to build an offli

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988538#comment-13988538 ] Suresh Srinivas commented on HDFS-6293: --- bq. I created two subtasks for PB and an HTT

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988498#comment-13988498 ] Andrew Wang commented on HDFS-6293: --- I created two subtasks for PB and an HTTP interface.

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988435#comment-13988435 ] Suresh Srinivas commented on HDFS-6293: --- bq. PB would be preferable to JSON. I'd be i

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988414#comment-13988414 ] Andrew Wang commented on HDFS-6293: --- Hey Suresh, This plan sounds generally good to me,

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988321#comment-13988321 ] Suresh Srinivas commented on HDFS-6293: --- Here is the summary of a quick call I had wi

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985707#comment-13985707 ] Marcelo Vanzin commented on HDFS-6293: -- [~ajisakaa] my modified parser does not have a

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985395#comment-13985395 ] Akira AJISAKA commented on HDFS-6293: - Created HDFS-6310 for output tokens, and attache

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985368#comment-13985368 ] Akira AJISAKA commented on HDFS-6293: - bq. It will be great if someone can come up with

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13984360#comment-13984360 ] Kihwal Lee commented on HDFS-6293: -- Another option may be adding an option for checkpointi

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13984353#comment-13984353 ] Kihwal Lee commented on HDFS-6293: -- bq. One solution we can do is - Add an option to print

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983734#comment-13983734 ] Suresh Srinivas commented on HDFS-6293: --- OfflineImageViewer just dumps the fsimage in

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983412#comment-13983412 ] Kihwal Lee commented on HDFS-6293: -- bq. This is by design. I understand that it has merits

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983362#comment-13983362 ] Kihwal Lee commented on HDFS-6293: -- [~va...@rededc.com.br]: Thanks for sharing your experi

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983361#comment-13983361 ] Haohui Mai commented on HDFS-6293: -- bq. Another issue is the complete change of format/con

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983288#comment-13983288 ] Marcelo Vanzin commented on HDFS-6293: -- Hi Kihwal, We have developed some code intern

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983131#comment-13983131 ] Kihwal Lee commented on HDFS-6293: -- The 2.4.0 pb-fsimage does contain tokens, but OIV does

[jira] [Commented] (HDFS-6293) Issues with OIV processing PB-based fsimages

[ https://issues.apache.org/jira/browse/HDFS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983095#comment-13983095 ] Kihwal Lee commented on HDFS-6293: -- Outputting in the new XML format is fast and consumes