[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565161#comment-14565161 ] Jing Zhao commented on HDFS-7609: - The 03 patch looks good to me. +1. I will commit it shor

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14564340#comment-14564340 ] Hadoop QA commented on HDFS-7609: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562543#comment-14562543 ] Jing Zhao commented on HDFS-7609: - Thanks for sharing the thoughts, Ming! Totally agree wit

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562214#comment-14562214 ] Ming Ma commented on HDFS-7609: --- Thanks [~jingzhao]. Good point about saveNamespace. Regardi

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14561372#comment-14561372 ] Jing Zhao commented on HDFS-7609: - bq. any ideas why the current implementation sets NN to

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14561354#comment-14561354 ] Jing Zhao commented on HDFS-7609: - Thanks Ming! The new patch looks good to me. One minor i

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560362#comment-14560362 ] Hadoop QA commented on HDFS-7609: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555402#comment-14555402 ] Jing Zhao commented on HDFS-7609: - Spent some further time digging into the issue. Besides

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555024#comment-14555024 ] Jing Zhao commented on HDFS-7609: - Thanks for the response, Ming! Yes I agree that in most

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555003#comment-14555003 ] Ming Ma commented on HDFS-7609: --- Thanks, [~jingzhao]. In the scenario you described, IIUC, in

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553292#comment-14553292 ] Jing Zhao commented on HDFS-7609: - Thanks for working on this, [~mingma]! Your analysis mak

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549939#comment-14549939 ] Tsz Wo Nicholas Sze commented on HDFS-7609: --- {quote} PriorityQueue#remove is O\(n

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527438#comment-14527438 ] Hadoop QA commented on HDFS-7609: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527030#comment-14527030 ] Ming Ma commented on HDFS-7609: --- #3 in the scenario description above should be "Before nn2 s

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14523562#comment-14523562 ] Ming Ma commented on HDFS-7609: --- Small retryCache size can also impact the correctness, given

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394599#comment-14394599 ] Kihwal Lee commented on HDFS-7609: -- We have seen a related case. In a relatively small clu

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318940#comment-14318940 ] Chris Nauroth commented on HDFS-7609: - I tried testing an old client without call ID su

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14292690#comment-14292690 ] Ming Ma commented on HDFS-7609: --- [~cnauroth], we upgraded from hadoop 2.0.5 to hadoop 2.4, so

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291438#comment-14291438 ] Carrey Zhan commented on HDFS-7609: --- same call stack is found in origin problem. sorry fo

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290735#comment-14290735 ] Chris Nauroth commented on HDFS-7609: - Specifically, the retry cache was added in 2.1.0

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290731#comment-14290731 ] Chris Nauroth commented on HDFS-7609: - [~kihwal] and [~mingma], thank you for the addit

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290453#comment-14290453 ] Ted Yu commented on HDFS-7609: -- If PriorityQueue.remove() took much time, can we utilize Prio

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289642#comment-14289642 ] Ming Ma commented on HDFS-7609: --- Yeah, we also had this issue. It appears somehow an entry wi

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289533#comment-14289533 ] Kihwal Lee commented on HDFS-7609: -- Compared to 0.23, edit replaying in 2.x is 5x-10x slow

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278877#comment-14278877 ] Daryn Sharp commented on HDFS-7609: --- I think there be a separate option to disable popula

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278085#comment-14278085 ] Carrey Zhan commented on HDFS-7609: --- Yes you are right. I know the meaning of retry cache

[jira] [Commented] (HDFS-7609) startup used too much time to load edits

[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277507#comment-14277507 ] Daryn Sharp commented on HDFS-7609: --- We've also noticed a huge performance degradation (1