[jira] [Commented] (HDFS-9766) TestDataNodeMetrics#testDataNodeTimeSpend fails intermittently

2016-03-01 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175110#comment-15175110 ] Xiao Chen commented on HDFS-9766: - Thanks [~ajisakaa] and [~liuml07]. > TestDataNodeMetric

[jira] [Created] (HDFS-9888) TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case setup

2016-03-02 Thread Xiao Chen (JIRA)
Xiao Chen created HDFS-9888: --- Summary: TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case setup Key: HDFS-9888 URL: https://issues.apache.org/jira/browse/HDFS-9888 Project: Hadoop H

[jira] [Updated] (HDFS-9888) TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case setup

2016-03-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9888: Attachment: HDFS-9888.01.patch > TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case

[jira] [Commented] (HDFS-9888) TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case setup

2016-03-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176311#comment-15176311 ] Xiao Chen commented on HDFS-9888: - Step 2 in the description is [here|https://github.com/a

[jira] [Updated] (HDFS-9888) TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case setup

2016-03-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9888: Status: Patch Available (was: Open) > TestBalancer#testBalancerWithKeytabs should reset KerberosName in test

[jira] [Commented] (HDFS-9888) TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case setup

2016-03-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176347#comment-15176347 ] Xiao Chen commented on HDFS-9888: - Thanks Zhe for the quick response. I just double checked

[jira] [Commented] (HDFS-9888) TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case setup

2016-03-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176362#comment-15176362 ] Xiao Chen commented on HDFS-9888: - Thanks! > TestBalancer#testBalancerWithKeytabs should r

[jira] [Commented] (HDFS-9888) TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case setup

2016-03-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176515#comment-15176515 ] Xiao Chen commented on HDFS-9888: - Sure Zhe, here's a sample error: {noformat} Error Messag

[jira] [Commented] (HDFS-9888) TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case setup

2016-03-03 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178743#comment-15178743 ] Xiao Chen commented on HDFS-9888: - The test failure and license warning looks unrelated. [~

[jira] [Commented] (HDFS-9888) TestBalancer#testBalancerWithKeytabs should reset KerberosName in test case setup

2016-03-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180223#comment-15180223 ] Xiao Chen commented on HDFS-9888: - Yes Zhe, HDFS-3016 is not in branch-2. Additionally, sin

[jira] [Commented] (HDFS-9888) Allow reseting KerberosName in unit tests

2016-03-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180281#comment-15180281 ] Xiao Chen commented on HDFS-9888: - Thanks very much [~zhz]! > Allow reseting KerberosName

[jira] [Commented] (HDFS-9888) Allow reseting KerberosName in unit tests

2016-03-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180302#comment-15180302 ] Xiao Chen commented on HDFS-9888: - FYI, I just noticed that the commit to trunk has a bunch

[jira] [Commented] (HDFS-9427) HDFS should not default to ephemeral ports

2016-03-07 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15183478#comment-15183478 ] Xiao Chen commented on HDFS-9427: - bq. I know nothing about KMS so not sure if it is safe t

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-07 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184370#comment-15184370 ] Xiao Chen commented on HDFS-9405: - Thanks all for the discussions and thoughts here. I'd li

[jira] [Assigned] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-07 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen reassigned HDFS-9405: --- Assignee: Xiao Chen > When starting a file, NameNode should generate EDEK in a separate thread > -

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-09 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.01.patch Patch 1 attached warms up the cache on NN startup/failover. IMHO daemon might

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-09 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Status: Patch Available (was: Open) > When starting a file, NameNode should generate EDEK in a separate thre

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-09 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.02.patch Thanks [~asuresh] for the comment, good point. Patch 2 retries every 1 second

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.03.patch Failed tests seems not related, but I figured we'd need some protection since

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189964#comment-15189964 ] Xiao Chen commented on HDFS-9405: - Thanks Arun and Mingliang for the review. Mingliang's c

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.04.patch > When starting a file, NameNode should generate EDEK in a separate thread > -

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-11 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.05.patch Thanks for explaining [~liuml07], I misread your previous comment. Agree the

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-11 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.06.patch Whitespace + checkstyle fix. > When starting a file, NameNode should generate

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-12 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15192175#comment-15192175 ] Xiao Chen commented on HDFS-9405: - Failed tests look unrelated. May I get another review? T

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-16 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.07.patch I took another pass on my patch, I think I should also log the IOException if

[jira] [Updated] (HDFS-9405) Warmup NameNode EDEK caches in background thread

2016-03-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.13.patch Patch 13 removes the 2 unneeded imports.. > Warmup NameNode EDEK caches in ba

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198027#comment-15198027 ] Xiao Chen commented on HDFS-9405: - Thanks for the review Andrew! All comments are addressed

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.10.patch Patch 10 for precommit. Now there should only be 80 char complaints about the

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.09.patch Patch 9 addresses above comments. Thanks again Andrew for the prompt and help

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198269#comment-15198269 ] Xiao Chen commented on HDFS-9405: - You're right Andrew, thanks for explaining. My previous

[jira] [Commented] (HDFS-9405) Warmup NameNode EDEK caches in background thread

2016-03-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202145#comment-15202145 ] Xiao Chen commented on HDFS-9405: - Hey Andrew, that's what I noticed and was trying to do i

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.11.patch checkstyle: New member var 'conf' hides a field in some methods me: sorry, ren

[jira] [Updated] (HDFS-9405) Warmup NameNode EDEK caches in background thread

2016-03-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.12.patch Patch 12 to fix the above. Thanks again Andrew. > Warmup NameNode EDEK caches

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198161#comment-15198161 ] Xiao Chen commented on HDFS-9405: - Thanks Andrew. Will address the comments, except: bq. Do

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-19 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.08.patch > When starting a file, NameNode should generate EDEK in a separate thread > -

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-20 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201072#comment-15201072 ] Xiao Chen commented on HDFS-9405: - Failed tests look unrelated, checkstyle is configuration

[jira] [Commented] (HDFS-9405) Warmup NameNode EDEK caches in background thread

2016-03-21 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204977#comment-15204977 ] Xiao Chen commented on HDFS-9405: - Thanks Andrew and all for the review! (Sorry for that mi

[jira] [Commented] (HDFS-9427) HDFS should not default to ephemeral ports

2016-03-25 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15212133#comment-15212133 ] Xiao Chen commented on HDFS-9427: - Thanks [~xiaobingo] for working on this. Have we consid

[jira] [Created] (HDFS-10233) Ability to list inode number with -ls

2016-03-29 Thread Xiao Chen (JIRA)
Xiao Chen created HDFS-10233: Summary: Ability to list inode number with -ls Key: HDFS-10233 URL: https://issues.apache.org/jira/browse/HDFS-10233 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-9427) HDFS should not default to ephemeral ports

2016-03-31 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15221156#comment-15221156 ] Xiao Chen commented on HDFS-9427: - Thanks [~xiaobingo] for the new rev, LGTM. +1 (non-bindi

[jira] [Assigned] (HDFS-9909) Can't read file after hdfs restart

2016-04-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen reassigned HDFS-9909: --- Assignee: Xiao Chen > Can't read file after hdfs restart > -- > >

[jira] [Commented] (HDFS-9909) Can't read file after hdfs restart

2016-04-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225381#comment-15225381 ] Xiao Chen commented on HDFS-9909: - Thanks for reporting this Bogdan. I'd like to work on th

[jira] [Assigned] (HDFS-6520) hdfs fsck -move not working

2016-04-05 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen reassigned HDFS-6520: --- Assignee: Xiao Chen (was: John Zhuge) > hdfs fsck -move not working > --- > >

[jira] [Updated] (HDFS-6520) hdfs fsck -move not working

2016-04-05 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-6520: Attachment: john.test.patch Seems John didn't get a complete test patch when uploading. Fortunately he hands

[jira] [Updated] (HDFS-6520) hdfs fsck -move not working

2016-04-05 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-6520: Status: Patch Available (was: In Progress) > hdfs fsck -move not working > --- > >

[jira] [Updated] (HDFS-6520) hdfs fsck -move not working

2016-04-05 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-6520: Attachment: HDFS-6520.01.patch > hdfs fsck -move not working > --- > >

[jira] [Commented] (HDFS-6520) hdfs fsck -move not working

2016-04-05 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227185#comment-15227185 ] Xiao Chen commented on HDFS-6520: - So the root cause is that {{fsck -move}} will throw exce

[jira] [Updated] (HDFS-6520) hdfs fsck -move not working

2016-04-05 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-6520: Attachment: HDFS-6520.02.patch Thanks [~cmccabe] for the review! Good idea - we should actively detect and p

[jira] [Commented] (HDFS-6520) hdfs fsck -move not working

2016-04-05 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227779#comment-15227779 ] Xiao Chen commented on HDFS-6520: - Test failure looks unrelated and passed locally. > hdfs

[jira] [Commented] (HDFS-6520) hdfs fsck passes invalid length value when creating BlockReader

2016-04-06 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15228843#comment-15228843 ] Xiao Chen commented on HDFS-6520: - Thanks very much [~cmccabe]! Maybe we can update the tit

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229404#comment-15229404 ] Xiao Chen commented on HDFS-10267: -- Thanks [~cmccabe] for working on this, patch looks gr

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229452#comment-15229452 ] Xiao Chen commented on HDFS-10267: -- Thanks for explaining Colin, that's very helpful. And

[jira] [Commented] (HDFS-8986) Add option to -du to calculate directory space usage excluding snapshots

2016-04-12 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237909#comment-15237909 ] Xiao Chen commented on HDFS-8986: - Thanks Gautam for creating this, and everyone for the di

[jira] [Commented] (HDFS-8986) Add option to -du to calculate directory space usage excluding snapshots

2016-04-12 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238288#comment-15238288 ] Xiao Chen commented on HDFS-8986: - Thanks Gautam for the response. I'm assigning this to my

[jira] [Assigned] (HDFS-8986) Add option to -du to calculate directory space usage excluding snapshots

2016-04-12 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen reassigned HDFS-8986: --- Assignee: Xiao Chen (was: Gautam Gopalakrishnan) > Add option to -du to calculate directory space usa

[jira] [Updated] (HDFS-8986) Add option to -du to calculate directory space usage excluding snapshots

2016-04-13 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-8986: Attachment: HDFS-8986.01.patch I'm attaching a preliminary patch 1 to show the general idea of this change.

[jira] [Updated] (HDFS-8986) Add option to -du to calculate directory space usage excluding snapshots

2016-04-15 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-8986: Status: Patch Available (was: Open) > Add option to -du to calculate directory space usage excluding snapsho

[jira] [Updated] (HDFS-8986) Add option to -du to calculate directory space usage excluding snapshots

2016-04-15 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-8986: Target Version/s: 2.8.0 > Add option to -du to calculate directory space usage excluding snapshots >

[jira] [Updated] (HDFS-8986) Add option to -du to calculate directory space usage excluding snapshots

2016-04-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-8986: Attachment: HDFS-8986.02.patch Patch 2 fixes all reported errors. The 2 added tests in {{DFSShell}} can pass

[jira] [Commented] (HDFS-8986) Add option to -du to calculate directory space usage excluding snapshots

2016-04-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246800#comment-15246800 ] Xiao Chen commented on HDFS-8986: - The left test failures are not related (HDFS-10291 fixes

[jira] [Created] (HDFS-10320) Rack failures may result in NN terminate

2016-04-20 Thread Xiao Chen (JIRA)
Xiao Chen created HDFS-10320: Summary: Rack failures may result in NN terminate Key: HDFS-10320 URL: https://issues.apache.org/jira/browse/HDFS-10320 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-04-20 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Description: If there're rack failures which end up leaving only 1 rack available, {{BlockPlacementPolicy

[jira] [Commented] (HDFS-10320) Rack failures may result in NN terminate

2016-04-20 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15250924#comment-15250924 ] Xiao Chen commented on HDFS-10320: -- The failure is due to a race, since in {{BPPD#chooseR

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-04-20 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Attachment: HDFS-10320.01.patch Patch 1 logs an error when {{NetworkTopology#chooseRandom}} cannot choose

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-04-20 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Status: Patch Available (was: Open) > Rack failures may result in NN terminate >

[jira] [Commented] (HDFS-9427) HDFS should not default to ephemeral ports

2016-04-22 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254398#comment-15254398 ] Xiao Chen commented on HDFS-9427: - Thanks Xiaobing for pushing this forward. We need a comm

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-04-25 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Attachment: HDFS-10324.002.patch I went through this again, and feels the current way of falling back to l

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-04-26 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Attachment: HDFS-10320.02.patch Thanks [~mingma] for the advice. I didn't understand the placement code c

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-04-26 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Attachment: (was: HDFS-10324.002.patch) > Rack failures may result in NN terminate > -

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-05-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Attachment: HDFS-10320.03.patch Hi [~mingma], Thanks a lot for the enlightening suggestion. Given we've fi

[jira] [Updated] (HDFS-9405) Warmup NameNode EDEK caches in background thread

2016-05-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Labels: supportability (was: ) > Warmup NameNode EDEK caches in background thread >

[jira] [Updated] (HDFS-9721) Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference

2016-05-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9721: Labels: supportability (was: ) > Allow Delimited PB OIV tool to run upon fsimage that contains INodeReferenc

[jira] [Updated] (HDFS-6520) hdfs fsck -move passes invalid length value when creating BlockReader

2016-05-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-6520: Labels: supportability (was: ) > hdfs fsck -move passes invalid length value when creating BlockReader > ---

[jira] [Commented] (HDFS-10320) Rack failures may result in NN terminate

2016-05-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270255#comment-15270255 ] Xiao Chen commented on HDFS-10320: -- Thank you [~mingma]! Patch 4 is attached: bq. Althoug

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-05-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Attachment: HDFS-10320.04.patch > Rack failures may result in NN terminate > -

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-05-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Attachment: HDFS-10320.05.patch > Rack failures may result in NN terminate > -

[jira] [Commented] (HDFS-10320) Rack failures may result in NN terminate

2016-05-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270938#comment-15270938 ] Xiao Chen commented on HDFS-10320: -- After a good night's sleep, I see the {{addToExcluded

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-05-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Affects Version/s: 2.6.0 Target Version/s: 2.8.0 > Rack failures may result in NN terminate > ---

[jira] [Created] (HDFS-10364) Log current node in reversexml tool when parse failed

2016-05-04 Thread Xiao Chen (JIRA)
Xiao Chen created HDFS-10364: Summary: Log current node in reversexml tool when parse failed Key: HDFS-10364 URL: https://issues.apache.org/jira/browse/HDFS-10364 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-10364) Log current node in reversexml tool when parse failed

2016-05-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10364: - Status: Patch Available (was: Open) > Log current node in reversexml tool when parse failed > ---

[jira] [Updated] (HDFS-10364) Log current node in reversexml tool when parse failed

2016-05-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10364: - Attachment: HDFS-10364.01.patch > Log current node in reversexml tool when parse failed >

[jira] [Updated] (HDFS-10320) Rack failures may result in NN terminate

2016-05-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10320: - Attachment: HDFS-10320.06.patch bq. Maybe rename the method or add more comments there. Sure, I updated th

[jira] [Commented] (HDFS-10320) Rack failures may result in NN terminate

2016-05-04 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271717#comment-15271717 ] Xiao Chen commented on HDFS-10320: -- Thanks _you_ [~mingma] for the valuable thoughts, rev

[jira] [Commented] (HDFS-10372) Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume

2016-05-06 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15274389#comment-15274389 ] Xiao Chen commented on HDFS-10372: -- Thanks for the contribution [~shahrs87] and [~kihwal]

[jira] [Commented] (HDFS-10372) Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume

2016-05-06 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15274482#comment-15274482 ] Xiao Chen commented on HDFS-10372: -- I see... thanks for the explanation! > Fix for faili

[jira] [Commented] (HDFS-4210) NameNode Format should not fail for DNS resolution on minority of JournalNode

2016-05-12 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15282111#comment-15282111 ] Xiao Chen commented on HDFS-4210: - Thanks all for the discussion so far, and [~clamb] for t

[jira] [Commented] (HDFS-9231) fsck doesn't explicitly list when Bad Replicas/Blocks are in a snapshot

2015-10-14 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14958010#comment-14958010 ] Xiao Chen commented on HDFS-9231: - The pre-patch Findbugs warning is unrelated. (see HDFS-9

[jira] [Created] (HDFS-9250) LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty

2015-10-15 Thread Xiao Chen (JIRA)
Xiao Chen created HDFS-9250: --- Summary: LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty Key: HDFS-9250 URL: https://issues.apache.org/jira/browse/HDFS-9250 Project: Hadoop HDFS

[jira] [Updated] (HDFS-9250) LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty

2015-10-15 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9250: Attachment: HDFS-9250.001.patch Patch 001 fixes the described exception. Added test case in a new test class

[jira] [Updated] (HDFS-9250) LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty

2015-10-15 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9250: Status: Patch Available (was: Open) > LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is

[jira] [Commented] (HDFS-9250) LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty

2015-10-15 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959574#comment-14959574 ] Xiao Chen commented on HDFS-9250: - The Findbugs warning and test failure are not relevant.

[jira] [Updated] (HDFS-3059) ssl-server.xml causes NullPointer

2015-10-15 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-3059: Status: Open (was: Patch Available) > ssl-server.xml causes NullPointer > -

[jira] [Updated] (HDFS-3059) ssl-server.xml causes NullPointer

2015-10-15 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-3059: Attachment: HDFS-3059.06.patch > ssl-server.xml causes NullPointer > - > >

[jira] [Commented] (HDFS-3059) ssl-server.xml causes NullPointer

2015-10-15 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960014#comment-14960014 ] Xiao Chen commented on HDFS-3059: - Thanks Yongjun for the comments. Attached patch 06 addre

[jira] [Updated] (HDFS-9231) fsck doesn't explicitly list when Bad Replicas/Blocks are in a snapshot

2015-10-15 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9231: Status: Open (was: Patch Available) > fsck doesn't explicitly list when Bad Replicas/Blocks are in a snapsho

[jira] [Commented] (HDFS-9250) LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty

2015-10-15 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960044#comment-14960044 ] Xiao Chen commented on HDFS-9250: - Hi Andrew, thanks for the comments and additional inform

[jira] [Updated] (HDFS-9250) LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty

2015-10-16 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9250: Status: Open (was: Patch Available) > LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is

[jira] [Updated] (HDFS-7087) Ability to list /.reserved

2015-10-16 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-7087: Status: Patch Available (was: Open) > Ability to list /.reserved > -- > >

[jira] [Updated] (HDFS-9250) LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty

2015-10-16 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9250: Attachment: HDFS-9250.002.patch > LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty

[jira] [Updated] (HDFS-9250) LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty

2015-10-16 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9250: Attachment: HDFS-9250.002.patch > LocatedBlock#addCachedLoc may throw ArrayStoreException when cache is empty

  1   2   3   4   5   6   7   8   9   10   >