[jira] [Commented] (HDFS-10263) Provide symmetric entries in reversed snapshot diff report

2016-04-14 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241707#comment-15241707 ] John Zhuge commented on HDFS-10263: --- Thanks [~yzhangal] for the great report. {{SnapshotDiffReport}} API

[jira] [Commented] (HDFS-7499) Add NFSv4 + Kerberos / client authentication support

2016-04-14 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241677#comment-15241677 ] John Zhuge commented on HDFS-7499: -- Wonder whether we can add an HDFS based layout type for

[jira] [Commented] (HDFS-10289) Balancer configures DNs directly

2016-04-14 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241396#comment-15241396 ] John Zhuge commented on HDFS-10289: --- Thanks [~anu], I will take a look. > Balancer configures DNs

[jira] [Updated] (HDFS-10216) distcp -diff relative path exception

2016-04-14 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10216: -- Hadoop Flags: Reviewed +1 LGTM > distcp -diff relative path exception >

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-04-14 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240779#comment-15240779 ] John Zhuge commented on HDFS-9940: -- Filed HDFS-10289 for the idea coming out of the discussion. I'd like

[jira] [Created] (HDFS-10289) Balancer configures DNs directly

2016-04-14 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10289: - Summary: Balancer configures DNs directly Key: HDFS-10289 URL: https://issues.apache.org/jira/browse/HDFS-10289 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Updated] (HDFS-9670) DistCp throws NPE when source is root

2016-04-14 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9670: - Component/s: distcp > DistCp throws NPE when source is root > - > >

[jira] [Updated] (HDFS-9670) DistCp throws NPE when source is root

2016-04-14 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9670: - Fix Version/s: 2.8.0 Affects Version/s: 2.6.0 Target Version/s: 2.8.0 Status:

[jira] [Updated] (HDFS-9670) DistCp throws NPE when source is root

2016-04-14 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9670: - Attachment: HDFS-9670.001.patch Patch 001: * {{computeSourceRootPath}} returns itself instead of its

[jira] [Updated] (HDFS-10288) MiniYARNCluster should implement AutoCloseable

2016-04-14 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10288: -- Description: {{MiniYARNCluster}} should implement {{AutoCloseable}} in order to support

[jira] [Created] (HDFS-10288) MiniYARNCluster should implement AutoCloseable

2016-04-14 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10288: - Summary: MiniYARNCluster should implement AutoCloseable Key: HDFS-10288 URL: https://issues.apache.org/jira/browse/HDFS-10288 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-9670) DistCp throws NPE when source is root

2016-04-14 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9670: - Summary: DistCp throws NPE when source is root (was: DistCp throws NPE when source is root and target

[jira] [Created] (HDFS-10287) MiniDFSCluster should implement AutoCloseable

2016-04-13 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10287: - Summary: MiniDFSCluster should implement AutoCloseable Key: HDFS-10287 URL: https://issues.apache.org/jira/browse/HDFS-10287 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-9670) DistCp throws NPE when source is root and target exists

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9670: - Summary: DistCp throws NPE when source is root and target exists (was: DistCp throws NPE when source is

[jira] [Commented] (HDFS-9670) DistCp throws NPE when source is root "/"

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240490#comment-15240490 ] John Zhuge commented on HDFS-9670: -- Source file system can also be different, e.g., {{s3a}}. > DistCp

[jira] [Work started] (HDFS-9670) DistCp throws NPE when source is root "/"

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-9670 started by John Zhuge. > DistCp throws NPE when source is root "/" > - >

[jira] [Commented] (HDFS-9670) DistCp throws NPE when source is root "/"

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240486#comment-15240486 ] John Zhuge commented on HDFS-9670: -- Reproduced test in different ways: * The target dir must exist. * The

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240461#comment-15240461 ] John Zhuge commented on HDFS-10216: --- Very nice. Consider reviewed after 2 tiny updates: * Line 709 is

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-04-13 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15239952#comment-15239952 ] John Zhuge commented on HDFS-10216: --- Great job. Just minor nitpicks: * {{TestDistCpSync.java}} ## 683:

[jira] [Commented] (HDFS-7499) Add NFSv4 + Kerberos / client authentication support

2016-04-11 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235557#comment-15235557 ] John Zhuge commented on HDFS-7499: -- Do not have bandwidth to work on it for now. > Add NFSv4 + Kerberos /

[jira] [Updated] (HDFS-7499) Add NFSv4 + Kerberos / client authentication support

2016-04-11 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-7499: - Assignee: (was: John Zhuge) > Add NFSv4 + Kerberos / client authentication support >

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-04-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234354#comment-15234354 ] John Zhuge commented on HDFS-10216: --- Thanks for enhancing the unit tests which will definitely improve

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-04-09 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233621#comment-15233621 ] John Zhuge commented on HDFS-10216: --- Great job with the patch [~bwtakacy]. I assume the ad-hoc test

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-04-08 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232890#comment-15232890 ] John Zhuge commented on HDFS-10216: --- Please go ahead. > distcp -diff relative path exception >

[jira] [Updated] (HDFS-10216) distcp -diff relative path exception

2016-04-08 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10216: -- Assignee: (was: John Zhuge) > distcp -diff relative path exception >

[jira] [Updated] (HDFS-6520) hdfs fsck -move not working

2016-04-01 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-6520: - Attachment: HDFS-6520-partial.001.patch Add a partial patch with a unit test that reproduces the problem.

[jira] [Commented] (HDFS-6520) hdfs fsck -move not working

2016-04-01 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15222114#comment-15222114 ] John Zhuge commented on HDFS-6520: -- In this test case, {{echo CORRUPT >/path/to/block}} is used to corrupt

[jira] [Assigned] (HDFS-10233) Ability to list inode number with -ls

2016-03-29 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-10233: - Assignee: John Zhuge > Ability to list inode number with -ls >

[jira] [Commented] (HDFS-10230) HDFS Native Client build failed

2016-03-29 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15216205#comment-15216205 ] John Zhuge commented on HDFS-10230: --- No problem with {{mvn clean install -DskipTests

[jira] [Created] (HDFS-10230) HDFS Native Client build failed

2016-03-29 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10230: - Summary: HDFS Native Client build failed Key: HDFS-10230 URL: https://issues.apache.org/jira/browse/HDFS-10230 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-7060) Avoid taking locks when sending heartbeats from the DataNode

2016-03-28 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15215413#comment-15215413 ] John Zhuge commented on HDFS-7060: -- If we assume dataset lock protects {{bpSlices}}. The following

[jira] [Commented] (HDFS-9939) Increase DecompressorStream skip buffer size

2016-03-28 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15215119#comment-15215119 ] John Zhuge commented on HDFS-9939: -- Thanks. That could have been caught by HADOOP-12701 since I ran

[jira] [Commented] (HDFS-6520) hdfs fsck -move not working

2016-03-28 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214966#comment-15214966 ] John Zhuge commented on HDFS-6520: -- Wrote the unit test that reproduced the problem. Got checksum

[jira] [Commented] (HDFS-10215) distcp throws vague exception without home directory

2016-03-28 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214228#comment-15214228 ] John Zhuge commented on HDFS-10215: --- Thanks [~steve_l]. Will do. > distcp throws vague exception

[jira] [Updated] (HDFS-9633) Add more SnapshotDiffReport unit tests

2016-03-25 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9633: - Description: Add more behavior-based unit tests to verify and document SnapshotDiffReport API behaviors:

[jira] [Commented] (HDFS-10216) distcp -diff relative path exception

2016-03-25 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212151#comment-15212151 ] John Zhuge commented on HDFS-10216: --- Did hit the exception twice on 2 different nodes when running

[jira] [Created] (HDFS-10216) distcp -diff relative path exception

2016-03-25 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10216: - Summary: distcp -diff relative path exception Key: HDFS-10216 URL: https://issues.apache.org/jira/browse/HDFS-10216 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-10215) distcp throws vague exception without home directory

2016-03-25 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10215: - Summary: distcp throws vague exception without home directory Key: HDFS-10215 URL: https://issues.apache.org/jira/browse/HDFS-10215 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-9939) Increase DecompressorStream skip buffer size

2016-03-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9939: - Fix Version/s: 2.8.0 Status: Patch Available (was: Open) > Increase DecompressorStream skip

[jira] [Updated] (HDFS-9939) Increase DecompressorStream skip buffer size

2016-03-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9939: - Attachment: HDFS-9939.001.patch Patch 001: * Set skip buffer size to 2048 (MAX_SKIP_BUFFER_SIZE) * Move

[jira] [Updated] (HDFS-9573) SnapshotDiffReport$DiffReportEntry$hashCode should use type field

2016-03-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9573: - Resolution: Won't Fix Status: Resolved (was: Patch Available) It is recommended for hashCode and

[jira] [Commented] (HDFS-9633) Test SnapshotDiffReport file path corner cases

2016-03-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209577#comment-15209577 ] John Zhuge commented on HDFS-9633: -- No, I am not trying fix any problem, just adding more test cases. >

[jira] [Updated] (HDFS-8496) Calling stopWriter() with FSDatasetImpl lock held may block other threads

2016-03-23 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-8496: - Summary: Calling stopWriter() with FSDatasetImpl lock held may block other threads (was: Should not call

[jira] [Updated] (HDFS-8496) Should not call stopWriter() with FSDatasetImpl lock held

2016-03-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-8496: - Summary: Should not call stopWriter() with FSDatasetImpl lock held (was: Calling stopWriter() with

[jira] [Updated] (HDFS-9939) Increase DecompressorStream skip buffer size

2016-03-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9939: - Summary: Increase DecompressorStream skip buffer size (was: Possible performance improvement by

[jira] [Updated] (HDFS-10190) Add DN FSDatasetImpl lock metrics

2016-03-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10190: -- Description: Add DN FSDatasetImpl lock metrics: * Number of lock calls * Contention rate * Average wait

[jira] [Created] (HDFS-10190) Expose FSDatasetImpl lock metrics

2016-03-21 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10190: - Summary: Expose FSDatasetImpl lock metrics Key: HDFS-10190 URL: https://issues.apache.org/jira/browse/HDFS-10190 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-10185) TestHFlushInterrupted verifies interrupt state incorrectly

2016-03-21 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205713#comment-15205713 ] John Zhuge commented on HDFS-10185: --- Close it then. Thanks for attempting to fix this unit test though,

[jira] [Commented] (HDFS-10185) TestHFlushInterrupted verifies interrupt state incorrectly

2016-03-21 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204427#comment-15204427 ] John Zhuge commented on HDFS-10185: --- bq. Yes, my new code is used to check if

[jira] [Commented] (HDFS-10185) TestHFlushInterrupted verifies interrupt state incorrectly

2016-03-20 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203699#comment-15203699 ] John Zhuge commented on HDFS-10185: --- Thanks [~linyiqun] for working on the unit test issue which is very

[jira] [Commented] (HDFS-9885) In distcp cmd ouput, Display name should be given for org.apache.hadoop.tools.mapred.CopyMapper$Counter.

2016-03-19 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200092#comment-15200092 ] John Zhuge commented on HDFS-9885: -- My bad, it seems "_" is necessary because

[jira] [Commented] (HDFS-9885) In distcp cmd ouput, Display name should be given for org.apache.hadoop.tools.mapred.CopyMapper$Counter.

2016-03-19 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200058#comment-15200058 ] John Zhuge commented on HDFS-9885: -- Thanks for the patch. Minor comments: * Replace "Distcp" and "distcp"

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-19 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198864#comment-15198864 ] John Zhuge commented on HDFS-9940: -- Thank you all for the great discussion ! [~yzhangal] like the design

[jira] [Commented] (HDFS-9885) In distcp cmd ouput, Display name should be given for org.apache.hadoop.tools.mapred.CopyMapper$Counter.

2016-03-19 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200929#comment-15200929 ] John Zhuge commented on HDFS-9885: -- OK > In distcp cmd ouput, Display name should be given for >

[jira] [Commented] (HDFS-10173) Typo in DataXceiverServer

2016-03-18 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197765#comment-15197765 ] John Zhuge commented on HDFS-10173: --- +1 LGTM > Typo in DataXceiverServer > - >

[jira] [Updated] (HDFS-9885) In distcp cmd ouput, Display name should be given for org.apache.hadoop.tools.mapred.CopyMapper$Counter.

2016-03-18 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9885: - Hadoop Flags: Reviewed +1 LGTM > In distcp cmd ouput, Display name should be given for >

[jira] [Resolved] (HDFS-10171) Balancer should log config values

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge resolved HDFS-10171. --- Resolution: Duplicate Fix Version/s: 2.8.0 {noformat} 2016-03-15 22:42:31,618 [Thread-0] INFO

[jira] [Updated] (HDFS-10171) Balancer should log config values

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10171: -- Description: To improve supportability, Balancer should log config values. (was: To improve

[jira] [Created] (HDFS-10171) Balancer should log config values and iteration termination reasons

2016-03-15 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10171: - Summary: Balancer should log config values and iteration termination reasons Key: HDFS-10171 URL: https://issues.apache.org/jira/browse/HDFS-10171 Project: Hadoop HDFS

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196712#comment-15196712 ] John Zhuge commented on HDFS-9940: -- I like 2) better because all rebalance configuration is done at

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196705#comment-15196705 ] John Zhuge commented on HDFS-9940: -- [~yzhangal], great idea if we don't have to manually config each DN

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196482#comment-15196482 ] John Zhuge commented on HDFS-9940: -- Balancer uses config {{dfs.datanode.balance.max.concurrent.moves}} to

[jira] [Commented] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-15 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196363#comment-15196363 ] John Zhuge commented on HDFS-9940: -- +1 HDFS-7466 > Rename dfs.balancer.max.concurrent.moves to avoid

[jira] [Created] (HDFS-9940) New property dfs.balancer.max.concurrent.moves for Balancer

2016-03-10 Thread John Zhuge (JIRA)
John Zhuge created HDFS-9940: Summary: New property dfs.balancer.max.concurrent.moves for Balancer Key: HDFS-9940 URL: https://issues.apache.org/jira/browse/HDFS-9940 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9940: - Summary: Rename dfs.balancer.max.concurrent.moves to avoid confusion (was: New property

[jira] [Assigned] (HDFS-9939) Possible performance improvement by increasing buf size in DecompressorStream in HDFS

2016-03-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-9939: Assignee: John Zhuge (was: Yongjun Zhang) > Possible performance improvement by increasing buf

[jira] [Commented] (HDFS-9913) DispCp doesn't use Trash with -delete option

2016-03-07 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15183607#comment-15183607 ] John Zhuge commented on HDFS-9913: -- The problem exists in CDH5.8.0 (2.6.0) based on my quick test. >

[jira] [Created] (HDFS-9915) Typo in DistCp.html

2016-03-07 Thread John Zhuge (JIRA)
John Zhuge created HDFS-9915: Summary: Typo in DistCp.html Key: HDFS-9915 URL: https://issues.apache.org/jira/browse/HDFS-9915 Project: Hadoop HDFS Issue Type: Improvement Components:

[jira] [Assigned] (HDFS-9913) DispCp doesn't use Trash with -delete option

2016-03-07 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-9913: Assignee: John Zhuge > DispCp doesn't use Trash with -delete option >

[jira] [Updated] (HDFS-9913) DispCp doesn't use Trash with -delete option

2016-03-06 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9913: - Description: Documentation for DistCp -delete option says

[jira] [Commented] (HDFS-7639) Remove the limitation imposed by dfs.balancer.moverThreads

2016-03-03 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179177#comment-15179177 ] John Zhuge commented on HDFS-7639: -- +1 Definitely wasteful to occupy a thread to send a request and wait

[jira] [Work started] (HDFS-6520) hdfs fsck -move not working

2016-03-03 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-6520 started by John Zhuge. > hdfs fsck -move not working > --- > > Key:

[jira] [Updated] (HDFS-6520) hdfs fsck -move not working

2016-03-03 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-6520: - Summary: hdfs fsck -move not working (was: Failed to run fsck -move) Was able to gain more insight after

[jira] [Assigned] (HDFS-6520) Failed to run fsck -move

2016-03-03 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-6520: Assignee: John Zhuge > Failed to run fsck -move > > > Key:

[jira] [Assigned] (HDFS-9896) WebHDFS API may return invalid JSON

2016-03-03 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-9896: Assignee: John Zhuge > WebHDFS API may return invalid JSON > --- >

[jira] [Assigned] (HDFS-9841) FileDiff's skipped by hdfs snapshotDiff

2016-03-02 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-9841: Assignee: John Zhuge > FileDiff's skipped by hdfs snapshotDiff >

[jira] [Created] (HDFS-9728) TestDFSShell should not create MiniDFSCluster in each test case

2016-01-31 Thread John Zhuge (JIRA)
John Zhuge created HDFS-9728: Summary: TestDFSShell should not create MiniDFSCluster in each test case Key: HDFS-9728 URL: https://issues.apache.org/jira/browse/HDFS-9728 Project: Hadoop HDFS

[jira] [Updated] (HDFS-9573) SnapshotDiffReport$DiffReportEntry$hashCode should use type field

2016-01-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9573: - Summary: SnapshotDiffReport$DiffReportEntry$hashCode should use type field (was:

[jira] [Reopened] (HDFS-9573) SnapshotDiffReport$DiffReportEntry$hashCode should use type field

2016-01-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reopened HDFS-9573: -- Fix the summary based on Nicholas's comment. > SnapshotDiffReport$DiffReportEntry$hashCode should use type

[jira] [Updated] (HDFS-9573) SnapshotDiffReport$DiffReportEntry$hashCode should use type field

2016-01-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9573: - Status: Patch Available (was: Reopened) Resubmit the patch. >

[jira] [Updated] (HDFS-9573) SnapshotDiffReport$DiffReportEntry$hashCode should use type field

2016-01-22 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9573: - Attachment: HDFS-9573-002.patch Patch 002: * Rebase * Update commit message >

[jira] [Commented] (HDFS-9655) NN should start JVM pause monitor before loading fsimage

2016-01-20 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109313#comment-15109313 ] John Zhuge commented on HDFS-9655: -- Unit test failures seem unrelated. > NN should start JVM pause

[jira] [Assigned] (HDFS-9670) DistCp throws NPE when source is root "/"

2016-01-20 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-9670: Assignee: John Zhuge (was: Yongjun Zhang) > DistCp throws NPE when source is root "/" >

[jira] [Commented] (HDFS-9655) NN should start JVM pause monitor before loading fsimage

2016-01-20 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109655#comment-15109655 ] John Zhuge commented on HDFS-9655: -- Thanks [~eddyxu]. > NN should start JVM pause monitor before loading

[jira] [Updated] (HDFS-9655) NN should start JVM pause monitor before loading fsimage

2016-01-16 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9655: - Status: Patch Available (was: Open) > NN should start JVM pause monitor before loading fsimage >

[jira] [Updated] (HDFS-9655) NN should start JVM pause monitor before loading fsimage

2016-01-16 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9655: - Attachment: HDFS-9655.001.patch Patch 001: * Start JVM pause monitor before loading fsimage during NN

[jira] [Created] (HDFS-9655) NN should start JVM pause monitor before loading fsimage

2016-01-16 Thread John Zhuge (JIRA)
John Zhuge created HDFS-9655: Summary: NN should start JVM pause monitor before loading fsimage Key: HDFS-9655 URL: https://issues.apache.org/jira/browse/HDFS-9655 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-9633) Test SnapshotDiffReport file path corner cases

2016-01-12 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094575#comment-15094575 ] John Zhuge commented on HDFS-9633: -- Diff report for testDiffReportWithDeleteCreateSameName: {noformat}

[jira] [Updated] (HDFS-9633) Test SnapshotDiffReport file path corner cases

2016-01-12 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9633: - Status: Patch Available (was: Open) > Test SnapshotDiffReport file path corner cases >

[jira] [Updated] (HDFS-9633) Test SnapshotDiffReport file path corner cases

2016-01-12 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9633: - Attachment: HDFS-9633-001.patch > Test SnapshotDiffReport file path corner cases >

[jira] [Reopened] (HDFS-9573) o.a.h.hdfs.protocol.SnapshotDiffReport$DiffReportEntry$hashCode inconsistent with equals

2016-01-12 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reopened HDFS-9573: -- Unlink this jira from HDFS-9522 for granularity. >

[jira] [Commented] (HDFS-9522) Cleanup o.a.h.hdfs.protocol.SnapshotDiffReport

2016-01-12 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094137#comment-15094137 ] John Zhuge commented on HDFS-9522: -- Thanks, [~drankye], it is not minor cleanup :) I will remove the

[jira] [Commented] (HDFS-9573) o.a.h.hdfs.protocol.SnapshotDiffReport$DiffReportEntry$hashCode inconsistent with equals

2016-01-12 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094186#comment-15094186 ] John Zhuge commented on HDFS-9573: -- Hi [~szetszwo], Thanks for looking at this jira. hashCode and equals

[jira] [Updated] (HDFS-9573) o.a.h.hdfs.protocol.SnapshotDiffReport$DiffReportEntry$hashCode inconsistent with equals

2016-01-12 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9573: - Attachment: HDFS-9573-001.patch Patch 001: * Implement hashCode and equals with Apache commons builders.

[jira] [Updated] (HDFS-9522) Cleanup o.a.h.hdfs.protocol.SnapshotDiffReport

2016-01-11 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9522: - Status: Open (was: Patch Available) Pondering more related cleanups in DiffInfo and DistCpSync. There is

[jira] [Updated] (HDFS-9522) Cleanup o.a.h.hdfs.protocol.SnapshotDiffReport

2016-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9522: - Attachment: HDFS-9522-006.patch Patch 006: * Use subclass hierarchy instead of C-style tagged union for

[jira] [Created] (HDFS-9633) Test SnapshotDiffReport file path corner cases

2016-01-09 Thread John Zhuge (JIRA)
John Zhuge created HDFS-9633: Summary: Test SnapshotDiffReport file path corner cases Key: HDFS-9633 URL: https://issues.apache.org/jira/browse/HDFS-9633 Project: Hadoop HDFS Issue Type: Test

[jira] [Commented] (HDFS-9522) Cleanup o.a.h.hdfs.protocol.SnapshotDiffReport

2016-01-09 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090838#comment-15090838 ] John Zhuge commented on HDFS-9522: -- The unit tests do not belong to this jira. Will move them to

[jira] [Updated] (HDFS-9522) Cleanup o.a.h.hdfs.protocol.SnapshotDiffReport

2016-01-08 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9522: - Attachment: HDFS-9522-005.patch Patch 005: * Use subclass hierarchy instead of C-style tagged union for

[jira] [Updated] (HDFS-9522) Cleanup o.a.h.hdfs.protocol.SnapshotDiffReport

2016-01-07 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9522: - Attachment: HDFS-9522-003.patch Patch 003: * Fix checkstyle and findbugs errors and warnings. * Abandon

<    4   5   6   7   8   9   10   >