[jira] [Commented] (HDFS-11088) Quash unnecessary safemode WARN message during NameNode startup

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631719#comment-15631719 ] Mingliang Liu commented on HDFS-11088: -- I think the proposal and the solution make sense. Thanks for

[jira] [Commented] (HDFS-10885) [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on

2016-11-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631635#comment-15631635 ] Kai Zheng commented on HDFS-10885: -- bq. One idea is just throw an error and fail SPS startup ... I'd like

[jira] [Commented] (HDFS-10531) Add EC policy and storage policy related usage summarization function to dfs du command

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631574#comment-15631574 ] Hadoop QA commented on HDFS-10531: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11079) TestQuotaByStorageType#setUp does not need refresh the cluster

2016-11-02 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-11079: - Status: Patch Available (was: Open) > TestQuotaByStorageType#setUp does not need refresh the

[jira] [Commented] (HDFS-11079) TestQuotaByStorageType#setUp does not need refresh the cluster

2016-11-02 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631530#comment-15631530 ] Xiaobing Zhou commented on HDFS-11079: -- Per the discussion, I posted a simple patch v000. Thanks

[jira] [Updated] (HDFS-11079) TestQuotaByStorageType#setUp does not need refresh the cluster

2016-11-02 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-11079: - Attachment: HDFS-11079.000.patch > TestQuotaByStorageType#setUp does not need refresh the cluster

[jira] [Commented] (HDFS-11088) Quash unnecessary safemode WARN message during NameNode startup

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631527#comment-15631527 ] Hadoop QA commented on HDFS-11088: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-10954) [SPS]: Provide mechanism to send blocks movement result back to NN from coordinator DN

2016-11-02 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-10954: Resolution: Fixed Fix Version/s: HDFS-10285 Target Version/s: (was: HDFS-10285)

[jira] [Commented] (HDFS-10531) Add EC policy and storage policy related usage summarization function to dfs du command

2016-11-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631433#comment-15631433 ] Andrew Wang commented on HDFS-10531: Thanks for working on this Wei-Chiu. I have some code review

[jira] [Commented] (HDFS-11060) make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable

2016-11-02 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631425#comment-15631425 ] Lantao Jin commented on HDFS-11060: --- Is it better to complete in another ticket? Seems not only missing

[jira] [Comment Edited] (HDFS-11088) Quash unnecessary safemode WARN message during NameNode startup

2016-11-02 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631353#comment-15631353 ] Yiqun Lin edited comment on HDFS-11088 at 11/3/16 3:05 AM: --- Thanks

[jira] [Updated] (HDFS-11088) Quash unnecessary safemode WARN message during NameNode startup

2016-11-02 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11088: - Attachment: HDFS-11088.002.patch Thanks [~andrew.wang] for the comments. {quote} but maybe we should add

[jira] [Commented] (HDFS-11080) Update HttpFS to use ConfigRedactor

2016-11-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631341#comment-15631341 ] Hudson commented on HDFS-11080: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10760 (See

[jira] [Comment Edited] (HDFS-11058) Implement 'hadoop fs -df' command for ViewFileSystem

2016-11-02 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631318#comment-15631318 ] Manoj Govindassamy edited comment on HDFS-11058 at 11/3/16 2:45 AM:

[jira] [Commented] (HDFS-9668) Optimize the locking in FsDatasetImpl

2016-11-02 Thread Jingcheng Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631332#comment-15631332 ] Jingcheng Du commented on HDFS-9668: It seems the failure is not related with this patch, it is caused

[jira] [Commented] (HDFS-11058) Implement 'hadoop fs -df' command for ViewFileSystem

2016-11-02 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631318#comment-15631318 ] Manoj Govindassamy commented on HDFS-11058: --- # {{ViewFsMountPoint}} : Yes, this replaces

[jira] [Assigned] (HDFS-11091) Implement a getTrashRoot that does not fall-back

2016-11-02 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu reassigned HDFS-11091: - Assignee: Yuanbo Liu > Implement a getTrashRoot that does not fall-back >

[jira] [Updated] (HDFS-10531) Add EC policy and storage policy related usage summarization function to dfs du command

2016-11-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10531: --- Status: Patch Available (was: Open) > Add EC policy and storage policy related usage summarization

[jira] [Updated] (HDFS-11080) Update HttpFS to use ConfigRedactor

2016-11-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-11080: --- Resolution: Fixed Fix Version/s: 3.0.0-alpha2 2.8.0 Status:

[jira] [Commented] (HDFS-11080) Update HttpFS to use ConfigRedactor

2016-11-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631241#comment-15631241 ] Andrew Wang commented on HDFS-11080: LGTM +1 will commit shortly > Update HttpFS to use

[jira] [Updated] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11095: - Priority: Minor (was: Major) > BlockManagerSafeMode should respect extension period default

[jira] [Commented] (HDFS-11088) Quash unnecessary safemode WARN message during NameNode startup

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631169#comment-15631169 ] Mingliang Liu commented on HDFS-11088: -- {quote} The default extension value

[jira] [Commented] (HDFS-11088) Quash unnecessary safemode WARN message during NameNode startup

2016-11-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631159#comment-15631159 ] Andrew Wang commented on HDFS-11088: This looks okay, since {{checkSafeMode}} behaves very similarly.

[jira] [Commented] (HDFS-10566) Submit plan request should throw exception if Datanode is in non-REGULAR status.

2016-11-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631123#comment-15631123 ] Hudson commented on HDFS-10566: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10758 (See

[jira] [Commented] (HDFS-11045) TestDirectoryScanner#testThrottling fails: Throttle is too permissive

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631116#comment-15631116 ] Hadoop QA commented on HDFS-11045: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-10566) Submit plan request should throw exception if Datanode is in non-REGULAR status.

2016-11-02 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-10566: Resolution: Fixed Hadoop Flags: Reviewed Target Version/s: 3.0.0-beta1

[jira] [Commented] (HDFS-11058) Implement 'hadoop fs -df' command for ViewFileSystem

2016-11-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631057#comment-15631057 ] Andrew Wang commented on HDFS-11058: Hi Manoj, thanks for working on this. As a high-level comment,

[jira] [Commented] (HDFS-10954) [SPS]: Provide mechanism to send blocks movement result back to NN from coordinator DN

2016-11-02 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631039#comment-15631039 ] Uma Maheswara Rao G commented on HDFS-10954: +1 on the latest patch > [SPS]: Provide

[jira] [Commented] (HDFS-10909) De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads

2016-11-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630973#comment-15630973 ] Hudson commented on HDFS-10909: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10757 (See

[jira] [Updated] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2016-11-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-11096: --- Priority: Blocker (was: Major) > Support rolling upgrade between 2.x and 3.x >

[jira] [Updated] (HDFS-10909) De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads

2016-11-02 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-10909: - Resolution: Fixed Fix Version/s: 3.0.0-alpha2 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-11010) Update to Protobuf 3 in trunk

2016-11-02 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630924#comment-15630924 ] Anu Engineer commented on HDFS-11010: - Sorry for the delay in replying. My original comment was in

[jira] [Updated] (HDFS-11045) TestDirectoryScanner#testThrottling fails: Throttle is too permissive

2016-11-02 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HDFS-11045: Attachment: HDFS-11045.004.patch Here's the current state of the patch. I don't have time

[jira] [Commented] (HDFS-11045) TestDirectoryScanner#testThrottling fails: Throttle is too permissive

2016-11-02 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630870#comment-15630870 ] Daniel Templeton commented on HDFS-11045: - OK, I think I have improved things substantially, but I

[jira] [Commented] (HDFS-10909) De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads

2016-11-02 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630842#comment-15630842 ] Manoj Govindassamy commented on HDFS-10909: --- [~eddyxu], TestMaintenanceState failure is not

[jira] [Commented] (HDFS-10909) De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630839#comment-15630839 ] Hadoop QA commented on HDFS-10909: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2016-11-02 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-11096: -- Summary: Support rolling upgrade between 2.x and 3.x Key: HDFS-11096 URL: https://issues.apache.org/jira/browse/HDFS-11096 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-11026) Convert BlockTokenIdentifier to use Protobuf

2016-11-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630809#comment-15630809 ] Andrew Wang commented on HDFS-11026: Filed HDFS-11096 and linked it here. There might be more to do to

[jira] [Commented] (HDFS-4101) ZKFC should implement zookeeper.recovery.retry like HBase to connect to ZooKeeper

2016-11-02 Thread Andres Perez (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630800#comment-15630800 ] Andres Perez commented on HDFS-4101: I think this issue can be approached the same way YARN-4243 did

[jira] [Commented] (HDFS-11026) Convert BlockTokenIdentifier to use Protobuf

2016-11-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630797#comment-15630797 ] Andrew Wang commented on HDFS-11026: This is probably related to HADOOP-13142, which bumped the min DN

[jira] [Commented] (HDFS-11094) TestLargeBlockReport fails intermittently

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630789#comment-15630789 ] Hadoop QA commented on HDFS-11094: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630771#comment-15630771 ] Hadoop QA commented on HDFS-11095: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11060) make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable

2016-11-02 Thread Ravi Prakash (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630734#comment-15630734 ] Ravi Prakash commented on HDFS-11060: - +1 for pagination > make

[jira] [Commented] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630726#comment-15630726 ] Hudson commented on HDFS-11095: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10756 (See

[jira] [Commented] (HDFS-11026) Convert BlockTokenIdentifier to use Protobuf

2016-11-02 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630708#comment-15630708 ] Daryn Sharp commented on HDFS-11026: bq. I couldn't get a 3.0 and 2.8 datanode/namenode to talk to

[jira] [Updated] (HDFS-11092) Stackoverflow if only root directory is used in Command Line

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11092: - Component/s: fs > Stackoverflow if only root directory is used in Command Line >

[jira] [Updated] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11095: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available)

[jira] [Updated] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11095: - Fix Version/s: 3.0.0-alpha2 2.9.0 > BlockManagerSafeMode should respect

[jira] [Commented] (HDFS-11010) Update to Protobuf 3 in trunk

2016-11-02 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630675#comment-15630675 ] Daryn Sharp commented on HDFS-11010: Yes, [~anu], please clarify what "we have broken so many of those

[jira] [Commented] (HDFS-11045) TestDirectoryScanner#testThrottling fails: Throttle is too permissive

2016-11-02 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630658#comment-15630658 ] Daniel Templeton commented on HDFS-11045: - Well, shite. The test failed in the Jenkins run. I'm

[jira] [Commented] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630647#comment-15630647 ] Hadoop QA commented on HDFS-11095: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11094) TestLargeBlockReport fails intermittently

2016-11-02 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630592#comment-15630592 ] Eric Badger commented on HDFS-11094: bq. Will this suppress the heartbeat interval config, and fail

[jira] [Updated] (HDFS-11094) TestLargeBlockReport fails intermittently

2016-11-02 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Badger updated HDFS-11094: --- Status: Patch Available (was: Open) > TestLargeBlockReport fails intermittently >

[jira] [Commented] (HDFS-11041) Unable to unregister FsDatasetState MBean if if DataNode is shutdown twice

2016-11-02 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630565#comment-15630565 ] Wei-Chiu Chuang commented on HDFS-11041: The test failure is unrelated to this patch. > Unable to

[jira] [Updated] (HDFS-10909) De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads

2016-11-02 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-10909: -- Attachment: HDFS-10909.02.patch Thanks for the review [~eddyxu]. Incorporated your

[jira] [Updated] (HDFS-11079) TestQuotaByStorageType#setUp does not need refresh the cluster

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11079: - Priority: Minor (was: Major) > TestQuotaByStorageType#setUp does not need refresh the cluster >

[jira] [Updated] (HDFS-11079) TestQuotaByStorageType#setUp does not need refresh the cluster

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11079: - Component/s: test > TestQuotaByStorageType#setUp does not need refresh the cluster >

[jira] [Commented] (HDFS-11031) Add additional unit test for DataNode startup behavior when volumes fail

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630503#comment-15630503 ] Mingliang Liu commented on HDFS-11031: -- Thanks for your continuous help and review, [~brahmareddy]!

[jira] [Commented] (HDFS-11045) TestDirectoryScanner#testThrottling fails: Throttle is too permissive

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630511#comment-15630511 ] Hadoop QA commented on HDFS-11045: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11094) TestLargeBlockReport fails intermittently

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630481#comment-15630481 ] Mingliang Liu commented on HDFS-11094: -- # Will this suppress the heartbeat interval config, and fail

[jira] [Commented] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630482#comment-15630482 ] Jing Zhao commented on HDFS-11095: -- +1 pending Jenkins. > BlockManagerSafeMode should respect extension

[jira] [Commented] (HDFS-11092) Stackoverflow if only root directory is used in Command Line

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630445#comment-15630445 ] Mingliang Liu commented on HDFS-11092: -- This is embarrassing. {{java.net.URI.create()}} factory

[jira] [Work started] (HDFS-11070) NPE in BlockSender due to race condition

2016-11-02 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-11070 started by Wei-Chiu Chuang. -- > NPE in BlockSender due to race condition >

[jira] [Assigned] (HDFS-11070) NPE in BlockSender due to race condition

2016-11-02 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HDFS-11070: -- Assignee: Wei-Chiu Chuang > NPE in BlockSender due to race condition >

[jira] [Commented] (HDFS-11070) NPE in BlockSender due to race condition

2016-11-02 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630441#comment-15630441 ] Wei-Chiu Chuang commented on HDFS-11070: The race condition is introduced by HDFS-7496. > NPE in

[jira] [Updated] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11095: - Attachment: HDFS-11095.001.patch The v1 patch update the config description. Note that the

[jira] [Commented] (HDFS-11090) Leave safemode immediately if all blocks have reported in

2016-11-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630365#comment-15630365 ] Andrew Wang commented on HDFS-11090: Thanks for the comments everyone! To provide a little more

[jira] [Commented] (HDFS-10909) De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads

2016-11-02 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630318#comment-15630318 ] Lei (Eddy) Xu commented on HDFS-10909: -- Hi, [~manojg] It looks good to me overall. Some nits:

[jira] [Updated] (HDFS-11079) TestQuotaByStorageType#setUp does not need refresh the cluster

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11079: - Target Version/s: 2.8.0, 3.0.0-alpha2 > TestQuotaByStorageType#setUp does not need refresh the

[jira] [Updated] (HDFS-11079) TestQuotaByStorageType#setUp does not need refresh the cluster

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11079: - Description: UPDATED: We don't need call {{refreshClusterState();}} after we build a brand new

[jira] [Updated] (HDFS-11094) TestLargeBlockReport fails intermittently

2016-11-02 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Badger updated HDFS-11094: --- Attachment: HDFS-11094.001.patch The problem is that the MiniDFSCluster doesn't wait for the DN(s) to

[jira] [Commented] (HDFS-11079) Accelerate TestQuotaByStorageType using shared MiniDFSCluster

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630303#comment-15630303 ] Mingliang Liu commented on HDFS-11079: -- You're correct [~arpitagarwal] I just had a look at the code

[jira] [Commented] (HDFS-11090) Leave safemode immediately if all blocks have reported in

2016-11-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630287#comment-15630287 ] Kihwal Lee commented on HDFS-11090: --- I agree with [~jingzhao]. If people don't want safe mode extension,

[jira] [Commented] (HDFS-11076) Add unit test for extended Acls

2016-11-02 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630276#comment-15630276 ] Arpit Agarwal commented on HDFS-11076: -- [~liuml07], please go ahead with the commit if it looks good

[jira] [Commented] (HDFS-11076) Add unit test for extended Acls

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630261#comment-15630261 ] Mingliang Liu commented on HDFS-11076: -- +1 [~arpitagarwal] do you need to have a look at this before

[jira] [Updated] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11095: - Status: Patch Available (was: Open) > BlockManagerSafeMode should respect extension period

[jira] [Updated] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11095: - Attachment: HDFS-11095.000.patch Ping [~jingzhao] for review. Thanks, > BlockManagerSafeMode

[jira] [Created] (HDFS-11095) BlockManagerSafeMode should respect extension period default config value (30s)

2016-11-02 Thread Mingliang Liu (JIRA)
Mingliang Liu created HDFS-11095: Summary: BlockManagerSafeMode should respect extension period default config value (30s) Key: HDFS-11095 URL: https://issues.apache.org/jira/browse/HDFS-11095

[jira] [Updated] (HDFS-9550) DiskBalancer: Add Run Command

2016-11-02 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-9550: Description: Run is kind of window dressing command that wraps plan and execute commands to make it

[jira] [Updated] (HDFS-11070) NPE in BlockSender due to race condition

2016-11-02 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11070: --- Attachment: HDFS-11070.001.patch Upload a trivial fix to remove the possible race condition.

[jira] [Commented] (HDFS-11056) Concurrent append and read operations lead to checksum error

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630215#comment-15630215 ] Hadoop QA commented on HDFS-11056: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11045) TestDirectoryScanner#testThrottling fails: Throttle is too permissive

2016-11-02 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HDFS-11045: Attachment: HDFS-11045.003.patch This patch fixes the one checkstyle issue and also

[jira] [Commented] (HDFS-11076) Add unit test for extended Acls

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630089#comment-15630089 ] Hadoop QA commented on HDFS-11076: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-6804) race condition between transferring block and appending block causes "Unexpected checksum mismatch exception"

2016-11-02 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15630020#comment-15630020 ] Wei-Chiu Chuang commented on HDFS-6804: --- I am pretty sure this is the same bug described in

[jira] [Commented] (HDFS-11079) Accelerate TestQuotaByStorageType using shared MiniDFSCluster

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629983#comment-15629983 ] Mingliang Liu commented on HDFS-11079: -- Is it possible to totally isolate each test in this case

[jira] [Commented] (HDFS-10566) Submit plan request should throw exception if Datanode is in non-REGULAR status.

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629986#comment-15629986 ] Hadoop QA commented on HDFS-10566: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11079) Accelerate TestQuotaByStorageType using shared MiniDFSCluster

2016-11-02 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1563#comment-1563 ] Arpit Agarwal commented on HDFS-11079: -- Hi [~liuml07], unfortunately that may still lead to

[jira] [Commented] (HDFS-11090) Leave safemode immediately if all blocks have reported in

2016-11-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629948#comment-15629948 ] Jing Zhao commented on HDFS-11090: -- If 100% block threshold has been met, this means all the blocks have

[jira] [Updated] (HDFS-11056) Concurrent append and read operations lead to checksum error

2016-11-02 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11056: --- Attachment: HDFS-11056.002.patch Upload patch v002. This version fixed the bug in

[jira] [Commented] (HDFS-11089) Accelerate TestDiskBalancerCommand using static shared MiniDFSCluster

2016-11-02 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629931#comment-15629931 ] Arpit Agarwal commented on HDFS-11089: -- I recommend leaving this as it is. [This

[jira] [Commented] (HDFS-11079) Accelerate TestQuotaByStorageType using shared MiniDFSCluster

2016-11-02 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629919#comment-15629919 ] Arpit Agarwal commented on HDFS-11079: -- Hi [~xiaobingo], I recommend leaving this as it is. Sharing

[jira] [Commented] (HDFS-11080) Update HttpFS to use ConfigRedactor

2016-11-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629921#comment-15629921 ] Hadoop QA commented on HDFS-11080: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Created] (HDFS-11094) TestLargeBlockReport fails intermittently

2016-11-02 Thread Eric Badger (JIRA)
Eric Badger created HDFS-11094: -- Summary: TestLargeBlockReport fails intermittently Key: HDFS-11094 URL: https://issues.apache.org/jira/browse/HDFS-11094 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-10566) Submit plan request should throw exception if Datanode is in non-REGULAR status.

2016-11-02 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629788#comment-15629788 ] Anu Engineer commented on HDFS-10566: - [~xiaobingo] Thank you for updating the patch. +1, pending

[jira] [Updated] (HDFS-11076) Add unit test for extended Acls

2016-11-02 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11076: -- Attachment: HDFS-11076.003.patch Thanks [~liuml07] for the review! Uploaded v003 patch for the check

[jira] [Commented] (HDFS-11090) Leave safemode immediately if all blocks have reported in

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629748#comment-15629748 ] Mingliang Liu commented on HDFS-11090: -- Ping [~jingzhao] for more input. > Leave safemode

[jira] [Commented] (HDFS-11090) Leave safemode immediately if all blocks have reported in

2016-11-02 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629742#comment-15629742 ] Mingliang Liu commented on HDFS-11090: -- {quote} Is this mean we should leave safemode when all the

[jira] [Updated] (HDFS-11080) Update HttpFS to use ConfigRedactor

2016-11-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11080: - Target Version/s: 3.0.0-alpha2 Status: Patch Available (was: Open) > Update HttpFS

[jira] [Updated] (HDFS-11080) Update HttpFS to use ConfigRedactor

2016-11-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11080: - Attachment: HDFS-11080.001.patch Attaching a patch. Some keys were already redacted, so added

[jira] [Commented] (HDFS-10756) Expose getTrashRoot to HTTPFS and WebHDFS

2016-11-02 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15629530#comment-15629530 ] Xiao Chen commented on HDFS-10756: -- Thanks Yuanbo, patch 7 LGTM. Won't fix the checkstyles to keep

  1   2   >