[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817542#comment-15817542 ] Hadoop QA commented on HDFS-11186: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-10 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817471#comment-15817471 ] Uma Maheswara Rao G commented on HDFS-11150: {quote} I was considering raising a JIRA to fix

[jira] [Commented] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2017-01-10 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817315#comment-15817315 ] Rakesh R commented on HDFS-11186: - Thank you [~zhouwei] for the updates. I'm almost OK with the changes.

[jira] [Commented] (HDFS-10733) NameNode terminated after full GC thinking QJM is unresponsive.

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817204#comment-15817204 ] Hadoop QA commented on HDFS-10733: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-10 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817184#comment-15817184 ] Manoj Govindassamy commented on HDFS-9391: -- * A clean build with the latest pull for both trunk

[jira] [Commented] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-10 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817174#comment-15817174 ] Manoj Govindassamy commented on HDFS-9391: -- Thanks [~mingma] for the review and commit. Much

[jira] [Updated] (HDFS-11186) [SPS]: Daemon thread of SPS should start only in Active NN

2017-01-10 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11186: Attachment: HDFS-11186-HDFS-10285.03.patch Thanks [~rakeshr] for reviewing the patch! {quote} Probably, we

[jira] [Commented] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-10 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817144#comment-15817144 ] Hudson commented on HDFS-9391: -- FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #11104 (See

[jira] [Commented] (HDFS-11121) Add assertions to BlockInfo#addStorage to protect from breaking reportedBlock-blockGroup mapping

2017-01-10 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817127#comment-15817127 ] Wei-Chiu Chuang commented on HDFS-11121: Sorry I missed your update. Will follow up soon. > Add

[jira] [Comment Edited] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-10 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817119#comment-15817119 ] Ming Ma edited comment on HDFS-9391 at 1/11/17 4:23 AM: Thanks [~manojg] for the

[jira] [Updated] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-10 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated HDFS-9391: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.9.0

[jira] [Commented] (HDFS-11296) Maintenance state expiry should be an epoch time and not jvm monotonic

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817108#comment-15817108 ] Hadoop QA commented on HDFS-11296: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11191) Datanode Capacity is misleading if the dfs.datanode.data.dir is configured with two directories from the same file system.

2017-01-10 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817066#comment-15817066 ] Weiwei Yang commented on HDFS-11191: Can someone help to review the v7 patch? The idea is: Added a

[jira] [Commented] (HDFS-10530) BlockManager reconstruction work scheduling should correctly adhere to EC block placement policy

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817065#comment-15817065 ] Hadoop QA commented on HDFS-10530: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9342) Erasure coding: client should update and commit block based on acknowledged size

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817063#comment-15817063 ] Hadoop QA commented on HDFS-9342: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11312) Discrepancy in nonDfsUsed index in protobuf

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817054#comment-15817054 ] Hadoop QA commented on HDFS-11312: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9822) Erasure Coding: Avoids scheduling multiple reconstruction tasks for a striped block at the same time

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817023#comment-15817023 ] Hadoop QA commented on HDFS-9822: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817020#comment-15817020 ] Hadoop QA commented on HDFS-9391: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9657) Schedule EC tasks at proper time to reduce the impact of recovery traffic

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817022#comment-15817022 ] Hadoop QA commented on HDFS-9657: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11306) Print remaining edit logs from buffer if edit log can't be rolled.

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816995#comment-15816995 ] Hadoop QA commented on HDFS-11306: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11312) Discrepancy in nonDfsUsed index in protobuf

2017-01-10 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-11312: Status: Patch Available (was: Open) [~mackrorysd] thanks for reporting this and

[jira] [Commented] (HDFS-11313) Segmented Block Reports

2017-01-10 Thread Vinitha Reddy Gankidi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816933#comment-15816933 ] Vinitha Reddy Gankidi commented on HDFS-11313: -- [~shv] This idea seems promising. I would

[jira] [Updated] (HDFS-11170) Add create API in filesystem public class to support assign parameter through builder

2017-01-10 Thread Wei Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Zhou updated HDFS-11170: Attachment: HDFS-11170-01.patch Hi [~andrew.wang], thanks for reviewing the patch and the great

[jira] [Resolved] (HDFS-8095) Allow to configure the system default EC schema

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-8095. --- Resolution: Not A Problem Resolving per above comments, since we think that this can be handled by a

[jira] [Commented] (HDFS-8095) Allow to configure the system default EC schema

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816879#comment-15816879 ] Andrew Wang commented on HDFS-8095: --- Cool, sounds good :) I filed HDFS-11314 to track this validation, I

[jira] [Created] (HDFS-11314) Validate client-provided EC schema on the NameNode

2017-01-10 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-11314: -- Summary: Validate client-provided EC schema on the NameNode Key: HDFS-11314 URL: https://issues.apache.org/jira/browse/HDFS-11314 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-8031) Follow-on work for erasure coding phase I (striping layout)

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816870#comment-15816870 ] Andrew Wang commented on HDFS-8031: --- Hi folks, I've gone through the 78 open subtasks and added the

[jira] [Updated] (HDFS-10530) BlockManager reconstruction work scheduling should correctly adhere to EC block placement policy

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10530: --- Labels: hdfs-ec-3.0-nice-to-have (was: ) > BlockManager reconstruction work scheduling should

[jira] [Updated] (HDFS-11124) Report blockIds of internal blocks for EC files in Fsck

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-11124: --- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Report blockIds of internal blocks for EC files in Fsck >

[jira] [Updated] (HDFS-11121) Add assertions to BlockInfo#addStorage to protect from breaking reportedBlock-blockGroup mapping

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-11121: --- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Add assertions to BlockInfo#addStorage to protect from

[jira] [Commented] (HDFS-11072) Add ability to unset and change directory EC policy

2017-01-10 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816856#comment-15816856 ] SammiChen commented on HDFS-11072: -- Thanks Andrew, for all the reviewing efforts and commit! > Add

[jira] [Updated] (HDFS-11066) Improve test coverage for Java coder/ISA-L native coder

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-11066: --- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Improve test coverage for Java coder/ISA-L native coder >

[jira] [Updated] (HDFS-10531) Add EC policy and storage policy related usage summarization function to dfs du command

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10531: --- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Add EC policy and storage policy related usage

[jira] [Updated] (HDFS-10983) OIV tool should make an EC file explicit

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10983: --- Labels: hdfs-ec-3.0-nice-to-have (was: ) > OIV tool should make an EC file explicit >

[jira] [Updated] (HDFS-11209) SNN can't checkpoint when rolling upgrade is not finalized

2017-01-10 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11209: -- Description: Similar problem has been fixed with HDFS-7185. Recent change in HDFS-8432 brings this

[jira] [Commented] (HDFS-9962) Erasure Coding: need a way to test multiple EC policies

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816851#comment-15816851 ] Andrew Wang commented on HDFS-9962: --- It looks like we still lack complete test coverage, I see that RS

[jira] [Commented] (HDFS-8095) Allow to configure the system default EC schema

2017-01-10 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816849#comment-15816849 ] Kai Zheng commented on HDFS-8095: - bq. When you say data corruption, do you mean data loss if the cluster

[jira] [Updated] (HDFS-9962) Erasure Coding: need a way to test multiple EC policies

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9962: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Erasure Coding: need a way to test multiple EC policies >

[jira] [Commented] (HDFS-9822) Erasure Coding: Avoids scheduling multiple reconstruction tasks for a striped block at the same time

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816843#comment-15816843 ] Andrew Wang commented on HDFS-9822: --- This one looks like a good bug, adding to "nice-to-have" and it

[jira] [Updated] (HDFS-9822) Erasure Coding: Avoids scheduling multiple reconstruction tasks for a striped block at the same time

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9822: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Erasure Coding: Avoids scheduling multiple reconstruction

[jira] [Commented] (HDFS-11313) Segmented Block Reports

2017-01-10 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816841#comment-15816841 ] Konstantin Shvachko commented on HDFS-11313: Full block report (FBR) processing on the

[jira] [Resolved] (HDFS-10258) Erasure Coding: support small cluster whose #DataNode < # (Blocks in a BlockGroup)

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-10258. Resolution: Later We also committed the XOR 2,1 policy, so I think the priority of this JIRA is

[jira] [Commented] (HDFS-9657) Schedule EC tasks at proper time to reduce the impact of recovery traffic

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816831#comment-15816831 ] Andrew Wang commented on HDFS-9657: --- Adding the "nice-to-have" label since this seems like a nice JIRA to

[jira] [Updated] (HDFS-9657) Schedule EC tasks at proper time to reduce the impact of recovery traffic

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9657: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Schedule EC tasks at proper time to reduce the impact of

[jira] [Updated] (HDFS-9826) Erasure Coding: Postpone the recovery work for a configurable time period

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9826: -- Resolution: Not A Problem Status: Resolved (was: Patch Available) Resolving per above, feel

[jira] [Commented] (HDFS-11209) SNN can't checkpoint when rolling upgrade is not finalized

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816828#comment-15816828 ] Hadoop QA commented on HDFS-11209: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9826) Erasure Coding: Postpone the recovery work for a configurable time period

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816825#comment-15816825 ] Andrew Wang commented on HDFS-9826: --- As Walter says above, we already prioritize recovery based on the

[jira] [Commented] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-10 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816821#comment-15816821 ] Yuanbo Liu commented on HDFS-11150: --- [~umamaheswararao] Thanks for your response. {code} When we finish

[jira] [Commented] (HDFS-9342) Erasure coding: client should update and commit block based on acknowledged size

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816813#comment-15816813 ] Andrew Wang commented on HDFS-9342: --- This one looks important for correctness. Could one of the watchers

[jira] [Created] (HDFS-11313) Segmented Block Reports

2017-01-10 Thread Konstantin Shvachko (JIRA)
Konstantin Shvachko created HDFS-11313: -- Summary: Segmented Block Reports Key: HDFS-11313 URL: https://issues.apache.org/jira/browse/HDFS-11313 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-9342) Erasure coding: client should update and commit block based on acknowledged size

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9342: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Erasure coding: client should update and commit block based

[jira] [Updated] (HDFS-10733) NameNode terminated after full GC thinking QJM is unresponsive.

2017-01-10 Thread Vinitha Reddy Gankidi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinitha Reddy Gankidi updated HDFS-10733: - Status: Patch Available (was: Open) > NameNode terminated after full GC thinking

[jira] [Commented] (HDFS-8095) Allow to configure the system default EC schema

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816802#comment-15816802 ] Andrew Wang commented on HDFS-8095: --- Got it, thanks for the context Kai. When you say data corruption, do

[jira] [Commented] (HDFS-10999) Use more generic "low redundancy" blocks instead of "under replicated" blocks

2017-01-10 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816787#comment-15816787 ] Yuanbo Liu commented on HDFS-10999: --- I think I've lost the context of this JIRA. Dismiss my ownership,

[jira] [Updated] (HDFS-10999) Use more generic "low redundancy" blocks instead of "under replicated" blocks

2017-01-10 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10999: -- Assignee: (was: Yuanbo Liu) > Use more generic "low redundancy" blocks instead of "under

[jira] [Commented] (HDFS-11268) Persist erasure coding policy ID in FSImage

2017-01-10 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816774#comment-15816774 ] SammiChen commented on HDFS-11268: -- Hi [~andrew.wang], I'am working on it. Will give an update soon. >

[jira] [Updated] (HDFS-11306) Print remaining edit logs from buffer if edit log can't be rolled.

2017-01-10 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11306: --- Attachment: HDFS-11306.002.patch Upload v002 patch to address [~yzhangal]'s comment. > Print

[jira] [Commented] (HDFS-8201) Refactor the end to end test for stripping file writing and reading

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816745#comment-15816745 ] Hadoop QA commented on HDFS-8201: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8095) Allow to configure the system default EC schema

2017-01-10 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816734#comment-15816734 ] Kai Zheng commented on HDFS-8095: - Thanks [~andrew.wang] for the thoughts. I'm not sure if we still need

[jira] [Resolved] (HDFS-8796) Erasure coding: merge HDFS-8499 to EC branch and refactor BlockInfoStriped

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-8796. --- Resolution: Invalid I think this JIRA is invalid now given that the EC branch has been merged to

[jira] [Commented] (HDFS-8387) Erasure Coding: Revisit the long and int datatypes usage in striping logic

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816731#comment-15816731 ] Andrew Wang commented on HDFS-8387: --- Hi [~rakeshr] do you think this JIRA is still important, and have

[jira] [Updated] (HDFS-8387) Erasure Coding: Revisit the long and int datatypes usage in striping logic

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8387: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Erasure Coding: Revisit the long and int datatypes usage in

[jira] [Updated] (HDFS-9345) Erasure Coding: create dummy coder and schema

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9345: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Erasure Coding: create dummy coder and schema >

[jira] [Updated] (HDFS-8201) Refactor the end to end test for stripping file writing and reading

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8201: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Refactor the end to end test for stripping file writing and

[jira] [Updated] (HDFS-8201) Refactor the end to end test for stripping file writing and reading

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8201: -- Target Version/s: 3.0.0-alpha2 (was: HDFS-7285) > Refactor the end to end test for stripping file

[jira] [Updated] (HDFS-8225) EC client code should not print info log message

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8225: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > EC client code should not print info log message >

[jira] [Commented] (HDFS-11302) Improve Logging for SSLHostnameVerifier

2017-01-10 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816714#comment-15816714 ] Chen Liang commented on HDFS-11302: --- The v001 patch is following the existing style of this class, which

[jira] [Updated] (HDFS-10733) NameNode terminated after full GC thinking QJM is unresponsive.

2017-01-10 Thread Vinitha Reddy Gankidi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinitha Reddy Gankidi updated HDFS-10733: - Attachment: HDFS-10733.002.patch > NameNode terminated after full GC thinking QJM

[jira] [Commented] (HDFS-10733) NameNode terminated after full GC thinking QJM is unresponsive.

2017-01-10 Thread Vinitha Reddy Gankidi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816705#comment-15816705 ] Vinitha Reddy Gankidi commented on HDFS-10733: -- [~shv] I agree. Attached a new patch with

[jira] [Updated] (HDFS-8295) Add MODIFY and REMOVE ECSchema editlog operations

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8295: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Add MODIFY and REMOVE ECSchema editlog operations >

[jira] [Commented] (HDFS-8095) Allow to configure the system default EC schema

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816693#comment-15816693 ] Andrew Wang commented on HDFS-8095: --- Adding the "nice-to-have" label, since I think it's worth having a

[jira] [Updated] (HDFS-8196) Erasure Coding related information on NameNode UI

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8196: -- Labels: NameNode WebUI hdfs-ec-3.0-nice-to-have (was: NameNode WebUI) > Erasure Coding related

[jira] [Updated] (HDFS-11296) Maintenance state expiry should be an epoch time and not jvm monotonic

2017-01-10 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-11296: -- Attachment: HDFS-11296-branch-2.01.patch > Maintenance state expiry should be an epoch

[jira] [Commented] (HDFS-8095) Allow to configure the system default EC schema

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816691#comment-15816691 ] Andrew Wang commented on HDFS-8095: --- Is this JIRA superceded by HDFS-7859? Also like [Vinay's

[jira] [Updated] (HDFS-8095) Allow to configure the system default EC schema

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8095: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Allow to configure the system default EC schema >

[jira] [Updated] (HDFS-8140) ECSchema supports for offline EditsVistor over an OEV XML file

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8140: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > ECSchema supports for offline EditsVistor over an OEV XML

[jira] [Updated] (HDFS-8140) ECSchema supports for offline EditsVisitor over an OEV XML file

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8140: -- Summary: ECSchema supports for offline EditsVisitor over an OEV XML file (was: ECSchema supports for

[jira] [Updated] (HDFS-8140) ECSchema supports for offline EditsVistor over an OEV XML file

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8140: -- Target Version/s: 3.0.0-alpha2 (was: HDFS-7285) > ECSchema supports for offline EditsVistor over an OEV

[jira] [Updated] (HDFS-7619) Erasure Coding: Handling truncated files in snapshots

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7619: -- Summary: Erasure Coding: Handling truncated files in snapshots (was: Erasure Coding: Handling files in

[jira] [Updated] (HDFS-7786) Handle slow writers for DFSStripedOutputStream

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7786: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Handle slow writers for DFSStripedOutputStream >

[jira] [Updated] (HDFS-9391) Update webUI/JMX to display maintenance state info

2017-01-10 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-9391: - Attachment: HDFS-9391-branch-2.02.patch

[jira] [Updated] (HDFS-7337) Configurable and pluggable Erasure Codec and schema

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7337: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Configurable and pluggable Erasure Codec and schema >

[jira] [Updated] (HDFS-7337) Configurable and pluggable Erasure Codec and schema

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7337: -- Target Version/s: 3.0.0-alpha2 (was: HDFS-7285) > Configurable and pluggable Erasure Codec and schema >

[jira] [Commented] (HDFS-11302) Improve Logging for SSLHostnameVerifier

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1581#comment-1581 ] Hadoop QA commented on HDFS-11302: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11028) libhdfs++: FileHandleImpl::CancelOperations needs to be able to cancel pending connections

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816662#comment-15816662 ] Hadoop QA commented on HDFS-11028: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-8672) Erasure Coding: Add EC-related Metrics to NN (seperate striped blocks count from UnderReplicatedBlocks count)

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8672: -- Labels: hdfs-ec-3.0-nice-to-have (was: ) > Erasure Coding: Add EC-related Metrics to NN (seperate

[jira] [Updated] (HDFS-10999) Use more generic "low redundancy" blocks instead of "under replicated" blocks

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10999: --- Labels: hdfs-ec-3.0-nice-to-have supportability (was: supportability) > Use more generic "low

[jira] [Resolved] (HDFS-7674) [umbrella] Adding metrics for Erasure Coding

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-7674. --- Resolution: Done I think we can close this since the subtasks are resolved. Thanks everyone for the

[jira] [Commented] (HDFS-11302) Improve Logging for SSLHostnameVerifier

2017-01-10 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816640#comment-15816640 ] Xiaoyu Yao commented on HDFS-11302: --- Thanks [~vagarychen] for working on this. Patch looks good to me.

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816623#comment-15816623 ] Andrew Wang commented on HDFS-11096: Thanks for checking all this Sean, a couple replies: * Nice find

[jira] [Updated] (HDFS-11209) SNN can't checkpoint when rolling upgrade is not finalized

2017-01-10 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11209: -- Attachment: HDFS-11209.03.patch Update patch for HDFS-11209.03.patch. Diff from prev: consolidate the

[jira] [Comment Edited] (HDFS-11209) SNN can't checkpoint when rolling upgrade is not finalized

2017-01-10 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816608#comment-15816608 ] Xiaoyu Yao edited comment on HDFS-11209 at 1/11/17 12:07 AM: - Update patch for

[jira] [Commented] (HDFS-11303) Hedged read might hang infinitely if read data from all DN failed

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816550#comment-15816550 ] Hadoop QA commented on HDFS-11303: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11312) Discrepancy in nonDfsUsed index in protobuf

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816546#comment-15816546 ] Andrew Wang commented on HDFS-11312: It looks like this discrepancy exists between 2.8 and 2.7, so

[jira] [Updated] (HDFS-11312) Discrepancy in nonDfsUsed index in protobuf

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-11312: --- Affects Version/s: 2.8.0 3.0.0-alpha1 Target Version/s: 2.8.0,

[jira] [Commented] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816514#comment-15816514 ] Hadoop QA commented on HDFS-10702: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-10 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816508#comment-15816508 ] Uma Maheswara Rao G commented on HDFS-11150: Latest patch almost looks good to me. A few minor

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-01-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816505#comment-15816505 ] Sean Mackrory commented on HDFS-11096: -- The getHdfsBlockLocations removal is documented in HDFS-8895

[jira] [Updated] (HDFS-11312) Discrepancy in nonDfsUsed index in protobuf

2017-01-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11312: - Attachment: HDFS-11312.001.patch > Discrepancy in nonDfsUsed index in protobuf >

[jira] [Created] (HDFS-11312) Discrepancy in nonDfsUsed index in protobuf

2017-01-10 Thread Sean Mackrory (JIRA)
Sean Mackrory created HDFS-11312: Summary: Discrepancy in nonDfsUsed index in protobuf Key: HDFS-11312 URL: https://issues.apache.org/jira/browse/HDFS-11312 Project: Hadoop HDFS Issue Type:

  1   2   >