[jira] [Commented] (HDFS-12731) Ozone: Fix inconsistent default value in ozone settings

2017-10-30 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224414#comment-16224414 ] Yiqun Lin commented on HDFS-12731: -- Thanks for uploading the patch, [~bharatviswa]. Looks like the failed

[jira] [Updated] (HDFS-12744) More logs when short-circuit read is failed and disabled

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12744: --- Attachment: HDFS-12744.001.patch Two line changes in this patch, 1) Added a warning message when SCR

[jira] [Updated] (HDFS-12744) More logs when short-circuit read is failed and disabled

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12744: --- Status: Patch Available (was: Open) > More logs when short-circuit read is failed and disabled >

[jira] [Updated] (HDFS-12732) Correct spellings of ramdomly to randomly in log.

2017-10-30 Thread hu xiaodong (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hu xiaodong updated HDFS-12732: --- Affects Version/s: 3.0.0-alpha3 > Correct spellings of ramdomly to randomly in log. >

[jira] [Commented] (HDFS-7060) Avoid taking locks when sending heartbeats from the DataNode

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224451#comment-16224451 ] Weiwei Yang commented on HDFS-7060: --- Hi [~yangjiandan] Your latest patch looks good to me. Thanks for

[jira] [Commented] (HDFS-12682) ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED

2017-10-30 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224526#comment-16224526 ] Kai Zheng commented on HDFS-12682: -- Thanks for the work and discussions here. I thought the smell of this

[jira] [Commented] (HDFS-12742) Add support for KSM --expunge command

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224626#comment-16224626 ] Weiwei Yang commented on HDFS-12742: Hi [~shashikant] I am confused with this patch, {{KSMExpunge}}

[jira] [Updated] (HDFS-12719) Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch

2017-10-30 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-12719: - Attachment: HDFS-12719-HDFS-7240.001.patch > Ozone: Fix checkstyle, javac, whitespace

[jira] [Commented] (HDFS-12742) Add support for KSM --expunge command

2017-10-30 Thread Shashikant Banerjee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224800#comment-16224800 ] Shashikant Banerjee commented on HDFS-12742: Hi [~cheersyang], The idea here is to have an

[jira] [Commented] (HDFS-12742) Add support for KSM --expunge command

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224811#comment-16224811 ] Weiwei Yang commented on HDFS-12742: Hi [~shashikant] Sounds nice to me, +[~anu], [~xyao] for their

[jira] [Commented] (HDFS-12742) Add support for KSM --expunge command

2017-10-30 Thread Shashikant Banerjee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224719#comment-16224719 ] Shashikant Banerjee commented on HDFS-12742: Hi [~cheersyang], Thanks for the review

[jira] [Commented] (HDFS-12717) 'hdfs namenode -help' command will not work when Namenode is Running

2017-10-30 Thread Harshakiran Reddy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224730#comment-16224730 ] Harshakiran Reddy commented on HDFS-12717: -- [~aw] any idea on this..? > 'hdfs namenode -help'

[jira] [Comment Edited] (HDFS-12742) Add support for KSM --expunge command

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224742#comment-16224742 ] Weiwei Yang edited comment on HDFS-12742 at 10/30/17 11:33 AM: --- Hi

[jira] [Commented] (HDFS-12474) Ozone: SCM: Handling container report with key count and container usage.

2017-10-30 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224827#comment-16224827 ] Yiqun Lin commented on HDFS-12474: -- Thanks for working on this, [~nandakumar131]. Some comments from me:

[jira] [Comment Edited] (HDFS-12474) Ozone: SCM: Handling container report with key count and container usage.

2017-10-30 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224827#comment-16224827 ] Yiqun Lin edited comment on HDFS-12474 at 10/30/17 12:32 PM: - Thanks for

[jira] [Commented] (HDFS-12742) Add support for KSM --expunge command

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224742#comment-16224742 ] Weiwei Yang commented on HDFS-12742: Hi [~shashikant] Sorry my earlier comment is misleading bq.

[jira] [Comment Edited] (HDFS-12443) Ozone: Improve SCM block deletion throttling algorithm

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224885#comment-16224885 ] Weiwei Yang edited comment on HDFS-12443 at 10/30/17 1:05 PM: -- Hi [~linyiqun]

[jira] [Updated] (HDFS-12443) Ozone: Improve SCM block deletion throttling algorithm

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12443: --- Attachment: HDFS-12443-SCM-blockdeletion-throttle.pdf > Ozone: Improve SCM block deletion throttling

[jira] [Commented] (HDFS-12443) Ozone: Improve SCM block deletion throttling algorithm

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224875#comment-16224875 ] Weiwei Yang commented on HDFS-12443: Hi [~linyiqun] Sorry for the late review, was pretty busy last

[jira] [Commented] (HDFS-12443) Ozone: Improve SCM block deletion throttling algorithm

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16224885#comment-16224885 ] Weiwei Yang commented on HDFS-12443: Hi [~linyiqun] Please take a look at the pdf I just uploaded

[jira] [Created] (HDFS-12745) Ozone: XceiverClientManager should cache objects based on pipeline name

2017-10-30 Thread Mukul Kumar Singh (JIRA)
Mukul Kumar Singh created HDFS-12745: Summary: Ozone: XceiverClientManager should cache objects based on pipeline name Key: HDFS-12745 URL: https://issues.apache.org/jira/browse/HDFS-12745

[jira] [Updated] (HDFS-12745) Ozone: XceiverClientManager should cache objects based on pipeline name

2017-10-30 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-12745: - Description: With just the standalone pipeline, a new pipeline was created for each and

[jira] [Commented] (HDFS-12116) BlockReportTestBase#blockReport_08 and #blockReport_08 intermittently fail

2017-10-30 Thread Gabor Bota (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225020#comment-16225020 ] Gabor Bota commented on HDFS-12116: --- Hi Xiao, What's the current status on this issue? I'm currently

[jira] [Updated] (HDFS-7240) Object store in HDFS

2017-10-30 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-7240: Attachment: HDFS-7240.004.patch > Object store in HDFS > > >

[jira] [Updated] (HDFS-7240) Object store in HDFS

2017-10-30 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-7240: Attachment: (was: HDFS-7240.004.patch) > Object store in HDFS > > >

[jira] [Updated] (HDFS-7240) Object store in HDFS

2017-10-30 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-7240: Attachment: (was: HDFS-7240.004.patch) > Object store in HDFS > > >

[jira] [Commented] (HDFS-12720) Ozone: Ratis options are not passed from KSM Client protobuf helper correctly.

2017-10-30 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225022#comment-16225022 ] Mukul Kumar Singh commented on HDFS-12720: -- Thanks for the review [~nandakumar131]. HDFS-12745

[jira] [Updated] (HDFS-12720) Ozone: Ratis options are not passed from KSM Client protobuf helper correctly.

2017-10-30 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-12720: - Attachment: HDFS-12720-HDFS-7240.003.patch > Ozone: Ratis options are not passed from KSM

[jira] [Commented] (HDFS-12667) KMSClientProvider#ValueQueue does synchronous fetch of edeks in background async thread.

2017-10-30 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225083#comment-16225083 ] Rushabh S Shah commented on HDFS-12667: --- bq. Some preliminary comments: [~xiaochen]: do you have any

[jira] [Commented] (HDFS-12724) Add retry logic when data node block moves while reading

2017-10-30 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225092#comment-16225092 ] James Clampffer commented on HDFS-12724: Nice [~mdeepak]! Overall the code change looks good.

[jira] [Comment Edited] (HDFS-206) Support for head in FSShell

2017-10-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225096#comment-16225096 ] Sean Mackrory edited comment on HDFS-206 at 10/30/17 3:11 PM: -- This looks

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-10-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225110#comment-16225110 ] Sean Mackrory commented on HDFS-11096: -- Re: test4tests, the whole patch is tests. Should figure out

[jira] [Updated] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-30 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12665: -- Attachment: HDFS-12665-HDFS-9806.003.patch Attaching patch that rebased on the latest HDFS-11902 patch.

[jira] [Updated] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-30 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12665: -- Status: Patch Available (was: Open) > [AliasMap] Create a version of the AliasMap that runs in memory

[jira] [Updated] (HDFS-12665) [AliasMap] Create a version of the AliasMap that runs in memory in the Namenode (leveldb)

2017-10-30 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12665: -- Status: Open (was: Patch Available) > [AliasMap] Create a version of the AliasMap that runs in memory

[jira] [Updated] (HDFS-12698) Ozone: Use time units in the Ozone configuration values

2017-10-30 Thread Elek, Marton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDFS-12698: Status: Open (was: Patch Available) Moving back to inprogress, to check the failing tests. >

[jira] [Updated] (HDFS-206) Support for head in FSShell

2017-10-30 Thread Gabor Bota (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HDFS-206: Attachment: HDFS-206.003.patch > Support for head in FSShell > --- > >

[jira] [Updated] (HDFS-206) Support for head in FSShell

2017-10-30 Thread Gabor Bota (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HDFS-206: Status: Open (was: Patch Available) > Support for head in FSShell > --- > >

[jira] [Updated] (HDFS-206) Support for head in FSShell

2017-10-30 Thread Gabor Bota (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HDFS-206: Status: Patch Available (was: Open) Thanks for the review Sean! I've removed the -f and added the finally

[jira] [Commented] (HDFS-12718) Ozone: fix thread number calculation in CBlockManager

2017-10-30 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225224#comment-16225224 ] Mukul Kumar Singh commented on HDFS-12718: -- Thanks for working on this issue [~elek]. The patch

[jira] [Updated] (HDFS-11294) libhdfs++: Segfault in HA failover if DNS lookup for both Namenodes fails

2017-10-30 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Clampffer updated HDFS-11294: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed this to

[jira] [Commented] (HDFS-206) Support for head in FSShell

2017-10-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225096#comment-16225096 ] Sean Mackrory commented on HDFS-206: This looks pretty good to me. A couple of nitpicks: * You're still

[jira] [Commented] (HDFS-12667) KMSClientProvider#ValueQueue does synchronous fetch of edeks in background async thread.

2017-10-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225234#comment-16225234 ] Hadoop QA commented on HDFS-12667: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12738) Namenode logs many "SSL renegotiate denied" warnings after enable HTTPS for HDFS

2017-10-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225248#comment-16225248 ] Kihwal Lee commented on HDFS-12738: --- How frequent is it relative to the request rate? > Namenode logs

[jira] [Commented] (HDFS-2139) Fast copy for HDFS.

2017-10-30 Thread feiwei (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225268#comment-16225268 ] feiwei commented on HDFS-2139: -- In FsDatasetImpl.java , You should modify to ensure synchronization public

[jira] [Commented] (HDFS-206) Support for head in FSShell

2017-10-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225278#comment-16225278 ] Sean Mackrory commented on HDFS-206: Ah - you know what? I didn't see that the last argument passed to

[jira] [Commented] (HDFS-12718) Ozone: fix thread number calculation in CBlockManager

2017-10-30 Thread Elek, Marton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225284#comment-16225284 ] Elek, Marton commented on HDFS-12718: - Yes, I also considered this option as well, and couldn't decide

[jira] [Issue Comment Deleted] (HDFS-2139) Fast copy for HDFS.

2017-10-30 Thread feiwei (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwei updated HDFS-2139: - Comment: was deleted (was: In FsDatasetImpl.java , You should modify to ensure synchronization public void

[jira] [Updated] (HDFS-12718) Ozone: fix thread number calculation in CBlockManager

2017-10-30 Thread Elek, Marton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDFS-12718: Attachment: HDFS-12718-HDFS-7240.002.patch Patch is improved according to the suggestion. I also

[jira] [Commented] (HDFS-206) Support for head in FSShell

2017-10-30 Thread Gabor Bota (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225717#comment-16225717 ] Gabor Bota commented on HDFS-206: - Ok, thanks! I also agree that closing the resource like this would give

[jira] [Comment Edited] (HDFS-12725) BlockPlacementPolicyRackFaultTolerant still fails with racks with very few nodes

2017-10-30 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225679#comment-16225679 ] Wei-Chiu Chuang edited comment on HDFS-12725 at 10/30/17 9:23 PM: -- Hi

[jira] [Commented] (HDFS-12746) DataNode Audit Logger

2017-10-30 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225651#comment-16225651 ] Anu Engineer commented on HDFS-12746: - [~xkrogen] It is an interesting idea. Recently I had an

[jira] [Commented] (HDFS-12743) Fix Ozone's env vars (again)

2017-10-30 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225676#comment-16225676 ] Anu Engineer commented on HDFS-12743: - +1, Thank you for catching this and fixing this. I will commit

[jira] [Updated] (HDFS-12743) Ozone: Fix Ozone's env vars (again)

2017-10-30 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-12743: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Target

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2017-10-30 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225724#comment-16225724 ] Steve Loughran commented on HDFS-7878: -- Bits I Understand are +1 from me > API - expose an unique

[jira] [Commented] (HDFS-12396) Webhdfs file system should get delegation token from kms provider.

2017-10-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225807#comment-16225807 ] Hadoop QA commented on HDFS-12396: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (HDFS-12474) Ozone: SCM: Handling container report with key count and container usage.

2017-10-30 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225705#comment-16225705 ] Xiaoyu Yao edited comment on HDFS-12474 at 10/30/17 9:54 PM: - Thanks

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2017-10-30 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225867#comment-16225867 ] Chris Douglas commented on HDFS-7878: - Thanks [~elgoiri] and [~ste...@apache.org]. If there's no other

[jira] [Commented] (HDFS-9914) Fix configurable WebhDFS connect/read timeout

2017-10-30 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225674#comment-16225674 ] Chris Douglas commented on HDFS-9914: - Odd, {{TestWebHdfsTimeouts}} always fails for me. Though when I

[jira] [Commented] (HDFS-12667) KMSClientProvider#ValueQueue does synchronous fetch of edeks in background async thread.

2017-10-30 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225821#comment-16225821 ] Xiao Chen commented on HDFS-12667: -- I haven't found time to look at what this means to hdfs startFiles

[jira] [Commented] (HDFS-12718) Ozone: fix thread number calculation in CBlockManager

2017-10-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225728#comment-16225728 ] Hadoop QA commented on HDFS-12718: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12746) DataNode Audit Logger

2017-10-30 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225750#comment-16225750 ] Anu Engineer commented on HDFS-12746: - bq. I'm not sure about machine readable. Did you have something

[jira] [Updated] (HDFS-12749) After SNN restart and DN re-register to SNN,some DN may not send block report to SNN if DN num is too large

2017-10-30 Thread TanYuxin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TanYuxin updated HDFS-12749: Affects Version/s: (was: 2.7.4) > After SNN restart and DN re-register to SNN,some DN may not send

[jira] [Updated] (HDFS-12749) After SNN restart and DN re-register to SNN,some DN may not send block report to SNN if DN num is too large

2017-10-30 Thread TanYuxin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TanYuxin updated HDFS-12749: Target Version/s: (was: 2.7.4) > After SNN restart and DN re-register to SNN,some DN may not send block

[jira] [Created] (HDFS-12749) After SNN restart and DN re-register to SNN,some DN may not send block report to SNN if DN num is too large

2017-10-30 Thread TanYuxin (JIRA)
TanYuxin created HDFS-12749: --- Summary: After SNN restart and DN re-register to SNN,some DN may not send block report to SNN if DN num is too large Key: HDFS-12749 URL: https://issues.apache.org/jira/browse/HDFS-12749

[jira] [Updated] (HDFS-12682) ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED

2017-10-30 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-12682: - Attachment: HDFS-12682.05.patch Thanks for the review Eddy! Patch 5 addressed all comments. Also fixed

[jira] [Commented] (HDFS-12744) More logs when short-circuit read is failed and disabled

2017-10-30 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226256#comment-16226256 ] John Zhuge commented on HDFS-12744: --- +1 LGTM > More logs when short-circuit read is failed and disabled

[jira] [Updated] (HDFS-12744) More logs when short-circuit read is failed and disabled

2017-10-30 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-12744: -- Labels: supportability (was: ) > More logs when short-circuit read is failed and disabled >

[jira] [Commented] (HDFS-12744) More logs when short-circuit read is failed and disabled

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226258#comment-16226258 ] Weiwei Yang commented on HDFS-12744: Thanks [~jzhuge] for the quick review, looks like the pre-commit

[jira] [Updated] (HDFS-12744) More logs when short-circuit read is failed and disabled

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12744: --- Attachment: HDFS-12744.002.patch > More logs when short-circuit read is failed and disabled >

[jira] [Commented] (HDFS-12711) deadly hdfs test

2017-10-30 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226299#comment-16226299 ] Allen Wittenauer commented on HDFS-12711: - For those playing at home: YETUS-570 (in development)

[jira] [Commented] (HDFS-12711) deadly hdfs test

2017-10-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226133#comment-16226133 ] Hadoop QA commented on HDFS-12711: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-12748) Standby NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY

2017-10-30 Thread Jiandan Yang (JIRA)
Jiandan Yang created HDFS-12748: Summary: Standby NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY Key: HDFS-12748 URL: https://issues.apache.org/jira/browse/HDFS-12748 Project: Hadoop

[jira] [Commented] (HDFS-12459) Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API

2017-10-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226242#comment-16226242 ] Hadoop QA commented on HDFS-12459: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226268#comment-16226268 ] Hadoop QA commented on HDFS-11902: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12725) BlockPlacementPolicyRackFaultTolerant still fails with racks with very few nodes

2017-10-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226227#comment-16226227 ] Hadoop QA commented on HDFS-12725: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12459) Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226260#comment-16226260 ] Weiwei Yang commented on HDFS-12459: Hi [~shahrs87], any updates on this? > Fix revert: Add new op

[jira] [Updated] (HDFS-12749) After SNN restart and DN re-register to SNN,some DN may not send block report to SNN if DN num is too large

2017-10-30 Thread TanYuxin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TanYuxin updated HDFS-12749: Description: After SNN restart,DN will call BPServiceActor#reRegister method to register. But when DN

[jira] [Updated] (HDFS-12749) DN may not send block report to SNN after SNN restart and DN re-register to SNN,

2017-10-30 Thread TanYuxin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TanYuxin updated HDFS-12749: Summary: DN may not send block report to SNN after SNN restart and DN re-register to SNN, (was: After SNN

[jira] [Updated] (HDFS-12749) DN may not send block report to NN after NN restart

2017-10-30 Thread TanYuxin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TanYuxin updated HDFS-12749: Summary: DN may not send block report to NN after NN restart (was: DN may not send block report to SNN

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2017-10-30 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225895#comment-16225895 ] Anu Engineer commented on HDFS-7878: Thanks for the heads up. Please go ahead. > API - expose an

[jira] [Updated] (HDFS-12705) WebHdfsFileSystem exceptions should retain the caused by exception

2017-10-30 Thread Hanisha Koneru (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanisha Koneru updated HDFS-12705: -- Attachment: HDFS-12705.002.patch Thanks [~nandakumar131] and [~daryn] for reviewing the patch.

[jira] [Updated] (HDFS-12499) dfs.namenode.shared.edits.dir property is currently namenode specific key

2017-10-30 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-12499: - Labels: (was: Incompatible) > dfs.namenode.shared.edits.dir property is currently namenode

[jira] [Updated] (HDFS-12499) dfs.namenode.shared.edits.dir property is currently namenode specific key

2017-10-30 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-12499: - Component/s: qjm > dfs.namenode.shared.edits.dir property is currently namenode specific key >

[jira] [Updated] (HDFS-12499) dfs.namenode.shared.edits.dir property is currently namenode specific key

2017-10-30 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-12499: - Hadoop Flags: Incompatible change > dfs.namenode.shared.edits.dir property is currently namenode

[jira] [Updated] (HDFS-12499) dfs.namenode.shared.edits.dir property is currently namenode specific key

2017-10-30 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-12499: - Target Version/s: 3.0.0 > dfs.namenode.shared.edits.dir property is currently namenode specific

[jira] [Comment Edited] (HDFS-12499) dfs.namenode.shared.edits.dir property is currently namenode specific key

2017-10-30 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226009#comment-16226009 ] Arpit Agarwal edited comment on HDFS-12499 at 10/30/17 11:59 PM: -

[jira] [Commented] (HDFS-12697) Ozone services must stay disabled in secure setup for alpha

2017-10-30 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226047#comment-16226047 ] Xiaoyu Yao commented on HDFS-12697: --- [~bharatviswa], can you fix the checkstyle and shell check

[jira] [Commented] (HDFS-12732) Correct spellings of ramdomly to randomly in log.

2017-10-30 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226060#comment-16226060 ] Akira Ajisaka commented on HDFS-12732: -- +1 pending Jenkins. > Correct spellings of ramdomly to

[jira] [Commented] (HDFS-12681) Fold HdfsLocatedFileStatus into HdfsFileStatus

2017-10-30 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226084#comment-16226084 ] Hadoop QA commented on HDFS-12681: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12638) NameNode exits due to ReplicationMonitor thread received Runtime exception in ReplicationWork#chooseTargets

2017-10-30 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226111#comment-16226111 ] Weiwei Yang commented on HDFS-12638: Hi [~shv] Thanks for the new patch. One question though, in

[jira] [Commented] (HDFS-206) Support for head in FSShell

2017-10-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225886#comment-16225886 ] Sean Mackrory commented on HDFS-206: Yetus never reviewed your 3rd patch, and Jenkins now appears to be

[jira] [Commented] (HDFS-12736) Enable SSL Documentation

2017-10-30 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225901#comment-16225901 ] Xiaoyu Yao commented on HDFS-12736: --- [~tasanuma0829], thanks for the comments. If I understand

[jira] [Commented] (HDFS-12730) Verify open files captured in the snapshots across config disable and enable

2017-10-30 Thread Hanisha Koneru (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225996#comment-16225996 ] Hanisha Koneru commented on HDFS-12730: --- Thanks for the patch, [~manojg]. This is what I understood

[jira] [Updated] (HDFS-12638) NameNode exits due to ReplicationMonitor thread received Runtime exception in ReplicationWork#chooseTargets

2017-10-30 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-12638: --- Attachment: HDFS-12638.002.patch You are right we are not collecting the

[jira] [Commented] (HDFS-12682) ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED

2017-10-30 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226023#comment-16226023 ] Lei (Eddy) Xu commented on HDFS-12682: -- Hi, [~xiaochen] Thanks for the updates. Overall LGTM. Minor

[jira] [Commented] (HDFS-12714) Hadoop 3 missing fix for HDFS-5169

2017-10-30 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226037#comment-16226037 ] John Zhuge commented on HDFS-12714: --- [~joemcdonnell] Thanks for the contribution! How did u test the

[jira] [Comment Edited] (HDFS-12714) Hadoop 3 missing fix for HDFS-5169

2017-10-30 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226037#comment-16226037 ] John Zhuge edited comment on HDFS-12714 at 10/31/17 1:00 AM: - [~joemcdonnell]

[jira] [Updated] (HDFS-12725) BlockPlacementPolicyRackFaultTolerant still fails with racks with very few nodes

2017-10-30 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-12725: - Attachment: HDFS-12725.03.patch Thanks for the review [~jojochuang]. Patch 3 to address the comments.

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2017-10-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225910#comment-16225910 ] Sean Mackrory commented on HDFS-7878: - No other feedback from me - +1 on the bits I understand, like

  1   2   >