[jira] [Commented] (HDFS-12521) Ozone: SCM should read all Container info into memory when booting up

2017-10-18 Thread Lokesh Jain (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209151#comment-16209151 ] Lokesh Jain commented on HDFS-12521: Thanks for the review [~linyiqun]. The v3 patch addresses the

[jira] [Updated] (HDFS-12673) Ozone: Log elapsed time for block deletion tasks

2017-10-18 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12673: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Status:

[jira] [Commented] (HDFS-12673) Ozone: Log elapsed time for block deletion tasks

2017-10-18 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208901#comment-16208901 ] Weiwei Yang commented on HDFS-12673: I have committed this to the feature branch, thanks [~linyiqun]

[jira] [Commented] (HDFS-12676) when blocks has corrupted replicas,throws Exception

2017-10-18 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209146#comment-16209146 ] Wei-Chiu Chuang commented on HDFS-12676: Correct. Please check out the attached source code patch

[jira] [Updated] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-12675: - Attachment: HDFS-12675-HDFS-7240.001.patch > Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases

[jira] [Updated] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-12675: - Status: Patch Available (was: Open) > Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

[jira] [Comment Edited] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208869#comment-16208869 ] Yiqun Lin edited comment on HDFS-12675 at 10/18/17 10:14 AM: - I just debugged

[jira] [Commented] (HDFS-12521) Ozone: SCM should read all Container info into memory when booting up

2017-10-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209135#comment-16209135 ] Hadoop QA commented on HDFS-12521: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-12420) Add an option to disallow 'namenode format -force'

2017-10-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12420: - Fix Version/s: 3.0.0 2.7.5 2.8.2 2.9.0 Thanks

[jira] [Updated] (HDFS-12677) Extend TestReconstructStripedFile with a random EC policy

2017-10-18 Thread Takanobu Asanuma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takanobu Asanuma updated HDFS-12677: Issue Type: Sub-task (was: Bug) Parent: HDFS-9962 > Extend

[jira] [Commented] (HDFS-12676) when blocks has corrupted replicas,throws Exception

2017-10-18 Thread lynn (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209044#comment-16209044 ] lynn commented on HDFS-12676: - hadoop-2.7.3,but i can't find the difference between the version 2.7.3 and the

[jira] [Updated] (HDFS-12521) Ozone: SCM should read all Container info into memory when booting up

2017-10-18 Thread Lokesh Jain (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDFS-12521: --- Attachment: HDFS-12521-HDFS-7240.003.patch > Ozone: SCM should read all Container info into memory

[jira] [Created] (HDFS-12676) when blocks has corrupted replicas,throws Exception

2017-10-18 Thread lynn (JIRA)
lynn created HDFS-12676: --- Summary: when blocks has corrupted replicas,throws Exception Key: HDFS-12676 URL: https://issues.apache.org/jira/browse/HDFS-12676 Project: Hadoop HDFS Issue Type: Bug

[jira] [Comment Edited] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208869#comment-16208869 ] Yiqun Lin edited comment on HDFS-12675 at 10/18/17 10:17 AM: - I just debugged

[jira] [Commented] (HDFS-12676) when blocks has corrupted replicas,throws Exception

2017-10-18 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209027#comment-16209027 ] Wei-Chiu Chuang commented on HDFS-12676: Thanks for reporting the issue. I am pretty sure this is

[jira] [Created] (HDFS-12677) Extend TestReconstructStripedFile with a random EC policy

2017-10-18 Thread Takanobu Asanuma (JIRA)
Takanobu Asanuma created HDFS-12677: --- Summary: Extend TestReconstructStripedFile with a random EC policy Key: HDFS-12677 URL: https://issues.apache.org/jira/browse/HDFS-12677 Project: Hadoop HDFS

[jira] [Commented] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210362#comment-16210362 ] Hadoop QA commented on HDFS-11902: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12682) ECAdmin -listPolicies will always show policy state as DISABLED

2017-10-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210317#comment-16210317 ] Xiao Chen commented on HDFS-12682: -- Had a quick discussion with Andrew, here's the outcome:

[jira] [Updated] (HDFS-12420) Add an option to disallow 'namenode format -force'

2017-10-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-12420: -- Fix Version/s: (was: 2.8.2) 2.8.3 > Add an option to disallow 'namenode format

[jira] [Commented] (HDFS-12420) Add an option to disallow 'namenode format -force'

2017-10-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210386#comment-16210386 ] Junping Du commented on HDFS-12420: --- Drop 2.8.2 from fix version as the patch only commit to branch-2.8

[jira] [Commented] (HDFS-12663) Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket

2017-10-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210511#comment-16210511 ] Anu Engineer commented on HDFS-12663: - bq. If we use it as it's now, should we explain what

[jira] [Updated] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-18 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12620: --- Attachment: HDFS-12620-branch-2.010.patch > Backporting HDFS-10467 to branch-2 >

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2017-10-18 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210392#comment-16210392 ] Chris Douglas commented on HDFS-7878: - Failure is due to HADOOP-14961 > API - expose an unique file

[jira] [Commented] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210503#comment-16210503 ] Anu Engineer commented on HDFS-12680: - [~nandakumar131] Thanks for getting this done. Probably one of

[jira] [Commented] (HDFS-12521) Ozone: SCM should read all Container info into memory when booting up

2017-10-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210591#comment-16210591 ] Anu Engineer commented on HDFS-12521: - [~ljain], Thanks for taking care of this. Some very minor

[jira] [Commented] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210346#comment-16210346 ] Hadoop QA commented on HDFS-12620: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-18 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210343#comment-16210343 ] Konstantin Shvachko commented on HDFS-12502: +1 for the latest patch. > nntop should support

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11902: -- Status: Patch Available (was: Open) > [READ] Merge BlockFormatProvider and

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11902: -- Attachment: (was: HDFS-11902-HDFS-9806.009.patch) > [READ] Merge

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11902: -- Attachment: HDFS-11902-HDFS-9806.009.patch > [READ] Merge BlockFormatProvider and

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11902: -- Status: Open (was: Patch Available) > [READ] Merge BlockFormatProvider and

[jira] [Updated] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-12680: Labels: ozoneMerge (was: ) > Ozone: SCM: Lease support for container creation >

[jira] [Commented] (HDFS-12638) NameNode exits due to ReplicationMonitor thread received Runtime exception in ReplicationWork#chooseTargets

2017-10-18 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210498#comment-16210498 ] Weiwei Yang commented on HDFS-12638: Hi [~daryn] bq. it sounds like you want to avoid the symptom

[jira] [Comment Edited] (HDFS-8921) Add an option to Balancer for specifying the k-most over-utilized DNs or all over-utilized DNs as sources.

2017-10-18 Thread Plamen Jeliazkov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210274#comment-16210274 ] Plamen Jeliazkov edited comment on HDFS-8921 at 10/18/17 11:26 PM: --- Yes,

[jira] [Updated] (HDFS-12495) TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently

2017-10-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-12495: -- Fix Version/s: (was: 2.8.2) > TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails

[jira] [Commented] (HDFS-12495) TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently

2017-10-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210288#comment-16210288 ] Junping Du commented on HDFS-12495: --- I don't find the patch landed in branch-2.8.2. Drop 2.8.2 in fix

[jira] [Commented] (HDFS-7337) Configurable and pluggable erasure codec and policy

2017-10-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210622#comment-16210622 ] Xiao Chen commented on HDFS-7337: - Hi [~drankye] and [~Sammi], Thanks a lot for working on this cool

[jira] [Commented] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209249#comment-16209249 ] Hadoop QA commented on HDFS-12675: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Assigned] (HDFS-12678) Ozone: Corona: Add statistical information to json output

2017-10-18 Thread Lokesh Jain (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain reassigned HDFS-12678: -- Assignee: Lokesh Jain > Ozone: Corona: Add statistical information to json output >

[jira] [Commented] (HDFS-11468) Ozone: SCM: Add Node Metrics for SCM

2017-10-18 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209291#comment-16209291 ] Weiwei Yang commented on HDFS-11468: Hi [~linyiqun] bq. Since metric is the data of aggregated

[jira] [Comment Edited] (HDFS-11468) Ozone: SCM: Add Node Metrics for SCM

2017-10-18 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209291#comment-16209291 ] Weiwei Yang edited comment on HDFS-11468 at 10/18/17 1:06 PM: -- Hi [~linyiqun]

[jira] [Commented] (HDFS-12674) hadoop fs -get does not throw permission denied in some cases

2017-10-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209450#comment-16209450 ] Daryn Sharp commented on HDFS-12674: We can't yet discount the jvm. The problem we debugged is when

[jira] [Updated] (HDFS-12678) Ozone: Corona: Add statistical information to json output

2017-10-18 Thread Lokesh Jain (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDFS-12678: --- Description: The Corona json output should contain statistical information like mean, quantiles and

[jira] [Updated] (HDFS-12678) Ozone: Corona: Add statistical information to json output

2017-10-18 Thread Lokesh Jain (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDFS-12678: --- Component/s: ozone > Ozone: Corona: Add statistical information to json output >

[jira] [Commented] (HDFS-12619) Do not catch and throw unchecked exceptions if IBRs fail to process

2017-10-18 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209165#comment-16209165 ] Wei-Chiu Chuang commented on HDFS-12619: Committing the patch based on [~xiaochen] and

[jira] [Created] (HDFS-12679) Namenode stale memory Datanode gets Wipes out

2017-10-18 Thread Vivek Ghatala (JIRA)
Vivek Ghatala created HDFS-12679: Summary: Namenode stale memory Datanode gets Wipes out Key: HDFS-12679 URL: https://issues.apache.org/jira/browse/HDFS-12679 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-12676) when blocks has corrupted replicas,throws Exception

2017-10-18 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-12676. Resolution: Duplicate Resolving it as a dup. Thanks @lynn for reporting it. > when blocks

[jira] [Created] (HDFS-12678) Ozone: Corona: Add statistical information to json output

2017-10-18 Thread Lokesh Jain (JIRA)
Lokesh Jain created HDFS-12678: -- Summary: Ozone: Corona: Add statistical information to json output Key: HDFS-12678 URL: https://issues.apache.org/jira/browse/HDFS-12678 Project: Hadoop HDFS

[jira] [Commented] (HDFS-12663) Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket

2017-10-18 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209657#comment-16209657 ] Nanda kumar commented on HDFS-12663: Thanks for the review [~anu] bq. I am wondering why? Does the

[jira] [Commented] (HDFS-11807) libhdfs++: Get minidfscluster tests running under valgrind

2017-10-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209678#comment-16209678 ] Hadoop QA commented on HDFS-11807: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-18 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12620: --- Attachment: HDFS-12620-branch-2.007.patch > Backporting HDFS-10467 to branch-2 >

[jira] [Commented] (HDFS-12638) NameNode exits due to ReplicationMonitor thread received Runtime exception in ReplicationWork#chooseTargets

2017-10-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209604#comment-16209604 ] Daryn Sharp commented on HDFS-12638: [~cheersyang], maybe I'm misinterpreting the proposal but it

[jira] [Commented] (HDFS-12624) Ozone: number of keys/values/buckets to KSMMetrics

2017-10-18 Thread Elek, Marton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209529#comment-16209529 ] Elek, Marton commented on HDFS-12624: - I just did a quick test. I have about 5 000 000 entries in the

[jira] [Updated] (HDFS-11807) libhdfs++: Get minidfscluster tests running under valgrind

2017-10-18 Thread Anatoli Shein (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anatoli Shein updated HDFS-11807: - Status: Patch Available (was: In Progress) libhdfs_mini_stress now runs under valgrind >

[jira] [Commented] (HDFS-12396) Webhdfs file system should get delegation token from kms provider.

2017-10-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209463#comment-16209463 ] Daryn Sharp commented on HDFS-12396: In {{KMSUtil}}, I'm not fond of returning null when passed null.

[jira] [Commented] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2017-10-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209570#comment-16209570 ] Daryn Sharp commented on HDFS-12574: This is bad: {{CryptoProtocolVersion.values()}}. The values

[jira] [Updated] (HDFS-11807) libhdfs++: Get minidfscluster tests running under valgrind

2017-10-18 Thread Anatoli Shein (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anatoli Shein updated HDFS-11807: - Attachment: HDFS-11807.HDFS-8707.000.patch First stab at running the mini dfs stress test under

[jira] [Assigned] (HDFS-11807) libhdfs++: Get minidfscluster tests running under valgrind

2017-10-18 Thread Anatoli Shein (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anatoli Shein reassigned HDFS-11807: Assignee: Anatoli Shein (was: James Clampffer) > libhdfs++: Get minidfscluster tests

[jira] [Work started] (HDFS-11807) libhdfs++: Get minidfscluster tests running under valgrind

2017-10-18 Thread Anatoli Shein (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-11807 started by Anatoli Shein. > libhdfs++: Get minidfscluster tests running under valgrind >

[jira] [Comment Edited] (HDFS-8921) Add an option to Balancer for specifying the k-most over-utilized DNs or all over-utilized DNs as sources.

2017-10-18 Thread Plamen Jeliazkov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209798#comment-16209798 ] Plamen Jeliazkov edited comment on HDFS-8921 at 10/18/17 6:40 PM: -- Would

[jira] [Created] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-18 Thread Nanda kumar (JIRA)
Nanda kumar created HDFS-12680: -- Summary: Ozone: SCM: Lease support for container creation Key: HDFS-12680 URL: https://issues.apache.org/jira/browse/HDFS-12680 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-8921) Add an option to Balancer for specifying the k-most over-utilized DNs or all over-utilized DNs as sources.

2017-10-18 Thread Plamen Jeliazkov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209798#comment-16209798 ] Plamen Jeliazkov commented on HDFS-8921: Would it be acceptable to say something "k-most

[jira] [Updated] (HDFS-12090) Handling writes from HDFS to Provided storages

2017-10-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12090: -- Attachment: HDFS-12090-Functional-Specification.002.pdf Attaching updated version of Functional

[jira] [Updated] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-18 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12680: --- Description: This brings in lease support for container creation. Lease should be give for a

[jira] [Updated] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-18 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12680: --- Description: This brings in lease support for container creation. Lease should be give for a

[jira] [Updated] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-18 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12680: --- Attachment: HDFS-12680-HDFS-7240.000.patch > Ozone: SCM: Lease support for container creation >

[jira] [Updated] (HDFS-12680) Ozone: SCM: Lease support for container creation

2017-10-18 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12680: --- Status: Patch Available (was: Open) > Ozone: SCM: Lease support for container creation >

[jira] [Commented] (HDFS-12673) Ozone: Log elapsed time for block deletion tasks

2017-10-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208860#comment-16208860 ] Hadoop QA commented on HDFS-12673: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-18 Thread Yiqun Lin (JIRA)
Yiqun Lin created HDFS-12675: Summary: Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails Key: HDFS-12675 URL: https://issues.apache.org/jira/browse/HDFS-12675 Project: Hadoop HDFS

[jira] [Updated] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-12675: - Description: Caught one UT failure {{TestLeaseManager#testLeaseCallbackWithMultipleLeases}}: {noformat}

[jira] [Commented] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208869#comment-16208869 ] Yiqun Lin commented on HDFS-12675: -- I am thinking one corner case leading for this: #

[jira] [Updated] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-12675: - Description: Caught one UT failure {{TestLeaseManager#testLeaseCallbackWithMultipleLeases}}: {noformat}

[jira] [Updated] (HDFS-12675) Ozone: TestLeaseManager#testLeaseCallbackWithMultipleLeases fails

2017-10-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-12675: - Description: Caught one UT failure {{TestLeaseManager#testLeaseCallbackWithMultipleLeases}}: {noformat}

[jira] [Commented] (HDFS-12673) Ozone: Log elapsed time for block deletion tasks

2017-10-18 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208874#comment-16208874 ] Weiwei Yang commented on HDFS-12673: LGTM, +1 to the latest patch, I will commit this shortly. Thanks

[jira] [Updated] (HDFS-11828) [WRITE] Refactor FsDatasetImpl to use the BlockAlias from ClientProtocol for PROVIDED blocks.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11828: -- Summary: [WRITE] Refactor FsDatasetImpl to use the BlockAlias from ClientProtocol for

[jira] [Updated] (HDFS-11639) [WRITE] Encode the BlockAlias in the client protocol

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11639: -- Summary: [WRITE] Encode the BlockAlias in the client protocol (was: [READ] Encode the

[jira] [Commented] (HDFS-12605) [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase

2017-10-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209864#comment-16209864 ] Ewan Higgs commented on HDFS-12605: --- +1 I'm running into this on the branch and this should fix the

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11902: -- Status: Open (was: Patch Available) > [READ] Merge BlockFormatProvider and

[jira] [Issue Comment Deleted] (HDFS-12605) [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase

2017-10-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-12605: -- Comment: was deleted (was: +1 I'm running into this on the branch and this should fix the issue.) >

[jira] [Commented] (HDFS-8921) Add an option to Balancer for specifying the k-most over-utilized DNs or all over-utilized DNs as sources.

2017-10-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209885#comment-16209885 ] Tsz Wo Nicholas Sze commented on HDFS-8921: --- It sounds like a good idea. Are you interested in

[jira] [Assigned] (HDFS-8921) Add an option to Balancer for specifying the k-most over-utilized DNs or all over-utilized DNs as sources.

2017-10-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze reassigned HDFS-8921: - Assignee: (was: Tsz Wo Nicholas Sze) > Add an option to Balancer for

[jira] [Commented] (HDFS-12605) [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase

2017-10-18 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209894#comment-16209894 ] Ewan Higgs commented on HDFS-12605: --- I tested this with a NN and DN on a local system and it worked. I

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11902: -- Attachment: (was: HDFS-11902-HDFS-9806.009.patch) > [READ] Merge

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11902: -- Attachment: HDFS-11902-HDFS-9806.009.patch > [READ] Merge BlockFormatProvider and

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11902: -- Status: Patch Available (was: Open) > [READ] Merge BlockFormatProvider and

[jira] [Updated] (HDFS-12607) [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12607: -- Description: When a DN configured with PROVIDED storage is marked as dead by the NN,

[jira] [Updated] (HDFS-12607) [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12607: -- Status: Open (was: Patch Available) > [READ] Even one dead datanode with PROVIDED

[jira] [Updated] (HDFS-12607) [READ] Even one dead datanode with PROVIDED storage results in ProvidedStorageInfo being marked as FAILED

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12607: -- Status: Patch Available (was: Open) > [READ] Even one dead datanode with PROVIDED

[jira] [Updated] (HDFS-12683) DFSZKFailOverController re-order logic for logging Exception

2017-10-18 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12683: -- Description: Log the exception before closing the connections and terminating server.

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11902: -- Status: Open (was: Patch Available) > [READ] Merge BlockFormatProvider and

[jira] [Updated] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11902: -- Status: Patch Available (was: Open) > [READ] Merge BlockFormatProvider and

[jira] [Commented] (HDFS-12605) [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210013#comment-16210013 ] Virajith Jalaparti commented on HDFS-12605: --- Thanks for testing this out [~ehiggs]. Committed

[jira] [Updated] (HDFS-12605) [READ] TestNameNodeProvidedImplementation#testProvidedDatanodeFailures fails after rebase

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12605: -- Resolution: Fixed Status: Resolved (was: Patch Available) > [READ]

[jira] [Commented] (HDFS-12620) Backporting HDFS-10467 to branch-2

2017-10-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210235#comment-16210235 ] Hadoop QA commented on HDFS-12620: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Assigned] (HDFS-8921) Add an option to Balancer for specifying the k-most over-utilized DNs or all over-utilized DNs as sources.

2017-10-18 Thread Plamen Jeliazkov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Plamen Jeliazkov reassigned HDFS-8921: -- Assignee: Plamen Jeliazkov > Add an option to Balancer for specifying the k-most

[jira] [Commented] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209957#comment-16209957 ] Hadoop QA commented on HDFS-11902: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12681) Fold HdfsLocatedFileStatus into HdfsFileStatus

2017-10-18 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210095#comment-16210095 ] Chris Douglas commented on HDFS-12681: -- Applications using {{instanceof}} to determine if the

[jira] [Updated] (HDFS-12682) ECAdmin -listPolicies will always show policy state as DISABLED

2017-10-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-12682: - Description: On a real cluster, {{hdfs ec -listPolicies}} will always show policy state as DISABLED.

[jira] [Commented] (HDFS-8921) Add an option to Balancer for specifying the k-most over-utilized DNs or all over-utilized DNs as sources.

2017-10-18 Thread Plamen Jeliazkov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210274#comment-16210274 ] Plamen Jeliazkov commented on HDFS-8921: Yes, I am interested. I have a rough first patch. Would

[jira] [Commented] (HDFS-11902) [READ] Merge BlockFormatProvider and FileRegionProvider.

2017-10-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209962#comment-16209962 ] Virajith Jalaparti commented on HDFS-11902: --- Thanks for taking a look at it [~ehiggs]! I am

  1   2   >