[jira] [Commented] (HDFS-10543) hdfsRead read stops at block boundary

2016-07-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15363214#comment-15363214 ] Colin Patrick McCabe commented on HDFS-10543: - One approach would be to try checking the

[jira] [Commented] (HDFS-10555) Unable to loadFSEdits due to a failure in readCachePoolInfo

2016-07-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15363005#comment-15363005 ] Colin Patrick McCabe commented on HDFS-10555: - Thanks, [~umamaheswararao], [~jingzhao], and

[jira] [Commented] (HDFS-10548) Remove the long deprecated BlockReaderRemote

2016-07-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15363003#comment-15363003 ] Colin Patrick McCabe commented on HDFS-10548: - Thanks for tackling this, guys. It is good to

[jira] [Commented] (HDFS-10543) hdfsRead read stops at block boundary

2016-07-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362997#comment-15362997 ] Colin Patrick McCabe commented on HDFS-10543: - Just to be clear, the existing HDFS Java client

[jira] [Updated] (HDFS-9805) TCP_NODELAY not set before SASL handshake in data transfer pipeline

2016-07-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9805: --- Resolution: Fixed Fix Version/s: 3.0.0-alpha1 Status: Resolved (was:

[jira] [Commented] (HDFS-9805) TCP_NODELAY not set before SASL handshake in data transfer pipeline

2016-07-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362983#comment-15362983 ] Colin Patrick McCabe commented on HDFS-9805: Thanks for the reminder, [~jzhuge]. I committed

[jira] [Updated] (HDFS-10594) HDFS-4949 should support recursive cache directives

2016-07-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10594: Summary: HDFS-4949 should support recursive cache directives (was:

[jira] [Commented] (HDFS-9700) DFSClient and DFSOutputStream should set TCP_NODELAY on sockets for DataTransferProtocol

2016-06-24 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348970#comment-15348970 ] Colin Patrick McCabe commented on HDFS-9700: Hmm. I think it's confusing to use a

[jira] [Commented] (HDFS-8940) Support for large-scale multi-tenant inotify service

2016-06-24 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348554#comment-15348554 ] Colin Patrick McCabe commented on HDFS-8940: bq. You mean reading inotify messages from the

[jira] [Commented] (HDFS-10328) Add per-cache-pool default replication num configuration

2016-06-22 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345791#comment-15345791 ] Colin Patrick McCabe commented on HDFS-10328: - Sorry for the breakage, [~kshukla]. HDFS-10555

[jira] [Commented] (HDFS-8940) Support for large-scale multi-tenant inotify service

2016-06-22 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345766#comment-15345766 ] Colin Patrick McCabe commented on HDFS-8940: I think Kafka would be a great choice for scaling

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-06-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340868#comment-15340868 ] Colin Patrick McCabe commented on HDFS-10301: - The "you" in that sentence was targetted at

[jira] [Commented] (HDFS-10448) CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1

2016-06-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340862#comment-15340862 ] Colin Patrick McCabe commented on HDFS-10448: - Committed to 2.8. Thanks, [~linyiqun]! Sorry

[jira] [Updated] (HDFS-10448) CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1

2016-06-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10448: Resolution: Fixed Fix Version/s: 2.8.0 Target Version/s: 2.8.0

[jira] [Updated] (HDFS-10448) CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1

2016-06-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10448: Summary: CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with

[jira] [Commented] (HDFS-10448) CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1

2016-06-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340851#comment-15340851 ] Colin Patrick McCabe commented on HDFS-10448: - Hi [~linyiqun], Sorry, I misread the patch the

[jira] [Updated] (HDFS-10328) Add per-cache-pool default replication num configuration

2016-06-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10328: Resolution: Fixed Target Version/s: 2.9.0 Status: Resolved

[jira] [Commented] (HDFS-10548) Remove the long deprecated BlockReaderRemote

2016-06-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340836#comment-15340836 ] Colin Patrick McCabe commented on HDFS-10548: - I would love to see this class go away. It is

[jira] [Updated] (HDFS-10328) Add per-cache-pool default replication num configuration

2016-06-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10328: Attachment: (was: HDFS-10328.004.patch) > Add per-cache-pool default

[jira] [Updated] (HDFS-10328) Add per-cache-pool default replication num configuration

2016-06-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10328: Attachment: HDFS-10328.004.patch Reposting patch 004 (and rebasing on trunk) to get

[jira] [Commented] (HDFS-10328) Add per-cache-pool default replication num configuration

2016-06-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336775#comment-15336775 ] Colin Patrick McCabe commented on HDFS-10328: - +1 pending jenkins. Thanks, [~xupener]. > Add

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-06-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335100#comment-15335100 ] Colin Patrick McCabe commented on HDFS-10301: - {code} +if (context.getTotalRpcs() ==

[jira] [Updated] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2016-06-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9466: --- Resolution: Fixed Fix Version/s: 2.8.0 Target Version/s: 2.8.0

[jira] [Updated] (HDFS-10525) Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap

2016-06-15 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10525: Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Commented] (HDFS-10525) Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap

2016-06-15 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333153#comment-15333153 ] Colin Patrick McCabe commented on HDFS-10525: - +1. Thanks, [~xiaochen]. > Fix NPE in

[jira] [Updated] (HDFS-10505) OIV's ReverseXML processor should support ACLs

2016-06-15 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10505: Resolution: Fixed Fix Version/s: 2.8.0 Target Version/s: 2.8.0

[jira] [Commented] (HDFS-10505) OIV's ReverseXML processor should support ACLs

2016-06-15 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333136#comment-15333136 ] Colin Patrick McCabe commented on HDFS-10505: - +1. Thanks, [~surendrasingh] > OIV's

[jira] [Commented] (HDFS-10525) Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap

2016-06-14 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330503#comment-15330503 ] Colin Patrick McCabe commented on HDFS-10525: - +1. Thanks, [~xiaochen]. > Fix NPE in

[jira] [Commented] (HDFS-10505) OIV's ReverseXML processor should support ACLs

2016-06-14 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330492#comment-15330492 ] Colin Patrick McCabe commented on HDFS-10505: - Thanks for this, [~surendrasingh]. It's good

[jira] [Commented] (HDFS-10525) Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap

2016-06-14 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330133#comment-15330133 ] Colin Patrick McCabe commented on HDFS-10525: - Thanks, [~xiaochen]. Can you add a

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-06-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322994#comment-15322994 ] Colin Patrick McCabe commented on HDFS-10301: - [~shv], comments about me "being on a -1 spree"

[jira] [Commented] (HDFS-10506) OIV's ReverseXML processor cannot reconstruct some snapshot details

2016-06-08 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321320#comment-15321320 ] Colin Patrick McCabe commented on HDFS-10506: - >From

[jira] [Commented] (HDFS-10505) OIV's ReverseXML processor should support ACLs

2016-06-08 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321317#comment-15321317 ] Colin Patrick McCabe commented on HDFS-10505: - >From

[jira] [Created] (HDFS-10506) OIV's ReverseXML processor cannot reconstruct some snapshot details

2016-06-08 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-10506: --- Summary: OIV's ReverseXML processor cannot reconstruct some snapshot details Key: HDFS-10506 URL: https://issues.apache.org/jira/browse/HDFS-10506

[jira] [Created] (HDFS-10505) OIV's ReverseXML processor should support ACLs

2016-06-08 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-10505: --- Summary: OIV's ReverseXML processor should support ACLs Key: HDFS-10505 URL: https://issues.apache.org/jira/browse/HDFS-10505 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-8061) Create an Offline FSImage Viewer tool

2016-06-08 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe resolved HDFS-8061. Resolution: Duplicate Fix Version/s: 2.8.0 Target Version/s: 2.8.0

[jira] [Updated] (HDFS-8306) Outputs Xattr in OIV XML format

2016-06-08 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8306: --- Resolution: Duplicate Fix Version/s: 2.8.0 Target Version/s: 2.8.0

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-06-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317644#comment-15317644 ] Colin Patrick McCabe commented on HDFS-10301: - Sorry for the slow reply. I was on vacation.

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-06-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317629#comment-15317629 ] Colin Patrick McCabe commented on HDFS-9924: +1 for a feature branch > [umbrella] Asynchronous

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-05-31 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308997#comment-15308997 ] Colin Patrick McCabe commented on HDFS-10301: - bq. Vinitha's patch adds one RPC only in the

[jira] [Comment Edited] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2016-05-31 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308980#comment-15308980 ] Colin Patrick McCabe edited comment on HDFS-9466 at 6/1/16 12:52 AM: -

[jira] [Commented] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2016-05-31 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308980#comment-15308980 ] Colin Patrick McCabe commented on HDFS-9466: Thanks for the explanation. It sounds like the

[jira] [Updated] (HDFS-10415) TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called

2016-05-31 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10415: Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Commented] (HDFS-10415) TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called

2016-05-31 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308928#comment-15308928 ] Colin Patrick McCabe commented on HDFS-10415: - The subclass can change the configuration that

[jira] [Comment Edited] (HDFS-10415) TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called

2016-05-31 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308928#comment-15308928 ] Colin Patrick McCabe edited comment on HDFS-10415 at 6/1/16 12:09 AM: --

[jira] [Updated] (HDFS-10415) TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called

2016-05-31 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10415: Summary: TestDistributedFileSystem#MyDistributedFileSystem attempts to set up

[jira] [Commented] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-31 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308894#comment-15308894 ] Colin Patrick McCabe commented on HDFS-10415: - It sounds like there are no strong objections

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-05-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304702#comment-15304702 ] Colin Patrick McCabe commented on HDFS-10301: - [~redvine], the fact that you are having

[jira] [Commented] (HDFS-7240) Object store in HDFS

2016-05-26 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303085#comment-15303085 ] Colin Patrick McCabe commented on HDFS-7240: bq. Correct me if I am wrong – before Andrew

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-05-26 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302426#comment-15302426 ] Colin Patrick McCabe commented on HDFS-10301: - I never said that patch 004 introduced

[jira] [Commented] (HDFS-7240) Object store in HDFS

2016-05-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301560#comment-15301560 ] Colin Patrick McCabe commented on HDFS-7240: bq. [~szetszwo] wrote: I seem to recall that you

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-05-24 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298675#comment-15298675 ] Colin Patrick McCabe commented on HDFS-10301: - Oh, sorry! I didn't realize we had added a new

[jira] [Updated] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-05-24 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10301: Attachment: HDFS-10301.005.patch Rebasing patch 003 on trunk. > BlockReport

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-05-24 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298565#comment-15298565 ] Colin Patrick McCabe commented on HDFS-10301: - Hi [~redvine], Thanks for your interest in

[jira] [Assigned] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-05-24 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe reassigned HDFS-10301: --- Assignee: Colin Patrick McCabe (was: Vinitha Reddy Gankidi) > BlockReport

[jira] [Commented] (HDFS-10448) CacheManager#checkLimit always assumes a replication factor of 1

2016-05-24 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298543#comment-15298543 ] Colin Patrick McCabe commented on HDFS-10448: - I think it should change {{computeNeeded}} to

[jira] [Updated] (HDFS-10448) CacheManager#checkLimit always assumes a replication factor of 1

2016-05-23 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10448: Summary: CacheManager#checkLimit always assumes a replication factor of 1 (was:

[jira] [Commented] (HDFS-10448) CacheManager#checkLimit not correctly

2016-05-23 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297068#comment-15297068 ] Colin Patrick McCabe commented on HDFS-10448: - This is a good find. I think that

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-05-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294073#comment-15294073 ] Colin Patrick McCabe commented on HDFS-9924: bq. Good point! On the other hand, programmers who

[jira] [Commented] (HDFS-7240) Object store in HDFS

2016-05-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293822#comment-15293822 ] Colin Patrick McCabe commented on HDFS-7240: bq. Another question about reading the ApacheCon

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-05-20 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293809#comment-15293809 ] Colin Patrick McCabe commented on HDFS-9924: I have to agree with [~andrew.wang] that it makes

[jira] [Commented] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289769#comment-15289769 ] Colin Patrick McCabe commented on HDFS-10415: - bq. As Steve Loughran's concern, if the stats

[jira] [Commented] (HDFS-10415) TestDistributedFileSystem#testDFSCloseOrdering() fails on branch-2

2016-05-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289312#comment-15289312 ] Colin Patrick McCabe commented on HDFS-10415: - Thanks for looking at this. So basically the

[jira] [Commented] (HDFS-8829) Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning

2016-05-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289239#comment-15289239 ] Colin Patrick McCabe commented on HDFS-8829: There was no need to make it public because it's

[jira] [Updated] (HDFS-10404) Fix formatting of CacheAdmin command usage help text

2016-05-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10404: Resolution: Fixed Fix Version/s: 2.9.0 Target Version/s: 2.9.0

[jira] [Updated] (HDFS-10404) Fix formatting of CacheAdmin command usage help text

2016-05-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10404: Summary: Fix formatting of CacheAdmin command usage help text (was: CacheAdmin

[jira] [Commented] (HDFS-10404) CacheAdmin command usage message not shows completely

2016-05-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287149#comment-15287149 ] Colin Patrick McCabe commented on HDFS-10404: - +1. Thanks, [~linyiqun]. > CacheAdmin command

[jira] [Updated] (HDFS-10276) HDFS throws AccessControlException when checking for the existence of /a/b when /a is a file

2016-05-12 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10276: Summary: HDFS throws AccessControlException when checking for the existence of /a/b

[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name

2016-05-12 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281676#comment-15281676 ] Colin Patrick McCabe commented on HDFS-10276: - Thanks for this, [~yuanbo]. Sorry for the

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-05-11 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280809#comment-15280809 ] Colin Patrick McCabe commented on HDFS-9924: bq. For some case like network connection errors,

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-05-11 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280617#comment-15280617 ] Colin Patrick McCabe commented on HDFS-9924: With regard to error handling, why not handle all

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-05-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278651#comment-15278651 ] Colin Patrick McCabe commented on HDFS-9924: Hi all, I am +1 for doing this work, based on

[jira] [Updated] (HDFS-10377) CacheReplicationMonitor shutdown log message should use INFO level.

2016-05-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10377: Resolution: Fixed Fix Version/s: 2.6.5 Target Version/s: 2.6.5

[jira] [Commented] (HDFS-10377) CacheReplicationMonitor shutdown log message should use INFO level.

2016-05-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278609#comment-15278609 ] Colin Patrick McCabe commented on HDFS-10377: - +1. Thanks, [~linyiqun]. >

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-05-04 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271863#comment-15271863 ] Colin Patrick McCabe commented on HDFS-10301: - Thanks for looking at this, [~daryn]. I'm not

[jira] [Commented] (HDFS-10328) Add per-cache-pool default replication num configuration

2016-05-03 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268954#comment-15268954 ] Colin Patrick McCabe commented on HDFS-10328: - Thanks for the patch, [~xupener]. {code} diff

[jira] [Commented] (HDFS-10352) Allow users to get last access time of a given directory

2016-05-02 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266999#comment-15266999 ] Colin Patrick McCabe commented on HDFS-10352: - -1. As [~linyiqun] commented, the performance

[jira] [Commented] (HDFS-10328) Add cache pool level replication managment

2016-04-29 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264841#comment-15264841 ] Colin Patrick McCabe commented on HDFS-10328: - Hi [~xupener], Interesting idea. However,

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-04-29 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264772#comment-15264772 ] Colin Patrick McCabe commented on HDFS-10301: - bq. You can think of it as a new operation

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260498#comment-15260498 ] Colin Patrick McCabe commented on HDFS-10175: - BTW, sorry for the last-minute-ness of this

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260487#comment-15260487 ] Colin Patrick McCabe commented on HDFS-10175: - Great. Let me add a webex > add per-operation

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-04-26 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259529#comment-15259529 ] Colin Patrick McCabe commented on HDFS-10301: - bq. Hey Colin, I reviewed your patch more

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-26 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259223#comment-15259223 ] Colin Patrick McCabe commented on HDFS-10175: - Hi [~steve_l], does 10:30AM work tomorrow?

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-26 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258726#comment-15258726 ] Colin Patrick McCabe commented on HDFS-10175: - bq. I see that, but stream-level counters are

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-26 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258706#comment-15258706 ] Colin Patrick McCabe commented on HDFS-10175: - bq. I prefer earlier (being in UK time and

[jira] [Commented] (HDFS-10305) Hdfs audit shouldn't log mkdir operaton if the directory already exists.

2016-04-26 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258603#comment-15258603 ] Colin Patrick McCabe commented on HDFS-10305: - bq. That's interesting then. hdfs dfs -mkdir

[jira] [Commented] (HDFS-10326) Disable setting tcp socket send/receive buffers for write pipelines

2016-04-26 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258592#comment-15258592 ] Colin Patrick McCabe commented on HDFS-10326: - bq. Some system may not support auto tuning,

[jira] [Commented] (HDFS-10305) Hdfs audit shouldn't log mkdir operaton if the directory already exists.

2016-04-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257383#comment-15257383 ] Colin Patrick McCabe commented on HDFS-10305: - Thanks, [~raviprak]. It's always good to have

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257074#comment-15257074 ] Colin Patrick McCabe commented on HDFS-10175: - We already have three statistics interfaces: 1.

[jira] [Commented] (HDFS-10323) transient deleteOnExit failure in ViewFileSystem due to close() ordering

2016-04-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256861#comment-15256861 ] Colin Patrick McCabe commented on HDFS-10323: - Thanks for the detailed bug report,

[jira] [Commented] (HDFS-10305) Hdfs audit shouldn't log mkdir operaton if the directory already exists.

2016-04-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256858#comment-15256858 ] Colin Patrick McCabe commented on HDFS-10305: - bq. I believe the audit log was supposed to

[jira] [Commented] (HDFS-10326) Disable setting tcp socket send/receive buffers for write pipelines

2016-04-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256848#comment-15256848 ] Colin Patrick McCabe commented on HDFS-10326: - bq. I think we can keep the configurability,

[jira] [Comment Edited] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256842#comment-15256842 ] Colin Patrick McCabe edited comment on HDFS-10175 at 4/25/16 7:20 PM: --

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256842#comment-15256842 ] Colin Patrick McCabe commented on HDFS-10175: - Thanks for commenting, [~steve_l]. It's great

[jira] [Comment Edited] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-04-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256791#comment-15256791 ] Colin Patrick McCabe edited comment on HDFS-10301 at 4/25/16 6:58 PM: --

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-04-25 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15256791#comment-15256791 ] Colin Patrick McCabe commented on HDFS-10301: - bq. [~shv] wrote: The last line is confusing,

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-22 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254503#comment-15254503 ] Colin Patrick McCabe commented on HDFS-10175: - bq. Can I also note that as the @Public @Stable

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-04-22 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254350#comment-15254350 ] Colin Patrick McCabe commented on HDFS-10301: - Yeah, perhaps we should file a follow-on JIRA

[jira] [Assigned] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-04-22 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe reassigned HDFS-10301: --- Assignee: Colin Patrick McCabe > BlockReport retransmissions may lead to

[jira] [Updated] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-04-21 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10301: Attachment: HDFS-10301.003.patch added a unit test > BlockReport retransmissions

  1   2   3   4   5   6   7   8   9   10   >