[jira] [Commented] (HDFS-5546) race condition crashes hadoop ls -R when directories are moved/removed

[ https://issues.apache.org/jira/browse/HDFS-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041159#comment-14041159 ] Colin Patrick McCabe commented on HDFS-5546: I think what Daryn is advocating

[jira] [Commented] (HDFS-6560) Byte array native checksumming on DN side

[ https://issues.apache.org/jira/browse/HDFS-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041477#comment-14041477 ] Colin Patrick McCabe commented on HDFS-6560: {code} + public static void

[jira] [Commented] (HDFS-6570) add api that enables checking if a user has certain permissions on a file

[ https://issues.apache.org/jira/browse/HDFS-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041492#comment-14041492 ] Colin Patrick McCabe commented on HDFS-6570: bq. Note that the man page for

[jira] [Commented] (HDFS-6596) Improve InputStream when read spans two blocks

[ https://issues.apache.org/jira/browse/HDFS-6596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042380#comment-14042380 ] Colin Patrick McCabe commented on HDFS-6596: What you are proposing is

[jira] [Commented] (HDFS-6597) New option for namenode upgrade

[ https://issues.apache.org/jira/browse/HDFS-6597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042384#comment-14042384 ] Colin Patrick McCabe commented on HDFS-6597: Concept seems reasonable, but I

[jira] [Commented] (HDFS-6387) HDFS CLI admin tool for creating deleting an encryption zone

[ https://issues.apache.org/jira/browse/HDFS-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042706#comment-14042706 ] Colin Patrick McCabe commented on HDFS-6387: {code} + private static

[jira] [Commented] (HDFS-6389) Rename restrictions for encryption zones

[ https://issues.apache.org/jira/browse/HDFS-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042718#comment-14042718 ] Colin Patrick McCabe commented on HDFS-6389: Looks good overall. {code} +

[jira] [Commented] (HDFS-6516) Persist and Restore the List of Encryption Zones

[ https://issues.apache.org/jira/browse/HDFS-6516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042766#comment-14042766 ] Colin Patrick McCabe commented on HDFS-6516: Is the

[jira] [Commented] (HDFS-6516) Persist and Restore the List of Encryption Zones

[ https://issues.apache.org/jira/browse/HDFS-6516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042767#comment-14042767 ] Colin Patrick McCabe commented on HDFS-6516: er, that should read inodes that

[jira] [Commented] (HDFS-6389) Rename restrictions for encryption zones

[ https://issues.apache.org/jira/browse/HDFS-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14043880#comment-14043880 ] Colin Patrick McCabe commented on HDFS-6389: OK, I re-checked this and there

[jira] [Commented] (HDFS-5546) race condition crashes hadoop ls -R when directories are moved/removed

[ https://issues.apache.org/jira/browse/HDFS-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14043956#comment-14043956 ] Colin Patrick McCabe commented on HDFS-5546: I agree with a lot of the stuff

[jira] [Commented] (HDFS-6389) Rename restrictions for encryption zones

[ https://issues.apache.org/jira/browse/HDFS-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046352#comment-14046352 ] Colin Patrick McCabe commented on HDFS-6389: Thanks. Looks better. I would

[jira] [Work started] (HDFS-6604) Disk space leak with shortcircuit

[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-6604 started by Colin Patrick McCabe. Disk space leak with shortcircuit - Key: HDFS-6604

[jira] [Updated] (HDFS-6604) Disk space leak with shortcircuit

[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6604: --- Attachment: HDFS-6604.001.patch The CacheCleaner thread is supposed to time out

[jira] [Assigned] (HDFS-6604) Disk space leak with shortcircuit

[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe reassigned HDFS-6604: -- Assignee: Colin Patrick McCabe Disk space leak with shortcircuit

[jira] [Updated] (HDFS-6604) Disk space leak with shortcircuit

[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6604: --- Attachment: HDFS-6604.002.patch Thanks, Chris. There is an existing unit test for

[jira] [Updated] (HDFS-6604) The short-circuit cache doesn't correctly time out replicas that haven't been used in a while

[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6604: --- Summary: The short-circuit cache doesn't correctly time out replicas that haven't

[jira] [Updated] (HDFS-6604) The short-circuit cache doesn't correctly time out replicas that haven't been used in a while

[ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6604: --- Resolution: Fixed Fix Version/s: 2.5.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-6482) Use block ID-based block layout on datanodes

[ https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050676#comment-14050676 ] Colin Patrick McCabe commented on HDFS-6482: I agree with the reasoning about

[jira] [Commented] (HDFS-6618) Remove deleted INodes from INodeMap right away

[ https://issues.apache.org/jira/browse/HDFS-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050803#comment-14050803 ] Colin Patrick McCabe commented on HDFS-6618: Ugh, I wrote a long comment and

[jira] [Commented] (HDFS-6622) Rename and AddBlock may race and produce invalid edits

[ https://issues.apache.org/jira/browse/HDFS-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050820#comment-14050820 ] Colin Patrick McCabe commented on HDFS-6622: The fix here has a problem. If

[jira] [Commented] (HDFS-6622) Rename and AddBlock may race and produce invalid edits

[ https://issues.apache.org/jira/browse/HDFS-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050843#comment-14050843 ] Colin Patrick McCabe commented on HDFS-6622: I think the best solution is to

[jira] [Commented] (HDFS-6516) Persist and Restore the List of Encryption Zones

[ https://issues.apache.org/jira/browse/HDFS-6516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050883#comment-14050883 ] Colin Patrick McCabe commented on HDFS-6516: {code} + private final MapINode,

[jira] [Commented] (HDFS-6625) Remove the Delete Encryption Zone function

[ https://issues.apache.org/jira/browse/HDFS-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14050932#comment-14050932 ] Colin Patrick McCabe commented on HDFS-6625: +1. Thanks, Charles. Remove the

[jira] [Commented] (HDFS-6618) Remove deleted INodes from INodeMap right away

[ https://issues.apache.org/jira/browse/HDFS-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14051901#comment-14051901 ] Colin Patrick McCabe commented on HDFS-6618: Thanks for that explanation. If

[jira] [Commented] (HDFS-6516) List of Encryption Zones should be based on inodes

[ https://issues.apache.org/jira/browse/HDFS-6516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14051926#comment-14051926 ] Colin Patrick McCabe commented on HDFS-6516: {code} assert hasWriteLock();

[jira] [Commented] (HDFS-6482) Use block ID-based block layout on datanodes

[ https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054008#comment-14054008 ] Colin Patrick McCabe commented on HDFS-6482: The new current directory will

[jira] [Commented] (HDFS-6482) Use block ID-based block layout on datanodes

[ https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054115#comment-14054115 ] Colin Patrick McCabe commented on HDFS-6482: bq. My read is, this will not work

[jira] [Commented] (HDFS-6482) Use block ID-based block layout on datanodes

[ https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055419#comment-14055419 ] Colin Patrick McCabe commented on HDFS-6482: bq. \[james wrote\]: Also, I don't

[jira] [Commented] (HDFS-6647) Edit log corruption when pipeline recovery occurs for deleted file present in snapshot

[ https://issues.apache.org/jira/browse/HDFS-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056517#comment-14056517 ] Colin Patrick McCabe commented on HDFS-6647: The simplest thing is probably

[jira] [Commented] (HDFS-6622) Rename and AddBlock may race and produce invalid edits

[ https://issues.apache.org/jira/browse/HDFS-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056596#comment-14056596 ] Colin Patrick McCabe commented on HDFS-6622: bq. I also wanted to reduce the

[jira] [Commented] (HDFS-6618) Remove deleted INodes from INodeMap right away

[ https://issues.apache.org/jira/browse/HDFS-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056608#comment-14056608 ] Colin Patrick McCabe commented on HDFS-6618: It looks like someone is updating

[jira] [Commented] (HDFS-6618) Remove deleted INodes from INodeMap right away

[ https://issues.apache.org/jira/browse/HDFS-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056610#comment-14056610 ] Colin Patrick McCabe commented on HDFS-6618: +1 for the patch. Just one small

[jira] [Commented] (HDFS-6634) inotify in HDFS

[ https://issues.apache.org/jira/browse/HDFS-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14057032#comment-14057032 ] Colin Patrick McCabe commented on HDFS-6634: I think it would be great to have

[jira] [Updated] (HDFS-6618) FSNamesystem#delete drops the FSN lock between removing INodes from the tree and deleting them from the inode map

[ https://issues.apache.org/jira/browse/HDFS-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6618: --- Summary: FSNamesystem#delete drops the FSN lock between removing INodes from the tree

[jira] [Updated] (HDFS-6618) FSNamesystem#delete drops the FSN lock between removing INodes from the tree and deleting them from the inode map

[ https://issues.apache.org/jira/browse/HDFS-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6618: --- Resolution: Fixed Fix Version/s: 2.5.0 Status: Resolved (was: Patch

[jira] [Updated] (HDFS-6622) Rename and AddBlock may race and produce invalid edits

[ https://issues.apache.org/jira/browse/HDFS-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6622: --- Resolution: Fixed Fix Version/s: 2.5.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-6647) Edit log corruption when pipeline recovery occurs for deleted file present in snapshot

[ https://issues.apache.org/jira/browse/HDFS-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14057103#comment-14057103 ] Colin Patrick McCabe commented on HDFS-6647: Thanks, Kihwal. These fixes are

[jira] [Updated] (HDFS-6647) Edit log corruption when pipeline recovery occurs for deleted file present in snapshot

[ https://issues.apache.org/jira/browse/HDFS-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6647: --- Status: Patch Available (was: Open) Edit log corruption when pipeline recovery

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061069#comment-14061069 ] Colin Patrick McCabe commented on HDFS-6658: 2**31 (2 billion) possible

[jira] [Commented] (HDFS-6114) Block Scan log rolling will never happen if blocks written continuously leading to huge size of dncp_block_verification.log.curr

[ https://issues.apache.org/jira/browse/HDFS-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061263#comment-14061263 ] Colin Patrick McCabe commented on HDFS-6114: I don't really see a good reason

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061374#comment-14061374 ] Colin Patrick McCabe commented on HDFS-6658: bq. As opposed to

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061381#comment-14061381 ] Colin Patrick McCabe commented on HDFS-6658: I think that we need to take a

[jira] [Updated] (HDFS-5809) BlockPoolSliceScanner and high speed hdfs appending make datanode to drop into infinite loop

[ https://issues.apache.org/jira/browse/HDFS-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5809: --- Description: {{BlockPoolSliceScanner#scan}} contains a while loop that continues to

[jira] [Commented] (HDFS-6114) Block Scan log rolling will never happen if blocks written continuously leading to huge size of dncp_block_verification.log.curr

[ https://issues.apache.org/jira/browse/HDFS-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062322#comment-14062322 ] Colin Patrick McCabe commented on HDFS-6114: bq. blockInfoSet is required to be

[jira] [Updated] (HDFS-5809) BlockPoolSliceScanner and high speed hdfs appending make datanode to drop into infinite loop

[ https://issues.apache.org/jira/browse/HDFS-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5809: --- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062514#comment-14062514 ] Colin Patrick McCabe commented on HDFS-6658: bq. If we don't have caching, we

[jira] [Commented] (HDFS-5464) Simplify block report diff calculation

[ https://issues.apache.org/jira/browse/HDFS-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062703#comment-14062703 ] Colin Patrick McCabe commented on HDFS-5464: {code} // collect blocks that

[jira] [Commented] (HDFS-5464) Simplify block report diff calculation

[ https://issues.apache.org/jira/browse/HDFS-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062867#comment-14062867 ] Colin Patrick McCabe commented on HDFS-5464: To be honest, I don't think this

[jira] [Commented] (HDFS-6114) Block Scan log rolling will never happen if blocks written continuously leading to huge size of dncp_block_verification.log.curr

[ https://issues.apache.org/jira/browse/HDFS-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063837#comment-14063837 ] Colin Patrick McCabe commented on HDFS-6114: {code} // add new blocks to scan

[jira] [Created] (HDFS-6695) Investigate using Java 7's nonblocking file I/O in BlockReaderLocal to implement read timeouts

Colin Patrick McCabe created HDFS-6695: -- Summary: Investigate using Java 7's nonblocking file I/O in BlockReaderLocal to implement read timeouts Key: HDFS-6695 URL:

[jira] [Commented] (HDFS-6695) Investigate using Java 7's nonblocking file I/O in BlockReaderLocal to implement read timeouts

[ https://issues.apache.org/jira/browse/HDFS-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063879#comment-14063879 ] Colin Patrick McCabe commented on HDFS-6695: There are some docs here:

[jira] [Commented] (HDFS-6450) Support non-positional hedged reads in HDFS

[ https://issues.apache.org/jira/browse/HDFS-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063892#comment-14063892 ] Colin Patrick McCabe commented on HDFS-6450: Yeah... if the first read, done

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063913#comment-14063913 ] Colin Patrick McCabe commented on HDFS-6658: bq. What you suggesting (if I

[jira] [Commented] (HDFS-6690) Deduplicate xattr names in memory

[ https://issues.apache.org/jira/browse/HDFS-6690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063972#comment-14063972 ] Colin Patrick McCabe commented on HDFS-6690: Looks good overall. Can we have

[jira] [Commented] (HDFS-6588) Investigating removing getTrueCause method in Server.java

[ https://issues.apache.org/jira/browse/HDFS-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063973#comment-14063973 ] Colin Patrick McCabe commented on HDFS-6588: bq. Rerunning the failed tests

[jira] [Commented] (HDFS-6690) Deduplicate xattr names in memory

[ https://issues.apache.org/jira/browse/HDFS-6690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064219#comment-14064219 ] Colin Patrick McCabe commented on HDFS-6690: +1, assuming jenkins failures are

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064241#comment-14064241 ] Colin Patrick McCabe commented on HDFS-6658: bq. I'm not totally convinced of

[jira] [Commented] (HDFS-6695) Investigate using Java 7's nonblocking file I/O in BlockReaderLocal to implement read timeouts

[ https://issues.apache.org/jira/browse/HDFS-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065288#comment-14065288 ] Colin Patrick McCabe commented on HDFS-6695: bq. I did not look through

[jira] [Commented] (HDFS-6482) Use block ID-based block layout on datanodes

[ https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14066988#comment-14066988 ] Colin Patrick McCabe commented on HDFS-6482: 1 second for 100k blocks is pretty

[jira] [Commented] (HDFS-6482) Use block ID-based block layout on datanodes

[ https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14066995#comment-14066995 ] Colin Patrick McCabe commented on HDFS-6482: So, when I try to apply your patch

[jira] [Commented] (HDFS-6482) Use block ID-based block layout on datanodes

[ https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14066997#comment-14066997 ] Colin Patrick McCabe commented on HDFS-6482: and also patch fails when you get

[jira] [Commented] (HDFS-6707) Intermittent failure of Symlink tests TestSymlinkLocalFSFileContext,TestSymlinkLocalFSFileSystem

[ https://issues.apache.org/jira/browse/HDFS-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067077#comment-14067077 ] Colin Patrick McCabe commented on HDFS-6707: I hope I didn't miss anything when

[jira] [Created] (HDFS-6709) Implement off-heap data structures for NameNode and other HDFS memory optimization

Colin Patrick McCabe created HDFS-6709: -- Summary: Implement off-heap data structures for NameNode and other HDFS memory optimization Key: HDFS-6709 URL: https://issues.apache.org/jira/browse/HDFS-6709

[jira] [Updated] (HDFS-6709) Implement off-heap data structures for NameNode and other HDFS memory optimization

[ https://issues.apache.org/jira/browse/HDFS-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6709: --- Attachment: HDFS-6709.001.patch This patch gives an idea of how we can use off-heap

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067231#comment-14067231 ] Colin Patrick McCabe commented on HDFS-6658: bq. Separately, I'll open a JIRA

[jira] [Commented] (HDFS-6570) add api that enables checking if a user has certain permissions on a file

[ https://issues.apache.org/jira/browse/HDFS-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068894#comment-14068894 ] Colin Patrick McCabe commented on HDFS-6570: bq. acl.proto: I'm not sure it's

[jira] [Commented] (HDFS-6707) Intermittent failure of Symlink tests TestSymlinkLocalFSFileContext,TestSymlinkLocalFSFileSystem

[ https://issues.apache.org/jira/browse/HDFS-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068935#comment-14068935 ] Colin Patrick McCabe commented on HDFS-6707: Good find, Yongjun. Parsing the

[jira] [Commented] (HDFS-6707) Intermittent failure of Symlink tests TestSymlinkLocalFSFileContext,TestSymlinkLocalFSFileSystem

[ https://issues.apache.org/jira/browse/HDFS-6707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068990#comment-14068990 ] Colin Patrick McCabe commented on HDFS-6707: +1 once you add a unit test to

[jira] [Commented] (HDFS-374) HDFS needs to support a very large number of open files.

[ https://issues.apache.org/jira/browse/HDFS-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069139#comment-14069139 ] Colin Patrick McCabe commented on HDFS-374: --- oh, and also, using short-circuit

[jira] [Commented] (HDFS-374) HDFS needs to support a very large number of open files.

[ https://issues.apache.org/jira/browse/HDFS-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069136#comment-14069136 ] Colin Patrick McCabe commented on HDFS-374: --- This is less of an issue than it once

[jira] [Commented] (HDFS-465) in branch-1, libhdfs makes jni lib calls after setting errno in some places

[ https://issues.apache.org/jira/browse/HDFS-465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069184#comment-14069184 ] Colin Patrick McCabe commented on HDFS-465: --- Yeah. This was fixed in HDFS-3579.

[jira] [Commented] (HDFS-97) DFS should detect slow links(nodes) and avoid them

[ https://issues.apache.org/jira/browse/HDFS-97?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069192#comment-14069192 ] Colin Patrick McCabe commented on HDFS-97: -- This is also related to hedged reads.

[jira] [Commented] (HDFS-6482) Use block ID-based block layout on datanodes

[ https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069631#comment-14069631 ] Colin Patrick McCabe commented on HDFS-6482: bq. One thread per storage

[jira] [Commented] (HDFS-6709) Implement off-heap data structures for NameNode and other HDFS memory optimization

[ https://issues.apache.org/jira/browse/HDFS-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070637#comment-14070637 ] Colin Patrick McCabe commented on HDFS-6709: The {{ByteBuffer}} interface

[jira] [Commented] (HDFS-6709) Implement off-heap data structures for NameNode and other HDFS memory optimization

[ https://issues.apache.org/jira/browse/HDFS-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072197#comment-14072197 ] Colin Patrick McCabe commented on HDFS-6709: If Unsafe is removed, then we'll

[jira] [Commented] (HDFS-6114) Block Scan log rolling will never happen if blocks written continuously leading to huge size of dncp_block_verification.log.curr

[ https://issues.apache.org/jira/browse/HDFS-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072218#comment-14072218 ] Colin Patrick McCabe commented on HDFS-6114: +1. Thanks, Vinayakumar. Block

[jira] [Commented] (HDFS-6698) try to optimize DFSInputStream.getFileLength()

[ https://issues.apache.org/jira/browse/HDFS-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072229#comment-14072229 ] Colin Patrick McCabe commented on HDFS-6698: Changing the locking model for the

[jira] [Commented] (HDFS-6735) A minor optimization to avoid pread() be blocked by read() inside the same DFSInputStream

[ https://issues.apache.org/jira/browse/HDFS-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072228#comment-14072228 ] Colin Patrick McCabe commented on HDFS-6735: Changing the locking model for the

[jira] [Commented] (HDFS-4257) The ReplaceDatanodeOnFailure policies could have a forgiving option

[ https://issues.apache.org/jira/browse/HDFS-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072325#comment-14072325 ] Colin Patrick McCabe commented on HDFS-4257: Nicholas, thank you for looking at

[jira] [Updated] (HDFS-6114) Block Scan log rolling will never happen if blocks written continuously leading to huge size of dncp_block_verification.log.curr

[ https://issues.apache.org/jira/browse/HDFS-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6114: --- Resolution: Fixed Fix Version/s: 2.6.0 Target Version/s: 2.6.0

[jira] [Commented] (HDFS-6698) try to optimize DFSInputStream.getFileLength()

[ https://issues.apache.org/jira/browse/HDFS-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072489#comment-14072489 ] Colin Patrick McCabe commented on HDFS-6698: {code} + public long

[jira] [Commented] (HDFS-6709) Implement off-heap data structures for NameNode and other HDFS memory optimization

[ https://issues.apache.org/jira/browse/HDFS-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073529#comment-14073529 ] Colin Patrick McCabe commented on HDFS-6709: bq. I thought RTTI is per class,

[jira] [Created] (HDFS-6750) The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale

Colin Patrick McCabe created HDFS-6750: -- Summary: The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale Key: HDFS-6750 URL:

[jira] [Updated] (HDFS-6750) The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale

[ https://issues.apache.org/jira/browse/HDFS-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6750: --- Status: Patch Available (was: Open) The DataNode should use its shared memory

[jira] [Updated] (HDFS-6750) The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale

[ https://issues.apache.org/jira/browse/HDFS-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6750: --- Attachment: HDFS-6750.001.patch The DataNode should use its shared memory segment to

[jira] [Commented] (HDFS-6750) The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale

[ https://issues.apache.org/jira/browse/HDFS-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074587#comment-14074587 ] Colin Patrick McCabe commented on HDFS-6750: Test failures are unrelated.

[jira] [Commented] (HDFS-6729) Support maintenance mode for DN

[ https://issues.apache.org/jira/browse/HDFS-6729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074651#comment-14074651 ] Colin Patrick McCabe commented on HDFS-6729: By default it takes 10 and a half

[jira] [Commented] (HDFS-583) HDFS should enforce a max block size

[ https://issues.apache.org/jira/browse/HDFS-583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074690#comment-14074690 ] Colin Patrick McCabe commented on HDFS-583: --- Most places where we refer to block

[jira] [Commented] (HDFS-6755) Make DFSOutputStream more efficient

[ https://issues.apache.org/jira/browse/HDFS-6755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074717#comment-14074717 ] Colin Patrick McCabe commented on HDFS-6755: The code here is using exponential

[jira] [Commented] (HDFS-6755) Make DFSOutputStream more efficient

[ https://issues.apache.org/jira/browse/HDFS-6755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074759#comment-14074759 ] Colin Patrick McCabe commented on HDFS-6755: Ah. I misinterpreted your

[jira] [Updated] (HDFS-6755) There is an unnecessary sleep in the code path where DFSOutputStream#close gives up its attempt to contact the namenode

[ https://issues.apache.org/jira/browse/HDFS-6755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6755: --- Description: DFSOutputStream#close has a loop where it tries to contact the NameNode,

[jira] [Updated] (HDFS-6755) There is an unnecessary sleep in the code path where DFSOutputStream#close gives up its attempt to contact the namenode

[ https://issues.apache.org/jira/browse/HDFS-6755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6755: --- Status: Patch Available (was: Open) There is an unnecessary sleep in the code path

[jira] [Updated] (HDFS-6755) There is an unnecessary sleep in the code path where DFSOutputStream#close gives up its attempt to contact the namenode

[ https://issues.apache.org/jira/browse/HDFS-6755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6755: --- Summary: There is an unnecessary sleep in the code path where DFSOutputStream#close

[jira] [Commented] (HDFS-223) Asynchronous IO Handling in Hadoop and HDFS

[ https://issues.apache.org/jira/browse/HDFS-223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074837#comment-14074837 ] Colin Patrick McCabe commented on HDFS-223: --- I think the existing thread-pool

[jira] [Commented] (HDFS-6755) There is an unnecessary sleep in the code path where DFSOutputStream#close gives up its attempt to contact the namenode

[ https://issues.apache.org/jira/browse/HDFS-6755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075029#comment-14075029 ] Colin Patrick McCabe commented on HDFS-6755: No new tests are needed, since

[jira] [Resolved] (HDFS-3607) log a message when fuse_dfs is not built

[ https://issues.apache.org/jira/browse/HDFS-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe resolved HDFS-3607. Resolution: Fixed Target Version/s: (was: ) log a message when

[jira] [Commented] (HDFS-3607) log a message when fuse_dfs is not built

[ https://issues.apache.org/jira/browse/HDFS-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075044#comment-14075044 ] Colin Patrick McCabe commented on HDFS-3607: We now do log a message when fuse

[jira] [Updated] (HDFS-3607) log a message when fuse_dfs is not built

[ https://issues.apache.org/jira/browse/HDFS-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-3607: --- Target Version/s: 2.0.2-alpha Fix Version/s: 2.0.2-alpha log a message when

[jira] [Commented] (HDFS-3607) log a message when fuse_dfs is not built

[ https://issues.apache.org/jira/browse/HDFS-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075060#comment-14075060 ] Colin Patrick McCabe commented on HDFS-3607: This was fixed by HADOOP-8368.

  1   2   3   4   5   6   7   8   9   10   >