[jira] [Commented] (HDFS-12348) disable removing blocks to trash while rolling upgrade

[ https://issues.apache.org/jira/browse/HDFS-12348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140074#comment-16140074 ] Koji Noguchi commented on HDFS-12348: - bq. 1) The blocks in the trash can be reduced, if datanode

[jira] [Commented] (HDFS-11111) Delete something in .Trash using "rm" should be forbidden without safety option

[ https://issues.apache.org/jira/browse/HDFS-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007106#comment-16007106 ] Koji Noguchi commented on HDFS-1: - Curious. Has anyone seen users accidentally wiping the files

[jira] [Commented] (HDFS-11357) Secure Delete

[ https://issues.apache.org/jira/browse/HDFS-11357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15834779#comment-15834779 ] Koji Noguchi commented on HDFS-11357: - Andrew, are your compliance/legal people ok that there still

[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error

[ https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290236#comment-15290236 ] Koji Noguchi commented on HDFS-10400: - My point was, the INFO message on the description {noformat}

[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error

[ https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286769#comment-15286769 ] Koji Noguchi commented on HDFS-10400: - I'm confused with this jira. Do we have a test case when hdfs

[jira] [Commented] (HDFS-8131) Implement a space balanced block placement policy

[ https://issues.apache.org/jira/browse/HDFS-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493504#comment-14493504 ] Koji Noguchi commented on HDFS-8131: Thanks for correcting my misunderstanding. I

[jira] [Commented] (HDFS-8131) Implement a space balanced block placement policy

[ https://issues.apache.org/jira/browse/HDFS-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492573#comment-14492573 ] Koji Noguchi commented on HDFS-8131: Hi Liu. Have you taken a look at [~kihwal]'s

[jira] [Commented] (HDFS-1161) Make DN minimum valid volumes configurable

[ https://issues.apache.org/jira/browse/HDFS-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13096475#comment-13096475 ] Koji Noguchi commented on HDFS-1161: bq. IIRC Koji's perspective was that an admin

[jira] [Commented] (HDFS-2126) Improve Namenode startup time [umbrella task]

[ https://issues.apache.org/jira/browse/HDFS-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13060681#comment-13060681 ] Koji Noguchi commented on HDFS-2126: Is it only us(Yahoo) namenodes which hits FullGC

[jira] [Commented] (HDFS-271) Metrics on Secondary namenode's activity

[ https://issues.apache.org/jira/browse/HDFS-271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13053328#comment-13053328 ] Koji Noguchi commented on HDFS-271: --- bq. Hey Koji, does the work done in HADOOP-3741 meet

[jira] [Commented] (HDFS-107) Data-nodes should be formatted when the name-node is formatted.

[ https://issues.apache.org/jira/browse/HDFS-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13040342#comment-13040342 ] Koji Noguchi commented on HDFS-107: --- bq. If the value is set to false then we can continue

[jira] [Commented] (HDFS-2006) ability to support storing extended attributes per file

[ https://issues.apache.org/jira/browse/HDFS-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13040058#comment-13040058 ] Koji Noguchi commented on HDFS-2006: For the API, would it help/affect HADOOP-7269?

[jira] [Reopened] (HDFS-157) dfs client -ls/-lsr outofmemory when one directory contained 2 million files.

[ https://issues.apache.org/jira/browse/HDFS-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi reopened HDFS-157: --- bq. Hairong just opened a better Jira. HADOOP:6732 This didn't change the memory footprint on the client

[jira] [Commented] (HDFS-1848) Datanodes should shutdown when a critical volume fails

[ https://issues.apache.org/jira/browse/HDFS-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13026340#comment-13026340 ] Koji Noguchi commented on HDFS-1848: bq. I am surprised that this problem is not solved

[jira] [Commented] (HDFS-1848) Datanodes should shutdown when a critical volume fails

[ https://issues.apache.org/jira/browse/HDFS-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13026125#comment-13026125 ] Koji Noguchi commented on HDFS-1848: bq. It seems like a good idea for a cluster

[jira] [Commented] (HDFS-1848) Datanodes should shutdown when a critical volume fails

[ https://issues.apache.org/jira/browse/HDFS-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13022799#comment-13022799 ] Koji Noguchi commented on HDFS-1848: In our tests with disk failures, We have verified

[jira] [Commented] (HDFS-1848) Datanodes should shutdown when a critical volume fails

[ https://issues.apache.org/jira/browse/HDFS-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13022815#comment-13022815 ] Koji Noguchi commented on HDFS-1848: bq. That was the problem earlier, Koji. With the

[jira] [Commented] (HDFS-1371) One bad node can incorrectly flag many files as corrupt

[ https://issues.apache.org/jira/browse/HDFS-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13020365#comment-13020365 ] Koji Noguchi commented on HDFS-1371: Can we take a similar approach to mapred

[jira] [Commented] (HDFS-1125) Removing a datanode (failed or decommissioned) should not require a namenode restart

[ https://issues.apache.org/jira/browse/HDFS-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13014718#comment-13014718 ] Koji Noguchi commented on HDFS-1125: Does HDFS-1773 help? Removing a datanode (failed

[jira] Commented: (HDFS-1609) rmr command is not displaying any error message when a path contains wildcard characters and does not exist.

[ https://issues.apache.org/jira/browse/HDFS-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002664#comment-13002664 ] Koji Noguchi commented on HDFS-1609: I see. tcsh and bash behaves differently for

[jira] Commented: (HDFS-1609) rmr command is not displaying any error message when a path contains wildcard characters and does not exist.

[ https://issues.apache.org/jira/browse/HDFS-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12998001#comment-12998001 ] Koji Noguchi commented on HDFS-1609: If we want to follow the glob handling in unix,

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12986788#action_12986788 ] Koji Noguchi commented on HDFS-1595: bq. So this faulty node F has no problem receiving

[jira] Commented: (HDFS-1595) DFSClient may incorrectly detect datanode failure

[ https://issues.apache.org/jira/browse/HDFS-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12985929#action_12985929 ] Koji Noguchi commented on HDFS-1595: Can we describe the problem users faced before

[jira] Commented: (HDFS-107) Data-nodes should be formatted when the name-node is formatted.

[ https://issues.apache.org/jira/browse/HDFS-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12980220#action_12980220 ] Koji Noguchi commented on HDFS-107: --- bq. The second approach looks fine to me. Second

[jira] Commented: (HDFS-1526) Dfs client name for a map/reduce task should have some randomness

[ https://issues.apache.org/jira/browse/HDFS-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966579#action_12966579 ] Koji Noguchi commented on HDFS-1526: Hairong, using a random number (with time as a

[jira] Created: (HDFS-1371) One bad node can incorrectly flag many files as corrupt

One bad node can incorrectly flag many files as corrupt --- Key: HDFS-1371 URL: https://issues.apache.org/jira/browse/HDFS-1371 Project: Hadoop HDFS Issue Type: Bug Components:

[jira] Commented: (HDFS-1371) One bad node can incorrectly flag many files as corrupt

[ https://issues.apache.org/jira/browse/HDFS-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12905952#action_12905952 ] Koji Noguchi commented on HDFS-1371: (you guys are too fast. I wanted the description

[jira] Commented: (HDFS-1158) HDFS-457 increases the chances of losing blocks

[ https://issues.apache.org/jira/browse/HDFS-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12905130#action_12905130 ] Koji Noguchi commented on HDFS-1158: bq. So this should just be closed then, right?

[jira] Commented: (HDFS-1094) Intelligent block placement policy to decrease probability of block loss

[ https://issues.apache.org/jira/browse/HDFS-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12882798#action_12882798 ] Koji Noguchi commented on HDFS-1094: Just curious. Are you getting missing/corrupt

[jira] Updated: (HDFS-1158) HDFS-457 increases the chances of losing blocks

[ https://issues.apache.org/jira/browse/HDFS-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HDFS-1158: --- Priority: Major (was: Critical) Lowering priority. As long as HDFS-1161 makes it to 0.21, this is

[jira] Commented: (HDFS-1161) Make DN minimum valid volumes configurable

[ https://issues.apache.org/jira/browse/HDFS-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12868705#action_12868705 ] Koji Noguchi commented on HDFS-1161: Thanks Eli, I think this would solve my worry

[jira] Created: (HDFS-1158) HDFS-457 increases the chances of losing blocks

HDFS-457 increases the chances of losing blocks Key: HDFS-1158 URL: https://issues.apache.org/jira/browse/HDFS-1158 Project: Hadoop HDFS Issue Type: Bug Components: data-node

[jira] Commented: (HDFS-1143) Implement Background deletion

[ https://issues.apache.org/jira/browse/HDFS-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12867935#action_12867935 ] Koji Noguchi commented on HDFS-1143: bq. Hey Koji, bq. HDFS-173 help other clients

[jira] Commented: (HDFS-1143) Implement Background deletion

[ https://issues.apache.org/jira/browse/HDFS-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12867201#action_12867201 ] Koji Noguchi commented on HDFS-1143: HDFS-173 didn't get us enough speedup?

[jira] Resolved: (HDFS-157) dfs client -ls/-lsr outofmemory when one directory contained 2 million files.

[ https://issues.apache.org/jira/browse/HDFS-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved HDFS-157. --- Resolution: Duplicate Hairong just opened a better Jira. HADOOP:6732 Thanks! dfs client -ls/-lsr

[jira] Resolved: (HDFS-1102) HftpFileSystem : errors during transfer result in truncated transfer

[ https://issues.apache.org/jira/browse/HDFS-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved HDFS-1102. Resolution: Duplicate Duplicate of HDFS-1085. It would be nice if we can fix this without

[jira] Commented: (HDFS-870) Topology is permanently cached

[ https://issues.apache.org/jira/browse/HDFS-870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12838583#action_12838583 ] Koji Noguchi commented on HDFS-870: --- If we're going to add this, please make it optional.

[jira] Resolved: (HDFS-23) many filelist calls to large directory killing the namenode

[ https://issues.apache.org/jira/browse/HDFS-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved HDFS-23. -- Resolution: Duplicate I'm hoping HDFS-985 is our answer. No more chasing down users hitting namenode with

[jira] Updated: (HDFS-761) Failure to process rename operation from edits log due to quota verification

[ https://issues.apache.org/jira/browse/HDFS-761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HDFS-761: -- Release Note: An error checking quota policy resulted in a failure to read the edits log, stopping the

[jira] Commented: (HDFS-173) Recursively deleting a directory with millions of files makes NameNode unresponsive for other commands until the deletion completes

[ https://issues.apache.org/jira/browse/HDFS-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12743285#action_12743285 ] Koji Noguchi commented on HDFS-173: --- bq. A related problem is that a Namenode server

[jira] Commented: (HDFS-525) ListPathsServlet.java uses static SimpleDateFormat that has threading issues

[ https://issues.apache.org/jira/browse/HDFS-525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12741394#action_12741394 ] Koji Noguchi commented on HDFS-525: --- Suresh, should we re-throw the exception after

[jira] Commented: (HDFS-531) Renaming of configuration keys

[ https://issues.apache.org/jira/browse/HDFS-531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12740226#action_12740226 ] Koji Noguchi commented on HDFS-531: --- Are we going to keep backward compatibility with