[jira] [Resolved] (HDFS-4434) Provide a mapping from INodeId to INode

2013-05-02 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HDFS-4434. --- Resolution: Fixed Fix Version/s: (was: 3.0.0) 2.0.5-beta

[jira] [Created] (HDFS-4787) Create a new HdfsConfiguration before each TestDFSClientRetries testcases

2013-05-02 Thread Tian Hong Wang (JIRA)
Tian Hong Wang created HDFS-4787: Summary: Create a new HdfsConfiguration before each TestDFSClientRetries testcases Key: HDFS-4787 URL: https://issues.apache.org/jira/browse/HDFS-4787 Project:

Re: Heads up - 2.0.5-beta

2013-05-02 Thread Konstantin Shvachko
I am not sure what was your point here. You seem to be assuming things I never mentioned. I am arguing against invasive and destructive features proposed for the release. Just to remind here they are again, since the history has been wiped out. # Snapshots # NFS gateway for HDFS # HDFS-347 unix

Re: Heads up - 2.0.5-beta

2013-05-02 Thread Konstantin Shvachko
On Thu, May 2, 2013 at 12:07 AM, Chris Douglas cdoug...@apache.org wrote: Can anyone remember why we vote on release plans? -C To vote on features to include in the release. Thanks, --Konstantin

Re: Is Hadoop SequenceFile binary safe?

2013-05-02 Thread Hs
Hi Chris, thanks for your replay. That's to say, SequenceFile is probabilistically binary safe. I notice a jira issue attempting to support append in existing SequenceFile( https://issues.apache.org/jira/browse/HADOOP-7139). It occurred to me that if some hacker reads the sync marker from the

[jira] [Created] (HDFS-4788) Fix bug in bestNode function which caused 'Could not reach the block' exception even when there're nodes available

2013-05-02 Thread Yuyang Lan (JIRA)
Yuyang Lan created HDFS-4788: Summary: Fix bug in bestNode function which caused 'Could not reach the block' exception even when there're nodes available Key: HDFS-4788 URL:

Jenkins build is back to stable : Hadoop-Hdfs-0.23-Build #599

2013-05-02 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/599/

Hadoop-Hdfs-trunk - Build # 1390 - Still Failing

2013-05-02 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1390/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 10251 lines...] Running

Re: DistributedFileSystem.listStatus() - Why does it do partial listings then assemble?

2013-05-02 Thread Todd Lipcon
Hi Brad, The reasoning is that the NameNode locking is somewhat coarse grained. In older versions of Hadoop, before it worked this way, we found that listing large directories (eg with 100k+ files) could end up holding the namenode's lock for a quite long period of time and starve other clients.

Re: DistributedFileSystem.listStatus() - Why does it do partial listings then assemble?

2013-05-02 Thread Suresh Srinivas
Additional reason, HDFS does not have limit on number of files in a directory. Some clusters had millions of files in a single directory. Listing such a directory resulted in very large responses, requiring large contiguous memory allocation in JVM (for the array) and unpredictable GC failures.

[jira] [Created] (HDFS-4789) redirectToRandomDataNode tries to get a delegation token without verifying security is enabled

2013-05-02 Thread Stephen Fritz (JIRA)
Stephen Fritz created HDFS-4789: --- Summary: redirectToRandomDataNode tries to get a delegation token without verifying security is enabled Key: HDFS-4789 URL: https://issues.apache.org/jira/browse/HDFS-4789

Re: Is Hadoop SequenceFile binary safe?

2013-05-02 Thread Colin McCabe
It seems like we could just set up an escape sequence and make it actually binary-safe, rather than just probabilistically. The escape sequence would only be inserted when there would otherwise be confusion between data and a sync marker. best, Colin On Thu, May 2, 2013 at 3:26 AM, Hs

[jira] [Created] (HDFS-4790) TestFsShellReturnCode may fail in branch-1

2013-05-02 Thread Tsz Wo (Nicholas), SZE (JIRA)
Tsz Wo (Nicholas), SZE created HDFS-4790: Summary: TestFsShellReturnCode may fail in branch-1 Key: HDFS-4790 URL: https://issues.apache.org/jira/browse/HDFS-4790 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-4774) Backport HDFS-4525 'Provide an API for knowing whether file is closed or not' to branch-1

2013-05-02 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE resolved HDFS-4774. -- Resolution: Fixed Fix Version/s: 1.2.0 I have committed this. Thanks,

[jira] [Created] (HDFS-4791) Update and fix deletion of reference inode

2013-05-02 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-4791: --- Summary: Update and fix deletion of reference inode Key: HDFS-4791 URL: https://issues.apache.org/jira/browse/HDFS-4791 Project: Hadoop HDFS Issue Type: Sub-task

Re: Heads up - 2.0.5-beta

2013-05-02 Thread Arun C Murthy
Konstantin, On May 2, 2013, at 2:08 AM, Konstantin Shvachko wrote: I am arguing against invasive and destructive features proposed for the release. Just to remind here they are again, since the history has been wiped out. # Snapshots # NFS gateway for HDFS # HDFS-347 unix domain socket

[jira] [Created] (HDFS-4792) remove unused import from TestEditLog in branch-1

2013-05-02 Thread Chris Nauroth (JIRA)
Chris Nauroth created HDFS-4792: --- Summary: remove unused import from TestEditLog in branch-1 Key: HDFS-4792 URL: https://issues.apache.org/jira/browse/HDFS-4792 Project: Hadoop HDFS Issue

Fwd: Heads up - 2.0.5-beta

2013-05-02 Thread Suresh Srinivas
Konstantin, I am arguing against invasive and destructive features proposed for the release. Your choice of words is deplorable, to say the least. Can you explain what do you mean by *destructive*? Please substantiate your claim on technical grounds. So far you have been quiet while we have

[jira] [Created] (HDFS-4793) uploading file larger than the spaceQuota limit should not create 0 byte file

2013-05-02 Thread yeshavora (JIRA)
yeshavora created HDFS-4793: --- Summary: uploading file larger than the spaceQuota limit should not create 0 byte file Key: HDFS-4793 URL: https://issues.apache.org/jira/browse/HDFS-4793 Project: Hadoop HDFS

Re: Heads up - 2.0.5-beta

2013-05-02 Thread Chris Douglas
On Thu, May 2, 2013 at 2:11 AM, Konstantin Shvachko shv.had...@gmail.com wrote: On Thu, May 2, 2013 at 12:07 AM, Chris Douglas cdoug...@apache.org wrote: Can anyone remember why we vote on release plans? -C To vote on features to include in the release. Since most features are developed in

[jira] [Resolved] (HDFS-4782) backport edit log corruption toleration to branch-1-win

2013-05-02 Thread Tsz Wo (Nicholas), SZE (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE resolved HDFS-4782. -- Resolution: Fixed Fix Version/s: 1-win I have committed this. Thanks,