Jenkins build is back to normal : Hadoop-Hdfs-trunk #1485

2013-08-08 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1485/changes

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-08-08 Thread Kihwal Lee
Another blocker, HADOOP-9850, has been committed. Kihwal From: Arun C Murthy a...@hortonworks.com To: Daryn Sharp da...@yahoo-inc.com Cc: hdfs-dev@hadoop.apache.org hdfs-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org mapreduce-...@hadoop.apache.org;

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

2013-08-08 Thread Kihwal Lee
Sorry to hijack the thread but, I also wanted to mention Avro. See HADOOP-9672. The version we are using has memory leak and inefficiency issues. We've seen users running into it. Kihwal From: Tsuyoshi OZAWA ozawa.tsuyo...@gmail.com To:

Re: Feature request to provide DFSInputStream subclassing mechanism

2013-08-08 Thread Matevz Tadel
Hi everybody, I'm jumping in as Jeff is away due to an unexpected annoyance involving Californian wildlife. On 8/7/13 7:47 PM, Andrew Wang wrote: Blocks are supposed to be an internal abstraction within HDFS, and aren't an inherent part of FileSystem (the user-visible class used to access

Re: Feature request to provide DFSInputStream subclassing mechanism

2013-08-08 Thread Colin McCabe
There is work underway to decouple the block layer and the namespace layer of HDFS from each other. Once this is done, block behaviors like the one you describe will be easy to implement. It's a use case very similar to the hierarchical storage management (HSM) use case that we've discussed

Re: Feature request to provide DFSInputStream subclassing mechanism

2013-08-08 Thread Steve Loughran
On 7 August 2013 10:59, Jeff Dost jd...@ucsd.edu wrote: Hello, We work in a software development team at the UCSD CMS Tier2 Center. We would like to propose a mechanism to allow one to subclass the DFSInputStream in a clean way from an external package. First I'd like to give some

Re: Feature request to provide DFSInputStream subclassing mechanism

2013-08-08 Thread Suresh Srinivas
This is being targeted for release 2.3. 2.1.x release stream is for stabilizing. When it reaches stability, 2.2 GA will be released. The current features in development will make it to 2.3, including HDFS-2832. On Thu, Aug 8, 2013 at 2:04 PM, Matevz Tadel mta...@ucsd.edu wrote: Thanks Colin,

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

2013-08-08 Thread Elliott Clark
In HBase land we've pretty well discovered that we'll need to have the same version of protobuf that the HDFS/Yarn/MR servers are running. That is to say there are issues with ever having 2.4.x and 2.5.x on the same class path. Upgrading to 2.5.x would be great, as it brings some new classes we

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

2013-08-08 Thread Alejandro Abdelnur
To move forward with this we need protoc 2.5.0 in the apache hadoop jenkins boxes. Who can help with this? I assume somebody at Y!, right? Thx On Thu, Aug 8, 2013 at 2:24 PM, Elliott Clark ecl...@apache.org wrote: In HBase land we've pretty well discovered that we'll need to have the same

[jira] [Created] (HDFS-5080) BootstrapStandby not working with QJM when the existing NN is active

2013-08-08 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-5080: --- Summary: BootstrapStandby not working with QJM when the existing NN is active Key: HDFS-5080 URL: https://issues.apache.org/jira/browse/HDFS-5080 Project: Hadoop HDFS

[jira] [Created] (HDFS-5081) DistributedFileSystem#listStatus() throws FileNotFoundException when target of symlink doesn't exist

2013-08-08 Thread Ted Yu (JIRA)
Ted Yu created HDFS-5081: Summary: DistributedFileSystem#listStatus() throws FileNotFoundException when target of symlink doesn't exist Key: HDFS-5081 URL: https://issues.apache.org/jira/browse/HDFS-5081

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

2013-08-08 Thread Andrew Purtell
On Thu, Aug 8, 2013 at 3:34 PM, Roman Shaposhnik r...@apache.org wrote: I'd be huge +1 on a flag day migrating us to 2.5.x. We can run a set of validation tests in Bigtop on all components that could be affected by it. Hadoop and HBase are the obvious suspects. What else in the ecosystem

Re: Feature request to provide DFSInputStream subclassing mechanism

2013-08-08 Thread Matevz Tadel
Hi Steve, Thank you very much for the reality check! Some more answers inline ... On 8/8/13 1:30 PM, Steve Loughran wrote: On 7 August 2013 10:59, Jeff Dost jd...@ucsd.edu wrote: Hello, We work in a software development team at the UCSD CMS Tier2 Center. We would like to propose a