[
https://issues.apache.org/jira/browse/HDFS-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726141#action_12726141
]
Todd Lipcon commented on HDFS-417:
----------------------------------
Woops, sorry, didn't realize I hadn't updated the JIRA a couple weeks ago.
With the current feature set it's possible to do so, since the NN plugin only
accesses the NameNode by making calls on it that are available over RPC. Doing
so would probably be a day's work, so I'd like to do it as a separate JIRA
later on. Additionally, I'm worried that making this architecture change will
tie our hands in the future if we want to make any calls from the NN into the
Plugin using hooks like in HADOOP-5640.
I think we had talked offline and decided that, as long as the old Thrift
server remains in tact, your use case won't be disturbed for now. Is that OK?
> Improvements to Hadoop Thrift bindings
> --------------------------------------
>
> Key: HDFS-417
> URL: https://issues.apache.org/jira/browse/HDFS-417
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: contrib/thriftfs
> Environment: Tested under Linux x86-64
> Reporter: Carlos Valiente
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: all.diff, BlockManager.java, build_xml.diff,
> DefaultBlockManager.java, DFSBlockManager.java, gen.diff,
> hadoop-4707-31c331.patch.gz, HADOOP-4707-55c046a.txt, hadoop-4707-6bc958.txt,
> hadoop-4707-867f26.txt.gz, HADOOP-4707.diff, HADOOP-4707.patch,
> HADOOP-4707.patch, hadoopfs_thrift.diff, hadoopthriftapi.jar,
> HadoopThriftServer.java, HadoopThriftServer_java.diff, hdfs.py,
> hdfs_py_venky.diff, libthrift.jar, libthrift.jar, libthrift.jar, libthrift.jar
>
>
> I have made the following changes to hadoopfs.thrift:
> # Added namespaces for Python, Perl and C++.
> # Renamed parameters and struct members to camelCase versions to keep them
> consistent (in particular FileStatus{blockReplication,blockSize} vs
> FileStatus.{block_replication,blocksize}).
> # Renamed ThriftHadoopFileSystem to FileSystem. From the perspective of a
> Perl/Python/C++ user, 1) it is already clear that we're using Thrift, and 2)
> the fact that we're dealing with Hadoop is already explicit in the namespace.
> The usage of generated code is more compact and (in my opinion) clearer:
> {quote}
> *Perl*:
> use HadoopFS;
> my $client = HadoopFS::FileSystemClient->new(..);
> _instead of:_
> my $client = HadoopFS::ThriftHadoopFileSystemClient->new(..);
> *Python*:
> from hadoopfs import FileSystem
> client = FileSystem.Client(..)
> _instead of_
> from hadoopfs import ThriftHadoopFileSystem
> client = ThriftHadoopFileSystem.Client(..)
> (See also the attached diff [^scripts_hdfs_py.diff] for the
> new version of 'scripts/hdfs.py').
> *C++*:
> hadoopfs::FileSystemClient client(..);
> _instead of_:
> hadoopfs::ThriftHadoopFileSystemClient client(..);
> {quote}
> # Renamed ThriftHandle to FileHandle: As in 3, it is clear that we're dealing
> with a Thrift object, and its purpose (to act as a handle for file
> operations) is clearer.
> # Renamed ThriftIOException to IOException, to keep it simpler, and
> consistent with MalformedInputException.
> # Added explicit version tags to fields of ThriftHandle/FileHandle, Pathname,
> MalformedInputException and ThriftIOException/IOException, to improve
> compatibility of existing clients with future versions of the interface which
> might add new fields to those objects (like stack traces for the exception
> types, for instance).
> Those changes are reflected in the attachment [^hadoopfs_thrift.diff].
> Changes in generated Java, Python, Perl and C++ code are also attached in
> [^gen.diff]. They were generated by a Thrift checkout from trunk
> ([http://svn.apache.org/repos/asf/incubator/thrift/trunk/]) as of revision
> 719697, plus the following Perl-related patches:
> * [https://issues.apache.org/jira/browse/THRIFT-190]
> * [https://issues.apache.org/jira/browse/THRIFT-193]
> * [https://issues.apache.org/jira/browse/THRIFT-199]
> The Thrift jar file [^libthrift.jar] built from that Thrift checkout is also
> attached, since it's needed to run the Java Thrift server.
> I have also added a new target to src/contrib/thriftfs/build.xml to build the
> Java bindings needed for org.apache.hadoop.thriftfs.HadoopThriftServer.java
> (see attachment [^build_xml.diff] and modified HadoopThriftServer.java to
> make use of the new bindings (see attachment [^HadoopThriftServer_java.diff]).
> The jar file [^lib/hadoopthriftapi.jar] is also included, although it can be
> regenerated from the stuff under 'gen-java' and the new 'compile-gen' Ant
> target.
> The whole changeset is also included as [^all.diff].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
