[
https://issues.apache.org/jira/browse/HDFS-8766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14964024#comment-14964024
]
James Clampffer commented on HDFS-8766:
---------------------------------------
Thanks Bob and Haohui for the feedback.
Bob, I'll make those changes first so we can get that landed, then right after
do the following:
1) I'm going to open up another jira for cli utilities in an effort to
eventually make a c++ implementation of the hadoop CLI tools. I'll turn this
stub into a unix-like cat executable so that it can be used for other things as
well. Then move onto stat and other simple fs utils.
2) I can get rid of the INVALIDATE_MEMBER_PTR macro and use unique pointers,
but I'd like to add a deleter to the smart pointers that nulls the pointer
value on destruction. This sort of debugging tool would be more useful in the
pipeline and block reader implementations because those are where most of the
dangling pointers and references tend to originate.
3) Opened HDFS-9265 for this. It's more broken than I originally thought which
explains all the double deletes that often manifest as calls to pure virtual
methods I've been seeing in crashes. Inputstream's state owns a unique_ptr to
the reader, and the reader is handing out shared_ptrs of itself to the
continuation pipeline. So it's double deleting by design rather than a memory
stomp. Fixing the make_shared_from_this usage alone isn't enough. I can take
a stab at refactoring and fixing this sometime this week.
4) This is the conventional posix signature, this was intentional because
google's guide says follow existing conventions if they are already well known.
5) Agree with Bob, he's already going to be refactoring a lot so I'll keep
everything in the same place and let him move that.
So the patch will cut out the c_api_test executable (1), add unique_ptrs (2) +
get rid of INVALIDATE_MEMBER_PTR, omit (3) because it's in another jira, not
change (4) because it falls into "exceptions to naming rules", and not change
(5) because it's going to be handled in HDFS-9144. I'll also get rid of hdfs.h
and use the one in the tree.
Haohui, can you think of anything else I'm going to need to change? I think
the current implementation, once rebased, should be sufficient to get on the
development branch so Bob can begin HDFS-9144, and then I'll submit another
patch with the changes I mentioned above tomorrow or Wednesday.
> Implement a libhdfs(3) compatible API
> -------------------------------------
>
> Key: HDFS-8766
> URL: https://issues.apache.org/jira/browse/HDFS-8766
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Reporter: James Clampffer
> Assignee: James Clampffer
> Attachments: HDFS-8766.HDFS-8707.000.patch,
> HDFS-8766.HDFS-8707.001.patch, HDFS-8766.HDFS-8707.002.patch,
> HDFS-8766.HDFS-8707.003.patch, HDFS-8766.HDFS-8707.004.patch,
> HDFS-8766.HDFS-8707.005.patch, HDFS-8766.HDFS-8707.006.patch
>
>
> Add a synchronous API that is compatible with the hdfs.h header used in
> libhdfs and libhdfs3. This will make it possible for projects using
> libhdfs/libhdfs3 to relink against libhdfspp with minimal changes.
> This also provides a pure C interface that can be linked against projects
> that aren't built in C++11 mode for various reasons but use the same
> compiler. It also allows many other programming languages to access
> libhdfspp through builtin FFI interfaces.
> The libhdfs API is very similar to the posix file API which makes it easier
> for programs built using posix filesystem calls to be modified to access HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
