[
https://issues.apache.org/jira/browse/HDFS-10188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410424#comment-16410424
]
Hudson commented on HDFS-10188:
-------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13869 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13869/])
HDFS-10188. libhdfs++: Implement debug allocators. Contributed by
(james.clampffer: rev 93382381f698ee525c3827be28f3ff35f2bd3c60)
* (edit)
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/rpc_engine.h
* (edit)
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filehandle.h
* (edit)
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filesystem.h
* (edit)
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/reader/block_reader.h
* (add)
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/new_delete.h
* (edit)
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/connection/datanodeconnection.h
> libhdfs++: Implement debug allocators
> -------------------------------------
>
> Key: HDFS-10188
> URL: https://issues.apache.org/jira/browse/HDFS-10188
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Reporter: James Clampffer
> Assignee: Xiaowei Zhu
> Priority: Major
> Attachments: HDFS-10188.HDFS-8707.000.patch,
> HDFS-10188.HDFS-8707.001.patch, HDFS-10188.HDFS-8707.002.patch,
> HDFS-10188.HDFS-8707.003.patch, HDFS-10188.HDFS-8707.004.patch
>
>
> I propose implementing a set of memory new/delete pairs with additional
> checking to detect double deletes, read-after-delete, and write-after-deletes
> to help debug resource ownership issues and prevent new ones from entering
> the library.
> One of the most common issues we have is use-after-free issues. The
> continuation pattern makes these really tricky to debug because by the time a
> segsegv is raised the context of what has caused the error is long gone.
> The plan is to add allocators that can be turned on that can do the
> following, in order of runtime cost.
> 1: no-op, forward through to default new/delete
> 2: make sure the memory given to the constructor is dirty, memset free'd
> memory to 0
> 3: implement operator new with mmap, lock that region of memory once it's
> been deleted; obviously this can't be left to run forever because the memory
> is never unmapped
> This should also put some groundwork in place for implementing specialized
> allocators for tiny objects that we churn through like std::string.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
