[
https://issues.apache.org/jira/browse/HDFS-10704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401181#comment-15401181
]
James Clampffer commented on HDFS-10704:
----------------------------------------
What are the main benefits of this? I have a feeling this will make the
implementation a bit more completed and a bit harder to debug. What's the
current issue preventing multiple users from using the same filesystem?
Kerberos tokens? The way I see it:
Pros:
-Can better represent some architectures which is pretty nice
-a few less io_service instances
-user explicitly manages more of the library.
Cons:
-io_service is where a lot of lock contention lives, if someone wants to avoid
it they'll end up using multiple io_services and some structure to track them
-each filesystem is most likely going to track it's worker threads anyway
-big one, as someone who's done a ton of the debugging inside asio, it makes
things harder to debug/fix
I've been thinking for a while that the RpcEngine should have it's own
asio::io_service. Most of the difficult bugs I've hit manifest as something
blowing up right around the io_service: dangling pointers to std::function
callbacks, asio throws an exeption that knocks out a worker thread, file
descriptor closed while another asio::socket object references it, so IMO more
io_services isolates issues. Performance should be pretty similar in either
case; the io_service is just a wrapper around epoll or select state + some
extra state to hold a fd condition->iovec/callback mapping. On the other hand
I'm all for moving things to requiring users to be smart about object
dependencies so we can start getting rid of shared_ptrs in other places so I'd
be happy if this pushes things in that direction.
> libhdfs++: FileSystem should not take ownership of IOService
> ------------------------------------------------------------
>
> Key: HDFS-10704
> URL: https://issues.apache.org/jira/browse/HDFS-10704
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Reporter: Bob Hansen
>
> The ctor for the filesystem currently takes ownership of the IOService passed
> in. There is a valid use case for a single IOService for multiple
> FileSystems (e.g. for different users).
> If an IOService is passed in, the consumer should be responsible for its
> lifetime.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
