We've definitely considered this. However, our measurements show that the performance degradation (from the extra copy) is around 2-3%. The nice thing about having the broker architecture is that it gets more people trying Hypertable in more environments, which helps shake out bugs. Another thing is that the broker provides an asynchronous interface to the FS, and most of these filesystems do not provide an asynchronous interface. This allows the application to "multiprogram", or get other work done while it is waiting for FS ops to complete.
- Doug On Thu, Jan 29, 2009 at 8:37 PM, jglim <[email protected]> wrote: > > I got it! > > BTW, I have one question. Since all data RangeServers are willing to > write pass through DfsBroker, > many superfluous memory copys occur in current architecture. For DFSes > supporting C++ client > libraries like KFS, is there any plan to make RangeServers manage > their own DFS clients, skipping the > communication with DfsBroker, for performance? > > On Jan 30, 11:33 am, Luke <[email protected]> wrote: > > > + fs().remove(path); > > > fs().rename(tmp, path); > > > } > > > > > the reason why I inserted fs().remove(path): in some filesystem, > > > renaming does not automatically remove the destination file > > > > See my previous post for a possibility of unrecoverable race > > condition. POSIX rename removes destination files. Since we have > > control over fs API. We can make sure a dfs broker removes destination > > files. > > > > __Luke > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en -~----------~----~----~----~------~----~------~--~---
