I think that optimal way which you can use here is ability to pass to a messagereader custom external segments provider. And that custom provider can implement lazy loading and should be thread safe. And by default the reader should use internal simpler segments provider which assumes that all segments are here and does not use locks.
Lazy loading.. may be it can be used for use capnproto as storage format for binary log files. When you append log records one by one and then mmap the file in the viewer. But i'm not sure. I can't explain you haw to build scheme with one we linked list of capnproto structures based on such lazy reader. :) пятница, 11 ноября 2016 г., 20:47:01 UTC+2 пользователь Kenton Varda написал: > > On Fri, Nov 11, 2016 at 8:03 AM, Vitaliy Bondarchuk < > [email protected] <javascript:>> wrote: > >> Honestly i think that even if there is a case when you have to use a >> reader from multiple threads I don't clearly understand why you use locks >> (by the way - exclusive lock here) in the ReaderArena::tryGetSegment >> method. >> imho, may be if it possible it's better to initialize reader in his >> constructor and then keep it immutable. >> Or. May be it will be nice if even on Vista and later I can create >> single-thread reader (may by by optional parameter in constructor). So the >> reader will not spend time for unnecessary lock functions >> > > The issue here is lazy loading. MessageReader can lazily load segments, so > that you can start processing the data in the first segment before the > other segments have been received. It's important for lazily-initialized > data structures to be thread-safe since programmers would reasonably expect > that the structure is immutable and therefore would expect it to be safe to > use in multiple threads concurrently. > > That said, this lazy-loading feature of MessageReader is probably a > misfeature and we should probably get rid of it. When reading from disk, > it's much cleaner to use mmap() to get the same effect. When reading from a > network, lazy loading is a bad idea because as soon as you try to follow a > pointer to a later segment it will block waiting for the segment to arrive, > which could be never. As a result we never actually use lazy loading for > network messages in practice. > > So, I think we'll remove this feature at some point. > > -Kenton > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/capnproto.
