Some operating systems (*cough* Windows *cough*) don't support non-blocking file (or inter-process pipe) I/O without some unreliable hacks. AFAICT.
~John On Wed, Mar 3, 2021 at 11:02 AM pepijn de vos <[email protected]> wrote: > Oh I just went hunting for the asynch bits and in the header it actually > say there is no such thing as async file IO > > // > ======================================================================================= > // The filesystem API > // > // This API is strictly synchronous because, unfortunately, there's no > such thing as asynchronous > // filesystem access in practice. The filesystem drivers on Linux are > written to assume they can > // block. The AIO API is only actually asynchronous for reading/writing > the raw file blocks, but if > // the filesystem needs to be involved (to allocate blocks, update > metadata, etc.) that will block. > // It's best to imagine that the filesystem is just another tier of > memory that happens to be > // slower than RAM (which is slower than L3 cache, which is slower than > L2, which is slower than > // L1). You can't do asynchronous RAM access so why asynchronous > filesystem? The only way to > // parallelize these is using threads. > // > // All KJ filesystem objects are thread-safe, and so all methods are > marked "const" (even write > // methods). Of course, if you concurrently write the same bytes of a > file from multiple threads, > // it's unspecified which write will "win". > > On Wed, Mar 3, 2021 at 7:57 PM pepijn de vos <[email protected]> > wrote: > >> Hey Kenton, >> >> Thanks for the answer, and sorry my frustration in trying to find out how >> to do async file IO in cap'n proto came across rude. >> >> What puzzles me is your suggestion that I don't need to use KJ. >> What puzzles me even more is that the KJ file I just obtained seems to >> not support async operations at all. >> Are you suggesting it's actually fine to just do blocking IO in the RPC >> eventloop? >> Is there some KJ thing to do async file IO, or a way to use other >> libraries for async file IO with the KJ eventloop? >> >> Thanks to all the people working on cap'n proto, it seems pretty neat :) >> >> Cheers, >> Pepijn >> >> On Wed, Mar 3, 2021 at 4:42 PM Kenton Varda <[email protected]> >> wrote: >> >>> Hi Pepijn, >>> >>> The full comment you refer to says: >>> >>> // DO NOT CALL THIS except at the top level of your program, e.g. in >>> main(). Anywhere else, you >>> // should instead have your caller pass in a Filesystem object, or a >>> specific Directory object, >>> // or whatever it is that your code needs. This ensures that your code >>> supports dependency >>> // injection, which makes it more reusable and testable. >>> >>> As the comment says, the idea is that you construct your Filesystem >>> object at the top level of your program, then you pass it (or specific File >>> or Directory objects) down to whatever parts of your program need it. That >>> ensures that those parts of your program can easily be unit-tested against >>> a fake in-memory filesystem, by swapping out the objects that you pass >>> down. This is trying to help you make your code more flexible, but you can >>> ignore it if you want. >>> >>> You also don't have to use the KJ filesystem API. In fact, nothing else >>> in KJ or Cap'n Proto cares what you use to open files. KJ is a toolkit, >>> meaning it provides a bunch of tools, but doesn't force you to use them if >>> you don't want to. >>> >>> KJ documentation can be found here: >>> https://github.com/capnproto/capnproto/tree/master/kjdoc >>> >>> Please note that Cap'n Proto and KJ are open source software provided >>> for free. I'd like them to be useful but I don't get any actual benefit >>> from you using them, and I'm not interested in helping people who are rude. >>> So, if you have further questions, please try to tone it down a bit. Thanks. >>> >>> -Kenton >>> >>> On Wed, Mar 3, 2021 at 5:54 AM [email protected] <[email protected]> >>> wrote: >>> >>>> Hey, >>>> >>>> I'm just getting started and pretty confused. >>>> The RPC server is all async and stuff, so no blocking IO in there. >>>> Then there are some vague mentions that you should use KJ for most >>>> things. >>>> But KJ documentation is nowhere to be found... >>>> >>>> All I want to do is just open a file and write to it. >>>> After resorting to reading the source code, I found >>>> >>>> https://github.com/capnproto/capnproto/blob/master/c%2B%2B/src/kj/filesystem.h >>>> So I managed to create a path so far. >>>> Then I started to look for how to open a file. >>>> I found some methods on Directory. >>>> So then I started to look how to make a directory. >>>> Then I found a Filesystem, which is abstract. >>>> Then I found newDiskFilesystem which tells me >>>> DO NOT CALL THIS except in main() >>>> Well **** how am I supposed to open a file inside my server handler >>>> then?? >>>> >>>> Pepijn >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Cap'n Proto" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/capnproto/cd5ce66c-b12e-4612-b383-32462f047f69n%40googlegroups.com >>>> <https://groups.google.com/d/msgid/capnproto/cd5ce66c-b12e-4612-b383-32462f047f69n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/CANPfQguo2bq4G0tWPeAgzphdSpU4isMkxRUzNmxjCjyvvaxO0w%40mail.gmail.com > <https://groups.google.com/d/msgid/capnproto/CANPfQguo2bq4G0tWPeAgzphdSpU4isMkxRUzNmxjCjyvvaxO0w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAOjmg%3DvK_NjbuD%2B2bYv26xEc5abd9faT1QOLD79%3D%3DBTRV9_shA%40mail.gmail.com.
