Hi Emyr, You can use capnp::writeMessage() to write to any kind of stream.
https://github.com/capnproto/capnproto/blob/7e2f70f354e2ca3d9de73ddaf974408ddbd98866/c++/src/capnp/serialize.h#L162 This function writes to a `kj::OutputStream`, which is an abstract interface that you can implement any way you want. It only has a couple methods: https://github.com/capnproto/capnproto/blob/7e2f70f354e2ca3d9de73ddaf974408ddbd98866/c++/src/kj/io.h#L77 (capnp::writeMessage() always calls the second version of write(), which takes an array of byte arrays, to write the entire message all at once.) In fact, there's even already an implementation of kj::OutputStream that gzip-compresses the output: https://github.com/capnproto/capnproto/blob/7e2f70f354e2ca3d9de73ddaf974408ddbd98866/c++/src/kj/compat/gzip.h#L69 So you don't even need to use boost for that. -Kenton On Sun, Jul 19, 2020 at 11:08 PM <[email protected]> wrote: > Hi, > > I have an application that is currently using gzipped tab seperated fields > file. The fields are a mix of strings and ints (it's the lstat info for > every file in a filesystem). To store the full path in this file I base64 > encode it so that the TSV format doesn't break when paths contain > unprintable characters. I load this into a clickhouse database for > reporting. Having to do base64 decoding in clickhouse is a pain and slows > down the queries. > I want to switch my lstap collector to output capnp messages to avoid > having to use base64 encoding. I currently use the boost gzip filter > streams to do on the fly compression for my tsv lines. I'd like to do the > same with capnp messages but it seems the only way to write to a file is > using file descriptors. How can I write messages to a > boost::io::filtering_ostream ? > > Many thanks, > > Emyr > > -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/7b25d572-414e-481a-a9e8-587b6ed522fco%40googlegroups.com > <https://groups.google.com/d/msgid/capnproto/7b25d572-414e-481a-a9e8-587b6ed522fco%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQnG3NSqJG1%3Dhr56%3DRY%3D9N4p1WDTW-bVk1JqqB9UcvwuUg%40mail.gmail.com.
