Hi Jonathan, I suppose the first question is: Does your database actually need to understand the dynamic fields at all? Or does it just need to be able to echo them back to the client later on, with only the client actually understanding them?
If only the client really needs to know about them, then I'd suggest using an `AnyPointer` field which the client fills in however they want. (You could also use generics for the same effect wrapped in nicer PL theory.) If the server needs to understand the fields -- perhaps, to index them, or something -- then one possibility is the client could upload its schema to the server in capnp schema format (schema.capnp). On the server, you could use the "dynamic API" (in C++, capnp/dynamic.h; some other implementations support it as well but not all) to load this schema dynamically and introspect the messages based on it. Alternatively, you could of course just have the client send names and values, as you suggest. Though at that point you might even consider just using JSON (or one of the myriad binary alternative encodings of JSON) as the Cap'n Proto encoding may actually be adding more bloat than it saves. It's hard to say what's the best approach without really understanding the details of the application, though. -Kenton On Sun, Sep 17, 2023 at 5:04 PM Jonathan Shapiro <s...@buttonsmith.com> wrote: > We're working on multiple applications with dynamic schemas: schemas where > customers need to be able to add or delete fields to meet their particular > requirements. This sort of thing impacts both the database layer and the > wire schema. I can think of an implementation that doesn't suck (see > below), I'm wondering what the best way would be to handle this sort of > thing in capnproto. > > The doesn't suck approach: > > In this application we can restrict in advance what the valid types are > for these new columns, which would allow us to handle the dynamic fields as > a list struct/union values, each carrying its field name as a string or > something similar. It's not an especially elegant way of describing the > protocol, but it covers the use cases. It isn't *necessary* that the bits > on the wire be "pretty", it's reasonably easy to decode into JavaScript > objects on the client, and it doesn't seem like a huge lift on the server > to validate that all expected extension fields are present and have values > of the expected type. > > Is there a better way to think about this? > > > Jonathan > > -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to capnproto+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/CAJdcQk3s9jnXR6VJEyhXSFHPO0qNNBBnRrm7A3Pr%2BukU5Q9DmQ%40mail.gmail.com > <https://groups.google.com/d/msgid/capnproto/CAJdcQk3s9jnXR6VJEyhXSFHPO0qNNBBnRrm7A3Pr%2BukU5Q9DmQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQkkAuc06H2ZzJuUPb8ir2M%2BzoAax0vXVuWUTOn2WdpsGg%40mail.gmail.com.
