I'm reaching out because I want to collect thoughts on this, but I'm going to enumerate the core points up front:
1: Code generation is better, but it's not helpful when you don't have control of the data structures being exchanged 2: It seems that the capnproto protocol could be used to encode information about a data structure that was not generated using a capnproto schema 3: If 2 is true, it seems that promise-driven capnproto RPC would be achievable for those same structures The question is: would it be worthwhile to construct a system that uses capnproto to exchange data between data structures that weren't built using a capnproto schema? Longer version: I'm working on a project that needs to exchange calls both inter-and-intra-host and needs to be able to do RPC on data structures that were not build to support RPC. For reasons, it's really hard to get the project owners to adjust their data structures (and it's not clear that doing so would provide a better customer experience anyway) If we were to imagine reflecting over any in-memory data structure, we could also imagine a system that enumerated those properties by name and type and was capable of building an in-memory parse-tree for that structure. With that, you could imagine constructing an in-memory data structure that performs the mapping that generated code would. If you had those systems, the encoder and decoder could exchange data between processes and that data could be used to map to another object that was, "close enough" (meaning that the property enumeration was the same, and the types lined up) With packing, and assuming empty buffer space, it seems like your enumeration strategy could be pretty sparse (let's assume a configurable key space of a 8-24 bit integer). If the properties weren't enumerated sequentially, but instead using some hashcode accounting for property name, property type, and nesting level, this would allow a lot of flexibility for type-to type mapping across processes. Assuming that the decoder was able to discretely specify which properties on the receiving type were candidates for data received from the message, it seems like this would be pretty secure. Ok, so now the question: does this make sense? It seems to me like this would provide a pretty generic and straightforward RPC and ser/deser solution, but am I missing something? It seems like the up-front cost of establishing the pointer maps using reflection might be high, and that cost might be incurred again whenever the key space in the messages changed, but still it seems like it helps a client accomplish a goal that was previously unachievable (sharing data that was stored in a structure it didn't own, with a system that also may not own the data structure) Do people know of another solution that achieves this? Can anyone think of something I'm just plain missing here? -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/capnproto. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/980a99b0-6fd3-4b7f-8b8b-7d515d311db9%40googlegroups.com.
