[capnproto] Re: Schemas on the wire Dynamic API

2023-02-27 Thread Dane
A follow up after simplifying the API to the core issue:

The data flow is now:
1. A client connects to a server, passing a Schema.Node as a parameter. 
2. A server loads the Schema.Node and creates a new message of the given 
schema, which is returned as an AnyStruct

Problem 1 above still exists.

We have figured out how to handle point 2, but not without a malloc. How 
can I init the result as a DynamicStruct, instead of copying as in the 
below snippet:
auto message = context.getParams();
auto schema_ = message.getSchema();
auto schema_loader = capnp::SchemaLoader();
auto loaded_schema = schema_loader.load(schema_);

// how to do below without malloc?

// cannot init result as a DynamicStruct
auto mb = capnp::MallocMessageBuilder();
auto msg = mb.initRoot(loaded_schema.asStruct());
msg.set("text", "abc");

auto results = context.initResults();
results.setValue(msg.asReader());


Would this work as above if we had nested structs? 

Aside: I am also trying to replicate this in pycapnp, which is proving 
deeply problematic since it skips out on a lot of ported functionality from 
capnp.

- Dane

On Monday, 27 February 2023 at 11:34:50 UTC+2 Dane wrote:

> Hi 
>
> I am trying to implement a dynamic API. I have followed the suggestions in 
> https://groups.google.com/g/capnproto/c/Ivl3u4JJoEQ/m/MIH2a2nPBQAJ and 
> https://groups.google.com/g/capnproto/c/fNnr3Om9haM/m/1TdBqwsNBwAJ
>
> My implementation attempt lies at 
> https://github.com/DaneSlattery/capnp_generic_poc .
>
> The desired data flow is as follows:
>
> 1. A client connects to a server, and receives a capability that allows 
> the client to load a schema (the InterfaceLoader). When connecting, the 
> client also passes a ClientInterface to the server.
> 2. The client registers a schema using the InterfaceLoader. The client 
> must send a Schema.Node parameter to the server. The server must load the 
> schema.
> 3. The server uses the given loaded schema to communicate with the client 
> interface, which takes a parameter of AnyStruct.
>
> I have a few problems:
>
> Firstly, when the client attempts to set the schema using  
> capnp::Schema::from().getProto() , line 683 of 
> capnproto-c++-0.10.3/src/capnp/dynamic.c++ blows up. I have changed this 
> line to, instead assessing the ID of the schema for equality :
> KJ_REQUIRE(structValue.getSchema().getProto().getId() == 
> structType.getProto().getId(), "Value type mismatch.")
> {
> return;
> }
> I think this change should perhaps go in a PR, unless there is another, 
> better way.
>
> Secondly, I can't figure out how to send messages of the loaded schema 
> back from the server to the client. Can anyone see if the POC linked is 
> viable, and how to get to the final step?
>
> - Dane
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/capnproto/568ea931-e1b3-4ed5-80d2-1074cf6cbce7n%40googlegroups.com.


[capnproto] Schemas on the wire Dynamic API

2023-02-27 Thread Dane
Hi 

I am trying to implement a dynamic API. I have followed the suggestions in 
https://groups.google.com/g/capnproto/c/Ivl3u4JJoEQ/m/MIH2a2nPBQAJ and 
https://groups.google.com/g/capnproto/c/fNnr3Om9haM/m/1TdBqwsNBwAJ

My implementation attempt lies at 
https://github.com/DaneSlattery/capnp_generic_poc .

The desired data flow is as follows:

1. A client connects to a server, and receives a capability that allows the 
client to load a schema (the InterfaceLoader). When connecting, the client 
also passes a ClientInterface to the server.
2. The client registers a schema using the InterfaceLoader. The client must 
send a Schema.Node parameter to the server. The server must load the schema.
3. The server uses the given loaded schema to communicate with the client 
interface, which takes a parameter of AnyStruct.

I have a few problems:

Firstly, when the client attempts to set the schema using  
capnp::Schema::from().getProto() , line 683 of 
capnproto-c++-0.10.3/src/capnp/dynamic.c++ blows up. I have changed this 
line to, instead assessing the ID of the schema for equality :
KJ_REQUIRE(structValue.getSchema().getProto().getId() == 
structType.getProto().getId(), "Value type mismatch.")
{
return;
}
I think this change should perhaps go in a PR, unless there is another, 
better way.

Secondly, I can't figure out how to send messages of the loaded schema back 
from the server to the client. Can anyone see if the POC linked is viable, 
and how to get to the final step?

- Dane

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/capnproto/4e26ccf6-67bf-43e5-8f31-f8b582a7c304n%40googlegroups.com.