[protobuf] Message References
Hey,
there are some threads in this group hinting to special solutions in
cases where you want to have some kind of references build in - or on
top of - protobuf messages. For example you would want that to model
DAG (or even cyclic graph) structures.
I was thinking about how I would want such techniques to be
implemented/usable and came up with this:
For messages types A and B where A messages are supposed to have
references to B messages:
message A {
required int32 ownvalue = 1;
required ptr_B bref = 2;
}
message B{
required bytes UUID = 1;
required int32 othervalue = 2;
}
message ptr_B {
required bytes refUUID = 1;
}
where ptr_B obviously is a reference variable message to a specific
B message (where the join of cause is on refUUID and UUID).
So far this is no problem, as I can easily have a script generate
ptr_X messages for each message X containing a defined attribute (e.g.
required bytes UUID).
The second part however is where I am not so sure. I would want to
have some type safe way of navigating/dereferencing those references.
Like:
A a; //this is what I received before
ptr_B reference = a.getRefUUID(); //get reference
B b = foo(reference); //dereference
A standard protocol buffer message type ptr_B from which a flat Java
class will be generated does not seem adequat to to so.
Instead some new abstract Class RefMessageT extends Message with a
method T getReferencedObject() would be the solution. ptr_X
messages simply had to be transformed to Java classes extending this
superclass.
My questions so far:
I could write my own code generator for that purpose, right?
Has anybody tried to implement some similar techniques for reference
types on top of protocol buffer?
Is there a completely different approach for this problem?
Am i getting the design and purpose of protobuf wrong when I need/want
such a solution?
Chris
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Message References
That is actually pretty similar to a scheme I had in mind to (opt-in only)
get protobuf-net working at graph serialization. I don't think it would be
easy to fit into the existing .proto or codegen, but protobuf-net doesn't
actually demand these.
Before I go on - the first thing I'll say is that I *haven't* done this (to
date) for exactly the reason that it would cripple interop with other
versions, which I do see as important, but it wouldn't be terribly tricky to
hook an object cache (to manage the unique keys etc) into my existing code.
Problems that I can foresee:
- it breaks the ability to simply append streams (I'm guessing the unique
ref would be an integer starting with 0/1 for space, which is ripe for
collissions)
- it breaks interop (or at least, makes it **really** hard for a regular
client to consume)
- the cache may be very busy for non-trivial sequences
- does it rule-out buffer-free object sequences? not sure
Anyway, those are my thoughts - certainly *possible* to hack into the
current protocol, but is it *desirable*?
You'd probably want to indicate treat as a graph via custom metadata (at
which level?)
Marc
2009/12/8 Chris hsifdr...@gmail.com
Hey,
there are some threads in this group hinting to special solutions in
cases where you want to have some kind of references build in - or on
top of - protobuf messages. For example you would want that to model
DAG (or even cyclic graph) structures.
I was thinking about how I would want such techniques to be
implemented/usable and came up with this:
For messages types A and B where A messages are supposed to have
references to B messages:
message A {
required int32 ownvalue = 1;
required ptr_B bref = 2;
}
message B{
required bytes UUID = 1;
required int32 othervalue = 2;
}
message ptr_B {
required bytes refUUID = 1;
}
where ptr_B obviously is a reference variable message to a specific
B message (where the join of cause is on refUUID and UUID).
So far this is no problem, as I can easily have a script generate
ptr_X messages for each message X containing a defined attribute (e.g.
required bytes UUID).
The second part however is where I am not so sure. I would want to
have some type safe way of navigating/dereferencing those references.
Like:
A a; //this is what I received before
ptr_B reference = a.getRefUUID(); //get reference
B b = foo(reference); //dereference
A standard protocol buffer message type ptr_B from which a flat Java
class will be generated does not seem adequat to to so.
Instead some new abstract Class RefMessageT extends Message with a
method T getReferencedObject() would be the solution. ptr_X
messages simply had to be transformed to Java classes extending this
superclass.
My questions so far:
I could write my own code generator for that purpose, right?
Has anybody tried to implement some similar techniques for reference
types on top of protocol buffer?
Is there a completely different approach for this problem?
Am i getting the design and purpose of protobuf wrong when I need/want
such a solution?
Chris
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
Regards,
Marc
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Message References
A standard protocol buffer message type ptr_B from which a flat Java
class will be generated does not seem adequat to to so.
Instead some new abstract Class RefMessageT extends Message with a
method T getReferencedObject() would be the solution. ptr_X
messages simply had to be transformed to Java classes extending this
superclass.
My questions so far:
I could write my own code generator for that purpose, right?
Has anybody tried to implement some similar techniques for reference
types on top of protocol buffer?
Is there a completely different approach for this problem?
Am i getting the design and purpose of protobuf wrong when I need/want
such a solution?
When I've seen a similar thing done, it's generally been done like this:
message A {
required int32 ownvalue = 1;
required int32 bref = 2;
}
message B {
required int32 othervalue = 1;
}
message Overall {
repeated B b = 1;
repeated A a = 2;
}
And then you do
Overall overall;
A a;
B b = overall.getB(a.getBref());
Basically, you first send a lookup table, then you send the remaining
items, with references as indexes into the table. To serialize a
graph using that technique, you could send a list of nodes and then a
list of edges. I don't know that it's the best way of doing it, but
it's worked out well where I've seen it.
- Adam
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
