Thanks so much for the reply Kenton :) I've inlined as you've covered a lot
On Thursday, 9 March 2017 20:49:49 UTC, Kenton Varda wrote:
>
> Hi Sam,
>
> Ross mostly covered this, but I'll give an official response as the author:
>
> Cap'n Proto has "presence", including hasX() accessors, for pointer types
> (structs, lists, text, data; anything that is variable-size) but not for
> primitives (integers, floats, bools).
>
Does this include arrays? (proto3 does not, so there is no workaround)
>
> The feature was not "removed" -- it never existed in the first place
> (though I suppose you could consider it "removed" if you think of Cap'n
> Proto as a successor to protobuf).
>
Sorry, yes I was referring to group of 'new age' formats (proto3/Cap'n
Proto) vs proto2
> Since the protobuf encoding is tag/value pairs, there's an obvious way to
> represent a missing field -- by not sending it on the wire. But Cap'n
> Proto's encoding is very different: primitive values are distinguished by
> their offsets from the beginning of a struct. There is nowhere to record
> whether the field is set or not. We would have to add more bits to the
> struct to store this, but the application could just as easily do that
> itself by adding `Bool` fields, e.g.:
>
> struct S {
> foo @0 :Int32;
> hasFoo @1 :Bool;
> }
>
> Cap'n Proto tightly packs boolean fields so this ends up just as efficient
> as if Cap'n Proto had allocated such bits automatically.
>
A good explanation, thanks. In one sense, as an API user, the wire format
is just magic to me but it is nice to know the underlying reason.
But for most apps, presence detection isn't needed, so if Cap'n Proto did
> it automatically, it would waste space.
>
>From the point of view of writing defensively against future regressions as
protos evolve in unpredictable ways, the presence detection is incredibly
useful.
> Also, as you noted, Cap'n Proto has strong support for unions (like
> protobuf's "oneof").
>
Small aside, this is the one upside of oneof - you can have a single member
of oneof and the unset case comes for free
> I find that many cases where I at first think I need field presence
> actually make more sense as a union -- I find that, surprisingly often,
> it's not just that the field is optional, but that there are several
> different conceivable "modalities" of the struct, where the field only
> makes sense in one of them. It's nice to encode this explicitly, and Cap'n
> Proto can then reuse space in the struct for mutually-exclusive fields.
>
As mentioned, I'm coming more from the angle of writing defensively. I
might not ever *want* to communicate an 'unset' case, I just want to detect
it and fail gracefully, early and safely. (i.e. I would want to union wrap
EVERY primitive field)
Finally, note that I would like to add a concept of "Maybe(T)" in the
> future, to explicitly state that the presence of absence of a field
> matters. I had intended to use this specifically for pointer types -- the
> effect would be that the `getX()` accessor would return a
> language-appropriate "Maybe" type (e.g. kj::Maybe<T> in C++), and then we
> could deprecate the `hasX()` accessors. I hadn't really thought about
> whether we should support Maybe(T) for primitives, which would require
> allocating an extra bit as described above (and would not be
> wire-compatible with the non-Maybe type, whereas for pointer types it would
> be; this might get confusing).
>
The support of generics in Cap'n Proto looks like it would lend itself to
this well. A firm thumbs up from me for this (including primitives) :)
>
> -Kenton
>
> On Thu, Mar 9, 2017 at 4:00 AM, <[email protected] <javascript:>> wrote:
>
>> I couldn't find an explicit mention of this on the website, curious if
>> CapNProto has also removed field presence detection - it's a really useful
>> feature to have!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> Visit this group at https://groups.google.com/group/capnproto.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
Visit this group at https://groups.google.com/group/capnproto.
