[protobuf] Re: (objective-c) Techniques for marshal/unpacking structured data from .proto bytes field
Thanks Jon, for pointing out what should have been clear was an unportable
approach.
I will try the fine-grained protos approach demonstrated by Jorge.
Thanks
On Thursday, November 12, 2015 at 2:48:20 AM UTC-5, Jon Skeet wrote:
>
> This is the step that makes everything completely unportable:
>
> Call ObjectToByteArray extension method, which uses the .Net
>> BinaryFormatter class (and MemoryStream) to convert the object to a byte
>> array (byte[] in .Net)
>
>
> That's like using ObjectOutputStream in Java. I would definitely *not* expect
> to be able to deserialize that data usefully in Objective C.
>
> It sounds like instead of a bytes field, you should probably use an Any
> field... or some other message type with a oneof { int32, int64, double,
> string} or whatever you need. Fundamentally, you'll need to stick to protos
> everywhere though.
>
> (As an aside, if you have any usability feedback on the C# protobuf
> experience, now is the time to tell me so I can try to smooth out any
> points of friction before release!)
>
> Jon
>
>
> On 12 November 2015 at 03:18, Thomas Van Lenten > wrote:
>
>> I can't say I know C# to comment much on that side of things. It does
>> sorta look like you are using something more native to those objects for
>> serializing into a blob, and simply using a proto to try and package the
>> row & col counts along with the blob. It probably would make more sense to
>> do something like Jorge suggests and use protos for the full packaging
>> rather than just the outer packaging.
>>
>> TVL
>>
>>
>> On Wed, Nov 11, 2015 at 8:42 PM, Rob Cecil > > wrote:
>>
>>>
>>>
>>> On Wednesday, November 11, 2015 at 8:41:36 PM UTC-5, Rob Cecil wrote:
Interesting solution. I will definitely try that approach as I'm
getting no where to continue with my current tack. I'm not sure how
familiar your are with the .Net side of things, but this how it is working
for me so far. I have a server implementation (currently just brutally
simple - using console projects) and a WPF test client that currently
works
with my defined *RangeData* protobuf, including the bytes field.
On the server side this what I do (narrating the
CommonTypeExtensions.ToRangeData method in the debugger screenshot -
attached)
public static RangeData ToRangeData(this object[] source)
{
var result = new RangeData();
result.Rows = (int)source[0];
result.Columns = (int)source[1];
var formatted = ObjectToByteArray(source[2]);
result.Data = ByteString.CopyFrom(formatted);
return result;
}
- Get an object "blob", just an untyped object array (from db source,
see attached image of VS debugger watch window showing "source" that is
visible when the debugger stops in the ToRangeData() extension method).
- Call ObjectToByteArray extension method, which uses the .Net
BinaryFormatter class (and MemoryStream) to convert the object to a
byte array (byte[] in .Net)
- Call Protobuf's ByteString.CopyFrom() factory method to generate the
ByteString data and stuff that into the *RangeData* object.
On the reverse side, in the WPF client (not completely shown in
screenshots):
var ba = rangeData.Data.ToByteArray();
var array = (Array)ba.FromByteArrayToObject();
ViewData = new ViewData((object[,])array);
- Get the *RangeData* instance from Protobufs, including ByteString
instance in the data field.
- Call rangeData.Data.ToByteArray() to convert ByteString to byte[]
array. ToByteArray() is Protobufs provided method.
- Call FromByteArrayToObject (in the CommonTypeExtensions.cs class,
partially shown in screenshot), which uses the BinaryFormatter (and
MemoryStream) in .Net to convert the byte[] array into a CLR "Object"
- Cast Object to Array, then cast to object[,] two-dimensional array
("ViewData", which makes the data more consumable to a WPF DataGrid).
That is how it is working in .Net/C#.
With the Objective-C side of things, I have a NSData, instead of the
ByteString. I know the #rows and #columns. I just don't know the
techniques to do the analogous thing in objective-c (or Swift).
Thanks
On Wednesday, November 11, 2015 at 7:42:21 PM UTC-5, Jorge Luis
Canizales Diaz wrote:
>
> [CC: Jon for the C# side and TVL for the ObjC side]
>
> Hi Rob!
>
> when you say:
>
> > I am currently taking a object array in C# and packing that into
> the ByteString.
>
> How are you accomplishing that? I see from your example that the
> elements of your array aren't proto messages, so maybe .Net has its own
> serialization format that non-.Net languages aren
[protobuf] Re: (objective-c) Techniques for marshal/unpacking structured data from .proto bytes field
This is the step that makes everything completely unportable:
Call ObjectToByteArray extension method, which uses the .Net
> BinaryFormatter class (and MemoryStream) to convert the object to a byte
> array (byte[] in .Net)
That's like using ObjectOutputStream in Java. I would definitely *not* expect
to be able to deserialize that data usefully in Objective C.
It sounds like instead of a bytes field, you should probably use an Any
field... or some other message type with a oneof { int32, int64, double,
string} or whatever you need. Fundamentally, you'll need to stick to protos
everywhere though.
(As an aside, if you have any usability feedback on the C# protobuf
experience, now is the time to tell me so I can try to smooth out any
points of friction before release!)
Jon
On 12 November 2015 at 03:18, Thomas Van Lenten wrote:
> I can't say I know C# to comment much on that side of things. It does
> sorta look like you are using something more native to those objects for
> serializing into a blob, and simply using a proto to try and package the
> row & col counts along with the blob. It probably would make more sense to
> do something like Jorge suggests and use protos for the full packaging
> rather than just the outer packaging.
>
> TVL
>
>
> On Wed, Nov 11, 2015 at 8:42 PM, Rob Cecil wrote:
>
>>
>>
>> On Wednesday, November 11, 2015 at 8:41:36 PM UTC-5, Rob Cecil wrote:
>>>
>>> Interesting solution. I will definitely try that approach as I'm getting
>>> no where to continue with my current tack. I'm not sure how familiar your
>>> are with the .Net side of things, but this how it is working for me so far.
>>> I have a server implementation (currently just brutally simple - using
>>> console projects) and a WPF test client that currently works with my
>>> defined *RangeData* protobuf, including the bytes field.
>>>
>>> On the server side this what I do (narrating the
>>> CommonTypeExtensions.ToRangeData method in the debugger screenshot -
>>> attached)
>>>
>>>public static RangeData ToRangeData(this object[] source)
>>> {
>>> var result = new RangeData();
>>> result.Rows = (int)source[0];
>>> result.Columns = (int)source[1];
>>> var formatted = ObjectToByteArray(source[2]);
>>> result.Data = ByteString.CopyFrom(formatted);
>>> return result;
>>> }
>>>
>>>
>>> - Get an object "blob", just an untyped object array (from db source,
>>> see attached image of VS debugger watch window showing "source" that is
>>> visible when the debugger stops in the ToRangeData() extension method).
>>> - Call ObjectToByteArray extension method, which uses the .Net
>>> BinaryFormatter class (and MemoryStream) to convert the object to a
>>> byte array (byte[] in .Net)
>>> - Call Protobuf's ByteString.CopyFrom() factory method to generate the
>>> ByteString data and stuff that into the *RangeData* object.
>>>
>>> On the reverse side, in the WPF client (not completely shown in
>>> screenshots):
>>>
>>> var ba = rangeData.Data.ToByteArray();
>>> var array = (Array)ba.FromByteArrayToObject();
>>> ViewData = new ViewData((object[,])array);
>>>
>>> - Get the *RangeData* instance from Protobufs, including ByteString
>>> instance in the data field.
>>> - Call rangeData.Data.ToByteArray() to convert ByteString to byte[]
>>> array. ToByteArray() is Protobufs provided method.
>>> - Call FromByteArrayToObject (in the CommonTypeExtensions.cs class,
>>> partially shown in screenshot), which uses the BinaryFormatter (and
>>> MemoryStream) in .Net to convert the byte[] array into a CLR "Object"
>>> - Cast Object to Array, then cast to object[,] two-dimensional array
>>> ("ViewData", which makes the data more consumable to a WPF DataGrid).
>>>
>>> That is how it is working in .Net/C#.
>>>
>>> With the Objective-C side of things, I have a NSData, instead of the
>>> ByteString. I know the #rows and #columns. I just don't know the
>>> techniques to do the analogous thing in objective-c (or Swift).
>>>
>>> Thanks
>>>
>>> On Wednesday, November 11, 2015 at 7:42:21 PM UTC-5, Jorge Luis
>>> Canizales Diaz wrote:
[CC: Jon for the C# side and TVL for the ObjC side]
Hi Rob!
when you say:
> I am currently taking a object array in C# and packing that into
the ByteString.
How are you accomplishing that? I see from your example that the
elements of your array aren't proto messages, so maybe .Net has its own
serialization format that non-.Net languages aren't aware of? In that case,
the only real solution I can think of would be to formalize that
serialization by using protos. For example:
message RangeData {
int32 num_rows = 1;
int32 num_columns = 2;
repeated Row row = 3;
}
message Row {
repeated Value value = 1;
}
message Value {
oneof value {
string text = 1;
double num
[protobuf] Re: (objective-c) Techniques for marshal/unpacking structured data from .proto bytes field
Interesting solution. I will definitely try that approach as I'm getting no
where to continue with my current tack. I'm not sure how familiar your are
with the .Net side of things, but this how it is working for me so far. I
have a server implementation (currently just brutally simple - using
console projects) and a WPF test client that currently works with my
defined *RangeData* protobuf, including the bytes field.
On the server side this what I do (narrating the
CommonTypeExtensions.ToRangeData method in the debugger screenshot -
attached)
public static RangeData ToRangeData(this object[] source)
{
var result = new RangeData();
result.Rows = (int)source[0];
result.Columns = (int)source[1];
var formatted = ObjectToByteArray(source[2]);
result.Data = ByteString.CopyFrom(formatted);
return result;
}
- Get an object "blob", just an untyped object array (from db source, see
attached image of VS debugger watch window showing "source" that is visible
when the debugger stops in the ToRangeData() extension method).
- Call ObjectToByteArray extension method, which uses the .Net
BinaryFormatter class (and MemoryStream) to convert the object to a byte
array (byte[] in .Net)
- Call Protobuf's ByteString.CopyFrom() factory method to generate the
ByteString data and stuff that into the *RangeData* object.
On the reverse side, in the WPF client (not completely shown in
screenshots):
var ba = rangeData.Data.ToByteArray();
var array = (Array)ba.FromByteArrayToObject();
ViewData = new ViewData((object[,])array);
- Get the *RangeData* instance from Protobufs, including ByteString
instance in the data field.
- Call rangeData.Data.ToByteArray() to convert ByteString to byte[] array.
ToByteArray() is Protobufs provided method.
- Call FromByteArrayToObject (in the CommonTypeExtensions.cs class,
partially shown in screenshot), which uses the BinaryFormatter (and
MemoryStream) in .Net to convert the byte[] array into a CLR "Object"
- Cast Object to Array, then cast to object[,] two-dimensional array
("ViewData", which makes the data more consumable to a WPF DataGrid).
That is how it is working in .Net/C#.
With the Objective-C side of things, I have a NSData, instead of the
ByteString. I know the #rows and #columns. I just don't know the
techniques to do the analogous thing in objective-c (or Swift).
Thanks
On Wednesday, November 11, 2015 at 7:42:21 PM UTC-5, Jorge Luis Canizales
Diaz wrote:
>
> [CC: Jon for the C# side and TVL for the ObjC side]
>
> Hi Rob!
>
> when you say:
>
> > I am currently taking a object array in C# and packing that into the
> ByteString.
>
> How are you accomplishing that? I see from your example that the elements
> of your array aren't proto messages, so maybe .Net has its own
> serialization format that non-.Net languages aren't aware of? In that case,
> the only real solution I can think of would be to formalize that
> serialization by using protos. For example:
>
> message RangeData {
> int32 num_rows = 1;
> int32 num_columns = 2;
> repeated Row row = 3;
> }
>
> message Row {
> repeated Value value = 1;
> }
>
> message Value {
> oneof value {
> string text = 1;
> double number = 2;
> }
> }
>
> or, alternatively, if you're not going to mix the strings and the numbers:
>
> message RangeData {
> repeated string row_name = 1;
> repeated string column_name = 2;
> repeated Row row = 3;
> }
>
> message Row {
> repeated double value = 1;
> }
>
> Hope that helps,
> Jorge
>
> On Wednesday, November 11, 2015 at 9:53:11 AM UTC-8, Rob Cecil wrote:
>>
>> I'm using the objective-c version Protobufs.
>>
>> I have a .proto defined as
>>
>> message RangeData {
>> int32 rows = 1;
>> int32 columns = 2;
>> bytes data = 3;
>> }
>>
>> The server-side has been developed in C#/.Net. I'm able to successfully
>> send/receive and unpack the data from the 'data' field 3 above. In the c#
>> port, 'bytes' are represented by a Google protobufs "ByteString" object.
>>
>> In objective-c, they're represented as NSData instance.
>>
>> I am currently taking a object array in C# and packing that into the
>> ByteString. I have a separate .Net WPF client that successfully retrieves
>> the data from the ByteString instance.
>>
>> The object array is essentially a two-dimensional (object[,]) array
>> instance whose elements can be numeric, or strings of varying length.
>>
>> ary[0, 0] = ""
>>
>> ary[0, 1] = "Jan 2010"
>>
>> ary[0, 2] = "Feb 2010"
>>
>> ...
>>
>> ary[0, 13] = "- Year 2010"
>>
>> ary[1, 0] = 89544.994
>>
>> ary[1, 1] = 93202.257
>>
>> ...
>>
>> ary[1, 13] = 492331.908
>>
>> ary[2, 0] = "Report A"
>>
>> ...
>>
>> ary[16, 13] = ...
>>
>>
>> The number of rows and columns for this two dimensional array are passed
>> in an outer (Message) context.
>>
>>
>> What is the technique for dealing with the NSData a
[protobuf] Re: (objective-c) Techniques for marshal/unpacking structured data from .proto bytes field
[CC: Jon for the C# side and TVL for the ObjC side]
Hi Rob!
when you say:
> I am currently taking a object array in C# and packing that into the
ByteString.
How are you accomplishing that? I see from your example that the elements
of your array aren't proto messages, so maybe .Net has its own
serialization format that non-.Net languages aren't aware of? In that case,
the only real solution I can think of would be to formalize that
serialization by using protos. For example:
message RangeData {
int32 num_rows = 1;
int32 num_columns = 2;
repeated Row row = 3;
}
message Row {
repeated Value value = 1;
}
message Value {
oneof value {
string text = 1;
double number = 2;
}
}
or, alternatively, if you're not going to mix the strings and the numbers:
message RangeData {
repeated string row_name = 1;
repeated string column_name = 2;
repeated Row row = 3;
}
message Row {
repeated double value = 1;
}
Hope that helps,
Jorge
On Wednesday, November 11, 2015 at 9:53:11 AM UTC-8, Rob Cecil wrote:
>
> I'm using the objective-c version Protobufs.
>
> I have a .proto defined as
>
> message RangeData {
> int32 rows = 1;
> int32 columns = 2;
> bytes data = 3;
> }
>
> The server-side has been developed in C#/.Net. I'm able to successfully
> send/receive and unpack the data from the 'data' field 3 above. In the c#
> port, 'bytes' are represented by a Google protobufs "ByteString" object.
>
> In objective-c, they're represented as NSData instance.
>
> I am currently taking a object array in C# and packing that into the
> ByteString. I have a separate .Net WPF client that successfully retrieves
> the data from the ByteString instance.
>
> The object array is essentially a two-dimensional (object[,]) array
> instance whose elements can be numeric, or strings of varying length.
>
> ary[0, 0] = ""
>
> ary[0, 1] = "Jan 2010"
>
> ary[0, 2] = "Feb 2010"
>
> ...
>
> ary[0, 13] = "- Year 2010"
>
> ary[1, 0] = 89544.994
>
> ary[1, 1] = 93202.257
>
> ...
>
> ary[1, 13] = 492331.908
>
> ary[2, 0] = "Report A"
>
> ...
>
> ary[16, 13] = ...
>
>
> The number of rows and columns for this two dimensional array are passed
> in an outer (Message) context.
>
>
> What is the technique for dealing with the NSData and extracting & parsing
> back into an objective-c (or Swift) arrays?
>
>
> Are there helper classes I should be aware of to help in these scenarios?
>
>
> Thanks
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
