Re: [protobuf] [Protobuf] Can protobuf serialize datatable (C#)
Yes, a `byte[]` can be declared as a `ProtoMember` - it maps to the `bytes`
type in the protobuf spec. I would strongly advise *against* DynamicType
here; that should be used as a last resort only. If you know the data you
want to serialize is actually a finite set of types, I would use that if
possible. For example, let's say you need to support `string`, `int` and
`DataTable`; you could do something like (this is completely untested -
writing in email client here, but the concept is sound):
[ProtoContact]
[ProtoInclude(10, typeof(ParameterValue)]
[ProtoInclude(11, typeof(ParameterValue)]
[ProtoInclude(12, typeof(ParameterValue)]
public abstract class ParameterValue {
public abstract Type Type {get;}
public abstract object Value {get;set}
}
[ProtoContract]
public sealed class ParameterValue : ParameterValue {
public override Type Type { get { return typeof(T); } }
[ProtoMember(1)]
public T TypedValue {get;set;}
public override object Value {
get { return TypedValue; }
set { TypedValue = (T)value; }
}
}
[ProtoContract]
public sealed class DataTableShim {
public DataTable Table {get;set;}
[ProtoMember(1)]
public byte[] TableSerialized {
get { /* serialize Table to a MemoryStream, and return
ms.ToArray(); */ }
set { /* create a new MemoryStream(value), and deserialize to
Table */ }
}
}
On 11 July 2014 07:54, Desmond Davids wrote:
> Hey Marc,
>
> Thanks for the reply. We not sending the actual SQL commands over the
> line. we sending the info in such a way that way know which commands to
> execute on which parameters. Problem is our parameters are almost always
> Table Valued Parameters (TVP). and needs to sit in the request as DataTable
> properties. This is our problem protobuf-net cannot serialize DataTable
> type. It also has a problem with object type but i think setting
> DynamicType = true does the trick here as the types will always be
> primitive types. Am i correct? So we need some way to get these TVPs over
> the line using protobuf-net as serializer in WCF.
>
> Can you tell if byte[] can be declared as a ProtoMember. I can therefore
> use protobuf-net-data to serialize the DataTable object and assign the this
> byte[] ProtoMember. When it comes to assigning my TVP I can just
> Deserialize with protobuf-net-data again.
>
> I am looking for the fastest way I can do this as the sole reason we using
> this is that we love the performance of protobuf-net. If meaning we have to
> sacrifice the performance to serialize every single TVP we might as well
> revert back to DataContractSerializer which is definitely not what we want.
>
> I think also you suggestion regarding converting DataTable to a DTO will
> be difficult as we dont really know what the schema of the DataTable will
> be. I will have to think about this more.
>
> Looking forward to hear from you.
>
> Regards
> Desmond
>
>
> On Thu, Jul 10, 2014 at 9:18 PM, Marc Gravell
> wrote:
>
>> Ooh, the idea of allowing SQL over a service boundary sends all kinds of
>> shivers up my back, and not the good kind of shivers. On your head be it,
>> but: I wouldn't do that myself.
>>
>> I once did some experimental datatable serialisation code - it isn't in
>> the release binary, but it is still inside the solution; but: I really
>> can't say how robust this would be, and you'd need to serialize it manually
>> and store it in a byte[]. Of course, by the time you've done all that you
>> might as well have simply asked the datatable to serialize itself using its
>> own internal storage format (make sure you switch from XML to binary) - and
>> again: shove it in a byte[] member. IIRC you can set the RemotingFormat and
>> use BinaryFormatter to get the internal binary storage format.
>>
>> Marc
>> On 10 Jul 2014 19:56, "Desmond Davids" wrote:
>>
>>> Hey Marc,
>>>
>>> I know this is an old article. but i need to ask you something regarding
>>> this.
>>>
>>> We have a WCF service which we use to execute SqlCommands to our DB. We
>>> send Commands with DataContractSerialization. I came across protobuf-net
>>> and I have converted my WCF service to use protobuf. We now sit with a
>>> problem. we use table valued parameters in our sqlcommands. which means we
>>> need to serialize DataTables as part of an object. I there a way we can
>>> achieve this with protobuf.
>>>
>>> Do you have any workaround/alternative or suggestion?
>>>
>>> Regards
>>> Desmond Davids
>>>
>>> On Friday, 16 July 2010 08:00:25 UTC+2, Marc Gravell wrote:
From the message, that *sounds* like protobuf-net...
There is no built-in handling of this, but it is possibly something
that could be added, especially in "v2" which has a much more flexible
model. For example, you *could* argue that there is an implicit schema that
uses a repeated element per row, and the colum
Re: [protobuf] [Protobuf] Can protobuf serialize datatable (C#)
Ooh, the idea of allowing SQL over a service boundary sends all kinds of shivers up my back, and not the good kind of shivers. On your head be it, but: I wouldn't do that myself. I once did some experimental datatable serialisation code - it isn't in the release binary, but it is still inside the solution; but: I really can't say how robust this would be, and you'd need to serialize it manually and store it in a byte[]. Of course, by the time you've done all that you might as well have simply asked the datatable to serialize itself using its own internal storage format (make sure you switch from XML to binary) - and again: shove it in a byte[] member. IIRC you can set the RemotingFormat and use BinaryFormatter to get the internal binary storage format. Marc On 10 Jul 2014 19:56, "Desmond Davids" wrote: > Hey Marc, > > I know this is an old article. but i need to ask you something regarding > this. > > We have a WCF service which we use to execute SqlCommands to our DB. We > send Commands with DataContractSerialization. I came across protobuf-net > and I have converted my WCF service to use protobuf. We now sit with a > problem. we use table valued parameters in our sqlcommands. which means we > need to serialize DataTables as part of an object. I there a way we can > achieve this with protobuf. > > Do you have any workaround/alternative or suggestion? > > Regards > Desmond Davids > > On Friday, 16 July 2010 08:00:25 UTC+2, Marc Gravell wrote: >> >> From the message, that *sounds* like protobuf-net... >> >> There is no built-in handling of this, but it is possibly something that >> could be added, especially in "v2" which has a much more flexible model. >> For example, you *could* argue that there is an implicit schema that uses a >> repeated element per row, and the column ordinal for a field. >> >> There are a couple of points here thought: >> >> - protobuf /normally/ doesn't include schema information; so when >> deserializing *either* you would have to set up the schema (columns etc) >> *first* and then use Merge, /or/ I could include schema information as an >> exception >> - it would need some thought re the difference between DataSet and >> DataTable - a data-set has relations etc, which don't necessarily fit very >> well here >> >> I'm interested in your thoughts with this; are you just after something >> to load/save the data? Or interop with other platforms? >> >> And the other important question: /an option/ is to move the data to/from >> a DTO during serialization; the DTO would serialize fine with protobuf-net. >> >> Marc >> >> On 15 July 2010 09:43, Ferryandi Chai wrote: >> >>> Hi All, >>> >>> Im just wondering can protobuf serialize datatable. >>> Im a c# developer, and when i tried to serialize datatable it prompt >>> this error: >>> >>> "Only data-contract classes (and lists/arrays of such) can be >>> processed (error processing DataTable)" >>> >>> Tried to google this problem but cannot find solutions. >>> >>> >>> Thx >>> F! >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Protocol Buffers" group. >>> To post to this group, send email to prot...@googlegroups.com. >>> To unsubscribe from this group, send email to protobuf+u...@ >>> googlegroups.com. >>> For more options, visit this group at http://groups.google.com/ >>> group/protobuf?hl=en. >>> >>> >> >> >> -- >> Regards, >> >> Marc >> > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to protobuf+unsubscr...@googlegroups.com. > To post to this group, send email to protobuf@googlegroups.com. > Visit this group at http://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] [Protobuf] Can protobuf serialize datatable (C#)
Hey Marc, I know this is an old article. but i need to ask you something regarding this. We have a WCF service which we use to execute SqlCommands to our DB. We send Commands with DataContractSerialization. I came across protobuf-net and I have converted my WCF service to use protobuf. We now sit with a problem. we use table valued parameters in our sqlcommands. which means we need to serialize DataTables as part of an object. I there a way we can achieve this with protobuf. Do you have any workaround/alternative or suggestion? Regards Desmond Davids On Friday, 16 July 2010 08:00:25 UTC+2, Marc Gravell wrote: > > From the message, that *sounds* like protobuf-net... > > There is no built-in handling of this, but it is possibly something that > could be added, especially in "v2" which has a much more flexible model. > For example, you *could* argue that there is an implicit schema that uses a > repeated element per row, and the column ordinal for a field. > > There are a couple of points here thought: > > - protobuf /normally/ doesn't include schema information; so when > deserializing *either* you would have to set up the schema (columns etc) > *first* and then use Merge, /or/ I could include schema information as an > exception > - it would need some thought re the difference between DataSet and > DataTable - a data-set has relations etc, which don't necessarily fit very > well here > > I'm interested in your thoughts with this; are you just after something to > load/save the data? Or interop with other platforms? > > And the other important question: /an option/ is to move the data to/from > a DTO during serialization; the DTO would serialize fine with protobuf-net. > > Marc > > On 15 July 2010 09:43, Ferryandi Chai > > wrote: > >> Hi All, >> >> Im just wondering can protobuf serialize datatable. >> Im a c# developer, and when i tried to serialize datatable it prompt >> this error: >> >> "Only data-contract classes (and lists/arrays of such) can be >> processed (error processing DataTable)" >> >> Tried to google this problem but cannot find solutions. >> >> >> Thx >> F! >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Protocol Buffers" group. >> To post to this group, send email to prot...@googlegroups.com >> . >> To unsubscribe from this group, send email to >> protobuf+u...@googlegroups.com . >> For more options, visit this group at >> http://groups.google.com/group/protobuf?hl=en. >> >> > > > -- > Regards, > > Marc > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at http://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] [Protobuf] Can protobuf serialize datatable (C#)
>From the message, that *sounds* like protobuf-net... There is no built-in handling of this, but it is possibly something that could be added, especially in "v2" which has a much more flexible model. For example, you *could* argue that there is an implicit schema that uses a repeated element per row, and the column ordinal for a field. There are a couple of points here thought: - protobuf /normally/ doesn't include schema information; so when deserializing *either* you would have to set up the schema (columns etc) *first* and then use Merge, /or/ I could include schema information as an exception - it would need some thought re the difference between DataSet and DataTable - a data-set has relations etc, which don't necessarily fit very well here I'm interested in your thoughts with this; are you just after something to load/save the data? Or interop with other platforms? And the other important question: /an option/ is to move the data to/from a DTO during serialization; the DTO would serialize fine with protobuf-net. Marc On 15 July 2010 09:43, Ferryandi Chai wrote: > Hi All, > > Im just wondering can protobuf serialize datatable. > Im a c# developer, and when i tried to serialize datatable it prompt > this error: > > "Only data-contract classes (and lists/arrays of such) can be > processed (error processing DataTable)" > > Tried to google this problem but cannot find solutions. > > > Thx > F! > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To post to this group, send email to proto...@googlegroups.com. > To unsubscribe from this group, send email to > protobuf+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/protobuf?hl=en. > > -- Regards, Marc -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
