[protobuf] Correct behaviour when encountering an unexpected enum on the wire

2011-05-11 Thread Marc Gravell
I'm doing some code maintenance on my protobuf library, and I have encountered a test that is... confusing me. So before I go crazy (/ crazier)... what should an implementation do if during deserialization it gets an enum it doesn't recognise? - to explode in sparks? - to ignore the data? - to bru

Re: [protobuf] Serialization of primitive types

2011-06-15 Thread Marc Gravell
It sounds to me like you are over-complicating things. It is not uncommon to have a separate DTO model for serialization, so simply write a little code to map from your domain model (the comple model described above) to the DTO model (close to the serialization format). It is possible to write

Re: [protobuf] Re: Is any method available to convert a protobuf object to XML?

2011-06-24 Thread Marc Gravell
The code shown uses XmlSerializer - it doesn't use protobuf-net at all. protobuf-net does tend to be friendly towards this, however you would: - deserialize with protobuf-net into objects - serialize with XmlSerializer The only point of co tact between the two is the object model in the middle.

Re: [protobuf] Protocol Buffers can offer annotation features

2011-06-29 Thread Marc Gravell
Right; what platform are you on? That approach is certainly viable - it is in fact how protobuf-net operates (via .NET attributes). In guessing you are on Java; I do not know of a similar Java implementation. Marc On 29 Jun 2011, at 03:55, ordinary wrote: > Coding use Protocol Buffers by w

Re: [protobuf] c++/CLI and protobuf-net

2011-07-05 Thread Marc Gravell
I am not able to advise on either C++ option. However, if your solution doesn't already use .NET I wouldn't introduce a .NET dependency just for this - just use the c++ version. If your solution already contains some .NET classes and your intention is to add some protobuf, then it perhaps is an

Re: [protobuf] tag number range explanation

2011-07-07 Thread Marc Gravell
I don't pretend to know the original thinking, but it would be very hard to add such now without breaking existing clients. However, note that if you *really* don't want to have to get the lengths, you could encode your data inside a "group", since this has a terminator rather than a length pref

Re: [protobuf] java and dotnet

2011-07-08 Thread Marc Gravell
I haven't used protostuff/IKVM, but I would *hope* that IKVM allows some kind of passing of either a Stream or byte[]. That would allow you to serialize/deserialize to swap between models. If you have access to *both* models at once, perhaps another possibility is AutoMapper on the .NET side.

Re: [protobuf] Is the Protocol Buffers serialization format standardized?

2011-07-15 Thread Marc Gravell
>From my perspective as a fellow user, the spec is very stable in terms of >breaking changes. The last notable wire change I can think of is packed >arrays, which would: A: only apply on an opt-in basis B: continue to be round-trip safe (via extension fields) for clients that didn't expect it.

Re: [protobuf] GPB object versioning(noobie question)

2011-07-25 Thread Marc Gravell
Golden rule: - don't change a field, or re-use a field-numer In particular, your string vs int *is* a breaking change. Most of the others are not; additional data can be ignored or handed at runtime via extensions if you wish. Of course, if you add a new field you should probably make it "opti

Re: [protobuf] protobuf-net, HttpServer report exceptions from server to client

2011-08-01 Thread Marc Gravell
If I understand the meaning, then I would tend to make the exception scenario mean "something is fundamentally wrong with the service", rather than "your request was invalid". The latter scenario is better handled by allow an error message as part of the standard API - which could be anything from

Re: [protobuf] Protobuf vs MessagePack

2011-08-02 Thread Marc Gravell
/msgpack/browse_thread/thread/db5e20aa64f3020d?pli=1 Marc Gravell On 2 August 2011 08:51, Canggih Wibowo wrote: > MessagePack claims that they 4 times faster than Protobuf on > serialization+deserialization and it also have RPC implementation already. > Anyone give response? I mean, w

Re: [protobuf] building .proto files from Visual Studio 2010 for C#

2011-09-05 Thread Marc Gravell
It is a pain that it is so hard to set up a "Custom Tool" in VS; I do have a custom tool working for protobuf-net, but it is annoying to have to install it manually (there's a download on the protobuf-net site). I pinged the nuget team to see if we can get something easier, but no hope there, for n

Re: [protobuf] building .proto files from Visual Studio 2010 for C#

2011-09-05 Thread Marc Gravell
Generator/ProtoBufTool.cs?r=258 > > > On Mon, Sep 5, 2011 at 9:28 PM, Lars Schouw wrote: > >> I found this link >> >> http://stackoverflow.com/questions/453820/protocol-buffers-in-c-projects-using-protobuf-net-best-practices-for-code-gene/1139289#1139289 >> >&

Re: [protobuf] Serialized Object as a value in Key-Value store.

2011-09-22 Thread Marc Gravell
This will depend on many factors: - how big is each fragment? Very small fragments of *anything* generally get bigger when compressed - what is the data? If it contains a lot of text data you might see benefits; however, many typical fragments will get bigger when compressed - it depends entire

Re: [protobuf] Serializing string arrays and List

2011-09-22 Thread Marc Gravell
Protocol buffers works best with structured and predictable data. "object" sounds overly vague IMO. Most protocol buffers will not handle that; due to demand, I *do* have a feature in protobuf-net that might work for that, but it basically breaks all the interop benefits of protocol buffers. So

Re: [protobuf] ProtoBuf: passing empty arrays

2011-09-27 Thread Marc Gravell
Ooh, that sounds like me, then. It works fine against the current v2 code; I do seem to recall there was a bug at some point in v2 that behaved like this. If you are using v2, please make sure you have a recent version. If you are using v1 and this happens /there/, then my flabber is officially gha

Re: [protobuf] Protocl under "Protocol Buffers"

2011-09-30 Thread Marc Gravell
Protocol buffers itself only (AFAIK) describes the serialisation format; it does not define RPC. If you add an http-based RPC stack, then it will be http-based, but that is nothing to do with protocol buffers, you could (and many do) use raw sockets just as well. Marc On 30 Sep 2011, at 10:14,

Re: [protobuf] Serialize protocol buffer file into xml/text format

2011-10-10 Thread Marc Gravell
I answered this where you cross-posted on stackoverflow: http://stackoverflow.com/questions/7717036/serialize-protocol-buffer-file-into-xml-text-format To repeat; in most cases, the generated code **should already** work perfectly well with XmlSerailizer (the inbuilt .NET xml serialization tool),

Re: [protobuf] TypeModel.DeserializeType(value) Exception Message request.

2011-10-11 Thread Marc Gravell
In standard use, protouf-net is fully contract based and doesn't care what *types* are involved; this only matters if you are using the DynamicType option (which is outside the core protobuf stuff). If the types aren't stable, there is an event on the TypeModel that can be used to map in both di

[protobuf] Re: TypeModel.DeserializeType(value) Exception Message request.

2011-10-12 Thread Marc Gravell
In the next build, this has been tweaked: type = source.DeserializeType(typeName); if(type == null) { throw new ProtoException("Unable to resolve type: " + typeName); } So,

Re: [protobuf] IEnumerable in Silverlight with "external" Add(T item) method

2011-10-17 Thread Marc Gravell
Addressed on stackoverflow: http://stackoverflow.com/questions/7793527/protobuf-net-serializing-ienumerablet Marc On 17 Oct 2011, at 12:35, Broken Pipe wrote: > I'm trying to use ProtoBuf-net in Silverlight project, I'm using my > existing domain objects, which define most collection's as: >

Re: [protobuf] decode messages that are in protocol buffer format

2011-10-18 Thread Marc Gravell
Well, firstly protobuf is not a text format, so UTF-8 is not the way to start. What is it you need? Note that the protobuf format is ambiguous unless you already know the schema (the same data can be interpreted in different ways). However, if you read the encoding spec, you should be able to gu

Re: [protobuf] Re: decode messages that are in protocol buffer format

2011-10-19 Thread Marc Gravell
gle.com/p/protobuf-wireshark/ > > > > Might get you want you need. > > > > -Aaron > > > > > > > > > > > > > > > > On Tue, Oct 18, 2011 at 11:02 AM, Marc Gravell > wrote: > > > Well, firstly protobuf is not a text for

Re: [protobuf] Non-byte aligned data

2011-10-20 Thread Marc Gravell
Inside any standard implementation? None. And I doubt that is something that would be added anytime soon (if ever). You could of corse create some private branch combining elements of protobuf with your own more specific needs. Marc On 20 Oct 2011, at 12:32, Phillip Dann Ward wrote: > The bac

Re: [protobuf] protosharp project seems stale

2011-10-31 Thread Marc Gravell
that could also just mean "stable and not needing massive work" - there haven't been any significant changes to the protobuf format since packed arrays (the most recent commit). (I honestly don't know either way; I just don't think it is quite safe to assume lack of activity means "abandoned" here

Re: [protobuf] Re: How can I get iOS(iDevice) build for r470

2011-12-07 Thread Marc Gravell
@Bob - based on the version numbers, Yury is talking about protobuf-net (one of the 3rd party .NET implementations), using it on iOS via MonoTouch or Unity @Yury - sorry, I completely didn't see this thread; but indeed, the iOS folder got accidentally dropped when I updated my build script, and wa

Re: [protobuf] Register protocolbuf-net in SQL CLR

2011-12-19 Thread Marc Gravell
ve got your email - would you like me to send you a copy of the current code compiled with FEAT_SAFE? Marc Gravell (protobuf-net) On 19 December 2011 19:11, Scott Moore wrote: > I readily admit that I am a bit of a novice developer (I work in a > small business and we all help as much as pos

[protobuf] Re: Help to use protobuff .net remote code

2012-01-03 Thread Marc Gravell
If you mean protobuf-remote (which I'm not personally familiar with), that is an RPC stack. It will help you send and receive messages. Any database requirements you must handle separately. There is a "manual" page linked for both C# and C++ - http://code.google.com/p/protobuf-remote/ Marc On Dec

Re: [protobuf] protobuf-net and c++ protocol buffer missing line feeds.

2012-03-07 Thread Marc Gravell
Well, protobuf-net certainly includes them (it just does a simple UTF-8 conversion, nothing more), and I'm pretty sure the C++ side will be handling them fine. My guess would be that they are being lost in your code with whatever file / network handling you have in place. In particular, any code t

Re: [protobuf] re: protobuf-net 480 for .net 4 - objects in a IList I serialize have all members serialized to 0

2012-03-07 Thread Marc Gravell
Sorry, I didn't see this at the time - do you have any kind of concrete, reproducible example here? Marc On 28 February 2012 20:48, costa wrote: > Have you seen this? > > The list is an IList where TestClass has the ProtoContract > attribute and all its members have the ProtoMember attribute. >

Re: [protobuf] Protobuf and entity framework - included entities serialization

2012-05-31 Thread Marc Gravell
This question relates to protobuf-net specifically, and is being addressed in http://stackoverflow.com/q/10831660/ Marc (protobuf-net) -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegrou

Re: [protobuf] Members Identifiers

2012-06-04 Thread Marc Gravell
it only has to be unique to the particular message - not unique globally. The "why" is simply: because that is what it uses on the wire to identify different members. If they weren't unique, clearly it wouldn't work. If they weren't explicit (but were, say, assumed positionally) then it would not

Re: [protobuf] Members Identifiers

2012-06-04 Thread Marc Gravell
gt; about shared ids. [?] > > > > On Mon, Jun 4, 2012 at 11:44 AM, Marc Gravell wrote: > >> it only has to be unique to the particular message - not unique globally. >> The "why" is simply: because that is what it uses on the wire to identify >> different

Re: [protobuf] Serialization Error

2012-06-04 Thread Marc Gravell
This relates to protobuf-net; you've already emailed me directly, and I replied. You raised an issue report: I replied. Please read (and act on) my existing replies. Marc On 4 Jun 2012, at 12:00, Farooq Mushtaq wrote: > While serializing I am getting error "Possible recursion detected > (offs

Re: [protobuf] repeated field problem with protobuf-net (other side: java protobuf)

2012-06-18 Thread Marc Gravell
I answered this at stackoverflow (http://stackoverflow.com/a/11083229/23354) The main problem was the data-types in V3DDelta not matching the contract (note: there are tools for generating classes from a .proto definition). The particular code for reading the data stored via "writeDelimitedTo" ne

Re: [protobuf] Serializing List of Objects

2012-06-20 Thread Marc Gravell
(note: this is specific to protobuf-net, not "protocol buffers" more widely), but yes: that (a generic list) would work fine, as long as the property has been marked for serialization and given a number. There also doesn't need to be a "set" accessor, although it can make full use of a "set" - i.e.

Re: [protobuf] Serializing List of Objects

2012-06-27 Thread Marc Gravell
ications) > > Joel > > > On Wednesday, June 20, 2012 9:05:49 AM UTC-4, Marc Gravell wrote: >> >> (note: this is specific to protobuf-net, not "protocol buffers" more >> widely), but yes: that (a generic list) would work fine, as long as the >> propert

Re: [protobuf] Serializing List of Objects

2012-06-28 Thread Marc Gravell
confirmation that this would probably not > work nicely interop-ing with other Protocol Buffer libraries. > > (Note: I'm new to this stuff, so if I'm not making any sense at all let me > know and I'll return with a smarter question.) > > > Joel > &

Re: [protobuf] Serializing List of Objects

2012-06-28 Thread Marc Gravell
taType. Marc On 28 June 2012 13:51, Joel Carrier wrote: > Yes it does. Thanks. > > Where can I read more about this disabling of list-handling and its > effects? > > > On Thu, Jun 28, 2012 at 8:31 AM, Marc Gravell wrote: > >> Well, until I get around to re-implemen

Re: [protobuf] Serialization of existing XML Type

2012-07-29 Thread Marc Gravell
protobuf-net indeed needs *some* way to associate protobuf numbers with members; one of the ways it supports is `[XmlElement(Order=n)]`, but to confirm: yes the "n" needs to be >= 1, and yes, since `[XmlAttribute]` doesn't specify any such number, protobuf-net can't use that in any meaningful way.

Re: [protobuf] Serialization of Object type without modifying C# class generated out of schema

2012-07-31 Thread Marc Gravell
And in this model, what is FloatData, StringData, IntData, etc? This is certainly solvable with protobuf-net, but to do a complete example I'd need to see those additional types. Marc (protobuf-net) On 31 July 2012 14:14, Shail wrote: > public class Param > { > > /// > [

Re: [protobuf] Serialization of Object type without modifying C# class generated out of schema

2012-08-01 Thread Marc Gravell
ute()] > public System.Single AdjustValue; > > > /// > [System.Xml.Serialization.XmlAttributeAttribute()] > public System.Single[] Values; > } > > Regards > Shailendra > > On Wednesday, 1 August 2012 11:51:01 UTC+5:30, Marc Gravell

Re: [protobuf] Serialization of Object type without modifying C# class generated out of schema

2012-08-01 Thread Marc Gravell
data.Values[1]); >> >> Which all works fine, demonstrating that we have serialized and >> deserialized the data correctly. >> >> Marc >> >> >> >> On 1 August 2012 07:52, Shail wrote: >> >>> Hi Marc, >>> Here FloatData, StringData,In

Re: [protobuf] Portable Build on NuGet

2012-08-29 Thread Marc Gravell
Sure; do you happen to know the NuGet folder-name for the portable framework? Marc On 28 Aug 2012, at 17:42, Luke Pulverenti wrote: > Hey Marc, > I noticed the NuGet package (both 480 and 580) do not contain the portable > class library build. The google code download has it though. > > Do yo

Re: [protobuf] Precompiler Serialization Error

2012-09-08 Thread Marc Gravell
This sounds very protobuf-net specific; it might be easier to take this off-group, but I think I'll need a bit more context; in particular, since it is a PCL type, what runtime is this on when erroring? Also, are you using Serializer.*, or are you creating an instance of the custom serializer cr

Re: [protobuf] protobuf.net is taking more time than the default DataContractSerializer

2012-12-07 Thread Marc Gravell
Hi again. I already replied with a few comments / questions on stack overflow. I'm happy to try to help, but at the moment you haven't given me much context to work on. Marc (protobuf-net) On 7 Dec 2012, at 11:28, Evangelist wrote: > Hello folks, > > I have a wcf service consumed by silverli

Re: [protobuf] protobuf-net precompile 20+ types

2013-02-06 Thread Marc Gravell
I think I hit "reply" instead of "reply all" - but: this has been investigated (thanks Andrew!) and resolved in protobuf-net r627 onwards. Marc On 5 February 2013 18:18, Andrew Sorrell wrote: > I was running into an issue when trying to precompile more then 20 types: > > Object reference not s

Re: [protobuf] Re: Serialize with Length Prefix

2013-02-20 Thread Marc Gravell
this seems to be a resurrection of a protobuf-net specific discussion, so let me jump in... The SerializeWithLengthPrefix method, by default, aims to represent data in a way that is a valid protobuf stream - in particular, as though it were simply a member of a parent object or list. As such, it e

Fwd: [protobuf] deserialization of a deleted sub class works...

2013-03-19 Thread Marc Gravell
Sorry - intended to cc the list for reference purposes --- This is protobuf-net specific; sorry for delay bit I'm on a family weekend and my wife deliberately picks destinations far far away from cell towers. To explain: protobuf-net always starts from the

Re: [protobuf] Strange behavior using C++ and C# (protobuf-net)

2013-03-23 Thread Marc Gravell
Protobuf-net does not swallow any errors - if bad things happen in shouts loudly. Additionally the API is thread safe - during deserialization no state is shared. The first thing I would look at is the code *around* protobuf - any IO code, for example - is there any chance different readers are a

Re: [protobuf] Strange behavior using C++ and C# (protobuf-net)

2013-03-25 Thread Marc Gravell
tch(_type) > { > case 1: > MessageType_1 msg_1 = > ProtoUtility.FromWire< MessageType_1 >(wireData, wireData.Length, 0); > OnMessage_1_Event( msg_1 ); > break; > > . > > default: > Debug.Assert(false, "Unsuppor

Re: [protobuf] how to calculate protocol buffer message size without parsing the full message?

2013-04-21 Thread Marc Gravell
The outermost message is not stored with a length; this allows multiple messages to be merged by concatenation. The default behaviour for a protobuf parser at the root-object is "read until you run out of data". If you have written multiple messages without some kind of "framing" (usually adding

Re: [protobuf] Export Control information for Protobuf

2013-05-07 Thread Marc Gravell
On 7 May 2013 19:40, "Marc Gravell" wrote: > I don't know about the other stuff. But: protobuf does not include any > form of encryption. Since it only handles serialization (not full RPC) it > does not include any authentication logic either. The data is seriali

Re: [protobuf] Can I use int16 instead of int32 to verify a variable?

2013-05-07 Thread Marc Gravell
On 7 May 2013 18:53, "Marc Gravell" wrote: > Protobuf has a fee encodings for integer data, which basically come down > to: > > - varint (with or without zigzag) > - fixed-32 (always 4 bytes) > - fixed-64 (always 8 bytes) > > For 16 bits, you will do best with v

Re: [protobuf] Deserializing How to?

2013-05-13 Thread Marc Gravell
"..." is the standard form of protobuf "repeated" data. A header value of 10 is the standard form for a length-prefixed field with key 1. If you want to read all the objects together, then write a wrapper message, i.e. message animals { repeated animal items = 1; } and treat it as a single "a

Re: [protobuf] Performance aspect of submessage serialization

2013-05-14 Thread Marc Gravell
I asked about this a few years ago (feel free to search the archive - I couldn't find it; I believe I used the term "subnormal forms" for this). IIRC the answer then was along the lines of "hmmm looking at the current implementation that will probably work, but it isn't guaranteed and won't be

Re: [protobuf] Performance aspect of submessage serialization

2013-05-14 Thread Marc Gravell
is of far less interest to me, since I don't use the Google API). Marc On 14 May 2013 17:47, "Marc Gravell" wrote: > I asked about this a few years ago (feel free to search the archive - I > couldn't find it; I believe I used the term "subnormal forms" for this).

Re: [protobuf] Performance aspect of submessage serialization

2013-05-15 Thread Marc Gravell
Prefixing a varint with zero bits would follow the protocol as I understand > it. I think it is more of a bug in the protoc implementation that it fail's > to parse such a message. > > Kind regards, > Jonas > > > > On Tuesday, May 14, 2013 6:50:00 PM UTC+2, Marc Gra

Re: [protobuf] Performance aspect of submessage serialization

2013-05-15 Thread Marc Gravell
age. >> > The answer to your question is the same as it was before: It probably work > but not guaranteed, and we don't have any plan to change it. > As to your performance concern, only benchmarks can tell. So far I haven't > seen such data showing that prefixing a fix

Re: [protobuf] List of projects/products using protobuf?

2013-05-21 Thread Marc Gravell
I don't know about project/product lists, but you could look to https://code.google.com/p/protobuf/downloads/list - 50,000 downloads of protoc 2.5 since late February, which suggests reasonable usage - and that doesn't include packages that either don't use protoc, or which embed protoc. You could

Re: [protobuf] How can I config the Attribute in RuntimeTypeModel.Default?

2013-05-31 Thread Marc Gravell
This is a protobuf-net specific question. The answer to that depends on what you want to do. And I should stress that using attribute decoration is a lot simpler (and is what you get if you start from .proto). But basically: RuntimeTypeModel.Default .Add(type(Customer), false)

Re: [protobuf] Support for required repeated field?

2013-06-11 Thread Marc Gravell
There is nothing UB the specification to enforce that. You would have to use your own checks. On 11 Jun 2013 01:55, "oxlc" wrote: > Hi everyone, > > Through some quick searching I couldn't find any discussion about > specifying a repeated field that must have at least one of them in a > message.

Re: [protobuf] Protocol Buffer Language inter-operability support

2013-06-13 Thread Marc Gravell
Protocol buffers is language and platform independent, so yes: you can share data happily without worrying about what each is. However, it is a serialisation layer *only*. It does not include RPC etc. If you want to get data from a to b, it is fine. Anything to do with relaying function calls you

Re: [protobuf] Types which can be used in Protocol buffer schema

2013-06-13 Thread Marc Gravell
Have you looked through the examples on the protobuf project site? On 13 Jun 2013 18:09, "Genius" wrote: > I want to use protocol buffer files. > I am trying to build schema and I have data of various types like int, > string, byte. > > so to declare the byte data of 4 bytes how do I write teh sc

Re: [protobuf] Can I decode the message just base on a bi

2013-06-13 Thread Marc Gravell
You can get a long way just using protoc --decode_raw For "bytes" you can try a few things - see if round-trips to utf-8 to see if it is a string: or try checking the contents to see if they are internally a valid protovuf message. "Packed" is a little trickier to detect robustly. On 13 Jun 2013 2

Re: [protobuf] No Model instance has been assigned

2013-06-19 Thread Marc Gravell
Hi - you also asked this on stackoverflow, and I asked you a few questions - can you answer them? Ta http://stackoverflow.com/questions/17173588/protobuf-wcf-no-model-instance-has-been-assigned Marc On 19 Jun 2013 21:59, "Andrew McCormack" wrote: > I am getting this error: > > No Model instance

Re: [protobuf] Protobuf order or serialization

2013-07-15 Thread Marc Gravell
This is specifically a protobuf-net question. In short, yes - that is fine... ish. If you add the numbers manually ***and get them right***, then it will work. However, your example actually gets them wrong: the protobuf-net library specifically assumes an *alphabetical* order for the properties /

Re: [protobuf] Protobuf order or serialization

2013-07-15 Thread Marc Gravell
a good choice) but the above is much easier for ad-hoc scenarios. Marc (protobuf-net) On 15 July 2013 19:48, Marc Gravell wrote: > This is specifically a protobuf-net question. > > In short, yes - that is fine... ish. If you add the numbers manually > ***and get them right***,

Re: [protobuf] Recursive Structures in protobuffers

2013-07-20 Thread Marc Gravell
Protobuf supports recursive schemes - but note that each object is separate (it is a tree, not a graph). For example, descriptor.proto includes the self-referential "DescriptorProto" (which is a "message" in language terms) // Describes a message type. message DescriptorProto { optional string

Re: [protobuf] "No suitable Default Object encoding found"

2013-09-16 Thread Marc Gravell
This is specifically a protobuf-net question. I am guessing that somewhere in your model is a property / field declared simply as "object". That isn't enough information for ptotobuf-net to work with, because the protobuf data format doesn't include type metadata. It would have no way of recreating

Re: [protobuf] Protobuf-net v2: what happened to Serializer.Serialize(), Deserialize(), etc?

2013-09-19 Thread Marc Gravell
That would depend entirely on what exact DLL you are using. Those methods exist on all "Full" builds. I'm guessing you have referenced one of the "CoreOnly" builds. The "CoreOnly" builds are intended for use with the precompiler ( http://marcgravell.blogspot.co.uk/2012/07/introducing-protobuf-net-p

Re: [protobuf] Does protobuf-net have any known problems with nested dictionaries?

2013-09-19 Thread Marc Gravell
In all honestly I can't answer that off the top of my head, and I'm not at a PC. I can investigate and get back to you. Marc On 19 Sep 2013 17:54, "David Deutsch" wrote: > I have the following member of a class: > > public SerializableDictionary SerializableDictionary> _test = new Se

Re: [protobuf] protobuf calling getter on deserialization?

2013-09-19 Thread Marc Gravell
What is the property? A sub-object? A List? If the serializer doesn't think it needs to call the setter: it won't. For example, the typical list handling code could be paraphrased (not the actual implementation) as: var list = obj.SomeList; bool setValue = false; if(list == null) {

Re: [protobuf] Re: protobuf calling getter on deserialization?

2013-09-19 Thread Marc Gravell
Yes, OverwriteList should fix this. IgnoreListHandling does something very different that doesn't apply here (see the intellisense comments for full usage Marc On 19 Sep 2013 17:21, "David Deutsch" wrote: > So I *think* what is happening is that protobuf does a get of the property > because it i

Re: [protobuf] Protobuf serializer corruption after a while

2013-09-29 Thread Marc Gravell
Yikes. Bug. Basically, the "position" field wasn't being reset when used from the pool. This field is used for two main purposes: - error reporting (telling the user at what offset it glitched) - tracking sub-object ranges I suspect that because it didn't reset, the field overflowed. This won't i

Re: [protobuf] Protobuf serializer corruption after a while

2013-09-30 Thread Marc Gravell
Build r668 went out earlier to both nuget and google-code. I guess the 666 build was doomed to have an evil bug hiding away in there somewhere. Marc On 29 September 2013 10:06, Marc Gravell wrote: > Yikes. Bug. Basically, the "position" field wasn't being reset when used >

Re: [protobuf] Issue with precomposed

2013-10-04 Thread Marc Gravell
This is specifically protobuf-net. I'd be happy to take a look, but is there any context you can add about your specific model? As far as I know there isn't a *general* reason for it to fail here. At the simplest : are you able to make your model DLL available to me to use as a repro? Note: due to

Re: [protobuf] Use protobuf-net in C++/CLI

2013-10-07 Thread Marc Gravell
I don't do much... no: *any* C++/CLI - but I guess the conflict here is that the c++ generated classes are not going to be CLI-friendly. So you need to either use a c++ library and c++ classes, or a managed library and managed classes. If your framework is unmanaged c++, then I *suspect* your best

Re: [protobuf] Use protobuf-net in C++/CLI

2013-10-07 Thread Marc Gravell
On 7 October 2013 10:45, Barzo wrote: > In a meanwhile I have built the .cs generated file into a separate DLL > assembly and I added it (linked) to my C++/CLI project. Indeed, compiling it as C# and referencing/linking was what I meant - i.e. using the C# *from* C++/CLI, rather than *in* C++/C

Re: [protobuf] segfault when deserializing under high concurrency

2013-10-15 Thread Marc Gravell
Firstly: exactly what version is this? There was a bug in 663 relating to threading (and which only exhibited after extended usage) that was fixed in something like 668. If you are using something >= 663 and < 668 then please update and retry. Marc (protobuf-net) On 15 Oct 2013 16:22, wrote: > S

Re: [protobuf] segfault when deserializing under high concurrency

2013-10-15 Thread Marc Gravell
the latest mono 3 master because of a > threading bug that was fixed 2 weeks ago, now this:) > > Chris > > On Tuesday, October 15, 2013 10:45:46 AM UTC-7, Marc Gravell wrote: >> >> Firstly: exactly what version is this? There was a bug in 663 relating to >> threading (a

Re: [protobuf] Getting TCP/IP message

2013-11-07 Thread Marc Gravell
On 7 Nov 2013 20:32, "Marc Gravell" wrote: > What platform? But basically protobuf should be treated as a BLOB. Then > you have two choices: > > - send it as a payload in an existing messaging stack > - write your own messaging layer on top of sockets - noting that be

Re: [protobuf] UTF-16 in Protocol Buffers

2013-11-14 Thread Marc Gravell
If your intent is to get text from a to b, then there won't be a problem. Protobuf uses utf-8, but that is an implementation detail that you should never see. If your concern is that it may take more bytes in utf-8 than utf-16 (for the codepoints in question), the you can always use a "bytes" type

Re: [protobuf] Http Request and Http Response in Android Using Protobuf

2013-11-19 Thread Marc Gravell
Can you be very specific with what you mean by "connect with .NET Database using Protocol Buffer"? What **exactly** are you trying to do? And since you are mentioning "Http Request", "Http Response" and "PHP" - does "Database" here really mean "web-server" ? Ultimately, the mechanism for getting pr

Re: [protobuf] protobuf-net has System.Xml reference?

2014-01-28 Thread Marc Gravell
This is specific to protobuf-net; the inclusion of System.Xml is basically linked to the PLAT_XMLSERIALIZER build symbol, which for the Full/Unity configuration is currently: TRACE;FEAT_COMPILER PLAT_BINARYFORMATTER PLAT_XMLSERIALIZER PLAT_NO_INTERLOCKED FEAT_SAFE You could try building local

Re: [protobuf] A first chance exception of type 'System.IO.EndOfStreamException' occurred in protobuf-net.DLL

2014-04-23 Thread Marc Gravell
This looks to be protobuf-net specific. Note that protobuf-net includes SerializeWithLengthPrefix and DeserializeWithLengthPrefix which can be used to simplify working with multiple separate logical messages on a single stream (such as against a socket), without ever over-reading. Also - it should

Re: [protobuf] I don't understand why we use protocol buffer

2014-05-07 Thread Marc Gravell
Protocol buffers in the public project consists primarily of the serialization framework; serialization is always necessary when communicating, and frameworks/formats are aplenty, including xml, json, etc. The primary features of protocol buffers (protobuf) are: - efficient binary on the wire (sma

Re: [protobuf] Use different encoding while serializing strings

2014-05-14 Thread Marc Gravell
protobuf is a binary-safe protocol, and is not impacted by contents such as \r, \n or \t. In particular, text content is utf-8 encoded and length-prefixed - it simply *does not care* what is inside the text. I suspect any problem you are having relates to how you are transporting and processing the

Re: [protobuf] What causes TargetInvocationException during serialization

2014-05-16 Thread Marc Gravell
Oops, meant to reply-all! On 16 May 2014 19:34, "Marc Gravell" wrote: > This is specifically protobuf-net; I have received a repro case from Marco > separately via email, so I will investigate and post back here and > directly, but short version "don't know off

Re: [protobuf] ProtoContractAttribute.Name Usage?

2014-05-27 Thread Marc Gravell
Oops; meant to press reply-all, not reply, but: This relates to protobuf-net. The only time the Name property is used is if you reverse-generate from code to .proto via Serializer.GetProto (or the similar method on RuntimeTypeModel). On 23 May 2014 13:23, Sam Eaton wrote: > Hello, > > I was wo

Re: [protobuf] Protobuf-net - list of objects with parent reference

2014-06-09 Thread Marc Gravell
The protobuf specification doesn't have the notion of object identity. protobuf-net *does* (as an optionally enabled feature), but it doesn't currently work for list items directly, although I suspect it probably should. Since it would break the format, though, it would need explicit enabling if I

Re: [protobuf] Re: Deserializing Corrupted data

2014-06-24 Thread Marc Gravell
As I said yesterday (dammit, I hit "reply" instead of "reply all", so it didn't go to the group): "That depends on the nature of the corruption. Maybe yes, maybe no. You can't guarantee it. I can think of many small changes I could make that wouldn't be noticed (except for returning different data

Re: [protobuf] Using a more secure encoding other than base64 for encoding.

2014-06-27 Thread Marc Gravell
Protobuf doesn't actually use base-64 *at all*. And base-64 (and other encodings) **are not security features**; they are not intended to be secure or insecure - that simply is an unrelated concern. I suspect any use of base-64 here is simply because protobuf is binary and cookies are text, and bas

Re: [protobuf] Re: Issue 269 in protobuf: Would like to have byte or int8 type for the message definition

2014-06-30 Thread Marc Gravell
The simple answer would be "use the existing oversized types, and cast at the caller". Varint data in particular will either be 1 byte or 2 (50% each) for byte values. For longer sequences (rgba etc) there are existing fixed-32 and bytes. If the intent is to add a wire type to precisely represent

Re: [protobuf] [Protobuf] Can protobuf serialize datatable (C#)

2014-07-10 Thread Marc Gravell
sit with a > problem. we use table valued parameters in our sqlcommands. which means we > need to serialize DataTables as part of an object. I there a way we can > achieve this with protobuf. > > Do you have any workaround/alternative or suggestion? > > Regards > Desmond Davids >

Re: [protobuf] [Protobuf] Can protobuf serialize datatable (C#)

2014-07-11 Thread Marc Gravell
Table will > be. I will have to think about this more. > > Looking forward to hear from you. > > Regards > Desmond > > > On Thu, Jul 10, 2014 at 9:18 PM, Marc Gravell > wrote: > >> Ooh, the idea of allowing SQL over a service boundary sends all kinds of >>

Re: [protobuf] Protogen and wcf json serialization

2014-07-15 Thread Marc Gravell
This is specifically protobuf-net; firstly, you should be able to add -p:lightFramework to the command - that omits a few things that don't work on all frameworks, including [Serializable] iirc. However, you can also just edit CSharp.xslt to make any necessary changes. Stick it alongside protogen.

Re: [protobuf] Newbie here.. pass serializer functions to protobuf-net at runtime?

2014-08-11 Thread Marc Gravell
This is specific to protobuf-net. The library does not currently expose any custom serialization extension point; to do that in a way that is genuinely useful, while not allowing the caller to break the wire format, is quite tricky. To date: it hasn't been necessary. If you can clarify *why* you fe

<    1   2   3   4