Protocol Buffers has a "bytes" type.  That's what it's talking about.  Just
change "string" to "bytes" in your .proto file.  (They work exactly the same
in C++ but are different in Java and Python.)
On Tue, May 12, 2009 at 6:47 AM, <dan.schmidt.va...@gmail.com> wrote:

>
> I am having a very similar problem. Just installed the 2.0.3 version
> and now all my serialisations complain.
>
> libprotobuf ERROR ./google/protobuf/wire_format_inl.h:138] Encountered
> string containing invalid UTF-8 data while parsing protocol buffer.
> Strings must contain only UTF-8; use the 'bytes' type for raw bytes.
>
> Now, C++ doesn't have a byte type. Just signed or unsigned chars, and
> string is an array of those. So, what does it need? Would I be better
> off serialising to a stream like the CodedStream?
>
> I am very confused on the issue. I have the horrible feeling now that
> I'm losing efficiency because serialising to string might mean that
> I'm losing my raw data.
>
> Otherwise, then the word ERROR on the output might be a bit too
> strong.
>
> If anybody can clarify, I'd be very grateful.
>
> Dan
>
> On May 10, 5:59 pm, Henner Zeller <h.zel...@acm.org> wrote:
> > On Sun, May 10, 2009 at 6:08 AM, edan <edan...@gmail.com> wrote:
> > > I have some fields that may contain non-UTF8 data.
> > > I understand that I just need to change their type from "string" to
> "bytes"
> > > and it should just work, transparently.
> >
> > yes. The're the same on the wire.
> >
> > > I have a few fields that probably will only contain ASCII i.e. legal
> UTF8,
> > > but I'm not 100% sure.
> > > I am tempted to just turn them all to "bytes".
> > > But this begs the question - what is the "string" type useful for, and
> why
> > > shouldn't I just always use "bytes" to be sure, all the time, and not
> both
> > > with "string" at all?
> > > Does "string" add anything besides validation that only valid UTF8 is
> > > passing over the wire?  Is there really a big benefit to this
> behavior?  Or
> > > is there some other advantage that I'll miss out on by changing all my
> > > "string"s to "bytes"?
> >
> > If you use the C++ api there is not much difference since both types
> > are represented as std::string in the API. It makes a big difference
> > for the Java API (and Python?), that have a native type for an UTF-8
> > string. In Java, if you deal with a protocol buffer 'string' type, the
> > generated API will return a java.lang.String while otherwise it will
> > return a ByteString. ByteString can hold any character while the
> > native Java String works only for UTF-8. So while 'ByteString' is more
> > flexible, 'String' is more convenient to deal with within Java code
> > because all string manipulation libraries can handle it.
> >
> > So the benefit is a more convenient Api in the generated Java code.
> > And as well documentation: if you use 'string' you emphasize that a
> > field only contains readable text while 'bytes' might contain any
> > binary blob.
> >
> > -h
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply via email to