Re: [protobuf] how can we deal with multi-lingual string? (c++)

2010-12-28 Thread Kenton Varda
The dirty little secret of std::wstring is that it does not actually deal
with non-ASCII characters on all platforms.  On some platforms wchar_t is
8-bit just like char!  You should avoid using wstring and wchar_t for this
reason; define your own types that are exactly what you need.

For protocol buffers, we take the convention of using regular 8-bit chars
but always using UTF-8 encoding.

On Sat, Dec 25, 2010 at 8:10 PM, alcohol  wrote:

> without std::wstring support,  how can we deal with strings consists
> of  Ascii, Chinese,Japanese, Korean characters?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to proto...@googlegroups.com.
> To unsubscribe from this group, send email to
> protobuf+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/protobuf?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.



Re: [protobuf] how can we deal with multi-lingual string? (c++)

2010-12-27 Thread Evan Jones

On Dec 25, 2010, at 21:10 , alcohol wrote:

without std::wstring support,  how can we deal with strings consists
of  Ascii, Chinese,Japanese, Korean characters?


It is expected that you put UTF-8 encoded characters into protocol  
buffer strings. Alternatively, you can use the bytes type and use any  
encoding you want, but then you'll need to handle the conversions,  
whereas if you use Java, the protocol buffer API handles conversions  
for you (to/from Java String, which uses Unicode). Hope that helps,


Evan

--
Evan Jones
http://evanjones.ca/

--
You received this message because you are subscribed to the Google Groups "Protocol 
Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.



[protobuf] how can we deal with multi-lingual string? (c++)

2010-12-26 Thread alcohol
without std::wstring support,  how can we deal with strings consists
of  Ascii, Chinese,Japanese, Korean characters?

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.