On Wed, Aug 21, 2013 at 11:30:12AM +0100, Gordon Sim wrote: > On 08/20/2013 09:46 PM, Jimmy Jones wrote: > >Hi, > > > >Darryl tells me Justin is going to post about handling utf8/binary > >strings in dynamic languages, > > Justin, when you do so, can I request that you do so on the user > list as there may be others there using these APIs with a view or > just interested in the discussion. > > >but I'm going to slightly jump the gun > >and post about what Darryl and I have been looking at. > > > >AFAIK in perl, if you include unicode characters in a string it'll > >set the utf8 flag. If you don't include any unicode characters (eg. 7 > >bit ascii, or raw bytes) the flag won't be set. So given a perl > >scalar that doesn't contain any utf8 characters, you don't know if > >its a textual string (str16) or a binary string (vbin). There is a > >is_utf8_string function, but that'll only tell you if the string > >would be valid utf8, but it could be a binary string that happens to > >be valid utf8, so that's not really safe. > > You can explicitly mark it as utf8 using utf8::upgrade() though, > right? Certainly I tried that in a simple test and the property in > question was then sent as str16.
I modified the spout.pl example to do this for both the name and value on properties pass in at the command line and they were seen by the Java drain tool (yay!). So, narrowing down the goal slightly, WRT QPID-5067, do we want to 1) require the users to explicitly called utf8::upgrade() on the property name and value and assume that, whatever they pass in is correct, or do we 2) assume it for them and upgrade the string in our porcelain layer? In this case, since message properties are always strings, the latter seems to me to be the right path. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/
pgpkAljAex6kP.pgp
Description: PGP signature
