[Flashcoders] Hands up who understands utf-8 encoding?

2011-05-27 Thread Glen Pike

Hi,

I have run into a problem that seems pretty bizarre, but may be a 
silly error on my part.


I have a function to encode strings in Flash to hex values, but it 
needs to deal with various languages and special characters, so needs to 
handle Unicode strings.


I had a function - it works fine in Windows and used to work on the 
target platform - running 10.0.45.2 standalone debug player on Gentoo Linux


All of a sudden - the encoding has stopped working and is creating 
garbage so is not working any more.


We have changed the hardware and probably the kernel version 
lately, so this is the only thing that I think which could effect it, 
but I maybe wrong.


I have an old function / workaround which I can use, but if someone 
could explain wtf is going on I would be greatful.  Code is posted below 
for your amusement.


Thanks

Glen
/*Example output (there is a newline \n between Simon and speech - 
note the disparity between the ByteArray version and the test / older 
function.


UniStr2UTF8Hex 4/4 read, hex is b4 9a 7e b5  from str voice=Simon
speech=My name is Robo Thespian.
UniStr2UTF8Hex Test str 76 6f 69 63 65 3d 53 69 6d 6f 6e 0a 73 70 65 65 
63 68 3d 4d 79 20 6e 61 6d 65 20 69 73 20 52 6f 62 6f 20 54 68 65 73 70 
6 9 61 6e 2e

*/
public function UniStr2UTF8Hex(str:String):String {
var hex:String = ;
var i:int = 0;

var bytes:ByteArray = new ByteArray();
bytes.writeMultiByte(str, utf-8);
bytes.position = 0;
var byte:String;
try {
while (i  bytes.length) {
byte = Number(bytes.readUnsignedByte()).toString(16);
if (1 == byte.length) {
byte = 0 + byte;
}
hex += byte +  ;
i++;
}
} catch (e:EOFError) {
debug(UniStr2UTF8Hex, finished at  + i +  len  + 
bytes.length, Dbg.DEBUG_MAX);

}
debug(UniStr2UTF8Hex, i + / + bytes.length + read, hex is 
 + hex +  from str  + str, Dbg.DEBUG_HIGH);


return hex.split( ).join();
}

In contrast, this function works fine:

public function UniStr2UTF8Hex(str:String):String {
var test:String = ;
for (i = 0; i  str.length; i++) {
var ucs:int = str.charCodeAt(i);
var ch:String;
if (128  ucs) {
ch = ucs.toString(16);
if (1 == ch.length) {
ch = 0 + ch;
}
test += ch +  ;
} else if (128 = ucs  2047 = ucs) {
ch = Number(192 + (ucs / 64)).toString(16);
if (1 == ch.length) {
ch = 0 + ch;
}
test += ch +  ;
ch = Number(128 + (ucs % 64)).toString(16);
if (1 == ch.length) {
ch = 0 + ch;
}
test += ch +  ;
}

}
debug(UniStr2UTF8Hex, Test str  + test, Dbg.DEBUG_HIGH);
return test.split( ).join();
}


___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders


Re: [Flashcoders] Hands up who understands utf-8 encoding?

2011-05-27 Thread Glen Pike

Hmm,

Think it's a bug in Flashplayer 10.0.45.2 which is not in a later 
version - 10.2.152.28


Glen

On 27/05/2011 12:09, Glen Pike wrote:

Hi,

I have run into a problem that seems pretty bizarre, but may be a 
silly error on my part.


I have a function to encode strings in Flash to hex values, but it 
needs to deal with various languages and special characters, so needs 
to handle Unicode strings.


I had a function - it works fine in Windows and used to work on 
the target platform - running 10.0.45.2 standalone debug player on 
Gentoo Linux


All of a sudden - the encoding has stopped working and is creating 
garbage so is not working any more.


We have changed the hardware and probably the kernel version 
lately, so this is the only thing that I think which could effect it, 
but I maybe wrong.


I have an old function / workaround which I can use, but if 
someone could explain wtf is going on I would be greatful.  Code is 
posted below for your amusement.


Thanks

Glen
/*Example output (there is a newline \n between Simon and speech - 
note the disparity between the ByteArray version and the test / older 
function.


UniStr2UTF8Hex 4/4 read, hex is b4 9a 7e b5  from str voice=Simon
speech=My name is Robo Thespian.
UniStr2UTF8Hex Test str 76 6f 69 63 65 3d 53 69 6d 6f 6e 0a 73 70 65 
65 63 68 3d 4d 79 20 6e 61 6d 65 20 69 73 20 52 6f 62 6f 20 54 68 65 
73 70 6 9 61 6e 2e

*/
public function UniStr2UTF8Hex(str:String):String {
var hex:String = ;
var i:int = 0;

var bytes:ByteArray = new ByteArray();
bytes.writeMultiByte(str, utf-8);
bytes.position = 0;
var byte:String;
try {
while (i  bytes.length) {
byte = Number(bytes.readUnsignedByte()).toString(16);
if (1 == byte.length) {
byte = 0 + byte;
}
hex += byte +  ;
i++;
}
} catch (e:EOFError) {
debug(UniStr2UTF8Hex, finished at  + i +  len  + 
bytes.length, Dbg.DEBUG_MAX);

}
debug(UniStr2UTF8Hex, i + / + bytes.length + read, hex is 
 + hex +  from str  + str, Dbg.DEBUG_HIGH);


return hex.split( ).join();
}

In contrast, this function works fine:

public function UniStr2UTF8Hex(str:String):String {
var test:String = ;
for (i = 0; i  str.length; i++) {
var ucs:int = str.charCodeAt(i);
var ch:String;
if (128  ucs) {
ch = ucs.toString(16);
if (1 == ch.length) {
ch = 0 + ch;
}
test += ch +  ;
} else if (128 = ucs  2047 = ucs) {
ch = Number(192 + (ucs / 64)).toString(16);
if (1 == ch.length) {
ch = 0 + ch;
}
test += ch +  ;
ch = Number(128 + (ucs % 64)).toString(16);
if (1 == ch.length) {
ch = 0 + ch;
}
test += ch +  ;
}

}
debug(UniStr2UTF8Hex, Test str  + test, Dbg.DEBUG_HIGH);
return test.split( ).join();
}


___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders




___
Flashcoders mailing list
Flashcoders@chattyfig.figleaf.com
http://chattyfig.figleaf.com/mailman/listinfo/flashcoders