Re: [Pharo-users] Problem with Traditional and Simplified Chinese parsing in Pharo

2018-01-29 Thread Clément Béra
Thanks all, Sven solution works. Yes this is not JSON it's some kind of JSON-like format (Is it yml ? I don't know it might be proprietary). I was naive and thought there was some metadata in files precising the encoding used, and that #readStream on FileReference was able to pick automatically

Re: [Pharo-users] Problem with Traditional and Simplified Chinese parsing in Pharo

2018-01-29 Thread Cyril Ferlicot D.
Le 29/01/2018 à 18:49, Clément Bera a écrit : > Hi, > > I am currently parsing lua and JSON-like file in Pharo. They contain > both Simplified and Traditional Chinese characters for comments and for > strings displayed in the UI. Lua files are parsed correctly. However the > JSON-like files

Re: [Pharo-users] Problem with Traditional and Simplified Chinese parsing in Pharo

2018-01-29 Thread Sven Van Caekenberghe
Your file is not in UTF-8 but in UTF-16 ! This will do: (FileLocator desktop / 'schinese.txt') readStreamDo: [ :in | (ZnCharacterReadStream on: in binary encoding: #utf16) upToEnd ]. BTW, this is not valid JSON. From Windows, for sure ... > On 29 Jan 2018, at 18:49, Clément Bera

Re: [Pharo-users] Problem with Traditional and Simplified Chinese parsing in Pharo

2018-01-29 Thread Guillermo Polito
Are you sure that the file is encoded in utf8? Can you try stream := ZnCharacterReadStream on: (File named: '...') readStream encoding: 'utf8'. stream upToEnd. ? If that does not work, it could mean that the file is in another encoding... 2018-01-29 18:49 GMT+01:00 Clément Bera

[Pharo-users] Problem with Traditional and Simplified Chinese parsing in Pharo

2018-01-29 Thread Clément Bera
Hi, I am currently parsing lua and JSON-like file in Pharo. They contain both Simplified and Traditional Chinese characters for comments and for strings displayed in the UI. Lua files are parsed correctly. However the JSON-like files aren't. In attachment I put one of the problematic file with