-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On Sat, Nov 22, 2008 at 8:46 AM, Eric Kow wrote: > On Sat, Nov 22, 2008 at 13:18:43 +1100, Trent W. Buck wrote: >> You're talking about removing src/UTF8.lhs, which is a convenience >> copy of a separate library (called "libghc6-utf8-string-dev" on >> Debian). Or at least, making the system copy of utf8-string be used >> in preference to src/UTF8.lhs. > > src/UTF8.lhs is a different library from utf8-string > I think the former was never cabalised or put onto hackage
Right. They have a few differences though; for example, utf8-string handles 'bad' sections of a string by adding error markers, and UTF8.lhs does... something else. I think it returns a messed up string if there are problems with it. But as I said before, so far as I know, we shouldn't have to care about possibly erroneous strings because the only function in UTF8.lhs is 'encode :: String -> [Word8]', and the Haskell runtime will never give us a malformed String. (If we do get a bad string, certainly nothing encode can do will make the situation worse.) > Thanks to Gwern, actually, we're only using half of the library, > the decoding functions having been stripped away. > > Also, another place where we have utf8 functionality to look > into is src/ByteStringUtils.hs and src/fpstring.c The crucial item in ByteStringUtils.hs is unpackPSfromUTF8 :: ByteString -> String. I believe utf8-string handles this: http://hackage.haskell.org/packages/archive/utf8-string/0.3.3/doc/html/Data-ByteString-UTF8.html#v%3AtoString 'Convert a UTF8 encoded bytestring into a Haskell string. Invalid characters are replaced with '\xFFFD'.' (Note what I said earlier about invalid characters.) unpackPSfromUTF8 is the only use of fpstring.h's 'int utf8_to_ints(HsInt *pwc, const unsigned char *s, int n);' - -- gwern -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEAREKAAYFAkkzBvMACgkQvpDo5Pfl1oKHqQCgnOgpl2Wb7A7gN+ttg+HNjGlv 3CoAn3Piy39dcDy5UTbmsxlDsbXHgCnJ =wUbT -----END PGP SIGNATURE----- _______________________________________________ darcs-users mailing list [email protected] http://lists.osuosl.org/mailman/listinfo/darcs-users
