Hi all! Last day I was trying to fix idiii library, because it uses utf8 for parsing non-unicode content. I found the functions
> -- | Parses one value and returns it as a 'String' > parseString :: CharEncoding -> TagParser String > parseString enc = do > v <- case enc of > 0x01 -> parseUntilWord16Null -- UTF-16 > 0x02 -> parseUntilWord16Null -- UTF-16 BOM > _ -> parseUntilWord8Null -- ISO-8859-1 or UTF-8 > return $ encPack enc v > > encPack :: CharEncoding -> [Token] -> String > encPack 0x00 s = Text.unpack $ decodeASCII $ BS.pack s > encPack 0x01 (0xFF:0xFE:s) = Text.unpack $ decodeUtf16LE $ BS.pack s > encPack 0x01 (0xFE:0xFF:s) = Text.unpack $ decodeUtf16BE $ BS.pack s > encPack 0x02 s = Text.unpack $ decodeUtf16BE $ BS.pack s > encPack _ s = Text.unpack $ decodeUtf8 $ BS.pack s updated the dependency from > import Data.Text.Encoding (decodeASCII, decodeUtf16LE, decodeUtf16BE, > decodeUtf8) to > import Data.Text.ICU.Convert and added implementation for decoding functions: > decodeAny :: String -> BS.ByteString -> Text.Text > decodeAny charset src = unsafePerformIO $ ((flip toUnicode) src) `fmap` open > charset (Just True) > > decodeASCII :: BS.ByteString -> Text.Text > decodeASCII = decodeAny "latin1" > > decodeUtf16LE = decodeAny "utf-16le" > > decodeUtf16BE = decodeAny "utf-16be" > > decodeUtf8 = decodeAny "utf-8" Now I want to add possibility to specify encoding to yse with decodeASCII. I was thinking of adding Reader monad and providing some sort of charset configuration there - but it will lead up to complicating the code, which uses this parseString function. And this code is used inside Parser of Text.ParserCombinators.Poly.State - so I will need to update all usages of this parser. Another approach might be to use IORef with encoding stored there, but I don't really like this solution. What would be the best way of refactoring of such kind? Thanks! -- Eugene N Dzhurinsky
pgp9sNFLGUKtm.pgp
Description: PGP signature
_______________________________________________ Haskell-Cafe mailing list [email protected] http://www.haskell.org/mailman/listinfo/haskell-cafe
