Aside: > lookPairs :: RqData [(String,String)] > lookPairs = asks fst >>= return . map (\(n,vbs)->(n,L.unpack $ inputValue vbs))
Looks like an opportunity for semantic editor combinators [1]. Something like > lookPairs = (fmap.fmap.fmap) (L.unpack . inputValue) (asks fst) Or specialize the edit path to (fmap.map.second) . - Conal [1] http://conal.net/blog/semantic-editor-combinators On Tue, Dec 30, 2008 at 6:14 AM, Jeremy Shaw <jer...@n-heptane.com> wrote: > Hello, > > I have not looked at the gitit source code, but I have had this > problem in other HAppS applications. The problem is that by default > HAppS does nothing about string encodings. The easy fix is to use > utf-8 and unicode everywhere. ('easy' compared to supporting multiple > encodings). > > The goal is to make sure that in gitit, a String is always a list of > unicode code points, and not a list of utf-8 encoded octets. This > means that whenever data comes in or goes out of gitit it needs to be > decoded or encoded. > > To transition you need to do atleast the following: > > 1. Set the charset of the outgoing pages so that the browser knows > that the pages is supposed to be utf-8: > > For html, this can be done by adding this meta to the <head> of each page: > > <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> > > However, for text/plain, etc, you must set it in the HTTP header > (which I will cover later). For html, it is still useful to set the > meta tag though, so that if the page is saved to disk, the encoding > is not lost. > > 2. use the utf8-string library, and make sure that all the > inputs/outputs are decoded/encoded properly. > > This probably means patching your copy of HAppS-Server (or copying the > modified functions into gitit). > > For example, lookPairs currently looks like this: > > > lookPairs :: RqData [(String,String)] > > lookPairs = asks fst >>= return . map (\(n,vbs)->(n,L.unpack $ inputValue > vbs)) > > As you can see, it just takes the incoming bytes and converts them to > a String, but without doing any decoding. You probably want something > more like: > > > lookPairs :: RqData [(String,String)] > > lookPairs = asks fst >>= return . map > (\(n,vbs)->(n,Data.ByteString.Lazy.UTF8.toString $ inputValue vbs)) > > Some of the other look* functions need patching as well. > > Similarily, the ToMessage instances need to encode the outgoing data. > Consider: > > > instance ToMessage Html where > > toContentType _ = B.pack "text/html" > > toMessage = L.pack . renderHtml > > We really want to make two changes: > > > instance ToMessage Html where > > toContentType _ = B.pack "text/html; charset=UTF-8" -- add > the encoding > > toMessage = Data.ByteString.Lazy.UTF8.fromString . renderHtml -- > encode the data > > 3. make sure that any I/O (readFile, writeFile, etc) uses the utf-8 > functions from utf8-string. > > If you don't want to patch HAppS-Server, then you could work around it by > doing silliness like: > > do pairs' <- lookPairs > let pairs = map (first toString . second toString) pairs' > > but that seems error prone and not a long term solution. The obvious > long term solution is for HAppS to fix its encoding issues. The simple > fix is to hardwire it for utf-8, but a system that would supports > arbitrary encodings might be nice? > > As far as I know, no one has even tried to submit a patch hardwiring > HAppS to use utf-8 -- which seems like a good short-term solution. You > might try posting on the HAppS mailing list and see if such a patch > would be welcome: > > http://groups.google.com/group/HAppS > > hope this helps. > - jeremy > > > At Tue, 30 Dec 2008 13:58:15 +0100, > Arnaud Bailly wrote: > > > > Hello, > > I have started using Gitit and I am very happy with it and eager to > > start hacking. I am running into a practical problem: characters > > encoding. When I edit pages using accented characters (I am french), > > the accents get mangled when the page come back from server. > > > > The raw files are incorrectly encoded. Where Shall I look for fixing > > this issue ? > > > > Thanks > > > > ps: the wiki is live at http://www.notre-ecole.org(some of the other > look funct > > > > -- > > Arnaud Bailly, PhD > > OQube - Software Engineering > > > > web> http://www.oqube.com > > _______________________________________________ > > Haskell-Cafe mailing list > > Haskell-Cafe@haskell.org > > http://www.haskell.org/mailman/listinfo/haskell-cafe > _______________________________________________ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe >
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe