Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Brandon S Allbery KF8NH
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/27/10 03:11 , Roman Beslik wrote: > No! The target encoding is the current locale. It is a no-brainer to find > it. Use your Unix. > $ man setlocale > $ locale So you want to use someone else's file, and their locale isn't the same as yours. Wha

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Alexey Khudyakov
On Sun, 27 Jun 2010 10:28:49 +0300 Roman Beslik wrote: > On 27.06.10 10:17, Bulat Ziganshin wrote: > > Sunday, June 27, 2010, 11:07:47 AM, you wrote: > >>> Currently, FilePath is an alias for String. Changing FilePath to a real > >>> type > >> Just do not change FilePath, what may be simpler?

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 11:37:24 AM, you wrote: >>> No! The target encoding is the current locale. It is a no-brainer to >> not necessarily. current locale, encoding of current terminal and >> encoding of every filesystem mounted are all different things > And we should stick to the c

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 11:28:49 AM, you wrote: >>> Just do not change FilePath, what may be simpler? >> if FilePath will become abstract type, it will break all programs >> that use it since they use it as String > Hello, do you read me? I said: "do not change FilePath". what you mea

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 11:24:16 AM, you wrote: > O'kay, but IMHO few people want to have a headache with recoding. You > knew that the implementation was incorrect, why you relied on it? what is alternative? :) on windows i've used low-level open()-styly APIs, on Linux i got the sa

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Roman Beslik
On 27.06.10 10:18, Bulat Ziganshin wrote: Hello Roman, Sunday, June 27, 2010, 11:11:59 AM, you wrote: No! The target encoding is the current locale. It is a no-brainer to not necessarily. current locale, encoding of current terminal and encoding of every filesystem mounted are all different

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Roman Beslik
On 27.06.10 10:17, Bulat Ziganshin wrote: Sunday, June 27, 2010, 11:07:47 AM, you wrote: Currently, FilePath is an alias for String. Changing FilePath to a real type Just do not change FilePath, what may be simpler? if FilePath will become abstract type, it will break all programs that use i

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Roman Beslik
On 27.06.10 09:38, Bulat Ziganshin wrote: Sunday, June 27, 2010, 3:52:54 AM, you wrote: I fail to see how it will brake programs. Current programs do not use Unicode because it is implemented incorrectly. i use it. current Linux implementation treats String as sequence of bytes, and with manua

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 11:11:59 AM, you wrote: > No! The target encoding is the current locale. It is a no-brainer to not necessarily. current locale, encoding of current terminal and encoding of every filesystem mounted are all different things -- Best regards, Bulat

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 11:07:47 AM, you wrote: >> Currently, FilePath is an alias for String. Changing FilePath to a real >> type > Just do not change FilePath, what may be simpler? if FilePath will become abstract type, it will break all programs that use it since they use it as S

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Roman Beslik
On 27.06.10 03:58, Felipe Lessa wrote: On Sun, Jun 27, 2010 at 02:55:33AM +0300, Roman Beslik wrote: On 26.06.10 15:44, Felipe Lessa wrote: However, suppose your program needs to create a file with a name based on a database information. Your database is UTF-8. How do you translate that UT

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Roman Beslik
On 27.06.10 04:07, Brandon S Allbery KF8NH wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/26/10 19:52 , Roman Beslik wrote: I fail to see how it will brake programs. Current programs do not use Unicode because it is implemented incorrectly. Currently, FilePath is an alias for Strin

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 3:52:54 AM, you wrote: > I fail to see how it will brake programs. Current programs do not use > Unicode because it is implemented incorrectly. i use it. current Linux implementation treats String as sequence of bytes, and with manual recoding it allows to use

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sat, Jun 26, 2010 at 10:01:57PM -0300, Felipe Lessa wrote: > The types are: > > getArgs :: IO [String] > writeFile :: FilePath -> String -> IO () On a similar note, getArgs probably suffers from the same problem. Which should it be? a) getArgs :: IO [String] b) getArgs :: IO [Word8]

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Brandon S Allbery KF8NH
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/26/10 19:52 , Roman Beslik wrote: > I fail to see how it will brake programs. Current programs do not use > Unicode because it is implemented incorrectly. Currently, FilePath is an alias for String. Changing FilePath to a real type will break pr

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sun, Jun 27, 2010 at 02:52:54AM +0300, Roman Beslik wrote: > On 26.06.10 16:34, Alexey Khudyakov wrote: > >On Sat, 26 Jun 2010 10:14:50 -0300 > >Felipe Lessa wrote: > >>On Sat, Jun 26, 2010 at 05:01:06PM +0400, Bulat Ziganshin wrote: > >>>but what you propose cannot be used in Windows at all!

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sun, Jun 27, 2010 at 02:55:33AM +0300, Roman Beslik wrote: > On 26.06.10 15:44, Felipe Lessa wrote: > >However, suppose your program needs to create a file with a name > >based on a database information. Your database is UTF-8. How do > >you translate that UTF-8 data into a filepath? This is

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Roman Beslik
On 26.06.10 15:44, Felipe Lessa wrote: On Sat, Jun 26, 2010 at 09:29:29AM +0300, Roman Beslik wrote: Incorrect encoding of filepaths is common in e.g. Cyrillic Linux (because of multiple possible encodings --- CP1251, KOI8-R, UTF-8) and is solved by fiddling with the current locale and media mo

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Roman Beslik
On 26.06.10 16:34, Alexey Khudyakov wrote: On Sat, 26 Jun 2010 10:14:50 -0300 Felipe Lessa wrote: On Sat, Jun 26, 2010 at 05:01:06PM +0400, Bulat Ziganshin wrote: but what you propose cannot be used in Windows at all! while current FilePath still works on Unix, with manual filenames en/decodi

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sat, Jun 26, 2010 at 05:34:49PM +0400, Alexey Khudyakov wrote: > It should solve most of problems I believe. But such change will break > a lot of programs maybe most of them. How could one introduce such a > change? One variant is to create new hierarchy and gradually deprecate > old. > > Also

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Alexey Khudyakov
On Sat, 26 Jun 2010 10:14:50 -0300 Felipe Lessa wrote: > On Sat, Jun 26, 2010 at 05:01:06PM +0400, Bulat Ziganshin wrote: > > but what you propose cannot be used in Windows at all! while current > > FilePath still works on Unix, with manual filenames en/decoding > > Now we got back on topic! :) >

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sat, Jun 26, 2010 at 05:01:06PM +0400, Bulat Ziganshin wrote: > >> > Even if we said "we don't care", we at least should change > >> > FilePath to be [Word8], and not [String]. Currently filepaths > > > other OSs worked fine, should I use this API (i.e. type FilePath > > = String) to its fulles

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Bulat Ziganshin
Hello Felipe, Saturday, June 26, 2010, 4:54:16 PM, you wrote: >> > Even if we said "we don't care", we at least should change >> > FilePath to be [Word8], and not [String]. Currently filepaths > other OSs worked fine, should I use this API (i.e. type FilePath > = String) to its fullest extent,

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sat, Jun 26, 2010 at 04:48:39PM +0400, Bulat Ziganshin wrote: > Saturday, June 26, 2010, 4:44:20 PM, Felipe Lessa wrote: > > Even if we said "we don't care", we at least should change > > FilePath to be [Word8], and not [String]. Currently filepaths > > are silently "truncated" if any codepoint

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Bulat Ziganshin
Hello Felipe, Saturday, June 26, 2010, 4:44:20 PM, you wrote: > Even if we said "we don't care", we at least should change > FilePath to be [Word8], and not [String]. Currently filepaths > are silently "truncated" if any codepoint is beyond 255. and there is no OS except Unix ;) -- Best rega

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sat, Jun 26, 2010 at 09:29:29AM +0300, Roman Beslik wrote: > Incorrect encoding of filepaths is common in e.g. Cyrillic Linux > (because of multiple possible encodings --- CP1251, KOI8-R, UTF-8) > and is solved by fiddling with the current locale and media mount > options. No need to change a pr

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Roman Beslik
On 25.06.10 20:09, Jason Dagit wrote: you got everything right here. So, as you said, there is a mismatch between representation in Haskell (list of code points) and representation in the operating system (list of bytes), so we need to know the encoding. Encoding is supplied by

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Jason Dagit
On Fri, Jun 25, 2010 at 3:15 PM, Brandon S Allbery KF8NH < allb...@ece.cmu.edu> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 6/25/10 17:56 , Roman Cheplyaka wrote: > > * Brandon S Allbery KF8NH [2010-06-25 > 05:00:08-0400] > >> You might want to look at how Python is dealing wi

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Brandon S Allbery KF8NH
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/25/10 17:56 , Roman Cheplyaka wrote: > * Brandon S Allbery KF8NH [2010-06-25 05:00:08-0400] >> You might want to look at how Python is dealing with this (including the >> pain involved; best to learn from example). > > Do you mean the pain when

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Roman Cheplyaka
* Brandon S Allbery KF8NH [2010-06-25 05:00:08-0400] > On 6/25/10 02:42 , Roman Cheplyaka wrote: > > * Jason Dagit [2010-06-24 20:52:03-0700] > >> On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka wrote: > >>> While ghc 6.12 finally has proper locale support, core packages (such as > >>> unix) st

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Brandon S Allbery KF8NH
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/25/10 17:05 , Roman Cheplyaka wrote: > By the way, GTK (which internally uses UTF-8 for strings) treats this > problem differently -- it has special variable G_FILENAME_ENCODING and > also G_BROKEN_FILENAMES (which means that filenames are encoded

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Roman Cheplyaka
* Jason Dagit [2010-06-25 10:09:21-0700] > On Thu, Jun 24, 2010 at 11:42 PM, Roman Cheplyaka wrote: > > > * Jason Dagit [2010-06-24 20:52:03-0700] > > > On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka > > wrote: > > > > > > > While ghc 6.12 finally has proper locale support, core packages (su

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Jason Dagit
On Thu, Jun 24, 2010 at 11:42 PM, Roman Cheplyaka wrote: > * Jason Dagit [2010-06-24 20:52:03-0700] > > On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka > wrote: > > > > > While ghc 6.12 finally has proper locale support, core packages (such > as > > > unix) still use withCString and therefore

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Brandon S Allbery KF8NH
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/25/10 02:42 , Roman Cheplyaka wrote: > * Jason Dagit [2010-06-24 20:52:03-0700] >> On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka wrote: >>> While ghc 6.12 finally has proper locale support, core packages (such as >>> unix) still use withCStri

Re: [Haskell-cafe] Core packages and locale support

2010-06-24 Thread Roman Cheplyaka
* Jason Dagit [2010-06-24 20:52:03-0700] > On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka wrote: > > > While ghc 6.12 finally has proper locale support, core packages (such as > > unix) still use withCString and therefore work incorrectly when argument > > (e.g. file path) is not ASCII. > > >

Re: [Haskell-cafe] Core packages and locale support

2010-06-24 Thread Jason Dagit
On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka wrote: > While ghc 6.12 finally has proper locale support, core packages (such as > unix) still use withCString and therefore work incorrectly when argument > (e.g. file path) is not ASCII. > Pardon me if I'm misunderstanding withCString, but my u

[Haskell-cafe] Core packages and locale support

2010-06-19 Thread Roman Cheplyaka
While ghc 6.12 finally has proper locale support, core packages (such as unix) still use withCString and therefore work incorrectly when argument (e.g. file path) is not ASCII. Is someone already working on this? If it's just a matter of time and manpower I can help but I need some guidance from a