Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 3:52:54 AM, you wrote: I fail to see how it will brake programs. Current programs do not use Unicode because it is implemented incorrectly. i use it. current Linux implementation treats String as sequence of bytes, and with manual recoding it allows to use

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Roman Beslik
On 27.06.10 04:07, Brandon S Allbery KF8NH wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/26/10 19:52 , Roman Beslik wrote: I fail to see how it will brake programs. Current programs do not use Unicode because it is implemented incorrectly. Currently, FilePath is an alias for

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Roman Beslik
On 27.06.10 03:58, Felipe Lessa wrote: On Sun, Jun 27, 2010 at 02:55:33AM +0300, Roman Beslik wrote: On 26.06.10 15:44, Felipe Lessa wrote: However, suppose your program needs to create a file with a name based on a database information. Your database is UTF-8. How do you translate that

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 11:07:47 AM, you wrote: Currently, FilePath is an alias for String. Changing FilePath to a real type Just do not change FilePath, what may be simpler? if FilePath will become abstract type, it will break all programs that use it since they use it as

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 11:11:59 AM, you wrote: No! The target encoding is the current locale. It is a no-brainer to not necessarily. current locale, encoding of current terminal and encoding of every filesystem mounted are all different things -- Best regards, Bulat

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Roman Beslik
On 27.06.10 09:38, Bulat Ziganshin wrote: Sunday, June 27, 2010, 3:52:54 AM, you wrote: I fail to see how it will brake programs. Current programs do not use Unicode because it is implemented incorrectly. i use it. current Linux implementation treats String as sequence of bytes, and with

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Roman Beslik
On 27.06.10 10:17, Bulat Ziganshin wrote: Sunday, June 27, 2010, 11:07:47 AM, you wrote: Currently, FilePath is an alias for String. Changing FilePath to a real type Just do not change FilePath, what may be simpler? if FilePath will become abstract type, it will break all programs that use

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Roman Beslik
On 27.06.10 10:18, Bulat Ziganshin wrote: Hello Roman, Sunday, June 27, 2010, 11:11:59 AM, you wrote: No! The target encoding is the current locale. It is a no-brainer to not necessarily. current locale, encoding of current terminal and encoding of every filesystem mounted are all different

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 11:24:16 AM, you wrote: O'kay, but IMHO few people want to have a headache with recoding. You knew that the implementation was incorrect, why you relied on it? what is alternative? :) on windows i've used low-level open()-styly APIs, on Linux i got the

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 11:28:49 AM, you wrote: Just do not change FilePath, what may be simpler? if FilePath will become abstract type, it will break all programs that use it since they use it as String Hello, do you read me? I said: do not change FilePath. what you mean by

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Bulat Ziganshin
Hello Roman, Sunday, June 27, 2010, 11:37:24 AM, you wrote: No! The target encoding is the current locale. It is a no-brainer to not necessarily. current locale, encoding of current terminal and encoding of every filesystem mounted are all different things And we should stick to the current

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Alexey Khudyakov
On Sun, 27 Jun 2010 10:28:49 +0300 Roman Beslik ber...@ukr.net wrote: On 27.06.10 10:17, Bulat Ziganshin wrote: Sunday, June 27, 2010, 11:07:47 AM, you wrote: Currently, FilePath is an alias for String. Changing FilePath to a real type Just do not change FilePath, what may be simpler?

Re: [Haskell-cafe] Core packages and locale support

2010-06-27 Thread Brandon S Allbery KF8NH
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/27/10 03:11 , Roman Beslik wrote: No! The target encoding is the current locale. It is a no-brainer to find it. Use your Unix. $ man setlocale $ locale So you want to use someone else's file, and their locale isn't the same as yours. What

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Roman Beslik
On 25.06.10 20:09, Jason Dagit wrote: you got everything right here. So, as you said, there is a mismatch between representation in Haskell (list of code points) and representation in the operating system (list of bytes), so we need to know the encoding. Encoding is supplied

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sat, Jun 26, 2010 at 09:29:29AM +0300, Roman Beslik wrote: Incorrect encoding of filepaths is common in e.g. Cyrillic Linux (because of multiple possible encodings --- CP1251, KOI8-R, UTF-8) and is solved by fiddling with the current locale and media mount options. No need to change a

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Bulat Ziganshin
Hello Felipe, Saturday, June 26, 2010, 4:44:20 PM, you wrote: Even if we said we don't care, we at least should change FilePath to be [Word8], and not [String]. Currently filepaths are silently truncated if any codepoint is beyond 255. and there is no OS except Unix ;) -- Best regards,

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sat, Jun 26, 2010 at 04:48:39PM +0400, Bulat Ziganshin wrote: Saturday, June 26, 2010, 4:44:20 PM, Felipe Lessa wrote: Even if we said we don't care, we at least should change FilePath to be [Word8], and not [String]. Currently filepaths are silently truncated if any codepoint is beyond

Re[2]: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Bulat Ziganshin
Hello Felipe, Saturday, June 26, 2010, 4:54:16 PM, you wrote: Even if we said we don't care, we at least should change FilePath to be [Word8], and not [String]. Currently filepaths other OSs worked fine, should I use this API (i.e. type FilePath = String) to its fullest extent, my

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sat, Jun 26, 2010 at 05:01:06PM +0400, Bulat Ziganshin wrote: Even if we said we don't care, we at least should change FilePath to be [Word8], and not [String]. Currently filepaths other OSs worked fine, should I use this API (i.e. type FilePath = String) to its fullest extent, my

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Alexey Khudyakov
On Sat, 26 Jun 2010 10:14:50 -0300 Felipe Lessa felipe.le...@gmail.com wrote: On Sat, Jun 26, 2010 at 05:01:06PM +0400, Bulat Ziganshin wrote: but what you propose cannot be used in Windows at all! while current FilePath still works on Unix, with manual filenames en/decoding Now we got

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sat, Jun 26, 2010 at 05:34:49PM +0400, Alexey Khudyakov wrote: It should solve most of problems I believe. But such change will break a lot of programs maybe most of them. How could one introduce such a change? One variant is to create new hierarchy and gradually deprecate old. Also same

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Roman Beslik
On 26.06.10 16:34, Alexey Khudyakov wrote: On Sat, 26 Jun 2010 10:14:50 -0300 Felipe Lessafelipe.le...@gmail.com wrote: On Sat, Jun 26, 2010 at 05:01:06PM +0400, Bulat Ziganshin wrote: but what you propose cannot be used in Windows at all! while current FilePath still works on Unix, with

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Roman Beslik
On 26.06.10 15:44, Felipe Lessa wrote: On Sat, Jun 26, 2010 at 09:29:29AM +0300, Roman Beslik wrote: Incorrect encoding of filepaths is common in e.g. Cyrillic Linux (because of multiple possible encodings --- CP1251, KOI8-R, UTF-8) and is solved by fiddling with the current locale and media

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sun, Jun 27, 2010 at 02:55:33AM +0300, Roman Beslik wrote: On 26.06.10 15:44, Felipe Lessa wrote: However, suppose your program needs to create a file with a name based on a database information. Your database is UTF-8. How do you translate that UTF-8 data into a filepath? This is the

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sun, Jun 27, 2010 at 02:52:54AM +0300, Roman Beslik wrote: On 26.06.10 16:34, Alexey Khudyakov wrote: On Sat, 26 Jun 2010 10:14:50 -0300 Felipe Lessafelipe.le...@gmail.com wrote: On Sat, Jun 26, 2010 at 05:01:06PM +0400, Bulat Ziganshin wrote: but what you propose cannot be used in

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Brandon S Allbery KF8NH
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/26/10 19:52 , Roman Beslik wrote: I fail to see how it will brake programs. Current programs do not use Unicode because it is implemented incorrectly. Currently, FilePath is an alias for String. Changing FilePath to a real type will break

Re: [Haskell-cafe] Core packages and locale support

2010-06-26 Thread Felipe Lessa
On Sat, Jun 26, 2010 at 10:01:57PM -0300, Felipe Lessa wrote: The types are: getArgs :: IO [String] writeFile :: FilePath - String - IO () On a similar note, getArgs probably suffers from the same problem. Which should it be? a) getArgs :: IO [String] b) getArgs :: IO [Word8] c)

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Roman Cheplyaka
* Jason Dagit da...@codersbase.com [2010-06-24 20:52:03-0700] On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka r...@ro-che.info wrote: While ghc 6.12 finally has proper locale support, core packages (such as unix) still use withCString and therefore work incorrectly when argument (e.g.

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Brandon S Allbery KF8NH
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/25/10 02:42 , Roman Cheplyaka wrote: * Jason Dagit da...@codersbase.com [2010-06-24 20:52:03-0700] On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka r...@ro-che.info wrote: While ghc 6.12 finally has proper locale support, core packages (such

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Jason Dagit
On Thu, Jun 24, 2010 at 11:42 PM, Roman Cheplyaka r...@ro-che.info wrote: * Jason Dagit da...@codersbase.com [2010-06-24 20:52:03-0700] On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka r...@ro-che.info wrote: While ghc 6.12 finally has proper locale support, core packages (such as

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Roman Cheplyaka
* Jason Dagit da...@codersbase.com [2010-06-25 10:09:21-0700] On Thu, Jun 24, 2010 at 11:42 PM, Roman Cheplyaka r...@ro-che.info wrote: * Jason Dagit da...@codersbase.com [2010-06-24 20:52:03-0700] On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka r...@ro-che.info wrote: While ghc

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Brandon S Allbery KF8NH
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/25/10 17:05 , Roman Cheplyaka wrote: By the way, GTK (which internally uses UTF-8 for strings) treats this problem differently -- it has special variable G_FILENAME_ENCODING and also G_BROKEN_FILENAMES (which means that filenames are encoded

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Roman Cheplyaka
* Brandon S Allbery KF8NH allb...@ece.cmu.edu [2010-06-25 05:00:08-0400] On 6/25/10 02:42 , Roman Cheplyaka wrote: * Jason Dagit da...@codersbase.com [2010-06-24 20:52:03-0700] On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka r...@ro-che.info wrote: While ghc 6.12 finally has proper locale

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Brandon S Allbery KF8NH
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/25/10 17:56 , Roman Cheplyaka wrote: * Brandon S Allbery KF8NH allb...@ece.cmu.edu [2010-06-25 05:00:08-0400] You might want to look at how Python is dealing with this (including the pain involved; best to learn from example). Do you mean

Re: [Haskell-cafe] Core packages and locale support

2010-06-25 Thread Jason Dagit
On Fri, Jun 25, 2010 at 3:15 PM, Brandon S Allbery KF8NH allb...@ece.cmu.edu wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/25/10 17:56 , Roman Cheplyaka wrote: * Brandon S Allbery KF8NH allb...@ece.cmu.edu [2010-06-25 05:00:08-0400] You might want to look at how Python is

Re: [Haskell-cafe] Core packages and locale support

2010-06-24 Thread Jason Dagit
On Sat, Jun 19, 2010 at 1:06 AM, Roman Cheplyaka r...@ro-che.info wrote: While ghc 6.12 finally has proper locale support, core packages (such as unix) still use withCString and therefore work incorrectly when argument (e.g. file path) is not ASCII. Pardon me if I'm misunderstanding

[Haskell-cafe] Core packages and locale support

2010-06-19 Thread Roman Cheplyaka
While ghc 6.12 finally has proper locale support, core packages (such as unix) still use withCString and therefore work incorrectly when argument (e.g. file path) is not ASCII. Is someone already working on this? If it's just a matter of time and manpower I can help but I need some guidance from