Re: ld memory usage
You could also have a look at the new 'Gold' linker that's part of binutils. Not sure if this has been tested before with ghc but its meant to have far greater performance (in time is what I've heard, not sure about memory) then 'ld' and be a drop in replacement. On 17 April 2010 00:00, Roman Beslik ber...@ukr.net wrote: Hi. This is not strictly a GHC question. I observed that ld when linking GHC-compiled programs eats 0.5 GB of resident memory. ~3 times more than GHC or Haddock. So if there is not enough free memory ld uses virtual memory and works very slowly (practically does not use CPU). I'd like to know, is anyone interested in researching and improving memory usage? Is it ld or GHC problem? I found a similar discussion here http://article.gmane.org/gmane.comp.lang.haskell.cafe/66189 but it was too broad and went nowhere. -- Best regards, Roman Beslik. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: FFI calls: is it possible to allocate a small memory block on a stack?
While alloca is not as cheap as, say, C's alloca, you should find that it is much quicker than C's malloc. I'm sure there's room for optimisation if it's critical for you. There may well be low-hanging fruit: take a look at the Core for alloca. Thank you, Simon. Indeed, there is a low-hanging fruit. alloca's type is Storable a = (Ptr a - IO b) - IO b and it is not inlined even though the function is small. And calls to functions of such signature are expensive (I suppose that's because of look-up into typeclass dictionary). However, when I added an INLINE pragma for the function into Foreign.Marshal.Alloc the time of execution dropped from 40 to 20 nanoseconds. I guess the same effect will take place if other similar functions get marked with INLINE. Is there a reason why we do not want small FFI-related functions with typeclass arguments be marked with INLINE pragma and gain a performance improvement? The only reason that comes to my mind is the size of code, but actually the resulting code looks very small and neat. With kind regards, Denys Rtveliashvili ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Output character encoding for ghc on OpenBSD
Hi, as some of you may know, I'm working on an update of OpenBSDs ghc port to 6.12.2, currently chasing down the last remaining testsuite failures. Yesterday, I ran into a problem which I have a fix for, but only a really ugly fix, and I need some opinions of what users would prefer. The problem is that Haskell uses unicode characters internally (ghc itself uses UTF-32 internally, where the endianess depends on the architecture it's running on), and that any Haskell program (including ghc and ghci) has to convert between the internal representation and the actual locale settings of the system it's running on. Unfortunately, OpenBSD is really bad if it comes to locale support; the only supported locales are the C and the POSIX locales, so even if you set LC_ALL or LC_CTYPE to something like, for example, de_DE.iso88591, this would have no effect on OpenBSD. Anyway, the short story is that I have to either hard-code the character set to something like utf-8, or ghc will start to behave really strange (for example, ghci would terminate immediately if you just *type* a non-ASCII character). So what would you prefer? - Use utf-8 and only utf-8 (i.e. hardcoded)? - Use something like iso-8859-15 (hardcoded)? - Make it configurable via some non-standard environment variable (GHC_CODESET, for example). If so, what should be the default if the environment variable isn't set? Back to 7 bit (ASCII)? utf-8? Some of the latin variants? Your suggestions are appreciated. Thanks in advance. Ciao, Kili ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Output character encoding for ghc on OpenBSD
On Sun, Apr 18, 2010 at 7:01 AM, Matthias Kilian k...@outback.escape.de wrote: Hi, as some of you may know, I'm working on an update of OpenBSDs ghc port to 6.12.2, currently chasing down the last remaining testsuite failures. Yesterday, I ran into a problem which I have a fix for, but only a really ugly fix, and I need some opinions of what users would prefer. The problem is that Haskell uses unicode characters internally (ghc itself uses UTF-32 internally, where the endianess depends on the architecture it's running on), and that any Haskell program (including ghc and ghci) has to convert between the internal representation and the actual locale settings of the system it's running on. Unfortunately, OpenBSD is really bad if it comes to locale support; the only supported locales are the C and the POSIX locales, so even if you set LC_ALL or LC_CTYPE to something like, for example, de_DE.iso88591, this would have no effect on OpenBSD. Anyway, the short story is that I have to either hard-code the character set to something like utf-8, or ghc will start to behave really strange (for example, ghci would terminate immediately if you just *type* a non-ASCII character). That sounds like it might be something to do with the haskeline package, which ghci uses for user interaction. Haskeline makes its own FFI calls to translate raw input bytes into Unicode Chars. Can you elaborate further on what exactly the issue is with OpenBSD's locale support? In particular, there's several components used by Haskeline: - call set_locale(LC_CTYPE) - call nl_langinfo(CODESET) - pass the resulting string (which should be, e.g., $LANG) to iconv_open - call iconv on user input (which may be malformed) Is the problem that setting $LC_ALL or $LANG has no effect on the string returned by nl_langinfo, so the translation fails? If so, haskeline is supposed to output ?s in that case, so there might be a bug in the package. Finally, when you say you have to hard-code the character set, are you talking about ghc, haskeline, the base library, or somewhere else? Best, -Judah ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Output character encoding for ghc on OpenBSD
Hi,
On Sun, Apr 18, 2010 at 10:53:22AM -0700, Judah Jacobson wrote:
Anyway, the short story is that I have to either hard-code the
character set to something like utf-8, or ghc will start to behave
really strange (for example, ghci would terminate immediately if
you just *type* a non-ASCII character).
That sounds like it might be something to do with the haskeline
package, which ghci uses for user interaction. Haskeline makes its
own FFI calls to translate raw input bytes into Unicode Chars.
Oh, this may indeed be a second problem. However, the encoding
problem itself also manifests in the `openTempFile001' test of the
testsuite. For example, with an unpatched ghc-6.12, the test fails
with the following output:
= openTempFile001(normal) 1048 of 2375 [0, 38, 0]
cd ./lib/IO '/usr/obj/ports/ghc-6.12.2/ghc-6.12.2/inplace/bin/ghc-stage2'
-fforce-recomp -dcore-lint -dcmm-lint -no-user-package-conf -dno-debug-output
-o openTempFile001 openTempFile001.hsopenTempFil
e001.comp.stderr 21
cd ./lib/IO ./openTempFile001/dev/null openTempFile001.run.stdout
2openTempFile001.run.stderr
Wrong exit code (expected 0 , actual 1 )
Stdout:
Stderr:
openTempFile001: ./test22236.txt: hClose: invalid argument (Illegal byte
sequence)
*** unexpected failure for openTempFile001(normal)
Can
you elaborate further on what exactly the issue is with OpenBSD's
locale support? In particular, there's several components used by
Haskeline:
- call set_locale(LC_CTYPE)
Problem number 1: set_locale(LC_CTYPE) fails (i.e. returns NULL)
for any locale except `C` or `POSIX'. Did I mention that OpenBSD
is really bad with locales? ;-)
- call nl_langinfo(CODESET)
Always returns `646' (ASCII). Duh.
- pass the resulting string (which should be, e.g., $LANG) to iconv_open
iconv_open appears to need the *codeset* name, not a complete locale.
Note that OpenBSD uses GNU libiconv-1.13, which AFAIK differs from
the one included in glibc. Even worse, I have to pass something
like UTF-8, whereas UTF8 doesn't work.
- call iconv on user input (which may be malformed)
I wrote a little C program that does the following (some error
checks omitted here):
char *inp, outp;
size_t insz, outsz;
unsigned char in[] = {0xa9, 0, 0, 0};
char out[512];
inp = in;
outp = out;
insz = sizeof(in);
outsz = sizeof(out) - 1;
setlocale(LC_CTYPE, );
ic = iconv_open(, UTF-32LE);
if (iconv(ic, inp, insz, outp, outsz) == -1) {
... bail out (perror() etc.) ...
}
iconv_close(ic);
*outp = 0;
puts(out);
And it just doesn't work, regardless what I set LC_CTYPE to. The
only way to get it printing the copyright symbol is to explicitely
use UTF-8 (or ISO-8859-1 or something else that knows about
that symbol) as the first argument to iconv_open().
Is the problem that setting $LC_ALL or $LANG has no effect on the
string returned by nl_langinfo, so the translation fails?
Yes, see above.
If so,
haskeline is supposed to output ?s in that case, so there might be a
bug in the package.
It fails (or rather: ghci fails, since I didn't yet do any separate
haskeline tests) with the same error as the test mentioned above,
with the difference that it fails on hPutChar instead of hClose for
obvious reasons.
Finally, when you say you have to hard-code the character set, are
you talking about ghc, haskeline, the base library, or somewhere else?
I'm talking about libraries/base/GHC/IO/Encoding/Iconv.hs
See? There just is no non-hackerish way to fix this (except of
course improving locale support on OpenBSD, but that's beyond my
scope currently).
Ciao,
Kili
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Can't install Criterion package on ghc ..
Hi,
I am trying to install Criterion package, but I keep getting an error and I
can't figure it out why it is like this !!
mozh...@mozhgan-kch:~$ cabal install Criterion
Resolving dependencies...
Configuring vector-algorithms-0.3...
Preprocessing library vector-algorithms-0.3...
Building vector-algorithms-0.3...
[1 of 9] Compiling Data.Vector.Algorithms.Common (
Data/Vector/Algorithms/Common.hs, dist/build/Data/Vector/Algorithms/Common.o
)
[2 of 9] Compiling Data.Vector.Algorithms.Search (
Data/Vector/Algorithms/Search.hs, dist/build/Data/Vector/Algorithms/Search.o
)
[3 of 9] Compiling Data.Vector.Algorithms.Radix (
Data/Vector/Algorithms/Radix.hs, dist/build/Data/Vector/Algorithms/Radix.o )
[4 of 9] Compiling Data.Vector.Algorithms.Optimal (
Data/Vector/Algorithms/Optimal.hs,
dist/build/Data/Vector/Algorithms/Optimal.o )
[5 of 9] Compiling Data.Vector.Algorithms.Insertion (
Data/Vector/Algorithms/Insertion.hs,
dist/build/Data/Vector/Algorithms/Insertion.o )
[6 of 9] Compiling Data.Vector.Algorithms.Merge (
Data/Vector/Algorithms/Merge.hs, dist/build/Data/Vector/Algorithms/Merge.o )
[7 of 9] Compiling Data.Vector.Algorithms.TriHeap (
Data/Vector/Algorithms/TriHeap.hs,
dist/build/Data/Vector/Algorithms/TriHeap.o )
[8 of 9] Compiling Data.Vector.Algorithms.Intro (
Data/Vector/Algorithms/Intro.hs, dist/build/Data/Vector/Algorithms/Intro.o )
ghc: panic! (the 'impossible' happened)
(GHC version 6.10.4 for i386-unknown-linux):
idInfo co{v a9WB} [tv]
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
cabal: Error: some packages failed to install:
criterion-0.5.0.0 depends on vector-algorithms-0.3 which failed to install.
statistics-0.5.1.0 depends on vector-algorithms-0.3 which failed to install.
vector-algorithms-0.3 failed during the building phase. The exception was:
exit: ExitFailure 1
I tried to install Statistics and vector-algorithms package separately
before installing the Criterion package itself .. but it wasn't successful.
Thanks,
Mozhgan
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: ld memory usage
I've tried. First error was that gold does not recognize -pthread option. If I remove -pthread, gold complains about many undefined references, strlen for instance. So, I did not succeed. I guess that GHC and gold need adjustments to work together. On 18.04.10 09:20, David Terei wrote: You could also have a look at the new 'Gold' linker that's part of binutils. -- Best regards, Roman Beslik. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: ld memory usage
Thank you very much. With your patch ld eats 3 times less resident memory when compiling Hackage packages. If someone here uses Arch Linux, I uploaded patched binutils to AUR with the name binutils-tune-bfd-hash. On 17.04.10 16:59, Bertram Felgenhauer wrote: I've been running binutils with this patch, http://int-e.home.tlink.de/haskell/binutils-2.18-tune-bfd-hash.patch (still applies to 2.20) -- Best regards, Roman Beslik. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
