Re: Directory-backed Handle?
Simon, Thanks for your reply. On 9/15/10, Simon Marlow wrote: > The FD implementation of Handles never makes a Handle with an IODeviceType > of Directory, because openFile is specified to raise an exception when > applied to a directory. Yes, that was the only place in the Libraries' code I found that mentions this device type (which means I haven't missed anything ;) > Alternatively we could give it a meaning as you suggest - newline-separated > file names seems like a reasonable interpretation. It looks like there is no comments in the code for the definition of IODeviceType. I have created a ticket #4317 http://hackage.haskell.org/trac/ghc/ticket/4317 providing the code comments proposed. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Directory-backed Handle?
Hi, In the GHC I/O system, a Handle may be backed by a "device" (IODevice instance) with IODeviceType = Directory There doesn't seem to be any (at least Google search did not give anything) code that uses this; however, what is supposed to be read from such a Handle: newline-separated list of filenames (like in getDirectoryContents), or some sort of serialized FileStatus structures, or anything else? Thanks. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Alternative (per-project) package database
Hi, Summarizing my findings (I couldn't find solution to this problem anywhere; maybe somebody else gets interested as well). Experimental setting: A simple package targeting a hello-world like application. We will keep all dependent upon packages in a subdir named .private-pkg All commands are run from within this project directory. Bootstrapping: Initialize packages database ghc-pkg init .private-pkg Clone few initial packages from the existing installation: ghc-pkg describe | (GHC_PACKAGE_PATH=`pwd`/.private-pkg/ ghc-pkg register -) These packages are (in this order): ffi, rts, ghc-prim, integer-gmp, base-4.2.0.1 (or whatever base-4 you have) If just "base" is specified, ghc-pkg will attempt to dump the description of base-3.x.x.x package which is not what we need. Check that packages are installed without problems. $ GHC_PACKAGE_PATH=`pwd`/.private-pkg/ ghc-pkg list /home/dima/src/hello-pvt/.private-pkg/ base-4.2.0.1 ffi-1.0 ghc-prim-0.2.0.0 integer-gmp-0.2.0.1 rts-1.0 Try to install some package (for simplicity, choose `bytestring' as it has only base in dependencies). (export GHC_PACKAGE_PATH=`pwd`/.private-pkg/; cabal install bytestring --package-db=$GHC_PACKAGE_PATH --prefix=`pwd`) $ GHC_PACKAGE_PATH=`pwd`/.private-pkg/ ghc-pkg list /home/dima/src/hello-pvt/.private-pkg/ base-4.2.0.1 bytestring-0.9.1.7 ffi-1.0 ghc-prim-0.2.0.0 integer-gmp-0.2.0.1 rts-1.0 Our test executable looks like this: --- module Main where import System.Directory main = do putStrLn "Hello Private World" getDirectoryContents "." >>= putStrLn . show --- So we have to include `directory' as dependency in the project's cabal file. Let's build the executable: (export GHC_PACKAGE_PATH=`pwd`/.private-pkg/; cabal install --package-db=$GHC_PACKAGE_PATH --prefix=`pwd`) Resolving dependencies... Configuring filepath-1.1.0.4... Preprocessing library filepath-1.1.0.4... Building filepath-1.1.0.4... [1 of 3] Compiling System.FilePath.Windows ( System/FilePath/Windows.hs, dist/build/System/FilePath/Windows.o ) [2 of 3] Compiling System.FilePath.Posix ( System/FilePath/Posix.hs, dist/build/System/FilePath/Posix.o ) [3 of 3] Compiling System.FilePath ( System/FilePath.hs, dist/build/System/FilePath.o ) Registering filepath-1.1.0.4... Installing library in /home/dima/src/hello-pvt/lib/filepath-1.1.0.4/ghc-6.12.2 ... Building directory-1.0.1.1... [1 of 1] Compiling System.Directory ( System/Directory.hs, dist/build/System/Directory.o ) Registering directory-1.0.1.1... Installing library in /home/dima/src/hello-pvt/lib/directory-1.0.1.1/ghc-6.12.2 Registering directory-1.0.1.1... Configuring hello-pvt-0.1... Preprocessing executables for hello-pvt-0.1... Building hello-pvt-0.1... Linking dist/build/hello-pvt/hello-pvt ... Installing executable(s) in /home/dima/src/hello-pvt/bin $ GHC_PACKAGE_PATH=`pwd`/.private-pkg/ ghc-pkg list /home/dima/src/hello-pvt/.private-pkg/ base-4.2.0.1 bytestring-0.9.1.7 directory-1.0.1.1 ffi-1.0 filepath-1.1.0.4 ghc-prim-0.2.0.0 integer-gmp-0.2.0.1 old-locale-1.0.0.2 old-time-1.0.0.5 rts-1.0 unix-2.4.0.2 Thus, everything needed got compiled and installed under the project's directory. So it will be easy to clean everything up just by removing bin, lib, and .private-pkg subdirs (or choose a better prefix for cabal install, so everything will install in a single location). So, cabal does work in project-private mode. Maybe developers of cabal-install would consider adding such mode to the program, so all these extra options will not be needed. Of course votes from other users will increase the chance of such addition ;) Thanks. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Alternative (per-project) GHC package database?
Marc, Marc Weber wrote: > If you want to give hack-nix a try you can get what you're looking for > by running these two commands: > hack-nix --build-env > . ./default-env > However you have to install Nix once. I thought I could stay within Cabal without adding Nix, however Nix was on my mind. My initial thought is: for bootstrapping, I could dump package descriptions from the global database for few basic packages (rts, ghc-prim, base, etc.) and run them through ghc-pkg register with GHC_PACKAGE_PATH set to cover only per-project package conf directory. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Alternative (per-project) GHC package database?
Hi, I would like to have a completely per-project configuration of GHC installed packages. Suppose I have a project (targeting one or more executables) directory: /my/project and a (initially empty) subdirectory in it: /my/project/packages and GHC and Cabal are run like cd /my/project ; GHC_PACKAGE_PATH=/my/project/packages cabal install so the default global and user package configurations are not visible when building the project. How can I bootstrap a fresh minimal working GHC package configuration at the empty location? Thanks. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
GHC.IO.Device and "ready" method?
Hi, Should an implementation of GHC.IO.Device expect the `ready' method to be called from a thread different than one waiting for an I/O operation on a handle to complete, while that operation is in progress? As a more general question, are GHC Handles (and underlying implementations of GHC.IO.Device and GHC.IO.BufferedIO) expected to be thread-safe? I would be happy to store the id of the thread that calls mkFileHandle somewhere in the underlying `dev' and simply reject all I/O requests that come from other threads, but is this correct/permitted behavior? Thanks. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Thread-backed handles: thread blocked indefinitely in an MVar operation
Simon, On Tue, Jun 22, 2010 at 4:31 AM, Simon Marlow wrote: > So the problem is that fillReadBuffer has to return 0 to indicate EOF, but > you are killing the slave thread as soon as it has reached the end of the > stream. So you get one call to fillReadBuffer that returns the data up to > the end of the stream, and the next call that should return 0 blocks on the > Chan indefinitely because the slave thread has already been killed. Thanks for your reply. Updated paste: http://hpaste.org/fastcgi/hpaste.fcgi/view?id=26455 I enclosed readChan in a catchException call which seems to have cured the problem: it just simulates EOF on the handle if the thread is blocked (channel vanished). Interestingly that in the failing version, handle closing was done in a separate thread (thread 3), but in the updated version it was the same thread 1. -- Dimitry Golubovsky Anywhere on the Web -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Thread-backed handles: thread blocked indefinitely in an MVar operation
Hi, I am experimenting with the new feature in GHC such as custom Handles. I use GHC 6.12.2, base 4.2.0.1 I have pasted the code in question as http://hpaste.org/fastcgi/hpaste.fcgi/view?id=26413#a26413 I am trying to create a Handle using a thread as data producer. In my example is is a union of directories, and the Handle contents is newline-separated list of full paths of all files in the union. This list is produced by a thread which reads contents of all directories in the union, turns them into absolute filenames, and sends to the channel. The method of BufferedIO responsible for filling in the Handle's buffer reads data from the channel and places it in the buffer. The problem is, after the contents of the Handle (via hGetContents) has been consumed, the program crashes with the following message: : thread blocked indefinitely in an MVar operation that is, the BlockedIndefinitelyOnMVar exception has occurred. What is done wrong in my code? The dirClose method seems to be called (its output shows up), however with or without the call to killThread, the exception occurs anyway. Thanks for any ideas. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Default value for numCapabilities?
Simon, Can the documentation for forkProcess be fixed then? Instead of "forkProcess comes with a giant warning..." it could say: "forkProcess is not supported by GHC in multicore setting". PS I have created ticket #4027 with more detailed proposal. On Wed, Apr 28, 2010 at 4:26 AM, Simon Marlow wrote: > On 28/04/2010 03:02, Dimitry Golubovsky wrote: >> this limitation? rawSystem works fine from within a thread started by >> forkIO, but why cannot I use forkProcess? > > No good reason, other than it's tremendously difficult to implement. Thank you. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Default value for numCapabilities?
Hi, Interesting: my program is compiled with -threaded, and when I run it with +RTS -N I get: : forking not supported with +RTS -N greater than 1 This is probably about forkProcess rather than forkIO/forkOS, but why this limitation? rawSystem works fine from within a thread started by forkIO, but why cannot I use forkProcess? On Tue, Apr 27, 2010 at 12:35 PM, Bryan O'Sullivan wrote: > With 6.12.1 and newer, no "-N" argument implies 1 core, "-Nk" implies > k cores, and "-N" without an argument means "use all". -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Default value for numCapabilities?
Bulat, All I need is to know how many cores I have. In my program, I can run some actions in parallel, and knowing the number of cores helps set the maximum number of parallel processes. These processes are in fact external programs (CPU-intensive but not memory intensive, so number of cores is main factor) rather than GHC threads. I think I just need to check this experimentally as I have a 4-core (Atom) and 1-core (Via) PCs and see what numCapabilities returns. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Default value for numCapabilities?
Hi, As a followup to the discussion [1] about the portable way to find the number of CPUs/cores: how is the default value of numCapabilities [2] set when +RTS -N is not on the command line? Does GHC runtime figure out the number of cores itself, or it is by default 1 unless specified on the command line? Thanks. --- [1] http://haskell.org/pipermail/haskell-cafe/2010-April/076911.html [2] http://hackage.haskell.org/packages/archive/base/latest/doc/html/GHC-Conc.html#v:numCapabilities -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Why hide Distribution.Compat.FilePath, or what to use instead?
Hi, I'm trying to bring HSFFIG in order and make it work with GHC 6.6 and the modern package structure. I have found out that the Distribution.Compat.FilePath module (which used to be System.FilePath in its former life) is hidden: programs/ffipkg.hs:11:7: Could not find module `Distribution.Compat.FilePath': it is hidden (in package Cabal-1.1.6.2) I cannot use the new FilePath package because it does not have function like exeExtension. Other functions (splitFileExt, parseSearchPath, joinFileName, join FileExt) probably have their analogs in the new package. What else can I use for exeExtension? Thanks. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Exception when compiling HAppS
Hi, On 1/7/07, Kirsten Chevalier <[EMAIL PROTECTED]> wrote: By way of figuring out which system call is getting interrupted, can you run your "runghc" command with strace, like so: bash$ strace [whatever you were typing to build happs before] and paste the last couple of lines of strace's output? Surprisingly, tonight I was able to get past that file, and compiled the whole package. So the question remains in semi-closed state. I hope that future versions of GHC will be more informative of which child process caused such error. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Exception when compiling HAppS
Hi, On 1/7/07, Kirsten Chevalier <[EMAIL PROTECTED]> wrote: > I am getting a strange error message when trying to compile > HAppS-0.8.4 with ghc-6.4.1always on the same file. > [snip] > *** Exception: waitForProcess: interrupted (Interrupted system call) > This could mean a lot of things. What OS and platform are you using? bash$ uname -a Linux dmghome 2.2.16 #9 Mon Sep 16 22:43:25 EDT 2002 i686 unknown -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Exception when compiling HAppS
Hi, I am getting a strange error message when trying to compile HAppS-0.8.4 with ghc-6.4.1always on the same file. bash$ runghc Setup.hs configure --prefix=/previous/public/HAppS The field "hs-source-dir" is deprecated, please use hs-source-dirs. Setup.hs: Warning: No license-file field. Configuring HAppS-0.8.4... configure: Using install prefix: /previous/public/HAppS configure: Using compiler: /previous/public/ghc-6.4.1/bin/ghc configure: Compiler flavor: GHC configure: Compiler version: 6.4.1 configure: Using package tool: /previous/public/ghc-6.4.1/bin/ghc-pkg configure: No haddock found configure: No happy found configure: No alex found configure: Using hsc2hs: /previous/public/ghc-6.4.1/bin/hsc2hs configure: No c2hs found configure: Using cpphs: /previous/public/nhc98/bin/cpphs configure: No greencard found configure: Dependency base-any: using base-1.0 configure: Dependency fps>=0.8: using fps-0.8 configure: Dependency HaXml>=1.13&&<1.14: using HaXml-1.13.2 configure: Dependency mtl-any: using mtl-1.0 configure: Dependency network-any: using network-1.0 configure: Dependency stm-any: using stm-1.0 configure: Dependency template-haskell-any: using template-haskell-1.0 bash$ runghc Setup.hs build The field "hs-source-dir" is deprecated, please use hs-source-dirs. Preprocessing library HAppS-0.8.4... Building HAppS-0.8.4... Chasing modules from: HAppS,HAppS.Agents.MailSender,HAppS.Agents.SessionKeeper,HAppS.Agents.SessionKeeperEx,HAppS.Agents.Users,HAppS.Agents.WithBlockingIO,HAppS.DBMS.Index,HAppS.DBMS.RSMap,HAppS.DBMS.Table,HAppS.MACID,HAppS.MACID.Saver,HAppS.Protocols.Base64,HAppS.Protocols.Cookie,HAppS.Protocols.DNS,HAppS.Protocols.DNS.HiWire,HAppS.Protocols.DNS.RR,HAppS.Protocols.DNS.Type,HAppS.Protocols.HMAC,HAppS.Protocols.HTTP,HAppS.Protocols.HTTP.LowLevel,HAppS.Protocols.SimpleHTTP,HAppS.Protocols.JSON,HAppS.Protocols.MessageWrap,HAppS.Protocols.MinHaXML,HAppS.Protocols.SHA1,HAppS.Protocols.SMTP,HAppS.Protocols.SURI,HAppS.Protocols.W64,HAppS.Protocols.XSLT,HAppS.Protocols.XSLTSMTP,HAppS.Util.Concurrent,HAppS.Util.Cron,HAppS.Util.Common,HAppS.Util.EventHandler,HAppS.Util.StdMain,HAppS.Util.StdMain.Config,HAppS.Util.StdMain.StartState,HAppS.Util.TimeOut,HAppS.MACID.Checkpoint,HAppS.MACID.Monad,HAppS.MACID.Saver.Impl.File,HAppS.MACID.Saver.Impl.Queue,HAppS.MACID.Saver.Types,HAppS.MACID.Serialize,HAppS.MACID.SideEffect,HAppS.MACID.Transaction,HAppS.MACID.Types,HAppS.MACID.Util,HAppS.MACID.Var,HAppS.Protocols.DES,HAppS.Protocols.DNS.Cache,HAppS.Protocols.DNS.ChooseMethod,HAppS.Protocols.DNS.LoWire,HAppS.Protocols.DNS.MutableEnv,HAppS.Protocols.DNS.MXClient,HAppS.Protocols.DNS.Name,HAppS.Protocols.DNS.NSTree,HAppS.Protocols.DNS.Util,HAppS.Protocols.HTTP.Clock,HAppS.Protocols.HTTP.FileServe,HAppS.Protocols.HTTP.Handler,HAppS.Protocols.HTTP.LazyLiner,HAppS.Protocols.HTTP.Listen,HAppS.Protocols.HTTP.ServerPart,HAppS.Protocols.HTTP.Types,HAppS.Protocols.SURI.ParseURI,HAppS.Util.StdMain.StartStateTH,HAppS.Util.ByteStringCompat Skipping HAppS.Util.ByteStringCompat ( src/HAppS/Util/ByteStringCompat.hs, dist/build/HAppS/Util/ByteStringCompat.o ) Skipping HAppS.Protocols.SURI.ParseURI ( src/HAppS/Protocols/SURI/ParseURI.hs, dist/build/HAppS/Protocols/SURI/ParseURI.o ) Compiling HAppS.Protocols.HTTP.LazyLiner ( src/HAppS/Protocols/HTTP/LazyLiner.hs, dist/build/HAppS/Protocols/HTTP/LazyLiner.o ) *** Exception: waitForProcess: interrupted (Interrupted system call) bash$ ghc --version The Glorious Glasgow Haskell Compilation System, version 6.4.1 bash$ ghc-pkg -l /previous/public/ghc-6.4.1//lib/ghc-6.4.1/package.conf: rts-1.0, base-1.0, haskell98-1.0, template-haskell-1.0, unix-1.0, parsec-1.0, haskell-src-1.0, network-1.0, QuickCheck-1.0, HUnit-1.1, mtl-1.0, fgl-5.2, X11-1.1, HGL-3.1, stm-1.0, readline-1.0, (lang-1.0), (concurrent-1.0), (posix-1.0), (util-1.0), (data-1.0), (text-1.0), (net-1.0), (hssource-1.0), NewBinary-0.1, Crypto-2.1.0, HTTP-2005.11.23, hxt-5.3, CabalFind-0.2, Cabal-1.1.3, FilePath-0.1.0, UNISTD-0.0, DB-4.2, SYSLOG-0.0, XPROTO-0.0, HSFFIG-1.1, BF-0.0, Fudgets-0.1, (cpphs-1.3), HaXml-1.13.2, fps-0.8 Has anybody encountered this? Thanks. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Packages: exposed, hidden, available (was: Packages and Modules)
Simon Marlow wrote: In fact, we can imagine three states that a package could be in: - exposed: the package's modules populate the global module namespace, explicit "from" imports may be used to resolve ambiguity - hidden: the package cannot be used at all - available: the package can be used only by an explicit "from" import (this is rather like a qualified import at the package level, and might be useful if you want to use a package with module names that overlap with local modules). - I'd suggest the following sub-definition for "available" (other name could be "transient"): A package which is "available" is not "installed" at (immediately known to) a particular developer's system, as opposed to "exposed" and "hidden" which are "installed". When a package/program is being built that requires a package which is not installed, three options are available: - a package is located (on the Internet, using Cabal-get or Cabalfind, or manual search), downloaded, and installed (i. e. becomes known to the developer's system), and gets into "exposed" or "hidden" category - a package is located, downloaded temporarily (i. e. cached) on the developer's system*, used for a particular package that requires it and discarded afterwards (or lives for some time in the cache).** - package cannot be found (Internet search fails) * just due to the fact that existing OS's generally distinguish between local and network resources. If we imagine some sort of "http file system" or "darcs file system" that mount URLs/repos to local file system entries then explicit downloading is not necessary. The latter may be (or already is) implemented as user-level file systems with recent Linux kernels. ** one might think about "lazy package resolution" i. e. only minimal subset of packages like base, mtl, network, etc that generally come with GHC/other compiler are installed on a developer's system, and the rest of packages are transient/cached on demand. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
[glasgow-haskell-users] Searchpath vs. Cabalfind (was: installing a Cabal package non-exposed)
Alex, S. Alexander Jacobson wrote: Searchpath allows you easily to give each application its own namespace and automates import chasing accross the internet. It is a work in progress and is definitely less refined than cabal, but if you are having this sort of issue you might want to try it out. See http://searchpath.org Looks very interesting. However I couldn't get anything like "https://hackage.org/modules.map"; in a browser (nor with http: nor from www.hackage.org): it says "unknown host". I am just curious: instead of querying a centralized database, have you tried any way to get modules' locations from search engines? I wish I could see what "modules.map" looks like, then I could see better whether what Cabalfind gets from Google could be an alternative? Not that I am pushing for Cabalfind in any way; just curious. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Memory leak in Fudgets
I wrote a very simple Fudgets program (just copies stdin to stdout, no graphics involved) leaktest.hs -- module Main where import Graphics.UI.Fudgets.Fudgets main = fudlogue (stdoutF >==< stdinF) and ran it like this: yes | leaktest I found out that the program grows in memory. Its "traditional" analog: leaktest2.hs - module Main where import System.IO main = do hGetLine stdin >>= hPutStrLn stdout main works in constant space. I tried to observe the GC realtime stats with +RTS -Sstderr -M600K It turns out, garbage is collected, number of allocated and collected bytes are mostly constant, and number of live bytes fluctuates, but averagely stays the same: AllocCollectLiveGCGC TOT TOT Page Flts bytes bytes bytes user elapuserelap 316720262144 9528 0.00 0.000.020.02 144 499 (Gen: 0) 262128262144 22188 0.01 0.010.030.0403 (Gen: 0) 262128262144 8276 0.00 0.000.040.0500 (Gen: 0) 262128262144 24612 0.01 0.010.050.0600 (Gen: 0) 262128262144 10680 0.00 0.000.060.0700 (Gen: 0) 262128262144 23376 0.00 0.000.060.0800 (Gen: 0) 262128262144 9444 0.00 0.000.070.0900 (Gen: 0) 262012262144 25492 0.00 0.000.070.1000 (Gen: 0) ... program grows to 10M . 262128262144 1684 0.01 0.01 42.33 66.1000 (Gen: 0) 262128262144 13396 0.01 0.01 42.34 66.1100 (Gen: 0) 262124262144 25540 0.01 0.01 42.35 66.1200 (Gen: 0) 262128262144 11988 0.00 0.00 42.35 66.1300 (Gen: 0) 262128262144 1760 0.00 0.00 42.36 66.1400 (Gen: 0) 262128262144 14464 0.00 0.00 42.37 66.1500 (Gen: 0) 262128262144 1656 0.00 0.00 42.37 66.1600 (Gen: 0) and was interrupted ... 1,614,769,480 bytes allocated in the heap 78,801,952 bytes copied during GC 25,624 bytes maximum residency (6160 sample(s)) 6160 collections in generation 0 ( 10.35s) 1 Mb total memory in use INIT time0.01s ( 0.00s elapsed) MUT time 32.01s ( 55.38s elapsed) GCtime 10.35s ( 10.78s elapsed) EXIT time0.00s ( 0.00s elapsed) Total time 42.37s ( 66.16s elapsed) %GC time 24.4% (16.3% elapsed) Alloc rate50,430,027 bytes per MUT second Productivity 75.5% of total user, 48.4% of total elapsed So, what could be the reason of such memory leak? What else may grow if the heap remains constant? How can it be observed? Any ideas are welcome. -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
The -static option of ghc, and choice between user-supplied library types (Unix)
The Paragraph 4.10.7 of the GHC users guide contains this: -static Tell the linker to avoid shared Haskell libraries, if possible. This is the default. -dynamic Tell the linker to use shared Haskell libraries, if available (this option is only supported on Mac OS X at the moment, and also note that your distribution of GHC may not have been supplied with shared libraries). As I understand, this is related to the GHC own libraries. What about user-supplied libraries (listed in the `extra-libraries' field of a Cabal package description) e. g.: extra-libraries: gd, jpeg, m from http://www.cs.chalmers.se/~bringert/darcs/haskell-gd/gd.cabal There are usually files like libm.a and libm.so together in the same directory where GHC looks for libraries. Does the -static / -dynamic ghc flag affect the choice which library to link against?Or, to control this, one has to descend to the linker options, and the default would be the linker default? I believe, this is Unix-specific: AFAIR on Windows (and OS/2) dynamic implibs were indistinguishable by their names from static libraries and had the .lib suffix anyway? -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Unboxed types and FFI (GHC Users Guide, Section 8.1.1)
The section 8.1.1 of the GHC User's Guide says this: The following unboxed types may be used as basic foreign types (see FFI Addendum, Section 3.2): Int#, Word#, Char#, etc Does this mean that when declaring a foreign function import for a C function int fun(int) I may write foreign import ccall "bla.h fun" fun :: Int# -> Int# instead of using CInt? Are there any examples of using unboxed types with FFI? -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Two issues bulding ghc 6.4.1
Dimitry Golubovsky wrote: 2. Error while assembling (or mangling?) ForeignObj.lhs; fragments of make output below (long lines wrapped/truncated, the whole file is too long to post it here but I'll provide it if needed): This one has been fixed by installing gcc-2.95.2 and building using it. I am not very much familiar with the new GHC Wiki; does there exist a page on "tools compatibility" or similar? Dimitry ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Two issues bulding ghc 6.4.1
I have encountered two issues trying to build ghc-6.4.1 from the source tarball. 1. (fixed) GNU make v 3.79 does not work (was discussed on irc yesterday: at least two people including myself got this issue), v 3.80 works. If not already in FAQ/wiki perhaps worth including. 2. Error while assembling (or mangling?) ForeignObj.lhs; fragments of make output below (long lines wrapped/truncated, the whole file is too long to post it here but I'll provide it if needed): === ../../ghc/compiler/ghc-inplace -M -optdep-f -optdep.depend -optdep-s -optdepp -osuf o-H16m -O -ign
C--
I am trying to understand the role of cmm files in the current GHC compiler build process (I am guessing, they have replaced .hc files like PrimOps and AutoApply). They are compiled by the stage1 compiler. Is ability to compile cmm preserved in the compiler finally built at stage3 and installed? What compiler flags are available for cmm files compilation? The User's guide says nothing specific. Are -E, -C, -S available (along with -c and -fvia-c seen in the nightly build log)? -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
-split-objs and --make together?
Is it possible to use --make when one or more of modules involved are split? I tried this with ghc 6.2.2 I do the following (doesn't work) Suppose we have syscall.hs and UNISTD_H.hs: the first contains Main and imports UNISTD_H, and the second is to be split. mkdir UNISTD_H_split ghc -c -split-objs UNISTD_H.hs ar cq libUNISTD_H.a UNISTD_H_split/*.o I get the library and the .hi file: no .o file ghc --make syscall.hs libUNISTD_H.a -o syscall It tries to recompile UNISTD_H.hs because there is no UNISTD_H.o, and links against the object file, not the library When I try ghc syscall.hs libUNISTD.a -o syscall it works, and only syscall.hs is recompiled, but I cannot use --make. I tried to name the library UNISTD_H.a, same result. Is it possible to get ghc recognize the library same way as if it were an object file (which corresponds to the interface file) and use --make? PS1 This is for that experiment with hsffig when the header file contains functions undefined in the default glibc. Linking against the library with split objs works fine, and executable size is much smaller, and unresolved references are gone. Only ability to use --make is missing here. PS2 It would be nice if the perl script splitting the assembler code also created the directory where the split object files go. Dimitry Golubovsky Middletown, CT ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Unicode in GHC 6.2.2 and 6.4.x (was: Re: [Haskell-cafe] Unicode.hs)
Dear List Subscribers, Simon Marlow wrote: On 30 June 2005 14:36, Dimitry Golubovsky wrote: It is in CVS now, and I believe will be in 6.4.1 Not planned for 6.4.1, but definitely in 6.6. I have put those files that work for me in GHC 6.2.2 (Unicode support) for download. Please read the Wiki page: http://haskell.org/hawiki/GhcUnicode for instructions. Any feedback will be appreciated. I believe, the code will work with GHC 6.4.x as well. Please let me know if there are any problems. Dimitry Golubovsky Middletown, CT ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Cabal and older versions of GHC
Is it possible to use Cabal with older versions (e. g. 6.0 to 6.2.2) of GHC? Some time later, I am going to release HSFFIG as a package. To be usable with GHC 6.2.2, can it be packaged with Cabal, or do I have to use ghc-pkg? If I use ghc-pkg, is there a way to convert such a package into Cabal? I am not going to upgrade to GHC 6.4.x any time soon, so what are my options in using Cabal with 6.2.2? -- Dimitry Golubovsky Anywhere on the Web ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Suggestion for hsc2hs (structures with commas)
I came across the following issue in hsc2hs:
when requesting a size of a C structure with #size (or similarly this
may apply to #peek, #poke, #offset macros), and the structure is
declared "anonymously", and contains multiple field declaration of the
same type such as:
struct {int ab,cd;}
then #size(struct {int ab,cd;}) will result in
hsc_size(struct {int ab,cd;})
which will fail because of the comma inside the structure makes the
macro recognize 2 arguments instead of 1.
GCC's preprocessor allows variadic macros defined like
#define __quote__(x...) x
If hsc_size is called:
hsc_size(__quote__(struct {int ab,cd;}))
then error does not occur.
Similarly:
hsc_peek(__quote__(struct {int ab,cd;}),cd)
because __quote__ is scanned first, and its result is treated as a
single argument.
I looked through cpp.info, but could not find any other way to quote
macro arguments.
Is it possible to update hsc2hs (Main.hs I believe) to emit macro
calls involving C types in the way described above?
i. e. #size(t) becomes hsc_size(__quote__(t)),
#peek(t,f) becomes hsc_peek(__quote__(t),f) etc.
when hsc2hs generates the C file from a hsc source.
PS I am working with hsc2hs from the GHC 6.2.2 distribution. I
couldn't find such fixes in the changelog since then.
I believe, the person maintaining hsc2hs might be able to implement
these fixes faster than if I tried.
PPS This will work with GCC, but I am not sure abouth other compilers
(like Microsoft C)
--
Dimitry Golubovsky
Anywhere on the Web
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unicode in GHC: need more advice
Hi, Simon Marlow wrote: You're doing fine - but a better place for the tables is as part of the base package, rather than the RTS. We already have some C files in the base package: see libraries/base/cbits, for example. I suggest just putting your code in there. I have done that - now GHCi recognizes those symbols and loads fine. The test program also works when compiled. I still got some messages about missing prototypes and implicitly declared functions that I defined instead of libc functions, especially during Stage 1. I need to check into that, but since all those functions are basically int -> int, it does not affect the result. The code I use is some draft code, based on what I submitted for Hugs (pure Unicode basically, even without extra space characters). Now I need more advice on which "flavor" of Unicode support to implement. In Haskell-cafe, there were 3 flavors summarized: I am reposting the table here (its latest version). |Sebastien's| Marcin's | Hugs ---+---+--+-- alnum | L* N* | L* N*| L*, M*, N* <1> alpha | L*| L* | L* <1> cntrl | Cc| Cc Zl Zp | Cc digit | N*| Nd | '0'..'9' lower | Ll| Ll | Ll <1> punct | P*| P* | P* upper | Lu| Lt Lu| Lu Lt <1> blank | Z* \t\n\r | Z*(except| ' ' \t\n\r\f\v U+00A0 U+00A0 U+2007 U+202F) \t\n\v\f\r U+0085 <1>: for characters outside Latin1 range. For Latin1 characters (0 to 255), there is a lookup table defined as "unsigned char charTable[NUM_LAT1_CHARS];" I did not post the contents of the table Hugs uses for the Latin1 part. However, with that table completely removed, Hugs did not work properly. So its contents somehow differs from what Unicode defines for that character range. If needed, I may decode that table and post its mapping of character categories (keeping in mind that those are Haskell-recognized character categories, not Unicode) I am not asking for discussion in this list again. I rather expect some suggestion from the GHC team leads, which flavor (of shown above, or some combination of the above) to implement. One more question that I had when experimenting with Hugs: if a character (like those extra blank chars) is forced into some category for the purposes of Haskell language compilation (per the Report), does this mean that any other Haskell application should recognize Haskell-defined category of that character rather than Unicode-defined? For Hugs, there were no choice but say Yes, because both compiler and interpreter used the same code to decide on character category. In GHC this may be different. Since Hugs got there first, does it make sense just follow what was done here, or will a different decision be adopted for GHC: say, for the Parser, extra characters are forced to be blank, but for the rest of the programs compiled by GHC, Unicode definitions are adhered to. PS The latest rebuild I did, used ghc with new code compiled in as Stage 1 compiler. Dimitry Golubovsky Middletown, CT ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Unicode in GHC: need some advice on building
Hi, Following up the discussion in Haskell-Cafe about ways to bring better Unicode support in GHC. I may take care on putting this into the GHC runtime, but I need some advice as I am completely new to this. What needs to be done primarily, is to replace the FFI calls made from GHC.Unicode (iswupper, islower etc.) with functions implemented directly in the runtime, or in an external library (independent from libc). I tried to do this in two ways. First, I made a shared object containing substitutes for these functions (draft code, based on what I submitted for Hugs some time ago), and LD_PRELOADed it. Everything went fine, and my small test program worked both compiled to binary and in GHCi (it just coredumped when towlower/towupper was called on non-ASCII character without the substitution library). Bad thing is, LD_PRELOAD does not work on all systems. So I tried to put the code directly into the runtime (where I believe it should be; the Unicode properties table is packed, and won't eat much space). I renamed foreign function names in GHC.Unicode (to avoid conflict with libc functions) adding u_ to them (so now they are u_iswupper, etc). I placed the new file into ghc/rts, and the include file into ghc/includes. I could not avoid messages about missing prototypes for u_... functions , but finally I was able to build ghc. Now when I compiled my test program with the rebuilt ghc, it worked without the LD_PRELOADed library. However, GHCi could not start complaining that it could not see these u_... symbols. I noticed some other entry points into the runtime like revertCAFs, or getAllocations, declared in the Haskell part of GHCi just as other foreign calls, so I just followed the same style - partly unsuccessfully. Where am I wrong? Another thing, this might be done without intervention into the sources, just as an external library. Which would be the best placement for the library, so it would load/compile in automatically? I tried to find information in the GHC Commentary, but it did not give me much. Dimitry Golubovsky Middletown, CT ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: To show or not to show french accents
I would support the point of view that show should output escapes when showing characters outside ASCII. This is sort of a "transport" format (together with read), therefore it must be a GCD for all possible input encodings. UTF-8 might be alternative, but it would require to be equally supported by all Haskell implementations. -- Dmitry M. Golubovsky South Lyon, MI ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Marshalling functions was: Transmitting Haskell values
Hi, Hal Daume III wrote: Hmm... I can write out functions using the "Show (a -> b)" instance, but there's no matching "Read (a -> b)". Show (a -> b) is a bogus instances -- you won't actually be able to use it for marshalling functions. Well, marshalling functions (or storing-restoring some internal forms of them) might be especially nice... This would mean I can declare and compile a function on my side of a network connection (for example), and then send it to the other end for evaluation, and then get the result. Like a database request. Is this something absolutely impossible in Haskell and by what reason? Just because of strong typing (forgive my stupidity ;)? Or are there some deeper theoretical limitations? -- Dmitry M. Golubovsky South Lyon, MI ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Sending wide characters over the network socket
Glynn Clements wrote: To be more general: how would I send arbitrary binary data (stream of octets) over a socket or a file handle? Should I always assume that only lower bytes would be sent, and this will be forever in ghc? Yes. Well, maybe not forever, but for the forseeable future. Well, if it were there forever then one might rely upon it and just construct appropriate String containing only bottom octets in proper sequence. Or is it a bug? No. It's just a fundamental design flaw in Haskell. Presumably someone thought that wide-character support was just a question of defining Char, and forgot about a minor issue called "I/O". Perhaps not Haskell itself; this is just a language (i. e. a tool to describe some program logic). I think the problem is: I/O functions expose the underlying OS (Unix, Windows) to a program, and that OS has absolutely no knowledge of what String is in Haskell (or Java for example). So data chunks must be presented to those functions in the way that the OS understands best. In other words, array of octets fits better than String. In Java runtime, this gap is filled with a bulky layer of various XXXOutputStream's and a bunch of encodings. I guess, Haskell implementation could just enhance the IO subsystem by adding functions taking arrays of octets as their arguments instead of String's. This would not break compatibility with existing programs using String's. What would be really nice to have is some sort of object input/output over network. Then, by sending a String, I would expect it to be recreated at the other end. And if the other end expects, say [Int] then an error would be detected. I haven't looked through the whole GHC runtime, and maybe something like this is already there, any clue is welcome. -- Dmitry M. Golubovsky South Lyon, MI ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Sending wide characters over the network socket
Hi, I have tried to send a string of Unicode characters over a socket (or to write it into a file handle). The result is strange: it looks like characters are truncated down to their least significant bytes. Here is my program (I am new in Haskell, so the code may look not so good, but it illustrates what I am trying to achieve) === module Main where import IO import Char import Control.Concurrent import Network.Socket -- A string of wide characters wide = [chr 0x1234, chr 0x5678, chr 0x4321, chr 0x8765, chr 0x102345] -- Function to "narrow" characters to their least significant byte narrow s = map (\c -> chr $ (ord c) `mod` 256) s -- Decode string printing all of its characters' order numbers strtodec str = foldr1 (++) ( map (\c -> " "++show(ord c)++" ") str) -- Server: receive a string and print it decoded, -- narrow it and print it again. -- It does not actually loop, though it could server=do sock <- socket AF_INET Stream 6 ia <- inet_addr "127.0.0.1" bindSocket sock $ SockAddrInet 2323 ia listen sock 15 srvloop sock where srvloop sock = do (acsock,from) <- accept sock instr <- recv acsock 128 putStrLn $ "Received: " ++(strtodec instr) putStrLn $ "Lower Bytes: " ++(strtodec $ narrow instr) sClose acsock sClose sock -- Client: send a string of wide characters client=do threadDelay 100 sock <- socket AF_INET Stream 6 ia <- inet_addr "127.0.0.1" connect sock $ SockAddrInet 2323 ia cnt <- send sock wide putStrLn $ "Sent "++(show cnt)++" bytes" putStrLn $ "Source was: "++(strtodec wide) sClose sock threadDelay 100 main=do forkIO (server) client === And here is its output === Sent 5 bytes Source was: 4660 22136 17185 34661 1057605 Received: 52 120 33 101 69 Lower Bytes: 52 120 33 101 69 === Honestly, I expected that 20 bytes were sent (or something smaller if they were sent in UTF), and "Received" be identical to "Source was". The last string of output is just to check whether those are indeed lower bytes shown, not some garbage. I am using a binary distribution of GHC 6.0 on Linux - are there any special conditions I have to enable for the source distribution to be able to send/receive Unicode characters? To be more general: how would I send arbitrary binary data (stream of octets) over a socket or a file handle? Should I always assume that only lower bytes would be sent, and this will be forever in ghc? Or is it a bug? The problem is, Handle/Socket functions require a String to be the type of data to exchange; not a, say [Int8]. Therefore, I need to be able to coerce my binary data buffer to a String. PS Of course, I could write my own socket functions, but I am looking for a more or less "pure" Haskell solution. PPS I tried this only with GHC. -- Dmitry M. Golubovsky South Lyon, MI ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Changes in the Core language file syntax since GHC5.02?
Hi, I asked this in the fa.haskell newsgroup, but since this is more GHC-specific, maybe this list is a better place to ask. I am trying to learn the syntax of Core language files using the paper by Andrew Tolmach named "An External Representation for the GHC Core Language". This was a draft for the version 5.02. Have there been any improvements to the Core language definitions since that version and during the v6.0 implementation? Are they summarized in a document? Or maybe a fresher version of the paper is available? Thanks, -- Dmitry M. Golubovsky South Lyon, MI ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
