Re: No last core parallel slowdown on OS X
My first post was comparing almost identical machines: Different Q6600 steppings (the earlier chip makes a good space heater!) on different motherboards, same memory, both stock speeds. In a few weeks when the semester ends, I'll be able to try Linux -vs- BSD -vs- OS X on identical hardware, and try Simon's settings. (I do love overclocking, but five minutes improving Haskell code is generally more effective than a day tweaking motherboard voltages. We're too green to use A/C in the hot California summer, and this computer exhausts through a dryer hose out my office window as it is. I don't want it any hotter, I just want more cores!) I do have some experience comparing this code on four different Linux boxes, and three different Macs, and Linux does consistently worse. I waited to post until I could compare 4 cores against 4 cores on nearly identical hardware. Also, I tried many approaches to this code, and what I've been testing is my best version, which also happens to be one of the simplest approaches to parallelism. (It so often works that way with Haskell.) In fairness, I should also run the standard test suite used in the paper. On Apr 21, 2009, at 10:14 AM, Tyson Whitehead wrote: Why not try booting a CD or thumb-drive linux distro (e.g., ubuntu live) on your 2.4 GHz Q6600 OS X box and see how things stack up. It would certainly eliminate any questions of hardware differences. Cheers! -Tyson I can do even better: This $65 bay device takes four 2.5 SATA or SAS drives: http://addonics.com/products/raid_system/ae4rcs25nsa.asp It has a surprising build quality, makes it trivial to juggle 2.5 SATA drives. Removing the high-low jumper disables the loud fan, which is probably only needed for SAS drives. My primary drive is an OCZ Vertex SSD, for which this is perfect. I also have an assortment of spare laptop drives I can use, so an OS survey will be easy. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: writeFile/readFile on multiple threads leads to error
Is it too difficult to try this on Linux or Mac, just to see if it shows up there as well? -- Jason Dusek ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: writeFile/readFile on multiple threads leads to error
Is it too difficult to try this on Linux or Mac, just to see if it shows up there as well? Yes ;-) I might be able to try it on Linux next week, but that's as far as I'm likely to be able to go. If I get any results I'll email them in. Thanks Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: writeFile/readFile on multiple threads leads to error
Hello Neil, Wednesday, April 22, 2009, 4:22:01 PM, you wrote: you can try to use POSIX open/read/write/close calls Hi, I've got a multi-threaded application which occasionally generates failures in openFile. I haven't been able to reproduce the errors reliably, the code is way too large to send over, and any small attempts at invoking the same problem don't seem to work. Despite the uselessness of the bug report, I thought I'd share what I've seen and how I fixed it. I have many threads, which read and write files. Every so often one thread will write a file, then another thread will read the same file - but fail during the open call. There are locks to ensure that the write call finishes before the read call begins. I modified the code to give: do print (READ START,x) ; res - readFile x ; print (READ STOP,x) ; return res do print (WRITE START,x); writeFile x src ; print (WRITE STOP,x) I then get on the console: WRITE START foo WRITE STOP foo READ START foo openFile doesn't have permission to open foo. The writeFile/readFile are happening in different threads, and they usually succeed - but not always. The bug seems to go away when I add performGC just after writeFile. My guess is that something in the openFile/hClose pair isn't really closed until a garbage collection happens. All this is using GHC 6.10.2 on XP through Cygwin. I'm happy to supply more details if you can think of anything that will help, Thanks Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users -- Best regards, Bulatmailto:bulat.zigans...@gmail.com ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: writeFile/readFile on multiple threads leads to error
do print (READ START,x) ; res - readFile x ; print (READ STOP,x) ; return res Unless you've defined your own version of 'readFile', to mean read entire file now, the first 'print' is optimistic and the second 'print' is a lie. Claus do print (WRITE START,x); writeFile x src ; print (WRITE STOP,x) I then get on the console: WRITE START foo WRITE STOP foo READ START foo openFile doesn't have permission to open foo. The writeFile/readFile are happening in different threads, and they usually succeed - but not always. The bug seems to go away when I add performGC just after writeFile. My guess is that something in the openFile/hClose pair isn't really closed until a garbage collection happens. All this is using GHC 6.10.2 on XP through Cygwin. I'm happy to supply more details if you can think of anything that will help, Thanks Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: writeFile/readFile on multiple threads leads to error
Hi Claus, do print (READ START,x) ; res - readFile x ; print (READ STOP,x) ; return res Unless you've defined your own version of 'readFile', to mean read entire file now, the first 'print' is optimistic and the second 'print' is a lie. readFile calls openFile = hGetContents. It's the openFile that causes the problem, so READ START happens before openFile and READ STOP happens after openFile. The lazy semantics of the actual reading don't seem to have an effect. I did try changing to the strict bytestring file read, and that gave exactly the same error - apart from openBinaryFile was crashing rather than openFile. Thanks Neil do print (WRITE START,x); writeFile x src ; print (WRITE STOP,x) I then get on the console: WRITE START foo WRITE STOP foo READ START foo openFile doesn't have permission to open foo. The writeFile/readFile are happening in different threads, and they usually succeed - but not always. The bug seems to go away when I add performGC just after writeFile. My guess is that something in the openFile/hClose pair isn't really closed until a garbage collection happens. All this is using GHC 6.10.2 on XP through Cygwin. I'm happy to supply more details if you can think of anything that will help, Thanks Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: writeFile/readFile on multiple threads leads to error
do print (READ START,x) ; res - readFile x ; print (READ STOP,x) ; return res Unless you've defined your own version of 'readFile', to mean read entire file now, the first 'print' is optimistic and the second 'print' is a lie. readFile calls openFile = hGetContents. It's the openFile that causes the problem, so READ START happens before openFile and READ STOP happens after openFile. The lazy semantics of the actual reading don't seem to have an effect. Just want to make sure that it isn't the latent hClose from readFile. Once upon a time, I spend a lot of time figuring out why nhc98 wouldn't work on windows, until I noticed that its code used to rely on unixy file system tricks like readFile followed by removeFile, followed by looking at the contents. main = do writeFile data file contents r - readFile data writeFile data different contents print r *Main main *** Exception: data: openFile: permission denied (Permission denied) I did try changing to the strict bytestring file read, and that gave exactly the same error - apart from openBinaryFile was crashing rather than openFile. Weren't there versions of bytestring that didn't close the file early enough? Just checking, Claus Thanks Neil do print (WRITE START,x); writeFile x src ; print (WRITE STOP,x) I then get on the console: WRITE START foo WRITE STOP foo READ START foo openFile doesn't have permission to open foo. The writeFile/readFile are happening in different threads, and they usually succeed - but not always. The bug seems to go away when I add performGC just after writeFile. My guess is that something in the openFile/hClose pair isn't really closed until a garbage collection happens. All this is using GHC 6.10.2 on XP through Cygwin. I'm happy to supply more details if you can think of anything that will help, Thanks Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: writeFile/readFile on multiple threads leads to error
Bulat: I haven't tried moving to Posix calls, I'll try that next - although I was hoping the application wouldn't be posix dependent. readFile calls openFile = hGetContents. It's the openFile that causes the problem, so READ START happens before openFile and READ STOP happens after openFile. The lazy semantics of the actual reading don't seem to have an effect. Just want to make sure that it isn't the latent hClose from readFile. Yeah, I've done that before! It seems to be the writeFile that isn't closing, not the readFile - hence my surprise. I did try changing to the strict bytestring file read, and that gave exactly the same error - apart from openBinaryFile was crashing rather than openFile. Weren't there versions of bytestring that didn't close the file early enough? I only used bytestring for reading, so it shouldn't have made any difference. Thanks Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: writeFile/readFile on multiple threads leads to error
2009/4/22 Neil Mitchell ndmitch...@gmail.com: I've got a multi-threaded application which occasionally generates failures in openFile. I haven't been able to reproduce the errors reliably, the code is way too large to send over, and any small attempts at invoking the same problem don't seem to work. Despite the uselessness of the bug report, I thought I'd share what I've seen and how I fixed it. I have many threads, which read and write files. Every so often one thread will write a file, then another thread will read the same file - but fail during the open call. There are locks to ensure that the write call finishes before the read call begins. I modified the code to give: do print (READ START,x) ; res - readFile x ; print (READ STOP,x) ; return res do print (WRITE START,x); writeFile x src ; print (WRITE STOP,x) I then get on the console: WRITE START foo WRITE STOP foo READ START foo openFile doesn't have permission to open foo. The writeFile/readFile are happening in different threads, and they usually succeed - but not always. The bug seems to go away when I add performGC just after writeFile. My guess is that something in the openFile/hClose pair isn't really closed until a garbage collection happens. All this is using GHC 6.10.2 on XP through Cygwin. The hClose really does close the file descriptor. The only thing left is the finalizer, but it is just a no-op on an already-closed Handle. I can't think of anything we're doing that could possibly cause this, but I have seen rogue permission denied errors on Windows from time to time, they're quite annoying. Here's a possibly-related ticket: http://hackage.haskell.org/trac/ghc/ticket/2924 You might want to run the process under ProcMon and see if you can figure out what's going on (if you can bear to use ProcMon, it's a very poor replacement for strace IMO). Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: writeFile/readFile on multiple threads leads to error
2009/4/22 Simon Marlow: You might want to run the process under ProcMon and see if you can figure out what's going on (if you can bear to use ProcMon, it's a very poor replacement for strace IMO). You could try StraceNT instead (not as good as strace though). ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
6.10.3 plans
Hi all, Due to some problems with the handling of ^C in ghci, we are planning to do a 6.10.3 release. The idea is that, rather than being a full-blown release process, including all the various fixes that have been made since 6.10.2, we will only do what is necessary to resolve the ^C issue. By keeping the changes to a minimum, we will minimise the amount of testing, release-note writing, etc that is necessary. An exception to this rule is that we will probably also rebundle time in the bindists, as that has little chance of breaking anything else. There are actually two problems with ^C. One is due to the changes in the signal handling in ghci, and we will fix that. The other is that, on some platforms (including x86/Linux), the interaction between editline and GHC means that pressing ^C causes a segfault. Given all the other issues that people have had with editline, we plan to use haskeline in 6.10.3 instead. We already planned to do this for 6.12, but we think that it makes sense to do it sooner rather than later. The plan is: * Add haskeline and deps as boot packages * Move the interactive modules from the ghc package to the ghc-bin package. This means that the ghc package will not depend on the new boot packages, so there will not be any problems with upgrading them. Any comments? Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: 6.10.3 plans
Hi Ian,
thanks for the update on plans and the willingness to jump in and do another
release cycle so soon after 6.10.2. The suggested fixes seem agreeable to
me, but I have one _minor_ additional request for 6.10.3 if you end having
to rebuild 'base' -- add a DEPRECATED (or some such) to
Foreign.ForeignPtr.{newForeignPtr,addForeignPtrFinalizer} to indicate
that the operational behaviour of these have changed.
Small change, but could be helpful to package usersauthors when migrating
beyond 6.10.1
thanks
--sigbjorn
On 4/22/2009 16:56, Ian Lynagh wrote:
Hi all,
Due to some problems with the handling of ^C in ghci, we are planning to
do a 6.10.3 release.
The idea is that, rather than being a full-blown release process,
including all the various fixes that have been made since 6.10.2, we
will only do what is necessary to resolve the ^C issue. By keeping the
changes to a minimum, we will minimise the amount of testing,
release-note writing, etc that is necessary.
An exception to this rule is that we will probably also rebundle time in
the bindists, as that has little chance of breaking anything else.
There are actually two problems with ^C. One is due to the changes in
the signal handling in ghci, and we will fix that. The other is that, on
some platforms (including x86/Linux), the interaction between editline
and GHC means that pressing ^C causes a segfault. Given all the other
issues that people have had with editline, we plan to use haskeline in
6.10.3 instead. We already planned to do this for 6.12, but we think
that it makes sense to do it sooner rather than later. The plan is:
* Add haskeline and deps as boot packages
* Move the interactive modules from the ghc package to the ghc-bin
package. This means that the ghc package will not depend on the new
boot packages, so there will not be any problems with upgrading them.
Any comments?
Thanks
Ian
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: 6.10.3 plans
On Wed, Apr 22, 2009 at 4:56 PM, Ian Lynagh ig...@earth.li wrote: Hi all, Due to some problems with the handling of ^C in ghci, we are planning to do a 6.10.3 release. The idea is that, rather than being a full-blown release process, including all the various fixes that have been made since 6.10.2, we will only do what is necessary to resolve the ^C issue. By keeping the changes to a minimum, we will minimise the amount of testing, release-note writing, etc that is necessary. An exception to this rule is that we will probably also rebundle time in the bindists, as that has little chance of breaking anything else. There are actually two problems with ^C. One is due to the changes in the signal handling in ghci, and we will fix that. The other is that, on some platforms (including x86/Linux), the interaction between editline and GHC means that pressing ^C causes a segfault. Given all the other issues that people have had with editline, we plan to use haskeline in 6.10.3 instead. We already planned to do this for 6.12, but we think that it makes sense to do it sooner rather than later. The plan is: * Add haskeline and deps as boot packages * Move the interactive modules from the ghc package to the ghc-bin package. This means that the ghc package will not depend on the new boot packages, so there will not be any problems with upgrading them. Any comments? Thanks Ian Would this be a good time to add the time package too, or has that issue been resolved? Alex ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: 6.10.3 plans
On Apr 22, 2009, at 22:00 , Alexander Dunlap wrote: On Wed, Apr 22, 2009 at 4:56 PM, Ian Lynagh ig...@earth.li wrote: An exception to this rule is that we will probably also rebundle time in the bindists, as that has little chance of breaking anything else. Would this be a good time to add the time package too, or has that issue been resolved? ... -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allb...@kf8nh.com system administrator [openafs,heimdal,too many hats] allb...@ece.cmu.edu electrical and computer engineering, carnegie mellon universityKF8NH PGP.sig Description: This is a digitally signed message part ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
