Re: BlockedIndefinitelyOnMVar exception
Hi Simon, My suspicion for the root cause of the problem is that Concurrent.Chan is incorrect. In the course of debugging this problem we found 2 bugs in Chan, and while I never tracked down any other bugs in Chan, I no longer trust it. By rewriting parts of the program, including avoiding Chan, the bugs disappeared.I don't think I'll be using Chan again until after someone has proven in correct. Considering Chan is 150 lines of code and has been around for many years, that's amazing! Did you report the bugs? Is it anything to do with asynchronous exceptions? Nothing to do with async exceptions. I found: http://hackage.haskell.org/trac/ghc/ticket/4154 http://hackage.haskell.org/trac/ghc/ticket/3527 Of course, there's also the async exceptions bug still around: http://hackage.haskell.org/trac/ghc/ticket/3160 However, even after having a program with no async exceptions (I never used them), and eliminating unGetChan and isEmpyChan, I still got bugs. I have no proof they came from the Chan module, and no minimal test case was ever able to recreate them, but the same program with my own Chan implementation worked. My Chan had different properties (it queues items randomly) and a subset of the Chan functions, so it still doesn't prove any issue with Chan - but I am now sceptical. You should have more luck with Control.Concurrent.STM.TChan, incedentally. It's much easier to get right, and when we benchmarked it, performance was about the same (all those withMVar/modifyMVars in Chan are quite expensive), plus you get to compose it easily: reading from either of 2 TChans is trivial. The performance of the Haskell is irrelevant - the program spends all its time invoking system calls. Looking at the implementation I am indeed much more trusting of TChan, I'll be using that in future if there is ever a need. Thanks, Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: BlockedIndefinitelyOnMVar exception
On 04/07/10 10:30, Neil Mitchell wrote: Hi Simon, My suspicion for the root cause of the problem is that Concurrent.Chan is incorrect. In the course of debugging this problem we found 2 bugs in Chan, and while I never tracked down any other bugs in Chan, I no longer trust it. By rewriting parts of the program, including avoiding Chan, the bugs disappeared.I don't think I'll be using Chan again until after someone has proven in correct. Considering Chan is150 lines of code and has been around for many years, that's amazing! Did you report the bugs? Is it anything to do with asynchronous exceptions? Nothing to do with async exceptions. I found: http://hackage.haskell.org/trac/ghc/ticket/4154 Yup, that's a bug. Not clear if it's fixable. http://hackage.haskell.org/trac/ghc/ticket/3527 That too. A very similar bug in fact, if there is a fix it will probably fix both of them. The problem is that readChan holds a lock on the read end of the Chan, so neither isEmptyChan nor unGetChan can work when a reader is blocked. Of course, there's also the async exceptions bug still around: http://hackage.haskell.org/trac/ghc/ticket/3160 Yes, that's a bug (though not in Chan). However, even after having a program with no async exceptions (I never used them), and eliminating unGetChan and isEmpyChan, I still got bugs. I have no proof they came from the Chan module, and no minimal test case was ever able to recreate them, but the same program with my own Chan implementation worked. My Chan had different properties (it queues items randomly) and a subset of the Chan functions, so it still doesn't prove any issue with Chan - but I am now sceptical. It's surprising how difficult it is to get these MVar-based abstractions right. Some thorough testing of Chan is probably in order. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: BlockedIndefinitelyOnMVar exception
http://hackage.haskell.org/trac/ghc/ticket/4154 Yup, that's a bug. Not clear if it's fixable. http://hackage.haskell.org/trac/ghc/ticket/3527 That too. A very similar bug in fact, if there is a fix it will probably fix both of them. The problem is that readChan holds a lock on the read end of the Chan, so neither isEmptyChan nor unGetChan can work when a reader is blocked. I wrote my Chan around the abstraction: data Chan a = Chan (MVar (Either [a] [MVar a])) The Chan either has elements in it (Left), or has readers waiting for elements (Right). To get the fairness properties on Chan you might want to make these two lists Queue's, but I think the basic principle still works. By using this abstraction my Chan was a lot simpler. With this scheme implementing isEmpyChan or unGetChan would both work nicely. My Chan was not designed for performance. (In truth I replaced the Left with IntMap a, and inserted elements with a randomly chosen key, but the basic idea is the same.) own Chan implementation worked. My Chan had different properties (it queues items randomly) and a subset of the Chan functions, so it still doesn't prove any issue with Chan - but I am now sceptical. It's surprising how difficult it is to get these MVar-based abstractions right. Some thorough testing of Chan is probably in order. Agreed! In this project I wrote 8 different concurrency abstractions. I had bugs in most. MVar is a great building block on which to put higher layered abstractions, but using it correctly is tricky. I found that I used MVar's in four ways: 1) MVar's which are always full, and are just locks around data for consistency. Created with newMVar, used with modifyMVar. 2) MVar's which contain unit and are used for locking something other than data (i.e. a file on disk). Created with newMVar, used with withMVar. 3) MVar's which are used to signal computation can begin, created with newMVarEmpty, given to someone who calls putMVar (), and waited on by the person who created them. 4) MVar's which go in a higher-level concurrency operation - CountVars (variables which wait until they have been signaled N times), RandChan (Chan but with randomness), Pool (thread pool) etc. Thanks, Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
