Re: [fpc-pascal] Semaphore
On 12 Feb 2008, at 14:20, Damien Gerard wrote: Le Feb 12, 2008 à 12:36 PM, Graeme Geldenhuys a écrit : procedure TtiPool.CreatePoolSemaphore; begin {$IFDEF MSWINDOWS} if FSemaphore 0 then CloseHandle(FSemaphore); FSemaphore := CreateSemaphore(nil, FMaxPoolSize, FMaxPoolSize, nil); {$ENDIF MSWINDOWS} {$IFDEF LINUX} sem_destroy(FSemaphore); if sem_init(FSemaphore, 0, FMaxPoolSize) 0 then raise Exception.Create('Failed to create the semaphore'); {$ENDIF LINUX} end; Thanks ! I think I can safely change LINUX by UNIX and it should work on OS X too. No, you can't, because sem_init does not exist on Mac OS X. Use the RTL implementation, it works on all platforms. Further, the above code looks buggy because it always calls sem_destroy for FSemaphore, even it was not allocated (and that may cause the program to crash, depending on the contents of FSemaphore at that point and the implementation of sem_init). Jonas___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore
On 11 Feb 2008, at 10:45, Damien Gerard wrote: I think it has been discussed before but I would like to use Semaphores, to allow multiple reads but 1 write at once (which is in fact commons operations in a multi-thread program). My program must work under OSX/Nux and Windows. I may have missed something but I can not find something suitable for me. Under Windows OpenSemaphore, under unixes CurrentTM.Semaphore*... CurrentTM.Semaphore* should also work under Windows (if it doesn't, please file a bug report). And the way to use those is by calling {$ifdef unix} uses cthreads; {$endif} var p: pointer; begin p:=currenttm.SemaphoreInit; currenttm.semaphorepost(p); currenttm.semaphorewait(p); currenttm.semaphoredestroy(p); end. Jonas ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore
Le Feb 12, 2008 à 12:36 PM, Graeme Geldenhuys a écrit : On 11/02/2008, Damien Gerard [EMAIL PROTECTED] wrote: My program must work under OSX/Nux and Windows. I may have missed something but I can not find something suitable for me. Under Windows OpenSemaphore, under unixes CurrentTM.Semaphore*... Is there any TSemaphore class of equivalent ? Or may be an example how to use Semaphores ? :) I've wondered the same thing In the end I had to write code like the following: Not idea, but it works. We needed to support Linux and Windows only. procedure TtiPool.CreatePoolSemaphore; begin {$IFDEF MSWINDOWS} if FSemaphore 0 then CloseHandle(FSemaphore); FSemaphore := CreateSemaphore(nil, FMaxPoolSize, FMaxPoolSize, nil); {$ENDIF MSWINDOWS} {$IFDEF LINUX} sem_destroy(FSemaphore); if sem_init(FSemaphore, 0, FMaxPoolSize) 0 then raise Exception.Create('Failed to create the semaphore'); {$ENDIF LINUX} end; Thanks ! I think I can safely change LINUX by UNIX and it should work on OS X too. I will made a TSemaphore :) Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal -- Damien Gerard [EMAIL PROTECTED] Le temps n'a pas d'importance. Seul le code est important -- (f00ty) ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore
On 11/02/2008, Damien Gerard [EMAIL PROTECTED] wrote: My program must work under OSX/Nux and Windows. I may have missed something but I can not find something suitable for me. Under Windows OpenSemaphore, under unixes CurrentTM.Semaphore*... Is there any TSemaphore class of equivalent ? Or may be an example how to use Semaphores ? :) I've wondered the same thing In the end I had to write code like the following: Not idea, but it works. We needed to support Linux and Windows only. procedure TtiPool.CreatePoolSemaphore; begin {$IFDEF MSWINDOWS} if FSemaphore 0 then CloseHandle(FSemaphore); FSemaphore := CreateSemaphore(nil, FMaxPoolSize, FMaxPoolSize, nil); {$ENDIF MSWINDOWS} {$IFDEF LINUX} sem_destroy(FSemaphore); if sem_init(FSemaphore, 0, FMaxPoolSize) 0 then raise Exception.Create('Failed to create the semaphore'); {$ENDIF LINUX} end; Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore
Le Feb 12, 2008 à 12:36 PM, Graeme Geldenhuys a écrit : On 11/02/2008, Damien Gerard [EMAIL PROTECTED] wrote: My program must work under OSX/Nux and Windows. I may have missed something but I can not find something suitable for me. Under Windows OpenSemaphore, under unixes CurrentTM.Semaphore*... Is there any TSemaphore class of equivalent ? Or may be an example how to use Semaphores ? :) I've wondered the same thing In the end I had to write code like the following: Not idea, but it works. We needed to support Linux and Windows only. procedure TtiPool.CreatePoolSemaphore; begin {$IFDEF MSWINDOWS} if FSemaphore 0 then CloseHandle(FSemaphore); FSemaphore := CreateSemaphore(nil, FMaxPoolSize, FMaxPoolSize, nil); {$ENDIF MSWINDOWS} {$IFDEF LINUX} sem_destroy(FSemaphore); if sem_init(FSemaphore, 0, FMaxPoolSize) 0 then raise Exception.Create('Failed to create the semaphore'); {$ENDIF LINUX} end; I realize you have already made a TSemaphore like. May be I can directly use it for my purposes ? -- Damien Gerard [EMAIL PROTECTED] Le temps n'a pas d'importance. Seul le code est important -- (f00ty) ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore
Le Feb 12, 2008 à 2:41 PM, Jonas Maebe a écrit : On 11 Feb 2008, at 10:45, Damien Gerard wrote: I think it has been discussed before but I would like to use Semaphores, to allow multiple reads but 1 write at once (which is in fact commons operations in a multi-thread program). My program must work under OSX/Nux and Windows. I may have missed something but I can not find something suitable for me. Under Windows OpenSemaphore, under unixes CurrentTM.Semaphore*... CurrentTM.Semaphore* should also work under Windows (if it doesn't, please file a bug report). And the way to use those is by calling {$ifdef unix} uses cthreads; {$endif} var p: pointer; begin p:=currenttm.SemaphoreInit; currenttm.semaphorepost(p); currenttm.semaphorewait(p); currenttm.semaphoredestroy(p); end. I though that too but currenttm can not be found under Windows. What is the good unit for it ? -- Damien Gerard [EMAIL PROTECTED] Le temps n'a pas d'importance. Seul le code est important -- (f00ty) ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore
On 12/02/2008, Jonas Maebe [EMAIL PROTECTED] wrote: CurrentTM.Semaphore* should also work under Windows (if it doesn't, please file a bug report). And the way to use those is by calling {$ifdef unix} uses cthreads; {$endif} var p: pointer; begin p:=currenttm.SemaphoreInit; currenttm.semaphorepost(p); currenttm.semaphorewait(p); currenttm.semaphoredestroy(p); end. Thanks Jonas. :) I didn't know about this. I'll update my old code to the above... Regards, - Graeme - ___ fpGUI - a cross-platform Free Pascal GUI toolkit http://opensoft.homeip.net/fpgui/ ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Vinzent Höfler wrote: because we assume it's non-recursive, that was the whole point. So we should *first* check the count and then may lock/unlock the mutex accordingly. Note that these two actions must be atomic. This is what the TryLock is for. Micha ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
On Tuesday 25 July 2006 06:40, Micha Nelissen wrote: Vinzent Höfler wrote: because we assume it's non-recursive, that was the whole point. So we should *first* check the count and then may lock/unlock the mutex accordingly. Note that these two actions must be atomic. Oh, really? This is what the TryLock is for. No, that's what the Owner_Check_Lock in my posted code would be for in the first place. But you're right, TryLock makes it possibly easier to detect/handle failures. So the code from // Now try locking the real mutex. should actually be more like: |// First encounter? So lock it actually. |if self.Count = 0 then |begin | // This should *always* succeed, because we're the owning | // thread (we've just checked it, right?) and we didn't lock it | // yet according to Count. | // Thinkable ways how this can fail is if somebody has direct | // access to the actual pthread mutex inside here or we | // fuc^Wmessed up the counter somehow ... | if pthread_mutex_trylock (self...) 0 then | // Our real mutex couldn't be locked!? This *is* strange. | exit (FAILURE); |end {if}; | |// Alright, we got it. Mutex is locked (or at least, it *should* be |// already), so count up. |self.Count := self.Count + 1; |self.Owner_Check.Unlock; |exit (SUCCESS); A pthread_mutex_trylock alone wouldn't actually help on a non-recursive real mutex. Because: if it's already locked I still don't know if it belongs to the current thread, but precisely that knowledge is needed to implement the recursive one on top of a non-recursive. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Vinzent Hoefler wrote: On Tuesday 25 July 2006 06:40, Micha Nelissen wrote: Vinzent Höfler wrote: because we assume it's non-recursive, that was the whole point. So we should *first* check the count and then may lock/unlock the mutex accordingly. Note that these two actions must be atomic. Oh, really? Ehm, no. Got confused a bit :-). Reread the thread, and think your latest implementation as posted here is ok. Micha ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
On Tuesday 25 July 2006 07:46, Micha Nelissen wrote: Vinzent Hoefler wrote: On Tuesday 25 July 2006 06:40, Micha Nelissen wrote: Vinzent Höfler wrote: because we assume it's non-recursive, that was the whole point. So we should *first* check the count and then may lock/unlock the mutex accordingly. Note that these two actions must be atomic. Oh, really? Ehm, no. Ehm, yes. I was being ironic here. Of course, the action of checking the counter and locking/unlocking the associated mutex must be atomic. Got confused a bit :-). Reread the thread, and think your latest implementation as posted here is ok. No, it's not. As usually I should sit down with pencil, paper, a cup of coffee and a cigarette before writing code. Anything else is hacking and takes more time and changes. (Why the hell don't I listen to myself?) Despite the fact that some minor details may be still missing, like setting the owner's ThreadId, for example, this implementation has one very major flaw. It doesn't suspend the calling thread if it's not already owned by it and if it is, it doesn't do that either, becaus it shouldn't. That makes it kind of useless, I think. :) Instead of exiting with (NOT_OWNED) the mutex should be locked, of course. But before doing that, the Owner_Check_Lock must be released. And now I start smelling a race condition. Back to the drawing board, I guess. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Vinzent Hoefler wrote: Ehm, no. Ehm, yes. I was being ironic here. Of course, the action of checking the counter and locking/unlocking the associated mutex must be atomic. But here they are not associated, they're protected by owner_lock, as you said. Got confused a bit :-). Reread the thread, and think your latest implementation as posted here is ok. No, it's not. As usually I should sit down with pencil, paper, a cup of I mean the idea for the recursive locking is ok. I was confused by the whole 'owning' part. Locks don't own threads in my mind, so I assumed you got that right. Alas, no :-). Also problematic is posting only partly updates to a function. Then we forget the other, also critical part. function Recursive_Mutex.Lock:...; begin // Owned by current thread? if CurrentThreadId self.ThreadId then begin result := pthread_mutex_lock (self...); if result 0 then exit; self.ThreadId := CurrentThreadId; self.Count := 1; end else begin Inc(self.Count); end; result := SUCCESS; end {Mutex.Lock}; function Recursive_Mutex.Unlock: ...; begin if CurrentThreadId self.ThreadId then exit(ENOTOWN); assert(self.Count 0); dec(self.Count); if self.Count = 0 then begin self.ThreadId := 0; // a threadid that does not occur in system pthread_mutex_unlock(self...); end; result := SUCCESS; end; I don't see the need for Owner_Check_Lock anymore ? :-) Micha ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Am Dienstag, 25. Juli 2006 10:46 schrieb Micha Nelissen: Vinzent Hoefler wrote: Ehm, no. Ehm, yes. I was being ironic here. Of course, the action of checking the counter and locking/unlocking the associated mutex must be atomic. But here they are not associated, they're protected by owner_lock, as you said. Got confused a bit :-). Reread the thread, and think your latest implementation as posted here is ok. No, it's not. As usually I should sit down with pencil, paper, a cup of I mean the idea for the recursive locking is ok. I was confused by the whole 'owning' part. Locks don't own threads in my mind, so I assumed you got that right. Alas, no :-). Also problematic is posting only partly updates to a function. Then we forget the other, also critical part. function Recursive_Mutex.Lock:...; begin // Owned by current thread? if CurrentThreadId self.ThreadId then begin result := pthread_mutex_lock (self...); if result 0 then exit; self.ThreadId := CurrentThreadId; self.Count := 1; end else begin Inc(self.Count); end; result := SUCCESS; end {Mutex.Lock}; function Recursive_Mutex.Unlock: ...; begin if CurrentThreadId self.ThreadId then exit(ENOTOWN); assert(self.Count 0); dec(self.Count); if self.Count = 0 then begin self.ThreadId := 0; // a threadid that does not occur in system pthread_mutex_unlock(self...); end; result := SUCCESS; end; I don't see the need for Owner_Check_Lock anymore ? :-) You have to prevent that 2 concurrent threads call lock simultanously, so you have to take an internal lock before touching internal vars like self.ThreadID ? Burkhard ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Burkhard Carstens wrote: I don't see the need for Owner_Check_Lock anymore ? :-) You have to prevent that 2 concurrent threads call lock simultanously, No, why? so you have to take an internal lock before touching internal vars like self.ThreadID ? No, why ? This is a common misconception I think. (Multi-threaded) Programs are about state, not about touching internal variables. Micha ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
On Tuesday 25 July 2006 09:04, Burkhard Carstens wrote: Am Dienstag, 25. Juli 2006 10:46 schrieb Micha Nelissen: function Recursive_Mutex.Lock:...; begin // Owned by current thread? if CurrentThreadId self.ThreadId then begin result := pthread_mutex_lock (self...); if result 0 then exit; self.ThreadId := CurrentThreadId; self.Count := 1; end else begin Inc(self.Count); end; result := SUCCESS; end {Mutex.Lock}; function Recursive_Mutex.Unlock: ...; begin if CurrentThreadId self.ThreadId then exit(ENOTOWN); assert(self.Count 0); dec(self.Count); if self.Count = 0 then begin self.ThreadId := 0; // a threadid that does not occur in system pthread_mutex_unlock(self...); end; result := SUCCESS; end; I don't see the need for Owner_Check_Lock anymore ? :-) You have to prevent that 2 concurrent threads call lock simultanously, No. That's not a problem. See the explanation below. Perhaps the line if result 0 then exit; may be kind of superfluous, or even dangerous, because it would return from the lock operation without actually locking it, but well - if it goes wrong here, we're screwed anyway. At least the caller could check, if it did. so you have to take an internal lock before touching internal vars like self.ThreadID ? First I thought that, too. But after looking at and thinking about it a while, the race you mention could only occur if the *same* thread would call Lock() simultaneously, and that can't happen. In the case two different threads enter here, they may both still see 0 and are trying to continue then, but only one of them will acquire the mutex sucessfully, leaving the other suspended. Once the suspended thread continues, it actually owns the mutex, so everything's ok then. Ok, there's a glitch: The read and write of self.ThreadId is required to be atomic, so that a thread entering may either see 0 or the owner's thread id when checking, otherwise it could be possible, it sees its own thread_id due to a partial update and *then* it goes really wrong. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
On Tuesday 25 July 2006 08:46, Micha Nelissen wrote: Vinzent Hoefler wrote: Ehm, no. Ehm, yes. I was being ironic here. Of course, the action of checking the counter and locking/unlocking the associated mutex must be atomic. But here they are not associated, they're protected by owner_lock, as you said. Yes. That's what would have made the whole operation atomic. I mean the idea for the recursive locking is ok. Well, in the end it's functionality that counts. Ideas I got. Lots. I was confused by the whole 'owning' part. Locks don't own threads in my mind, so I assumed you got that right. Yes, that's right, locks don't own threads. But threads own locks. :) Alas, no :-). Also problematic is posting only partly updates to a function. Then we forget the other, also critical part. Yepp. It wasn't yet meant to be committed to svn, you know? ;) function Recursive_Mutex.Lock:...; [...] function Recursive_Mutex.Unlock: ...; [...] I don't see the need for Owner_Check_Lock anymore ? :-) Yes, that looks about right. ;-) Except perhaps for the possible glitch I mentioned in my other mail. Don't know, if we can ignore it for now or simply assume that (up to) 32-Bit-variables always *are* atomic types. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Am Dienstag, 25. Juli 2006 11:59 schrieb Vinzent Hoefler: On Tuesday 25 July 2006 09:04, Burkhard Carstens wrote: Am Dienstag, 25. Juli 2006 10:46 schrieb Micha Nelissen: function Recursive_Mutex.Lock:...; begin // Owned by current thread? if CurrentThreadId self.ThreadId then begin result := pthread_mutex_lock (self...); if result 0 then exit; self.ThreadId := CurrentThreadId; self.Count := 1; end else begin Inc(self.Count); end; result := SUCCESS; end {Mutex.Lock}; function Recursive_Mutex.Unlock: ...; begin if CurrentThreadId self.ThreadId then exit(ENOTOWN); assert(self.Count 0); dec(self.Count); if self.Count = 0 then begin self.ThreadId := 0; // a threadid that does not occur in system pthread_mutex_unlock(self...); end; result := SUCCESS; end; I don't see the need for Owner_Check_Lock anymore ? :-) You have to prevent that 2 concurrent threads call lock simultanously, No. That's not a problem. See the explanation below. Perhaps the line if result 0 then exit; may be kind of superfluous, or even dangerous, because it would return from the lock operation without actually locking it, but well - if it goes wrong here, we're screwed anyway. At least the caller could check, if it did. so you have to take an internal lock before touching internal vars like self.ThreadID ? First I thought that, too. But after looking at and thinking about it a while, the race you mention could only occur if the *same* thread would call Lock() simultaneously, and that can't happen. In the case two different threads enter here, they may both still see 0 and are trying to continue then, but only one of them will acquire the mutex sucessfully, leaving the other suspended. Once the suspended thread continues, it actually owns the mutex, so everything's ok then. Ok, there's a glitch: The read and write of self.ThreadId is required to be atomic, so that a thread entering may either see 0 or the owner's thread id when checking, otherwise it could be possible, it sees its own thread_id due to a partial update and *then* it goes really wrong. To prevent this very rare but indeed possible case, just use interlockedexchange to assign anything to self.threadid ? Beside this, I like this implementation! This way, we have a recursive mutex, without depending on pthread recursive extension. Burkhard ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Vinzent Hoefler wrote: Ok, there's a glitch: The read and write of self.ThreadId is required to be atomic, so that a thread entering may either see 0 or the owner's thread id when checking, otherwise it could be possible, it sees its own thread_id due to a partial update and *then* it goes really wrong. It's not a glitch but an assumption I'd say. The compiler devels should be able to answer whether we can assume this, or not ? Micha ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
On Tuesday 25 July 2006 14:51, Micha Nelissen wrote: Vinzent Hoefler wrote: Ok, there's a glitch: The read and write of self.ThreadId is required to be atomic, so that a thread entering may either see 0 or the owner's thread id when checking, otherwise it could be possible, it sees its own thread_id due to a partial update and *then* it goes really wrong. It's not a glitch but an assumption I'd say. Well, it will become a glitch, if the assumption about the atomic read/ write turns out to be wrong. ;-) The compiler devels should be able to answer whether we can assume this, or not ? Well, I suppose so. But in the end it depends on the target architecture, not the compiler in general. That's BTW, one of the reasons I'd like to have a Pascal's equivalent of Ada's Pragma Atomic(...) and Pragma Volatile(...) for certain types and variables. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
On Monday 24 July 2006 13:34, Marco van de Voort wrote: When I run the Unit Tests and create a single Lock and the Unlock it, all works fine. If I then iterrate that test by creating 10 locks and then call unlock 10 times, the Unit Tests freeze on the following line with the second iteration: Sounds like some recursion property is not set. I don't think, semaphores have recursion properties. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
On Mon, 24 Jul 2006, Vinzent Hoefler wrote: On Monday 24 July 2006 13:34, Marco van de Voort wrote: When I run the Unit Tests and create a single Lock and the Unlock it, all works fine. If I then iterrate that test by creating 10 locks and then call unlock 10 times, the Unit Tests freeze on the following line with the second iteration: Sounds like some recursion property is not set. I don't think, semaphores have recursion properties. They do, in some implementations. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
On Monday 24 July 2006 14:09, Michael Van Canneyt wrote: On Mon, 24 Jul 2006, Vinzent Hoefler wrote: On Monday 24 July 2006 13:34, Marco van de Voort wrote: When I run the Unit Tests and create a single Lock and the Unlock it, all works fine. If I then iterrate that test by creating 10 locks and then call unlock 10 times, the Unit Tests freeze on the following line with the second iteration: Sounds like some recursion property is not set. I don't think, semaphores have recursion properties. They do, in some implementations. Ok, granted. :) But not in POSIX, AFAICS. And they were never designed that way. I think, that's why someone invented the mutex. ;) Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Am Montag, 24. Juli 2006 16:19 schrieb Vinzent Hoefler: On Monday 24 July 2006 14:09, Michael Van Canneyt wrote: On Mon, 24 Jul 2006, Vinzent Hoefler wrote: On Monday 24 July 2006 13:34, Marco van de Voort wrote: When I run the Unit Tests and create a single Lock and the Unlock it, all works fine. If I then iterrate that test by creating 10 locks and then call unlock 10 times, the Unit Tests freeze on the following line with the second iteration: Sounds like some recursion property is not set. I don't think, semaphores have recursion properties. They do, in some implementations. Ok, granted. :) But not in POSIX, AFAICS. And they were never designed that way. I think, that's why someone invented the mutex. ;) hmm, so a mutex can be recursive, if the type is set correctly, which is not supportet on all platforms implementing pthread (man-pages of pthread). I vote for more pascal based versions of TMutex, TSemaphore and TEvent, that behaves just like the ones in Delphi and use the system specific functions (like pthread stuff) only internally in their simplest/most portable form. Currently, I have the problem to get a working TEvent: - The one in syncobjs returns wrError, when it's waitfor is called with anything else tah infinite timeout - same goes for rtlbasicevent (cthreads) - the rtlEvent works even with timeout (with my patch posted on mantis), but this one clears the signaled state, when wait is called. IOW. if I call waitfor on the allready signalled event, it won't return before timeout. Also it doesn't inform the caller, wheter it returnt due to signalled or to timeout .. regards Burkhard ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
On Monday 24 July 2006 14:57, Burkhard Carstens wrote: I vote for more pascal based versions of TMutex, TSemaphore and TEvent, that behaves just like the ones in Delphi and use the system specific functions (like pthread stuff) only internally in their simplest/most portable form. I'd vote for a more abstract version. :) The more or less basic things needed in multi-threaded programing are - semaphore - persistent signal - transient signal (pulse) - event There's some more higher level stuff (like broadcasts, for example), but that's about all you usually need. Currently we have the semaphore (more or less as SyncObjs.tCriticalSection), the persistent and the transient signal (SyncObjs.tEvent), although without timeouts (at least when trying to be portable), the event is somewhat implemented, but IIRC it misses a Toggle property, which can be used for simple intertask communication (Now I'm blocking, you're on.). Currently, I have the problem to get a working TEvent: - The one in syncobjs returns wrError, when it's waitfor is called with anything else tah infinite timeout Yes, that's not implemented, although it should be easily added by using condition variables of pthreads. - the rtlEvent works even with timeout (with my patch posted on mantis), but this one clears the signaled state, when wait is called. *oh* Before the wait? That's not good. This cries for race conditions and subtle deadlock situations. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Am Montag, 24. Juli 2006 17:27 schrieb Vinzent Hoefler: On Monday 24 July 2006 14:57, Burkhard Carstens wrote: I vote for more pascal based versions of TMutex, TSemaphore and TEvent, that behaves just like the ones in Delphi and use the system specific functions (like pthread stuff) only internally in their simplest/most portable form. I'd vote for a more abstract version. :) The more or less basic things needed in multi-threaded programing are - semaphore - persistent signal - transient signal (pulse) - event There's some more higher level stuff (like broadcasts, for example), but that's about all you usually need. Currently we have the semaphore (more or less as SyncObjs.tCriticalSection), the persistent and the transient signal (SyncObjs.tEvent), although without timeouts (at least when trying to be portable), the event is somewhat implemented, but IIRC it misses a Toggle property, which can be used for simple intertask communication (Now I'm blocking, you're on.). Currently, I have the problem to get a working TEvent: - The one in syncobjs returns wrError, when it's waitfor is called with anything else tah infinite timeout Yes, that's not implemented, although it should be easily added by using condition variables of pthreads. ..which currently doesn't work, unless someone applies my patch (or a better one) .. - the rtlEvent works even with timeout (with my patch posted on mantis), but this one clears the signaled state, when wait is called. *oh* Before the wait? That's not good. This cries for race conditions and subtle deadlock situations. thread synchronization uses this and takes care of it by calling RTLeventstartwait which locks the associated mutex. So I don't expect a race condition there, but don't know about other places. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Burkhard Carstens wrote: Am Montag, 24. Juli 2006 17:27 schrieb Vinzent Hoefler: - the rtlEvent works even with timeout (with my patch posted on mantis), but this one clears the signaled state, when wait is called. *oh* Before the wait? That's not good. This cries for race conditions and subtle deadlock situations. thread synchronization uses this and takes care of it by calling RTLeventstartwait which locks the associated mutex. As far as I just read code and comments, it does it for Windows to get Unix behaviour, although I don't see the point here, because this just forces the race condition the POSIX-Threads implementation actually tries to circumvent with this associated mutex. I'm not even sure, if Unix actually clears the event. The man-pages don't seem to indicate so. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Am Montag, 24. Juli 2006 19:22 schrieb Vinzent Höfler: Burkhard Carstens wrote: Am Montag, 24. Juli 2006 17:27 schrieb Vinzent Hoefler: - the rtlEvent works even with timeout (with my patch posted on mantis), but this one clears the signaled state, when wait is called. *oh* Before the wait? That's not good. This cries for race conditions and subtle deadlock situations. thread synchronization uses this and takes care of it by calling RTLeventstartwait which locks the associated mutex. As far as I just read code and comments, it does it for Windows to get Unix behaviour, although I don't see the point here, because this just forces the race condition the POSIX-Threads implementation actually tries to circumvent with this associated mutex. I don't think so. In unix, the mutex is locked, which ensures, there is not setevent between startwait and actual call to wait, because the call to wait seems to reset the cond, then releases the mutex and starts waiting. In windows, this doesn't matter (resetting the event on startwait), because it preserves the signaled state of the event. The wait call just returns immediately, if the event is allready signaled. To me, this is fine. As long as rtlevent is used with this in mind, it should give a consistent behaviour on windows and linux. (see classes.inc thread synchronization) I'm not even sure, if Unix actually clears the event. The man-pages don't seem to indicate so. It does, at least with the timedwait. I tested that extensively. Burkhard ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
[..] ... As long as rtlevent is used with this in mind, it should give a consistent behaviour on windows and linux. this in mind means: The only way how rtlevent should be used is: 1. call rtlstartwait 2. start the code (e.g. start/resume a thread), that signals the event. 3. call rtlwaitevent ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Burkhard Carstens wrote: Am Montag, 24. Juli 2006 19:22 schrieb Vinzent Höfler: Burkhard Carstens wrote: Am Montag, 24. Juli 2006 17:27 schrieb Vinzent Hoefler: - the rtlEvent works even with timeout (with my patch posted on mantis), but this one clears the signaled state, when wait is called. *oh* Before the wait? That's not good. This cries for race conditions and subtle deadlock situations. thread synchronization uses this and takes care of it by calling RTLeventstartwait which locks the associated mutex. As far as I just read code and comments, it does it for Windows to get Unix behaviour, although I don't see the point here, because this just forces the race condition the POSIX-Threads implementation actually tries to circumvent with this associated mutex. I don't think so. Well, in that case either I am totally stupid (won't be news, anyway) or I must have different source code: From rtl/unix/cthreads.inc: |procedure intRTLEventStartWait(AEvent: PRTLEvent); |var p:pintrtlevent; | |begin | p:=pintrtlevent(aevent); | pthread_mutex_lock(@p^.mutex); |end; | |procedure intRTLEventWaitFor(AEvent: PRTLEvent); |var p:pintrtlevent; | |begin | p:=pintrtlevent(aevent); | pthread_cond_wait(@p^.condvar, @p^.mutex); | pthread_mutex_unlock(@p^.mutex); |end; Which is pretty much what you described and if pthread_cond_wait() works as described, there's no race condition here. But now rtl/win/systhrd.inc: |procedure intRTLEventStartWait(AEvent: PRTLEvent); |begin | { this is to get at least some common behaviour on unix and win32: |events before startwait are lost on unix, so reset the event on |win32 as well } | ResetEvent(THANDLE(AEvent)); |end; | |procedure intRTLEventWaitFor(AEvent: PRTLEvent); |const | INFINITE=dword(-1); |begin | WaitForSingleObject(THANDLE(AEvent), INFINITE); |end; I don't see any locking in intRTLEventStartWait(), but I see the signal being cleared in an unprotected way. So the signal can still get set between the calls to intRTLStartWait() and intRTLEventWaitFor() which makes the clearing quite useless. In unix, the mutex is locked, which ensures, there is not setevent between startwait and actual call to wait, because the call to wait seems to reset the cond, then releases the mutex and starts waiting. Yes, and now I seem to understand why. Unix only implements transient signals. So if there's noone waiting on the signal, it gets lost. Windows OTOH seems to implement persistent signals. That's all the difference. In windows, this doesn't matter (resetting the event on startwait), because it preserves the signaled state of the event. It preserves the signalled state? By resetting it? I don't think so. AFAICS, it resets the signal before actually starting to wait for it. So it still can sneak in if the timing is right. That's what I'd call a race condition. The wait call just returns immediately, if the event is allready signaled. To me, this is fine. As long as rtlevent is used with this in mind, it should give a consistent behaviour on windows and linux. (see classes.inc thread synchronization) You mean code like: |if timeout0 then | begin |RtlEventStartWait(SynchronizeTimeoutEvent); |RtlEventWaitFor(SynchronizeTimeoutEvent,timeout); | end | else |RtlEventResetEvent(SynchronizeTimeoutEvent); So where's the code that makes sure that the thread signalling the event is (only) resumed between the two calls? I'm not even sure, if Unix actually clears the event. The man-pages don't seem to indicate so. It does, at least with the timedwait. I tested that extensively. Yes, and now I think we both know why. I seem to repeat myself, but it might be important: The difference is all between implementing transient and persistent signals. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Burkhard Carstens wrote: [..] ... As long as rtlevent is used with this in mind, it should give a consistent behaviour on windows and linux. this in mind means: The only way how rtlevent should be used is: 1. call rtlstartwait 2. start the code (e.g. start/resume a thread), that signals the event. 3. call rtlwaitevent Great. ;( But there's a flaw in this: If I'd actually know that thread A had been executed rtlstartwait() before thread B tries signalling the condition, I wouldn't need the whole synchronization at all. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Vinzent Höfler wrote: I didn't understand and follow the whole thread but please submit a bug report if something in FPC needs to be fixed :) ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Vinzent Höfler wrote: Currently all I see is a subtle semantic difference between Windows- and Unix-Event's implementation. AFAICS, it's as close as it can get. You mean subtle as in, unimportant, or as in, possibly fatal ? :-) If possibly fatal, describe when/how. So for now all I can say is that this should either a) be documented, *Could* be a good idea, yes ;-). b) changed to a common semantic behaviour, or They're both more or less 'transient' now. I am not sure how to emulate windows behavior on unix, although that behavior is easier to use I think, so would be desirable. Micha ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Burkhard Carstens wrote: Am Montag, 24. Juli 2006 20:33 schrieb Vinzent Höfler: Burkhard Carstens wrote: [..] ... As long as rtlevent is used with this in mind, it should give a consistent behaviour on windows and linux. this in mind means: The only way how rtlevent should be used is: 1. call rtlstartwait 2. start the code (e.g. start/resume a thread), that signals the event. 3. call rtlwaitevent Great. ;( But there's a flaw in this: If I'd actually know that thread A had been executed rtlstartwait() before thread B tries signalling the condition, I wouldn't need the whole synchronization at all. You want to be sure, a thread started in step 2 has finished, before you proceed to step 4 .. Ahh, yes. Now I see where you're going. What you're saying is that RTLEvent can't be used asynchronously (i.e. from different threads signalling each other) in a consistent way. Well, *that* should be documented. I thought, trying to achieve exactly that behaviour was the whole idea. btw. this is not TEvent, we are talking about, it's rtlevent .. TEvent (in synconjs) is implemented differently, and should work like delphi/ windows TEvent, except that it is not capable of timedwait .. Well, if I'm looking at it correctly that's only because its implementation is based on the POSIX semaphore. It could be using the phtread_mutex stuff instead. The pthread-primitives are used anyway, so I don't see problems with dependencies on libpthread/libc or so... Hmm, *looking closer* ... are there any differences between Darwin/BSD/Linux/Solaris code at all? AFAICS, all these implementations could be merged into a single one for Unix/POSIX targets. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Micha Nelissen wrote: Vinzent Höfler wrote: Currently all I see is a subtle semantic difference between Windows- and Unix-Event's implementation. AFAICS, it's as close as it can get. You mean subtle as in, unimportant, or as in, possibly fatal ? :-) If possibly fatal, describe when/how. If used incorrectly, i.e. asynchronously. So for now all I can say is that this should either a) be documented, *Could* be a good idea, yes ;-). b) changed to a common semantic behaviour, or They're both more or less 'transient' now. Depends on the view point. Doing two consecutive waits() might do very different things. :) So I'd say, the behaviour and intended use should definitely be documented. I am not sure how to emulate windows behavior on unix, although that behavior is easier to use I think, so would be desirable. You'd have to create an own abstraction about the useful primitives. That's what I was initially talking about. :) Dunno, if that's good for the RTL, because you'd probably loose /some/ speed on the way... Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Am Montag, 24. Juli 2006 22:05 schrieb Vinzent Höfler: Burkhard Carstens wrote: Am Montag, 24. Juli 2006 20:33 schrieb Vinzent Höfler: Burkhard Carstens wrote: [..] ... As long as rtlevent is used with this in mind, it should give a consistent behaviour on windows and linux. this in mind means: The only way how rtlevent should be used is: 1. call rtlstartwait 2. start the code (e.g. start/resume a thread), that signals the event. 3. call rtlwaitevent Great. ;( But there's a flaw in this: If I'd actually know that thread A had been executed rtlstartwait() before thread B tries signalling the condition, I wouldn't need the whole synchronization at all. You want to be sure, a thread started in step 2 has finished, before you proceed to step 4 .. Ahh, yes. Now I see where you're going. What you're saying is that RTLEvent can't be used asynchronously (i.e. from different threads signalling each other) in a consistent way. Well, *that* should be documented. I thought, trying to achieve exactly that behaviour was the whole idea. btw. this is not TEvent, we are talking about, it's rtlevent .. TEvent (in synconjs) is implemented differently, and should work like delphi/ windows TEvent, except that it is not capable of timedwait .. Well, if I'm looking at it correctly that's only because its implementation is based on the POSIX semaphore. It could be using the phtread_mutex stuff instead. The pthread-primitives are used anyway, so I don't see problems with dependencies on libpthread/libc or so... My idea was to use a rtlevent additionally and only when waitfor is called with timeout AND event is not signalled yet. Hmm, *looking closer* ... are there any differences between Darwin/BSD/Linux/Solaris code at all? AFAICS, all these implementations could be merged into a single one for Unix/POSIX targets. Don't know, maybe. However, we use PTHREAD_MUTEX_RECURSIVE_NP, where NP stands for NotPortable.. don't know which platforms support it and which not. That's why I voted for implementing most of the functionality in pascal and use pthread primitives only in their basic (i.e. mutex in non-recursive) form.. Burkhard ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Burkhard Carstens wrote: Am Montag, 24. Juli 2006 22:05 schrieb Vinzent Höfler: Burkhard Carstens wrote: Am Montag, 24. Juli 2006 20:33 schrieb Vinzent Höfler: Burkhard Carstens wrote: [..] btw. this is not TEvent, we are talking about, it's rtlevent .. TEvent (in synconjs) is implemented differently, and should work like delphi/ windows TEvent, except that it is not capable of timedwait .. Well, if I'm looking at it correctly that's only because its implementation is based on the POSIX semaphore. It could be using the phtread_mutex stuff instead. The pthread-primitives are used anyway, so I don't see problems with dependencies on libpthread/libc or so... My idea was to use a rtlevent additionally and only when waitfor is called with timeout AND event is not signalled yet. Why additionally? Do I miss something here? If done the proper way, everything in SyncObjs should be based on the basic RTL stuff. This would make the SyncObjs unit system independent with clearly defined semantics on all targets. Of course, the original implementation (I suppose it came from Delphi?) was very windows-centric and made use of the available Windows-primitives directly. So maybe for Win32 it should even be kept this way, if not for anything else, somebody may say, for performance reasons at least. Hmm, *looking closer* ... are there any differences between Darwin/BSD/Linux/Solaris code at all? AFAICS, all these implementations could be merged into a single one for Unix/POSIX targets. Don't know, maybe. However, we use PTHREAD_MUTEX_RECURSIVE_NP, where NP stands for NotPortable.. don't know which platforms support it and which not. Well, those flags are somhow there, but there marked XSI in the OpenGroup's current specification, so probably most of our targets implement it - but it's still an extension. That's why I voted for implementing most of the functionality in pascal and use pthread primitives only in their basic (i.e. mutex in non-recursive) form. Hmm. So we'd need a mutex inside a mutex. Now I know why they call it recursive. ;) So it'll be something like that: function Recursive_Mutex.Lock : ...; begin // Lock mutex inside mutex. self.Owner_Check_Lock.Lock; // Owned by current thread? if CurrentThreadId self.ThreadId then begin // Nope! Get the hell outta here. self.Owner_Check_Lock.Unlock; exit (NOT_OWNED); end {if}; // Now try locking the real mutex. if pthread_mutex_lock (self...) = 0 then begin self.Count := self.Count + 1; self.Owner_Check.Unlock; exit (SUCCESS); end {if}; self.Owner_Check.Unlock; exit (FAILURE); end {Mutex.Lock}; Something like that. Don't nail me on that, it's quite late and the heat is still killing me. ;) Well, a simple spinlock would even be better here for performance reasons, because first the locking time should be quite small, and second usually this code will be called in a sequential manner from within the same thread anyway. This would make the lock always being in an unlocked state under all sane circumstances. Maybe even some overflow check in case someone tries to lock the mutex more than 2**32 times in a row? ;) Same for unlock etc. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Vinzent Höfler wrote: Hmm. So we'd need a mutex inside a mutex. Now I know why they call it recursive. ;) So it'll be something like that: function Recursive_Mutex.Lock : ...; begin // Lock mutex inside mutex. self.Owner_Check_Lock.Lock; // Owned by current thread? if CurrentThreadId self.ThreadId then begin // Nope! Get the hell outta here. self.Owner_Check_Lock.Unlock; exit (NOT_OWNED); end {if}; // Now try locking the real mutex. if pthread_mutex_lock (self...) = 0 then begin self.Count := self.Count + 1; self.Owner_Check.Unlock; exit (SUCCESS); end {if}; self.Owner_Check.Unlock; exit (FAILURE); end {Mutex.Lock}; Something like that. Don't nail me on that, it's quite late and the heat is still killing me. ;) I knew it. Of course this is wrong. We can only lock the mutex once, because we assume it's non-recursive, that was the whole point. So we should *first* check the count and then may lock/unlock the mutex accordingly. Vinzent. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Semaphore problems
Am Montag, 24. Juli 2006 23:38 schrieb Vinzent Höfler: Vinzent Höfler wrote: Hmm. So we'd need a mutex inside a mutex. Now I know why they call it recursive. ;) So it'll be something like that: function Recursive_Mutex.Lock : ...; begin // Lock mutex inside mutex. self.Owner_Check_Lock.Lock; // Owned by current thread? if CurrentThreadId self.ThreadId then begin // Nope! Get the hell outta here. self.Owner_Check_Lock.Unlock; exit (NOT_OWNED); end {if}; // Now try locking the real mutex. if pthread_mutex_lock (self...) = 0 then begin self.Count := self.Count + 1; self.Owner_Check.Unlock; exit (SUCCESS); end {if}; self.Owner_Check.Unlock; exit (FAILURE); end {Mutex.Lock}; Something like that. Don't nail me on that, it's quite late and the heat is still killing me. ;) I knew it. Of course this is wrong. We can only lock the mutex once, because we assume it's non-recursive, that was the whole point. So we should *first* check the count and then may lock/unlock the mutex accordingly. yea, this stuff is allways really brain cracking ;-) .. heat kills me, too, so I will re-read these mails tomorrow and think about it again.. good night Burkhard ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-pascal