Re: [fpc-devel] Threading support and C library under Linux/Unix
Michael Schnell schrieb: From some ARM manual: The ARM7500FE does not use the lock feature available in the ARM701 macrocell You must take care to ensure that control of the memory is not removed from the ARM processor while it is performing this instruction. This seems to apply only to systems with multiple processors, and perhaps with DMA controllers. Who else could obtain control of the memory? And who would build a multi-processor system with that inappropriate processor? But sh*t happens, of course ;-) DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Michael Schnell schrieb: > On 06/23/2010 04:49 PM, Henry Vermaak wrote: >> No, SWP is atomic, so the implementation looks good (at a glance). >> > > Seemingly not on all ARM sub-archs: > >>From some ARM manual: > > The ARM7500FE does not use the lock feature available in the ARM701 > macrocell > You must take care to ensure that control of the memory is not > removed from the ARM > processor while it is performing this instruction. I'am sure if it's a problem for somebody, he will find a solution and send a patch. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Specialized Generics and Abstract methods
Can un-specialized generics have abstract methods and have the class implement of the specialized generic contain the actual methods? I'm getting abstract errors presently but want to know if it is even possible... Thanks ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23/06/10 16:19, Michael Schnell wrote: On 06/23/2010 04:49 PM, Henry Vermaak wrote: No, SWP is atomic, so the implementation looks good (at a glance). Seemingly not on all ARM sub-archs: From some ARM manual: The ARM7500FE does not use the lock feature available in the ARM701 macrocell You must take care to ensure that control of the memory is not removed from the ARM processor while it is performing this instruction. That only means that it doesn't send the lock signal to the external memory manager, so you can't have a multi processor implementation using this processor. It doesn't affect the working of the swp instruction. Henry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Michael Schnell wrote: On 06/23/2010 04:49 PM, Henry Vermaak wrote: No, SWP is atomic, so the implementation looks good (at a glance). Seemingly not on all ARM sub-archs: >From some ARM manual: The ARM7500FE does not use the lock feature available in the ARM701 macrocell You must take care to ensure that control of the memory is not removed from the ARM processor while it is performing this instruction. the ARM7500FE is ARMv3. I think fpc supports ARMv4 and higher. Regards, Bernd. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Op Wed, 23 Jun 2010, schreef Michael Schnell: On 06/23/2010 05:18 PM, Mattias Gaertner wrote: What system has such an unfair scheduler? A "fair" Scheduler only can be fair if time slices are to be managed. If you set a thread to a "realtime priority" (OK, you need to be root to be allowed to do that :) ) it's priority is above all the normal processes. That is quite normal if you do embedded Linux systems. It's a non-issue. If you set a thread realtime you know what you are doing and can take care you give up the CPU when feasible, or you simply don't enter mutuxes shared with non-realtime threads. Of course if there is a simple fix... Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Sorry if I have been sounding rude:(. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 05:18 PM, Mattias Gaertner wrote: > What system has such an unfair scheduler? A "fair" Scheduler only can be fair if time slices are to be managed. If you set a thread to a "realtime priority" (OK, you need to be root to be allowed to do that :) ) it's priority is above all the normal processes. That is quite normal if you do embedded Linux systems. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 05:15 PM, Micha Nelissen wrote: > > Priority inversion? Isn't this a problem with any InterlockedC/E? > Priority inversion here leads to an unavoidable deadlock. "Normal" priority inversion only reduces the priority of a high priority thread to that of a lower priority thread that holds the lock. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 04:49 PM, Henry Vermaak wrote: > > No, SWP is atomic, so the implementation looks good (at a glance). > Seemingly not on all ARM sub-archs: >From some ARM manual: The ARM7500FE does not use the lock feature available in the ARM701 macrocell You must take care to ensure that control of the memory is not removed from the ARM processor while it is performing this instruction. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On Wed, 23 Jun 2010 17:12:29 +0200 Michael Schnell wrote: > On 06/23/2010 04:49 PM, Henry Vermaak wrote: > > On 23/06/10 15:26, Michael Schnell wrote: > >> > >> Other ARMs (including thumb2) uses a loop with result compare. AFAIK > >> this does not work reliably. I am not sure it it's really true that ARM > > > > No, SWP is atomic, so the implementation looks good (at a glance). > > > > even if that is true there is a problem: > > // lock > ldr r12, .Lfpc_system_lock > mov r3, #1 > .Lloop: > swp r3, r3, [r12] > cmp r3, #0 > bne .Lloop > // do the job > > - thread 1 comes in and sets fpc_system_lock to 1 > - now thread 2 is scheduled and comes in > - fpc_system_lock is set so it enters a busy spinning > > unfortunately we are in a "realtime" system and the priority of thread 2 > is set higher than that of thread 1 and thus thread 1 never gets a CPU > cycle as thread 2 is busy and thus we have a dead-lock. What system has such an unfair scheduler? Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Michael Schnell wrote: unfortunately we are in a "realtime" system and the priority of thread 2 is set higher than that of thread 1 and thus thread 1 never gets a CPU Priority inversion? Isn't this a problem with any InterlockedC/E? Micha ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 04:49 PM, Henry Vermaak wrote: > On 23/06/10 15:26, Michael Schnell wrote: >> >> Other ARMs (including thumb2) uses a loop with result compare. AFAIK >> this does not work reliably. I am not sure it it's really true that ARM > > No, SWP is atomic, so the implementation looks good (at a glance). > even if that is true there is a problem: // lock ldr r12, .Lfpc_system_lock mov r3, #1 .Lloop: swp r3, r3, [r12] cmp r3, #0 bne .Lloop // do the job - thread 1 comes in and sets fpc_system_lock to 1 - now thread 2 is scheduled and comes in - fpc_system_lock is set so it enters a busy spinning unfortunately we are in a "realtime" system and the priority of thread 2 is set higher than that of thread 1 and thus thread 1 never gets a CPU cycle as thread 2 is busy and thus we have a dead-lock. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 04:38 PM, Florian Klaempfl wrote: > > I can only speak for sparc (because I implemented them initially): they > are implemented according to official sparc docs. Do you know more then > them? > Sorry if I have been sounding rude :(. I learned that If there is no hardware support, it's not possible to create thread-safe or smp safe atomic operations. Maybe the sequence in fact performs a "load locked, save conditional" paradigm, which I did not see. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23/06/10 15:26, Michael Schnell wrote: Other ARMs (including thumb2) uses a loop with result compare. AFAIK this does not work reliably. I am not sure it it's really true that ARM No, SWP is atomic, so the implementation looks good (at a glance). Henry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Michael Schnell schrieb: > PPC, PPC64, Sparc: loop: not reliable I can only speak for sparc (because I implemented them initially): they are implemented according to official sparc docs. Do you know more then them? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23 Jun 2010, at 16:26, Michael Schnell wrote: PPC, PPC64, Sparc: loop: not reliable interlocked* is implemented correctly in the RTL and functions 100% reliably on the above architectures. Please don't make such categorical statements if you are unfamiliar with the architectures (and I guess that the version for ARMv5 and below is probably also completely correct). Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 03:56 PM, Henry Vermaak wrote:
>
> We've got this already, then:
>
> http://www.freepascal.org/docs-html/rtl/system/interlockedcompareexchange.html
>
nice !
X86, X68-64: good.
"ARMV6" uses ldrex (seemingly = "load locked"): good
Other ARMs (including thumb2) uses a loop with result compare. AFAIK
this does not work reliably. I am not sure it it's really true that ARM
implements the "atomic region" but if FUTEX is available in the Kernel a
set of reliable atomic user space instructions is provided as well. So
we would need to use this.
PPC, PPC64, Sparc: loop: not reliable
In theory a decent Futex implementation could be done on top of this
(see "futexes are tricky") for X86, X86-64 and ARMV6. For ARMV5 we
should take a look at the implementation of the atomic functions for
Futex ("pthread_mutex...()") done in libc.
-Michael
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] About ppudump program
On 23 Jun 2010, at 16:14, Doriano Blengino wrote: Jonas wrote: First of all: please mention in the future that you are not subscribed (also in replies, because people may forget), so that people who reply know that they should CC you. Sorry - I didn't notice (is it mentioned somewhere?). I think it's normal to expect that people replying to a message on a mailing list may only reply to the mailing list if they don't know that you are not subscribed. Well, as far as I know, ppu simply skips blocks that it does not know, so I would think it is safe to use. First of all, it is not safe. While ppudump indeed skips blocks that it doesn't know, the contents of the blocks can also change between different versions. I made some tries with two or three different releases, making ppudump parse "its own" ppu files and ppu files from other releases. At different extents, different problems arose. In one bad case, ppudump (version about 2.2.0) crashed while parsing ppu files from its own release, trying to allocate an enourmous block of memory. A self compiled copy, took from 2.4.0 sources, did the same. Many times it complained about "!! has more information" (who cares), "!! refid out of range" or similar (critical). ppudump is indeed often out of date because it's seldom used by compiler developers and not automatically tested. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] About ppudump program
Jonas wrote: First of all: please mention in the future that you are not subscribed (also in replies, because people may forget), so that people who reply know that they should CC you. Sorry - I didn't notice (is it mentioned somewhere?). ** I'am not subscribed to this list, at present ** >/ Now, I would ask an advice: is it better to stick to ppudump, at a price of />/ some reduction in speed, and the risk that its output could change in future, />/ invalidating the IDE parsing routines? Or is it better to use ppu.pas in the />/ IDE, having confidence that ppu.pas will parse PPU file generated by />/ different versions/releases of freepascal? / Well, as far as I know, ppu simply skips blocks that it does not know, so I would think it is safe to use. I made some tries with two or three different releases, making ppudump parse "its own" ppu files and ppu files from other releases. At different extents, different problems arose. In one bad case, ppudump (version about 2.2.0) crashed while parsing ppu files from its own release, trying to allocate an enourmous block of memory. A self compiled copy, took from 2.4.0 sources, did the same. Many times it complained about "!! has more information" (who cares), "!! refid out of range" or similar (critical). Anyway, using external ppudump, seems to be the safest way to go... the only problem is that ppudump takes binary informations and converts them in a human-readable form, which are later parsed and converted again in binary form... it takes several seconds to parse the big gtk2.ppu file. Regards, Doriano ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23 June 2010 13:58, Michael Schnell wrote: > On 06/23/2010 02:45 PM, Henry Vermaak wrote: >> I think you'll first have to worry porting fpc to those architectures. >> > Right you are :) > > But anyway, if not using libc, you need to do the FUTEX user space part > in Pascal/Assembler > > - With X86 using the appropriate assembler instructions (such as "lock > cmpxchg") > - with ARM11 using "load locked / store conditional" > - with ARM9 using an appropriate call to a fixed Kernel defined address > to have the atomic instruction executed. > > tricky but doable. We've got this already, then: http://www.freepascal.org/docs-html/rtl/system/interlockedcompareexchange.html ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] About ppudump program
Michael Van Canneyt schrieb: At this time I am trying to finish a good auto-completion, so I need to extract symbols from pre-compiled PPUs (especially the runtime standard libraries). Right now I use ppudump(1) to extract symbols, but I could Interesting, is that actually faster/better than parsing the sources? Yes. Additionally, it also works if no sources are available. ...and also deserve no knowledge about the compiler settings at compile time. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Marco van de Voort schrieb: The Kylix-like addition of an C compiler would help to solve such issues, that arise from the use of *any* C code with FPC. Well, since this is about a maintenance problem, it would only solve it if the maintenance of the C compiler would be lower than the maintenance of fixing the headers/RTL.Which I doubt strongly, even if we forget about the gigantic initial costs (making a production C compiler) The C compiler IMO won't cost much, since it can use the OPL code generation. ToPas can produce an FPC compatible AST for that purpose. It also is already configurable, to emulate almost any existing C compiler. I've tested with CBuilder, MSVC and gcc, and their header files, so that such existing header files can be used instead of e.g. the MinGW or Cygwin rewrites. I'm just about to make the FPC compiler usable as a mere syntax parser, with all semantic actions, like code generation, moved into (exchangable) units. The C front-end could create the tree(s) for immediate use by these semantic procedures. Since the separation of the semantic code is already sponsored by Adem, I will do it anyways without burdening the FPC team. The result only depends on the goodwill of the FPC developers, to accept that separation and put it into the FPC trunk, so that the future development can be done without syncing different codebases. I'll start an according thread as soon as I have some more concrete ideas and results. If a real systematic problem here would ever appear, I would go more in the direction of a configure like hack. Characterising the system by compiling small C programs. I don't like that hack, and have ideas also for this problem. IMO a header file with the description of the capabilities of a concrete platform will be sufficient, with low maintenance requirements. Something like the autobloat tools try to achieve, but with a better implementation. Here the outstanding FPC handling of platform specific differences will be very helpful. The major problem of the open source (GNU/Linux...) world is the lack of cooperation between the maintainers of the various platforms - they always ask for perfect implementations which they can use off-the-shelf, but are not willing to spend a few hours even in the specification, adaptation or configuration of a new model. Since we already have a team of platform maintainers in the FPC project, it should be much easier to specify and implement a model for handling the platform differences. In the best case we could end up in an simplified and improved autobloat replacement, which all the C experts could not provide in past decades. Perhaps in a few years nobody will remember gcc and autobloat any more, because then everybody uses our superior fcc ;-) I think I don't entirely understand what problem this paragraph is about. Yes, we have two errnos now, but not doing that means that you essentially have to create a configure like layer. (since this is not even automagical for C programs) My idea is an abstraction at the header-file level, eliminating the need for an adaptation layer in code. It's not a very concrete project right now, since I could not yet identify all the possible problems. In the light of that, I have no problem with the current solution. I have no problem with the existing solution, too, but together we possibly could find easier and more consistent solutions for all these problems. Okay, it's time to wake up and stop dreaming, there remains enough higher priority work for everybody right now... DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Am 23.06.2010 15:34, schrieb Henry Vermaak: On 23 June 2010 14:22, Henry Vermaak wrote: On 23 June 2010 14:13, Daniël Mantione wrote: Op Wed, 23 Jun 2010, schreef Henry Vermaak: A futex syscall doesn't know anything about threads, it's for locking. Perhaps I'm misunderstanding you? I have should have written "futex" instead of "threads". With a Futex, you only call the Futex syscall, if the Futex is locked. You still have to test wether the Futex is locked in userspace, and you that with the interlocked assembler instructions. This assembler implementation must be implemented for each architecture (though abstractions can probably be used, we already have many interlockedx procedures). Ah, I see. I've seen the interlocked procedures in the rtl, we seem to have them for quite a couple arches (at least all the ones that support linux). Ops, except m68k. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel And that one is currently not supported by FPC (at least from 2.x on, because no one updated the cpu units for this architecture). Please correct me if I'm wrong. Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23 June 2010 14:22, Henry Vermaak wrote: > On 23 June 2010 14:13, Daniël Mantione wrote: >> >> >> Op Wed, 23 Jun 2010, schreef Henry Vermaak: >> >>> A futex syscall doesn't know anything about threads, it's for locking. >>> Perhaps I'm misunderstanding you? >> >> I have should have written "futex" instead of "threads". With a Futex, you >> only call the Futex syscall, if the Futex is locked. You still have to test >> wether the Futex is locked in userspace, and you that with the interlocked >> assembler instructions. This assembler implementation must be implemented >> for each architecture (though abstractions can probably be used, we already >> have many interlockedx procedures). > > Ah, I see. I've seen the interlocked procedures in the rtl, we seem > to have them for quite a couple arches (at least all the ones that > support linux). Ops, except m68k. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23 June 2010 14:13, Daniël Mantione wrote: > > > Op Wed, 23 Jun 2010, schreef Henry Vermaak: > >> A futex syscall doesn't know anything about threads, it's for locking. >> Perhaps I'm misunderstanding you? > > I have should have written "futex" instead of "threads". With a Futex, you > only call the Futex syscall, if the Futex is locked. You still have to test > wether the Futex is locked in userspace, and you that with the interlocked > assembler instructions. This assembler implementation must be implemented > for each architecture (though abstractions can probably be used, we already > have many interlockedx procedures). Ah, I see. I've seen the interlocked procedures in the rtl, we seem to have them for quite a couple arches (at least all the ones that support linux). Henry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 03:08 PM, Henry Vermaak wrote: > A futex syscall doesn't know anything about threads, it's for locking. > Perhaps I'm misunderstanding you? > The Futex syscall does not lock a Mutex at all. It's a helper for the user space locking action and performs the putting a thread to sleep (called by userspace, if userspace detects that the Mutex to be locked is already taken) and waking up a thread (called by userspace if userspace detects that a thread is waiting on the Mutex it is about to unlock) and moreover the Futex syscall takes care that the system can do some housekeeping in case a thread dies that still has a lock. The user space stuff is usually done in a library (such as pthread lib in libc. But of course it can be done natively the FPC RTL, as well. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Op Wed, 23 Jun 2010, schreef Henry Vermaak: A futex syscall doesn't know anything about threads, it's for locking. Perhaps I'm misunderstanding you? I have should have written "futex" instead of "threads". With a Futex, you only call the Futex syscall, if the Futex is locked. You still have to test wether the Futex is locked in userspace, and you that with the interlocked assembler instructions. This assembler implementation must be implemented for each architecture (though abstractions can probably be used, we already have many interlockedx procedures). Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 03:00 PM, Henry Vermaak wrote: > Why would you want to do that? Just use the futex syscall. Here you are definitely wrong. Futex is all about _not_ doing a syscall (i.e. change from user space to Kernel space and back) but doing the Mutex operation in user space, if possible (i.e. if not putting a thread to sleep due to Mutex already taken or to wake a thread if the Mutex is freed and at least one thread is waiting. As in most cases a Futex (TCtriticalSection) is only locked for a short time, performance is increased if no syscall needs to be done if no other thread tries to lock it while it's already taken. So userspace needs to do some non-trivial stuff without doing the FUTEX syscall and only do the system call if necessary (see "Futexes are tricky" I mentioned above). This includes doing some critical operations in an atomic way (Thread safe and SMP safe if appropriate). -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23 June 2010 14:03, Daniël Mantione wrote: > > > Op Wed, 23 Jun 2010, schreef Henry Vermaak: > >> On 23 June 2010 13:58, Michael Schnell wrote: >>> >>> On 06/23/2010 02:45 PM, Henry Vermaak wrote: I think you'll first have to worry porting fpc to those architectures. >>> Right you are :) >>> >>> But anyway, if not using libc, you need to do the FUTEX user space part >>> in Pascal/Assembler >> >> Why would you want to do that? Just use the futex syscall. > > Because the Futex syscall won't give you threads, it requires userspace > assistance. A futex syscall doesn't know anything about threads, it's for locking. Perhaps I'm misunderstanding you? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Op Wed, 23 Jun 2010, schreef Henry Vermaak: On 23 June 2010 13:58, Michael Schnell wrote: On 06/23/2010 02:45 PM, Henry Vermaak wrote: I think you'll first have to worry porting fpc to those architectures. Right you are :) But anyway, if not using libc, you need to do the FUTEX user space part in Pascal/Assembler Why would you want to do that? Just use the futex syscall. Because the Futex syscall won't give you threads, it requires userspace assistance. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23 June 2010 13:58, Michael Schnell wrote: > On 06/23/2010 02:45 PM, Henry Vermaak wrote: >> I think you'll first have to worry porting fpc to those architectures. >> > Right you are :) > > But anyway, if not using libc, you need to do the FUTEX user space part > in Pascal/Assembler Why would you want to do that? Just use the futex syscall. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 02:45 PM, Henry Vermaak wrote: > I think you'll first have to worry porting fpc to those architectures. > Right you are :) But anyway, if not using libc, you need to do the FUTEX user space part in Pascal/Assembler - With X86 using the appropriate assembler instructions (such as "lock cmpxchg") - with ARM11 using "load locked / store conditional" - with ARM9 using an appropriate call to a fixed Kernel defined address to have the atomic instruction executed. tricky but doable. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23 June 2010 13:23, Michael Schnell wrote: > > If they show up that does not mean there really is a valid > implementation. I don't know which archs are provided with the "Linus > Tree", but at least with some "embedded" Kernels trees there only is > some dummy code, just providing an error. I think you'll first have to worry porting fpc to those architectures. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 01:37 PM, Henry Vermaak wrote: > > I still can't find which arches miss futex, and I can't find this > information in the manpages, either. Can you point me to where you've > found this information? At least in the Linux Kernel Mailing list there are several discussions on Futex (e.g. non-existing Futex). Astonishingly, even for x86 archs there have been several patches to cure problems. > > I've had a look at the kernel source, go to the arch directory and > execute: > > grep -Rl NR_futex * > > All the architectures show up. If they show up that does not mean there really is a valid implementation. I don't know which archs are provided with the "Linus Tree", but at least with some "embedded" Kernels trees there only is some dummy code, just providing an error. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/23/2010 01:21 PM, Henry Vermaak wrote: > There are no magic NPTL APIs: both NPTL and LinuxThreads use the > "clone" system call. Yep. Additional "clone flags" have been added with newer Kernels (see "man clone"). Moreover for NPTL there are thread IDs additional to process IDs Same can be obtained from the Kernel by some kind of NPTL specific API. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] About ppudump program
On Wed, 23 Jun 2010, Graeme Geldenhuys wrote: Op 2010-06-22 14:40, Doriano Blengino het geskryf: At this time I am trying to finish a good auto-completion, so I need to extract symbols from pre-compiled PPUs (especially the runtime standard libraries). Right now I use ppudump(1) to extract symbols, but I could Interesting, is that actually faster/better than parsing the sources? Yes. Additionally, it also works if no sources are available. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] About ppudump program
Op 2010-06-22 14:40, Doriano Blengino het geskryf: > > At this time I am trying to finish a good auto-completion, so I need to > extract symbols from pre-compiled PPUs (especially the runtime standard > libraries). Right now I use ppudump(1) to extract symbols, but I could Interesting, is that actually faster/better than parsing the sources? Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/22/2010 03:12 PM, Henry Vermaak wrote: > > No, the syscall api is very stable, so minimum maintenance is > required. Which architectures don't provide futex? I'm not aware of > any. > The FUTEX Kernel API only makes sense, if the arch provides the appropriate atomic operations to perform the FUTEX semaphore handling in user space. Some archs (like X86 and the new ARMs' "load locked / store conditional" instructions) provide assembler instructions for this. Some archs (e.g. older ARMs and BlackFin) provide the "atomic area" function calls (procedures located at at fixed locations) for this (only doable with non-SMP implementations). Some archs don't provide atomic instructions at all (e.g. the current NIOS arch, "atomic region" is planned to be implemented".) Here enabling the FUTEX API in the Kernel does not make sense. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
In our previous episode, Hans-Peter Diettrich said: > > start with a translated glibc in a good month), but the continuous > > maintenance (for every distribution separately, since they could use > > different kernels, options etc) , and interoperability with C code would be > > killing. > > The Kylix-like addition of an C compiler would help to solve such > issues, that arise from the use of *any* C code with FPC. Well, since this is about a maintenance problem, it would only solve it if the maintenance of the C compiler would be lower than the maintenance of fixing the headers/RTL.Which I doubt strongly, even if we forget about the gigantic initial costs (making a production C compiler) If a real systematic problem here would ever appear, I would go more in the direction of a configure like hack. Characterising the system by compiling small C programs. > > This is all more trouble than interface glibc, even longterm. FPC already > > supports this (FPC_USE_LIBC) but it is not maintained/used much for > > Linux/FreeBSD. Because Darwin and Solaris use it, it is not totally outdated > > though. > > It may be a bit harder to unify the libc procedures, so that their use > does not duplicate already existing FPC library code, resulting in > confusion like with the errno/cerrno variables. This could be done by a > set of recognized platform-(in)dependent procedures, so that the basic > platform-specific procedures are translated from the (libc...) source > code, while the platform-independent procedures are replaced by the > according FPC implementation. I think I don't entirely understand what problem this paragraph is about. Yes, we have two errnos now, but not doing that means that you essentially have to create a configure like layer. (since this is not even automagical for C programs) In the light of that, I have no problem with the current solution. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23/06/10 12:13, Michael Schnell wrote: On 06/22/2010 02:55 PM, Michael Van Canneyt wrote: For the kernel interface: Not so much. (on some archs, FUTEX is missing). Yep. I still can't find which arches miss futex, and I can't find this information in the manpages, either. Can you point me to where you've found this information? I've had a look at the kernel source, go to the arch directory and execute: grep -Rl NR_futex * All the architectures show up. Henry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 23/06/10 11:44, Michael Schnell wrote: On 06/22/2010 05:18 PM, Florian Klaempfl wrote: At least according to Linus, the kernel api/syscall interface will never break. Regarding Threads, as far as I understand, the thread aware APIs (e.g. NPTL and FUTEX) have been added, while the other APIs were not affected. (No wonder, as I understand that the "Linux Threads" were constructed on top of the non-thread "process" APIs (e.g. fork and Inter-Process Semaphores) by the pthread library. These "APIs" you're talking about, are just system calls. There are no magic NPTL APIs: both NPTL and LinuxThreads use the "clone" system call. In order to implement Posix threads, some things needed to be improved in the kernel, though, and there were also some new system calls (tls for x86/64, exit_group). futex was added before NPTL, so NPTL uses them. Read this article if you'd like to find out how this was implemented: people.redhat.com/drepper/nptl-design.pdf Henry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/22/2010 02:57 PM, Graeme Geldenhuys wrote: > >> Of course you can avoid using the library, do the stuff in Pascal and >> directly access the Kernel API, but this might be dangerous in case the >> API might be modified some time in the future >> > Wouldn't the same apply to the pthreads library? From what I understand the > currently implementation of NTPL is very good, so I doubt it will change > any time soon. > I suppose that the Kernel Thread API (NPTL, FUTEX) in fact is here to stay, but especially _using_ the FUTEX API in a user land application is tricky, so relying on libc (on that behalf) seems like a good idea, as it's "pthread...()" api is easy to use. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/22/2010 02:55 PM, Michael Van Canneyt wrote: > > > For the kernel interface: Not so much. (on some archs, FUTEX is missing). Yep. > > The problem starts if you start linking to C libraries. In that case > you're > pretty much forced to use pthreads; each distribution can make different > choices when compiling libc. If not using libc, _we_ must do the decision (e.g. whether to use FUTEX or not. i.e..: Archs that don't provide FUTEX, do not have atomic user space instructions, which are necessary to make use of the FUTEX Kernel API. But some archs (some ARM types) do provide a "atomic region" interface to simulate atomic user space instruction,. so Futex is possible without hardware support of "atomic". > > It would be nice to have a kernel interface, though. That way you > could have > threading support without pulling in libc. Right but not easy to do and creates very arch- and sub-arch depending code. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] About ppudump program
First of all: please mention in the future that you are not subscribed (also in replies, because people may forget), so that people who reply know that they should CC you. On 22 Jun 2010, at 14:40, Doriano Blengino wrote: At this time I am trying to finish a good auto-completion, so I need to extract symbols from pre-compiled PPUs (especially the runtime standard libraries). Right now I use ppudump(1) to extract symbols, but I could switch in future to the use of ppu.pas. I noticed that different freepascal releases do not share a common PPU file format, so using ppu.pas inside the IDE would force the user to use the same compiler version as the one used to compile the IDE. On the other hand, I noticed that the release 2.4.x is the first one which reports "v100" for a PPU file, which hopefully indicates that future releases could be PPU-compatible... The "version number" in PPU files is just an increasing byte. 100 is not "1.00", it's "one hundred". It has no special meaning. Now, I would ask an advice: is it better to stick to ppudump, at a price of some reduction in speed, and the risk that its output could change in future, invalidating the IDE parsing routines? You should use ppudump. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/22/2010 02:49 PM, Graeme Geldenhuys wrote: > http://linas.org/linux/threads-faq.html This is _very_ old (1996). they say: "Kernel-Threads Kernel-level threads often are implemented in the kernel using several tables" This might be true, but we now have a ubiquitous and stable NPTL implementation in the Kernel. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] About ppudump program
On Tue, 22 Jun 2010, Doriano Blengino wrote: Hi, I am writing a Freepascal IDE in Linux+GTK2 (using freepascal, of course!). I attach a screenshot for your curiosity, just in case. At this time I am trying to finish a good auto-completion, so I need to extract symbols from pre-compiled PPUs (especially the runtime standard libraries). Right now I use ppudump(1) to extract symbols, but I could switch in future to the use of ppu.pas. I noticed that different freepascal releases do not share a common PPU file format, so using ppu.pas inside the IDE would force the user to use the same compiler version as the one used to compile the IDE. On the other hand, I noticed that the release 2.4.x is the first one which reports "v100" for a PPU file, which hopefully indicates that future releases could be PPU-compatible... Now, I would ask an advice: is it better to stick to ppudump, at a price of some reduction in speed, and the risk that its output could change in future, invalidating the IDE parsing routines? Or is it better to use ppu.pas in the IDE, having confidence that ppu.pas will parse PPU file generated by different versions/releases of freepascal? Well, as far as I know, ppu simply skips blocks that it does not know, so I would think it is safe to use. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/22/2010 03:35 PM, Henry Vermaak wrote: > > So maybe the fpc syscall interface needs to be updated. Can you give > any examples of what got reworked? I don't suppose we already have NPTL and FUTEX, as same are used by libc, anyway. If we want perfect performance and good (Posix-like) behavior of the threaded applications, but no libc binding, NPTL and FUTEX API calls need to be done instead of old-style clone() and interprocess semaphores. Especially FUTEX is tricky. (read "Futexes are tricky" by U. Drepper: http://people.redhat.com/drepper/futex.pdf) -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
On 06/22/2010 05:18 PM, Florian Klaempfl wrote: > At least according to Linus, the kernel api/syscall interface will never > break. > Regarding Threads, as far as I understand, the thread aware APIs (e.g. NPTL and FUTEX) have been added, while the other APIs were not affected. (No wonder, as I understand that the "Linux Threads" were constructed on top of the non-thread "process" APIs (e.g. fork and Inter-Process Semaphores) by the pthread library. When NPTL, FUTEX, etc appears in one arch, the pthread library is modified to use it, without changing the the user "pthread." interface of libc. Very nice for applications that use libc dynamically. So Kylix applications automatically take advantage of NPTL and FUTEX, behaving better and running faster. But of course using libc does provide problems in certain issues -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Marco van de Voort schrieb: The problem is not the programming (since a dedicated person could probably start with a translated glibc in a good month), but the continuous maintenance (for every distribution separately, since they could use different kernels, options etc) , and interoperability with C code would be killing. The Kylix-like addition of an C compiler would help to solve such issues, that arise from the use of *any* C code with FPC. An integration of such an compiler (or front-end) into FPC could be based on my ToPas project, that already parses all C syntax and allows to translate it into OPL, as far as that language allows for. It should not be so hard to add the few missing syntax constructs (anonymous unions, bitfields...) to the FPC compiler. This is all more trouble than interface glibc, even longterm. FPC already supports this (FPC_USE_LIBC) but it is not maintained/used much for Linux/FreeBSD. Because Darwin and Solaris use it, it is not totally outdated though. It may be a bit harder to unify the libc procedures, so that their use does not duplicate already existing FPC library code, resulting in confusion like with the errno/cerrno variables. This could be done by a set of recognized platform-(in)dependent procedures, so that the basic platform-specific procedures are translated from the (libc...) source code, while the platform-independent procedures are replaced by the according FPC implementation. A solution may be platform-specific header files, that #define macros for the compatible procedures, redirecting all calls to the existing FPC library procedures. The then obsolete implementations in the C sources can be ignored by the compiler, or can be removed by the linker. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Op Wed, 23 Jun 2010, schreef Graeme Geldenhuys: Op 2010-06-22 15:31, Marco van de Voort het geskryf: Same problem. You still have to interface with the kernel, and it would be incompatible with C libraries that use threads. Well, if I don't use C libraries (only Object Pascal code), then it should still be beneficial. I can then write pure Object Pascal code/applications and still have multi-threading support. It's not just about eliminating libc dependencies. I believe it would be beneficial in most cases, even when using C libraries, since the Linux kernels does not expose a Posix threads like API, Posix threads are emulated. The FPC model is again unlike the Posix model, so you have again glue code. Directly calling the kernel would remove this overhead and allow for more efficient threading. The fact that the C libraries you call are written for libpthread does not need to be a problem: - You would not be able to share a variable between Pascal and C and have threads in both worlds wait on each other and two threads may not be able to call C code at the same time, you would need to protect that. - However, C libraries could safely launch pthread based threads without getting in the wheels of Pascal based threads. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Threading support and C library under Linux/Unix
Op 2010-06-22 15:31, Marco van de Voort het geskryf: > > Same problem. You still have to interface with the kernel, and it would be > incompatible with C libraries that use threads. Well, if I don't use C libraries (only Object Pascal code), then it should still be beneficial. I can then write pure Object Pascal code/applications and still have multi-threading support. Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
