Re: [Xenomai-core] [pull request] Signals support.
Gilles Chanteperdrix wrote: Hi, here come the pull request for user-space signals support. The simple solution; handling signals upon system call return, has been implemented since the other solution (handling signals upon any return to user-space) required to change the I-pipe patch, and so made the upcoming 2.5 only compatible with newer patches. We pass to kernel-space a sixth argument which is a pointer where information about received signals is stored by kernel. The only architecture for which the implementation is peculiar is x86_32, because the register used as sixth argument is ebp, also used for the libc backtrace function implementation, so I tried to find a solution which makes backtracing still possible (otherwise we would have said bye-bye to involuntary mode changes chasing with SIGXCPU) without breaking too many things. I'm still digging through the code. A few minor remarks regarding the user space side so far: XENOMAI_DO_SYSCALL becomes quite bloated now, and it's inlined. Did you check that the fast path (ie. no signal) only contains one conditional branch when the compiler is done with optimizing? If not (I suspect so), the code should be refactored to look more like restart: do_syscall if unlikely(sigs.nsigs) res = handle_signals if res == -ERESTART goto restart Moreover, the inner while loop over sigs.remaining should be moved into a shared function as well. I don't see why it should be instantiated at each and every syscall invocation site. Am I right, there is no skin (except the test skin) using this feature so far? Jan signature.asc Description: OpenPGP digital signature ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] [pull request] Signals support.
Jan Kiszka wrote: Gilles Chanteperdrix wrote: Hi, here come the pull request for user-space signals support. The simple solution; handling signals upon system call return, has been implemented since the other solution (handling signals upon any return to user-space) required to change the I-pipe patch, and so made the upcoming 2.5 only compatible with newer patches. We pass to kernel-space a sixth argument which is a pointer where information about received signals is stored by kernel. The only architecture for which the implementation is peculiar is x86_32, because the register used as sixth argument is ebp, also used for the libc backtrace function implementation, so I tried to find a solution which makes backtracing still possible (otherwise we would have said bye-bye to involuntary mode changes chasing with SIGXCPU) without breaking too many things. I'm still digging through the code. A few minor remarks regarding the user space side so far: XENOMAI_DO_SYSCALL becomes quite bloated now, and it's inlined. Did you check that the fast path (ie. no signal) only contains one conditional branch when the compiler is done with optimizing? If not (I suspect so), the code should be refactored to look more like restart: do_syscall if unlikely(sigs.nsigs) res = handle_signals if res == -ERESTART goto restart The test for sigs.nsigs is done in the inline xnsig_dispatch function, which jumps to the out-of-line dispatch function only if there are really some signals to handle. Then comes the test while (sigs.nsigs sigs.remaining), so my hope was that the compiler would optimize the jump-after-conditional-to-a-conditional-with-same-condition, this is a pretty standard optimization, but have not checked that it did so. As for the goto, I have nothing against gotos when they are needed, but in that case we can easily avoid it with a do while loop. Moreover, the inner while loop over sigs.remaining should be moved into a shared function as well. I don't see why it should be instantiated at each and every syscall invocation site. The point is that when there are more than a fixed amount of signals queued (16, arbitrarily fixed to allow for some queueing, but avoid taking too much room on stack), we emit a second inner syscall to get them. And the way to emit this second syscall is platform-dependent. The code at this point is pretty much the same on all platforms... except on x86_32, though it is probably possible to factor this out with clever macros. Am I right, there is no skin (except the test skin) using this feature so far? Yes, we break the ABI first, and we will implement the posix skin signals after the initial 2.5 release. Because the signals data union is part of the ABI, it has already been given the posix siginfo structure size. Since this structure is pretty large, I expected no skin to need more room. -- Gilles. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] [pull request] Signals support.
Jan Kiszka wrote: Gilles Chanteperdrix wrote: Hi, here come the pull request for user-space signals support. The simple solution; handling signals upon system call return, has been implemented since the other solution (handling signals upon any return to user-space) required to change the I-pipe patch, and so made the upcoming 2.5 only compatible with newer patches. We pass to kernel-space a sixth argument which is a pointer where information about received signals is stored by kernel. The only architecture for which the implementation is peculiar is x86_32, because the register used as sixth argument is ebp, also used for the libc backtrace function implementation, so I tried to find a solution which makes backtracing still possible (otherwise we would have said bye-bye to involuntary mode changes chasing with SIGXCPU) without breaking too many things. I'm still digging through the code. A few minor remarks regarding the user space side so far: XENOMAI_DO_SYSCALL becomes quite bloated now, and it's inlined. Did you check that the fast path (ie. no signal) only contains one conditional branch when the compiler is done with optimizing? If not (I suspect so), the code should be refactored to look more like restart: do_syscall if unlikely(sigs.nsigs) res = handle_signals if res == -ERESTART goto restart Moreover, the inner while loop over sigs.remaining should be moved into a shared function as well. I don't see why it should be instantiated at each and every syscall invocation site. Am I right, there is no skin (except the test skin) using this feature so far? Next question: The signal delivery latency is naturally affected by the syscall invocation frequency of the target thread, right? Ie. no syscall, no signal. Once we offer RT signals via the skin, this limitation should be prominently documented. Jan signature.asc Description: OpenPGP digital signature ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
[Xenomai-core] Realtime gettimeofday()
Hi, we'd like to implement a gettimeofday() mechanism that works in realtime context, both from user- and kernelland. Most importantly, the correctins made to the wall time by the NTP protcol on Linux must be transferred into the Xenomai domain. We have a preliminary implementation (with several weaknesses) ready, but before we go into the details of writing a complete solution, it would be great to have the community's opinion on how to proceed best -- there are a number of alternatives, each with specific strenghts and weaknesses. Any comments on the following proposals are very welcome! Essentially, getting the NTP-corrected time requires three ingredients: - A clock source. - A wall time t_{0} as basis, together with the value of the clocksource at that moment. - An NTP-corrected conversion factor between clock ticks and walltime. Since the NTP correction information holds wrt. a specific clock source, we have to use the clock source used by Linux. The Linux kernel provides a vsyscall based solution to the problem: The base data are periodically copied to a userland-accessible page, and a vsyscall reads the information using seqlock protection (i.e., rereading the information when it was changed during the read operation). This way, the data can be locklessly relayed into userland, and perturbation for the kernel is minimal as no lock to write the data is required. However, things get a little more involved when realtime comes into play: Using a simple seqlock that might require rereading the data an arbitrary number of times contrary to the determinism requirements of realtime. So far, we can think of three alternatives: 1.) Re-use the vsyscall data from the kernel with a buffered seqlock replacement. We cannot directly use the data provided by the Linux kernel because Xenomai might have interrupted the update process, which leads to an inconsistent state. Instead, we would use a buffered seqlock that uses two more copies of the data that are accompanied by a flag which indicates if some other Xenomai thread is currently reading or writing to the data: They are both filled with an initial valid copy of the data during Xenomai initialisation. When the data are requested in primary mode, Xenomai first checks if the Linux seqlock is taken: - If the lock is taken, the data provided by Linux may be in an inconsistent state, and the backup copies are used. - If the lock is not taken, check with the aforementioned flag if another Xenomai task is currently updating one of the copies. - (A) If yes, then read the data from the other copy. - (B) If no, perform at most one read try of the Linux data -- another CPU running the Linux kernel might update them in the meantime. If they have been read without perturbation, mark one copy as being currently updated (naturally, we need to flip the flag atomically), update the buffer, and use the data. If the data were modified during the read phase, discard the result and continue as in (A) Note that the flags may also be extended with version information to check which buffer has been updated more recently. While Xenomai might block the update process for a while, we always have at least a consistent copy of the data, and there is a fixed upper bound on the time required to compute the current NTP-corrected wall time. 2.) Use a transactional mechanism as outlined above to relay the data into userland, make sure that the triple (sec,nsec,ticks) is copied atomically to avoid an invalid state, and live with the fact that if the correction factor is not uptodate, we are only in danger of using outdated information, but won't get a completely bogous result (a userland application might start to read a buffer that is concurrently being updated by another CPU) 3.) Use explicit locking between the Linux kernel and Xenomai to update respectively access the data, and partially eliminate the performance advantages of the vsyscall mechanism. This would have the usual drawbacks associated with locks in primary mode. Besides, since a kernel lock is involved, it would not allow for a pure userland solution without kernel intervention. Are there preferences and/or arguments for/against any of these solutions? Or any better ideas? Thanks in advance for your comments! Best, Wolfgang ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Wolfgang Mauerer wrote: Hi, we'd like to implement a gettimeofday() mechanism that works in realtime context, both from user- and kernelland. Most importantly, the correctins made to the wall time by the NTP protcol on Linux must be transferred into the Xenomai domain. Yes, the real issue is NTP, because other than that, you can simply implement gettimeofday in terms of clock_gettime(CLOCK_REALTIME). This issue has been discussed several times, but never a lot. We have a simple solution: starting with 2.4 (if I remember correctly) xenomai provides clock_settime, so you can simply rely on clock_gettime, and call clock_settime from time to time to resync the Xenomai clock with Linux NTP-enhanced clock. This is not pretty, you get a drift increasing over time, then suddenly been reset. But I guess this has been enough for current users, until now. You control the maximum drift by how often you call clock_settime. Would not it be enough for your use of xenomai? -- Gilles ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Gilles Chanteperdrix wrote: Wolfgang Mauerer wrote: Hi, we'd like to implement a gettimeofday() mechanism that works in realtime context, both from user- and kernelland. Most importantly, the correctins made to the wall time by the NTP protcol on Linux must be transferred into the Xenomai domain. Yes, the real issue is NTP, because other than that, you can simply implement gettimeofday in terms of clock_gettime(CLOCK_REALTIME). Not truly as Linux processes that tune the host clock do not affect the Xenomai world. This issue has been discussed several times, but never a lot. We have a simple solution: starting with 2.4 (if I remember correctly) xenomai provides clock_settime, so you can simply rely on clock_gettime, and call clock_settime from time to time to resync the Xenomai clock with Linux NTP-enhanced clock. This is not pretty, you get a drift increasing over time, then suddenly been reset. But I guess this has been enough for current users, until now. You control the maximum drift by how often you call clock_settime. Would not it be enough for your use of xenomai? Nope for the reasons you mentioned. Moreover, reading the Xenomai real-time clock requires a syscall, gettimeofday can work without that penalty (given non-broken TSC). Next issue is that the different clock bases of clock_gettime(CLOCK_REALTIME) and (real) gettimeofday + the switch-back or even lock-up side-effects of the latter have been quite a PITA in our project. And finally, the vision is to have NTP-accurate (or whatever the sync source is, eg. IEEE1588) absolute timing also for the nucleus one day. Having an RT-safe way to obtain the NTP parameters from any context is the first step. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Jan Kiszka wrote: Gilles Chanteperdrix wrote: Wolfgang Mauerer wrote: Hi, we'd like to implement a gettimeofday() mechanism that works in realtime context, both from user- and kernelland. Most importantly, the correctins made to the wall time by the NTP protcol on Linux must be transferred into the Xenomai domain. Yes, the real issue is NTP, because other than that, you can simply implement gettimeofday in terms of clock_gettime(CLOCK_REALTIME). Not truly as Linux processes that tune the host clock do not affect the Xenomai world. This issue has been discussed several times, but never a lot. We have a simple solution: starting with 2.4 (if I remember correctly) xenomai provides clock_settime, so you can simply rely on clock_gettime, and call clock_settime from time to time to resync the Xenomai clock with Linux NTP-enhanced clock. This is not pretty, you get a drift increasing over time, then suddenly been reset. But I guess this has been enough for current users, until now. You control the maximum drift by how often you call clock_settime. Would not it be enough for your use of xenomai? Nope for the reasons you mentioned. Moreover, reading the Xenomai real-time clock requires a syscall, gettimeofday can work without that penalty (given non-broken TSC). Next issue is that the different clock bases of clock_gettime(CLOCK_REALTIME) and (real) gettimeofday + the switch-back or even lock-up side-effects of the latter have been quite a PITA in our project. And finally, the vision is to have NTP-accurate (or whatever the sync source is, eg. IEEE1588) absolute timing also for the nucleus one day. Having an RT-safe way to obtain the NTP parameters from any context is the first step. clock_gettime(CLOCK_MONOTONIC) works without syscall too. And note that the real-time clock could work without a syscall for the current scheme, but you were the one to explicitly refuse this. My opinion with regard to these issues is that the I-pipe patch should send us an event when the parameters are adjusted (the wall clock time changed by Linux, or the slow down/acceleration caused by the NTP daemon). And that we should duplicate the processing done by the Linux kernel for the Xenomai clock. Doing this in user-space without a syscall is another issue that will be possible to handle when we have solved the kernel-space clock issue. Of course, it supposes that Xenomai should use the same clocksource as Linux, so, that we should implement the HPET clock source for x86 (on other architectures, we already use the same clock source). Another possibility is to implement syscalls to allow adjusting the nucleus real-time clock parameters the same way ntp does, and to require a special ntp daemon that would use these syscalls instead of the linux kernel syscalls. After all, if we adjust Xenomai clock, Linux clock will follow without any adjustment. On an embedded system, requiring a recompiled NTP is not much of an issue. So, to summarize, here is what I think we should do: - implement the HPET clock source - implement the I-pipe events signalling wall clock adjustment and NTP related adjustments - or implement syscalls to allow NTP related adjustments (we already have the syscall allowing to change wall clock) - rework the nucleus real-time clock to use these adjustements - implement the user-space syscall-less version of the real-time clock read. Trying to go directly to the last point looks premature to me. -- Gilles ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Gilles Chanteperdrix wrote: Jan Kiszka wrote: Gilles Chanteperdrix wrote: Wolfgang Mauerer wrote: Hi, we'd like to implement a gettimeofday() mechanism that works in realtime context, both from user- and kernelland. Most importantly, the correctins made to the wall time by the NTP protcol on Linux must be transferred into the Xenomai domain. Yes, the real issue is NTP, because other than that, you can simply implement gettimeofday in terms of clock_gettime(CLOCK_REALTIME). Not truly as Linux processes that tune the host clock do not affect the Xenomai world. This issue has been discussed several times, but never a lot. We have a simple solution: starting with 2.4 (if I remember correctly) xenomai provides clock_settime, so you can simply rely on clock_gettime, and call clock_settime from time to time to resync the Xenomai clock with Linux NTP-enhanced clock. This is not pretty, you get a drift increasing over time, then suddenly been reset. But I guess this has been enough for current users, until now. You control the maximum drift by how often you call clock_settime. Would not it be enough for your use of xenomai? Nope for the reasons you mentioned. Moreover, reading the Xenomai real-time clock requires a syscall, gettimeofday can work without that penalty (given non-broken TSC). Next issue is that the different clock bases of clock_gettime(CLOCK_REALTIME) and (real) gettimeofday + the switch-back or even lock-up side-effects of the latter have been quite a PITA in our project. And finally, the vision is to have NTP-accurate (or whatever the sync source is, eg. IEEE1588) absolute timing also for the nucleus one day. Having an RT-safe way to obtain the NTP parameters from any context is the first step. clock_gettime(CLOCK_MONOTONIC) works without syscall too. And note that the real-time clock could work without a syscall for the current scheme, but you were the one to explicitly refuse this. Yes, because we need a scheme like Wolfgang suggests to ensure consistent states during updates. And we didn't had such thing back then. My opinion with regard to these issues is that the I-pipe patch should send us an event when the parameters are adjusted (the wall clock time changed by Linux, or the slow down/acceleration caused by the NTP daemon). And that we should duplicate the processing done by the Linux kernel for the Xenomai clock. That may be a solution for only half of the problem. Doing this in user-space without a syscall is another issue that will be possible to handle when we have solved the kernel-space clock issue. Of course, it supposes that Xenomai should use the same clocksource as Linux, so, that we should implement the HPET clock source for x86 (on other architectures, we already use the same clock source). Solving the user space transfer automatically includes a solution for the Linux-to-Xenomai transfer. I really suggest that we discuss a complete solution to avoid leaving design issues behind that require another ABI change later on. Another possibility is to implement syscalls to allow adjusting the nucleus real-time clock parameters the same way ntp does, and to require a special ntp daemon that would use these syscalls instead of the linux kernel syscalls. After all, if we adjust Xenomai clock, Linux clock will follow without any adjustment. On an embedded system, requiring a recompiled NTP is not much of an issue. So, to summarize, here is what I think we should do: - implement the HPET clock source What for? The days of HPET are counted (both as clock and event source), modern systems provide proper TSCs. - implement the I-pipe events signalling wall clock adjustment and NTP related adjustments - or implement syscalls to allow NTP related adjustments (we already have the syscall allowing to change wall clock) - rework the nucleus real-time clock to use these adjustements - implement the user-space syscall-less version of the real-time clock read. Trying to go directly to the last point looks premature to me. Why? It delivers us the core mechanism we need for the rest as well - and it does not require fancy I-pipe hooks. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Jan Kiszka wrote: Why? It delivers us the core mechanism we need for the rest as well - and it does not require fancy I-pipe hooks. Because relying on the vdso/vsyscall only works on x86. Whereas implementing clock slew down/acceleration at nucleus level and simply sharing data between kernel and user through the the global sem heap, works for all architectures. -- Gilles ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Gilles Chanteperdrix wrote: Jan Kiszka wrote: Why? It delivers us the core mechanism we need for the rest as well - and it does not require fancy I-pipe hooks. Because relying on the vdso/vsyscall only works on x86. Whereas implementing clock slew down/acceleration at nucleus level and simply sharing data between kernel and user through the the global sem heap, works for all architectures. There are three kind of archs: - those that already support vgettimeofday friends (x86, powerpc, maybe more) - those that do not yet though they could (I strongly suspect arm falls into this category as well) - those that never will (due to lacking user-readable time sources) We need temporary/permanent i-pipe workarounds for the last two, but I see no point in complicating the first category. This design aims at a longer term. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Jan Kiszka wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: Why? It delivers us the core mechanism we need for the rest as well - and it does not require fancy I-pipe hooks. Because relying on the vdso/vsyscall only works on x86. Whereas implementing clock slew down/acceleration at nucleus level and simply sharing data between kernel and user through the the global sem heap, works for all architectures. There are three kind of archs: - those that already support vgettimeofday friends (x86, powerpc, maybe more) - those that do not yet though they could (I strongly suspect arm falls into this category as well) - those that never will (due to lacking user-readable time sources) We need temporary/permanent i-pipe workarounds for the last two, but I see no point in complicating the first category. This design aims at a longer term. Well, I may be wrong, but I prefer generic code to arch-specific code. Nucleus code to handle clock slow down/acceleration would be generic; I-pipe code to signal NTP syscalls would be generic (and yes, even if I-pipe patches are generated for all architectures, whether the code is generic or specific makes a big difference); User-space code to implement clock_gettime(CLOCK_REALTIME) using data shared through the global sem heap would be generic. So, I think this design is future proof and easy to maintain. And I do not see how it complicates x86 situation, since it is only made of generic code. Looks like we found a troll to occupy this afternoon :-). -- Gilles ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Jan Kiszka wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: Why? It delivers us the core mechanism we need for the rest as well - and it does not require fancy I-pipe hooks. Because relying on the vdso/vsyscall only works on x86. Whereas implementing clock slew down/acceleration at nucleus level and simply sharing data between kernel and user through the the global sem heap, works for all architectures. There are three kind of archs: - those that already support vgettimeofday friends (x86, powerpc, maybe more) - those that do not yet though they could (I strongly suspect arm falls into this category as well) - those that never will (due to lacking user-readable time sources) We need temporary/permanent i-pipe workarounds for the last two, but I see no point in complicating the first category. This design aims at a longer term. Well, I may be wrong, but I prefer generic code to arch-specific code. Nucleus code to handle clock slow down/acceleration would be generic; I-pipe code to signal NTP syscalls would be generic (and yes, even if I-pipe patches are generated for all architectures, whether the code is generic or specific makes a big difference); User-space code to implement clock_gettime(CLOCK_REALTIME) using data shared through the global sem heap would be generic. So, I think this design is future proof and easy to maintain. And I do not see how it complicates x86 situation, since it is only made of generic code. Well, OK, then place a small optional I-pipe hook into that part that normally writes the update into the vdso page (I think that is arch-specific anyway), replicating it into a specified page the nucleus may set up on a globally shared heap. That hook also has to maintain a seqlock like Linux does, ie. generating the same layout and semantics. It's just the transport mechanism, we can easily select it based on the arch's level of support. But I'm against any needless redirection through the nucleus (including potential nklocks etc.). I do not really understand why you want to use the vdso page, since we have the global sem heap anyway. clock_gettime already has a mean to read a clock source and the frequency of this clock source, this is guaranteed on all platforms, so I think the correction code can be made generic. However, duplicating the ntp related kernel code may be the real issue. I have to look at that code to see how complex it is. For the locking, well, if we have variables on a shared area to update every time ntp corrects the clock, we will have problems doing it under nklock anyway (the irq locking prevents from preemption on the local cpu, but does not prevent the remote cpus running user-space programs from accessing the shared area). So, we will have to devise some locking mechanisms. But I do not see the reason for making this linux compatible. If we keep the nucleus business separated, the nucleus shared area will never have to be accessed by plain Linux, which will access its area in the vdso, the Linux kernel doing its house keeping. For the HPET clocksource, well, I do not share your enthusiasm, Linux keeps complaining about the stability of the tsc of my laptop, though it is a fairly recent core2 with the constant_tsc flag. -- Gilles ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] [pull request] Signals support.
Jan Kiszka wrote: Jan Kiszka wrote: Gilles Chanteperdrix wrote: Hi, here come the pull request for user-space signals support. The simple solution; handling signals upon system call return, has been implemented since the other solution (handling signals upon any return to user-space) required to change the I-pipe patch, and so made the upcoming 2.5 only compatible with newer patches. We pass to kernel-space a sixth argument which is a pointer where information about received signals is stored by kernel. The only architecture for which the implementation is peculiar is x86_32, because the register used as sixth argument is ebp, also used for the libc backtrace function implementation, so I tried to find a solution which makes backtracing still possible (otherwise we would have said bye-bye to involuntary mode changes chasing with SIGXCPU) without breaking too many things. I'm still digging through the code. A few minor remarks regarding the user space side so far: XENOMAI_DO_SYSCALL becomes quite bloated now, and it's inlined. Did you check that the fast path (ie. no signal) only contains one conditional branch when the compiler is done with optimizing? If not (I suspect so), the code should be refactored to look more like restart: do_syscall if unlikely(sigs.nsigs) res = handle_signals if res == -ERESTART goto restart Moreover, the inner while loop over sigs.remaining should be moved into a shared function as well. I don't see why it should be instantiated at each and every syscall invocation site. Ok. The second syscall is now done in the out-of-line signal handling function. Do you prefer this? It is only implemented for x86_32 since it is the problematic architecture, but if you are ok with it, I will change the other architectures. Am I right, there is no skin (except the test skin) using this feature so far? Next question: The signal delivery latency is naturally affected by the syscall invocation frequency of the target thread, right? Ie. no syscall, no signal. Once we offer RT signals via the skin, this limitation should be prominently documented. Well, if you make no syscall, your box is basically a brick. We need suspensions from time to time to get Linux running anyway. -- Gilles ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Gilles Chanteperdrix wrote: Jan Kiszka wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: Why? It delivers us the core mechanism we need for the rest as well - and it does not require fancy I-pipe hooks. Because relying on the vdso/vsyscall only works on x86. Whereas implementing clock slew down/acceleration at nucleus level and simply sharing data between kernel and user through the the global sem heap, works for all architectures. There are three kind of archs: - those that already support vgettimeofday friends (x86, powerpc, maybe more) - those that do not yet though they could (I strongly suspect arm falls into this category as well) - those that never will (due to lacking user-readable time sources) We need temporary/permanent i-pipe workarounds for the last two, but I see no point in complicating the first category. This design aims at a longer term. Well, I may be wrong, but I prefer generic code to arch-specific code. Nucleus code to handle clock slow down/acceleration would be generic; I-pipe code to signal NTP syscalls would be generic (and yes, even if I-pipe patches are generated for all architectures, whether the code is generic or specific makes a big difference); User-space code to implement clock_gettime(CLOCK_REALTIME) using data shared through the global sem heap would be generic. So, I think this design is future proof and easy to maintain. And I do not see how it complicates x86 situation, since it is only made of generic code. Well, OK, then place a small optional I-pipe hook into that part that normally writes the update into the vdso page (I think that is arch-specific anyway), replicating it into a specified page the nucleus may set up on a globally shared heap. That hook also has to maintain a seqlock like Linux does, ie. generating the same layout and semantics. It's just the transport mechanism, we can easily select it based on the arch's level of support. But I'm against any needless redirection through the nucleus (including potential nklocks etc.). I do not really understand why you want to use the vdso page, since we have the global sem heap anyway. clock_gettime already has a mean to read a clock source and the frequency of this clock source, this is guaranteed on all platforms, so I think the correction code can be made generic. Again, the page is not the point, what it contains is important. Our currently published data is not sufficient to support dynamic updates, but the vdso _contains_ all the data we could reuse with an enhanced algorithm to provide a dynamically adjusted time base. However, duplicating the ntp related kernel code may be the real issue. I have to look at that code to see how complex it is. I surely don't want to duplicate ntp code, just the results it spits out (e.g. into the vdso page). For the locking, well, if we have variables on a shared area to update every time ntp corrects the clock, we will have problems doing it under nklock anyway (the irq locking prevents from preemption on the local cpu, but does not prevent the remote cpus running user-space programs from accessing the shared area). So, we will have to devise some locking mechanisms. But I do not see the reason for making this linux compatible. If we keep the nucleus business separated, the nucleus shared area will never have to be accessed by plain Linux, which will access its area in the vdso, the Linux kernel doing its house keeping. The kernel need to synchronize with user land, so the problem is (almost) the same as with obtaining the original data from Linux directly: user land can only use a retry or a smart fall-back mechanism to obtain consistent data. For the HPET clocksource, well, I do not share your enthusiasm, Linux keeps complaining about the stability of the tsc of my laptop, though it is a fairly recent core2 with the constant_tsc flag. My point is that we offer Xenomai without a real alternative to tsc for many moons, and no one really stood up so far and complained about missing hpet support. Just like you cannot use every SMI-infected box for RT, you can't do so when the tsc is unfixably broken. I'm not sure how hard hpet addition and maintenance would actually be, if it's trivial, I'm fine. But I think we have more important issues to solve. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] [pull request] Signals support.
Gilles Chanteperdrix wrote: Jan Kiszka wrote: Jan Kiszka wrote: Gilles Chanteperdrix wrote: Hi, here come the pull request for user-space signals support. The simple solution; handling signals upon system call return, has been implemented since the other solution (handling signals upon any return to user-space) required to change the I-pipe patch, and so made the upcoming 2.5 only compatible with newer patches. We pass to kernel-space a sixth argument which is a pointer where information about received signals is stored by kernel. The only architecture for which the implementation is peculiar is x86_32, because the register used as sixth argument is ebp, also used for the libc backtrace function implementation, so I tried to find a solution which makes backtracing still possible (otherwise we would have said bye-bye to involuntary mode changes chasing with SIGXCPU) without breaking too many things. I'm still digging through the code. A few minor remarks regarding the user space side so far: XENOMAI_DO_SYSCALL becomes quite bloated now, and it's inlined. Did you check that the fast path (ie. no signal) only contains one conditional branch when the compiler is done with optimizing? If not (I suspect so), the code should be refactored to look more like restart: do_syscall if unlikely(sigs.nsigs) res = handle_signals if res == -ERESTART goto restart Moreover, the inner while loop over sigs.remaining should be moved into a shared function as well. I don't see why it should be instantiated at each and every syscall invocation site. Ok. The second syscall is now done in the out-of-line signal handling function. Do you prefer this? The smaller the inlined code gets, the better. It is only implemented for x86_32 since it is the problematic architecture, but if you are ok with it, I will change the other architectures. Will check, thanks. Am I right, there is no skin (except the test skin) using this feature so far? Next question: The signal delivery latency is naturally affected by the syscall invocation frequency of the target thread, right? Ie. no syscall, no signal. Once we offer RT signals via the skin, this limitation should be prominently documented. Well, if you make no syscall, your box is basically a brick. We need suspensions from time to time to get Linux running anyway. For sure. My point is that people may get the idea to build time-critical event delivery based on signals. In such cases it would make a big difference how often the destination thread issues a syscall. Also forced preemptions can extend the delivery latency, only the user space workload counts. I would not recommend such application designs, but people may get that idea when they once read RT-safe signals. :) Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] [pull request] Signals support.
Jan Kiszka wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: Jan Kiszka wrote: Gilles Chanteperdrix wrote: Hi, here come the pull request for user-space signals support. The simple solution; handling signals upon system call return, has been implemented since the other solution (handling signals upon any return to user-space) required to change the I-pipe patch, and so made the upcoming 2.5 only compatible with newer patches. We pass to kernel-space a sixth argument which is a pointer where information about received signals is stored by kernel. The only architecture for which the implementation is peculiar is x86_32, because the register used as sixth argument is ebp, also used for the libc backtrace function implementation, so I tried to find a solution which makes backtracing still possible (otherwise we would have said bye-bye to involuntary mode changes chasing with SIGXCPU) without breaking too many things. I'm still digging through the code. A few minor remarks regarding the user space side so far: XENOMAI_DO_SYSCALL becomes quite bloated now, and it's inlined. Did you check that the fast path (ie. no signal) only contains one conditional branch when the compiler is done with optimizing? If not (I suspect so), the code should be refactored to look more like restart: do_syscall if unlikely(sigs.nsigs) res = handle_signals if res == -ERESTART goto restart Moreover, the inner while loop over sigs.remaining should be moved into a shared function as well. I don't see why it should be instantiated at each and every syscall invocation site. Ok. The second syscall is now done in the out-of-line signal handling function. Do you prefer this? The smaller the inlined code gets, the better. It is only implemented for x86_32 since it is the problematic architecture, but if you are ok with it, I will change the other architectures. Will check, thanks. Am I right, there is no skin (except the test skin) using this feature so far? Next question: The signal delivery latency is naturally affected by the syscall invocation frequency of the target thread, right? Ie. no syscall, no signal. Once we offer RT signals via the skin, this limitation should be prominently documented. Well, if you make no syscall, your box is basically a brick. We need suspensions from time to time to get Linux running anyway. For sure. My point is that people may get the idea to build time-critical event delivery based on signals. In such cases it would make a big difference how often the destination thread issues a syscall. Also forced preemptions can extend the delivery latency, only the user space workload counts. I would not recommend such application designs, but people may get that idea when they once read RT-safe signals. :) Ok. But if the thread receiving the syscall is suspended in primary mode, it will wake up ASAP to handle the signal, so the latency is not that great. Of course it will work if the target thread is suspended most of the time. -- Gilles ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Jan Kiszka wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: Why? It delivers us the core mechanism we need for the rest as well - and it does not require fancy I-pipe hooks. Because relying on the vdso/vsyscall only works on x86. Whereas implementing clock slew down/acceleration at nucleus level and simply sharing data between kernel and user through the the global sem heap, works for all architectures. There are three kind of archs: - those that already support vgettimeofday friends (x86, powerpc, maybe more) - those that do not yet though they could (I strongly suspect arm falls into this category as well) - those that never will (due to lacking user-readable time sources) We need temporary/permanent i-pipe workarounds for the last two, but I see no point in complicating the first category. This design aims at a longer term. Well, I may be wrong, but I prefer generic code to arch-specific code. Nucleus code to handle clock slow down/acceleration would be generic; I-pipe code to signal NTP syscalls would be generic (and yes, even if I-pipe patches are generated for all architectures, whether the code is generic or specific makes a big difference); User-space code to implement clock_gettime(CLOCK_REALTIME) using data shared through the global sem heap would be generic. So, I think this design is future proof and easy to maintain. And I do not see how it complicates x86 situation, since it is only made of generic code. Well, OK, then place a small optional I-pipe hook into that part that normally writes the update into the vdso page (I think that is arch-specific anyway), replicating it into a specified page the nucleus may set up on a globally shared heap. That hook also has to maintain a seqlock like Linux does, ie. generating the same layout and semantics. It's just the transport mechanism, we can easily select it based on the arch's level of support. But I'm against any needless redirection through the nucleus (including potential nklocks etc.). I do not really understand why you want to use the vdso page, since we have the global sem heap anyway. clock_gettime already has a mean to read a clock source and the frequency of this clock source, this is guaranteed on all platforms, so I think the correction code can be made generic. Again, the page is not the point, what it contains is important. Our currently published data is not sufficient to support dynamic updates, but the vdso _contains_ all the data we could reuse with an enhanced algorithm to provide a dynamically adjusted time base. However, duplicating the ntp related kernel code may be the real issue. I have to look at that code to see how complex it is. I surely don't want to duplicate ntp code, just the results it spits out (e.g. into the vdso page). Ok, good point, we can avoid duplicating ntp code, but the vdso page trick only works on two architectures. So, IMO, the nucleus should get the infos, and copies them to the shared area. The question is to know if the infos are portable/in usable units such as nanoseconds or clock source ticks. For the locking, well, if we have variables on a shared area to update every time ntp corrects the clock, we will have problems doing it under nklock anyway (the irq locking prevents from preemption on the local cpu, but does not prevent the remote cpus running user-space programs from accessing the shared area). So, we will have to devise some locking mechanisms. But I do not see the reason for making this linux compatible. If we keep the nucleus business separated, the nucleus shared area will never have to be accessed by plain Linux, which will access its area in the vdso, the Linux kernel doing its house keeping. The kernel need to synchronize with user land, so the problem is (almost) the same as with obtaining the original data from Linux directly: user land can only use a retry or a smart fall-back mechanism to obtain consistent data. Well, ok, I imagine something like a revision counter. But I do not see how it would work on an SMP system (kernel writing on one CPU, user-space reading on another cpu). For the HPET clocksource, well, I do not share your enthusiasm, Linux keeps complaining about the stability of the tsc of my laptop, though it is a fairly recent core2 with the constant_tsc flag. My point is that we offer Xenomai without a real alternative to tsc for many moons, and no one really stood up so far and complained about missing hpet support. Just like you cannot use every SMI-infected box for RT, you can't do so when the tsc is unfixably broken. I'm not sure how hard hpet addition and maintenance would actually be, if it's trivial, I'm fine. But I think we have more important issues to solve. If we do not support HPET, we should have a way to know that Linux is currently not using tsc as
Re: [Xenomai-core] Realtime gettimeofday()
Gilles Chanteperdrix wrote: Jan Kiszka wrote: I surely don't want to duplicate ntp code, just the results it spits out (e.g. into the vdso page). Ok, good point, we can avoid duplicating ntp code, but the vdso page trick only works on two architectures. So, IMO, the nucleus should get the infos, and copies them to the shared area. The question is to know if the infos are portable/in usable units such as nanoseconds or clock source ticks. Ok. Had a look, the info is pretty much generic. We can use it on all architectures. The point which would generate the I-pipe event is vsyscall_update. The kernel need to synchronize with user land, so the problem is (almost) the same as with obtaining the original data from Linux directly: user land can only use a retry or a smart fall-back mechanism to obtain consistent data. Well, ok, I imagine something like a revision counter. But I do not see how it would work on an SMP system (kernel writing on one CPU, user-space reading on another cpu). Ok, I had a look at seqlock.h, we can do it the same way, in an area of the global heap. We will have to provide our own implementation since we can not include linux/seqlock.h in user-space. update_vsyscall on x86_64 turns off the irqs, to protect against the same code or some reader being called in an interrupt handler. We will have to do the same. For the HPET clocksource, well, I do not share your enthusiasm, Linux keeps complaining about the stability of the tsc of my laptop, though it is a fairly recent core2 with the constant_tsc flag. My point is that we offer Xenomai without a real alternative to tsc for many moons, and no one really stood up so far and complained about missing hpet support. Just like you cannot use every SMI-infected box for RT, you can't do so when the tsc is unfixably broken. I'm not sure how hard hpet addition and maintenance would actually be, if it's trivial, I'm fine. But I think we have more important issues to solve. If we do not support HPET, we should have a way to know that Linux is currently not using tsc as its clock source, and so that the nucleus should ignore the corrections, otherwise we may end up with a system drifting even more than if Linux was not running ntp. From the ABI point of view, we can add a member in the sysinfo structure, giving the address where the kernel updates the clock related data, if that member is NULL, the kernel does not update the clock related data (it can be a compile-time decision because using a too old I-pipe patch, or a run-time decision because Linux does not use the tsc as its clocksource), and clock_gettime(CLOCK_REALTIME) uses the syscall. We would then postpone the implementation of the non-NULL case (both in user-space and kernel-space) to after the initial 2.5 release. Mixing kernel and user from either version would not cause any issue. This still stands. But we have to handle the disabled tsc differently, probably with a flag in the shared area. Because this can happen after the sys info have been retrieved. -- Gilles. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Gilles Chanteperdrix wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: I surely don't want to duplicate ntp code, just the results it spits out (e.g. into the vdso page). Ok, good point, we can avoid duplicating ntp code, but the vdso page trick only works on two architectures. So, IMO, the nucleus should get the infos, and copies them to the shared area. The question is to know if the infos are portable/in usable units such as nanoseconds or clock source ticks. Ok. Had a look, the info is pretty much generic. We can use it on all architectures. The point which would generate the I-pipe event is vsyscall_update. completely agreed. But this required arch-specific changes to the I-pipe patch and cannot be done in a generic fashion. The kernel need to synchronize with user land, so the problem is (almost) the same as with obtaining the original data from Linux directly: user land can only use a retry or a smart fall-back mechanism to obtain consistent data. Well, ok, I imagine something like a revision counter. But I do not see how it would work on an SMP system (kernel writing on one CPU, user-space reading on another cpu). Ok, I had a look at seqlock.h, we can do it the same way, in an area of the global heap. We will have to provide our own implementation since we can not include linux/seqlock.h in user-space. update_vsyscall on x86_64 turns off the irqs, to protect against the same code or some reader being called in an interrupt handler. We will have to do the same. But then the code can loop potentially unbounded, can't it? It may be that I'm missing something, but this was the initial reasaon for the three-buffer idea I've presented initially. Best, Wolfgang For the HPET clocksource, well, I do not share your enthusiasm, Linux keeps complaining about the stability of the tsc of my laptop, though it is a fairly recent core2 with the constant_tsc flag. My point is that we offer Xenomai without a real alternative to tsc for many moons, and no one really stood up so far and complained about missing hpet support. Just like you cannot use every SMI-infected box for RT, you can't do so when the tsc is unfixably broken. I'm not sure how hard hpet addition and maintenance would actually be, if it's trivial, I'm fine. But I think we have more important issues to solve. If we do not support HPET, we should have a way to know that Linux is currently not using tsc as its clock source, and so that the nucleus should ignore the corrections, otherwise we may end up with a system drifting even more than if Linux was not running ntp. From the ABI point of view, we can add a member in the sysinfo structure, giving the address where the kernel updates the clock related data, if that member is NULL, the kernel does not update the clock related data (it can be a compile-time decision because using a too old I-pipe patch, or a run-time decision because Linux does not use the tsc as its clocksource), and clock_gettime(CLOCK_REALTIME) uses the syscall. We would then postpone the implementation of the non-NULL case (both in user-space and kernel-space) to after the initial 2.5 release. Mixing kernel and user from either version would not cause any issue. This still stands. But we have to handle the disabled tsc differently, probably with a flag in the shared area. Because this can happen after the sys info have been retrieved. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Wolfgang Mauerer wrote: Gilles Chanteperdrix wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: I surely don't want to duplicate ntp code, just the results it spits out (e.g. into the vdso page). Ok, good point, we can avoid duplicating ntp code, but the vdso page trick only works on two architectures. So, IMO, the nucleus should get the infos, and copies them to the shared area. The question is to know if the infos are portable/in usable units such as nanoseconds or clock source ticks. Ok. Had a look, the info is pretty much generic. We can use it on all architectures. The point which would generate the I-pipe event is vsyscall_update. completely agreed. But this required arch-specific changes to the I-pipe patch and cannot be done in a generic fashion. Well, it can be done in many ways. One of them is to create a function static inline ipipe_vsyscall_update(whatever) { vsyscall_update(whatever); ipipe_dispatch_event(whatever); } in linux/clocksource.h and replace calls to vsyscall_update in generic code with a call to ipipe_vsyscall_update. So, no, this does not necessarily requires arch-specific changes. However, changing it in the arch-specific way is probably the easier to maintain on the long run (if we use the method just sketched, we will have to check with every release if a new call to vsyscall_update appears). Besides, for the architectures which do not implement vsyscall_update (the majority, xenomai-wise, only x86 and ppc implement vsyscall_update), the call to ipipe_dispatch_event may be put in the empty implementation which may be found in linux/clocksource.h. The kernel need to synchronize with user land, so the problem is (almost) the same as with obtaining the original data from Linux directly: user land can only use a retry or a smart fall-back mechanism to obtain consistent data. Well, ok, I imagine something like a revision counter. But I do not see how it would work on an SMP system (kernel writing on one CPU, user-space reading on another cpu). Ok, I had a look at seqlock.h, we can do it the same way, in an area of the global heap. We will have to provide our own implementation since we can not include linux/seqlock.h in user-space. update_vsyscall on x86_64 turns off the irqs, to protect against the same code or some reader being called in an interrupt handler. We will have to do the same. But then the code can loop potentially unbounded, can't it? It may be that I'm missing something, but this was the initial reasaon for the three-buffer idea I've presented initially. Well, on a normally running system, the update takes place for every Linux timer interrupt (or is it only for the periodic tick ?), anyway, if real-time code is running, there is no chance that it can be preempted by a Linux timer interrupt, and if the code is called from non real-time context, we do not care much. Note that we already use such code for tsc emulation on some platforms, and well, it works. And this is normal, the chances to get preempted several times in a row by a timer interrupt without being able to complete the loop are pretty low. Otherwise it would mean that there are plenty of timer interrupts. -- Gilles. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core
Re: [Xenomai-core] Realtime gettimeofday()
Gilles Chanteperdrix wrote: Wolfgang Mauerer wrote: Gilles Chanteperdrix wrote: Gilles Chanteperdrix wrote: Jan Kiszka wrote: I surely don't want to duplicate ntp code, just the results it spits out (e.g. into the vdso page). Ok, good point, we can avoid duplicating ntp code, but the vdso page trick only works on two architectures. So, IMO, the nucleus should get the infos, and copies them to the shared area. The question is to know if the infos are portable/in usable units such as nanoseconds or clock source ticks. Ok. Had a look, the info is pretty much generic. We can use it on all architectures. The point which would generate the I-pipe event is vsyscall_update. completely agreed. But this required arch-specific changes to the I-pipe patch and cannot be done in a generic fashion. Well, it can be done in many ways. One of them is to create a function static inline ipipe_vsyscall_update(whatever) { vsyscall_update(whatever); ipipe_dispatch_event(whatever); } in linux/clocksource.h and replace calls to vsyscall_update in generic code with a call to ipipe_vsyscall_update. So, no, this does not necessarily requires arch-specific changes. However, changing it in the arch-specific way is probably the easier to maintain on the long run (if we use the method just sketched, we will have to check with every release if a new call to vsyscall_update appears). Besides, for the architectures which do not implement vsyscall_update (the majority, xenomai-wise, only x86 and ppc implement vsyscall_update), the call to ipipe_dispatch_event may be put in the empty implementation which may be found in linux/clocksource.h. The kernel need to synchronize with user land, so the problem is (almost) the same as with obtaining the original data from Linux directly: user land can only use a retry or a smart fall-back mechanism to obtain consistent data. Well, ok, I imagine something like a revision counter. But I do not see how it would work on an SMP system (kernel writing on one CPU, user-space reading on another cpu). Ok, I had a look at seqlock.h, we can do it the same way, in an area of the global heap. We will have to provide our own implementation since we can not include linux/seqlock.h in user-space. update_vsyscall on x86_64 turns off the irqs, to protect against the same code or some reader being called in an interrupt handler. We will have to do the same. But then the code can loop potentially unbounded, can't it? It may be that I'm missing something, but this was the initial reasaon for the three-buffer idea I've presented initially. Well, on a normally running system, the update takes place for every Linux timer interrupt (or is it only for the periodic tick ?), anyway, if real-time code is running, there is no chance that it can be preempted by a Linux timer interrupt, and if the code is called from non real-time context, we do not care much. Note that we already use such code for tsc emulation on some platforms, and well, it works. And this is normal, the chances to get preempted several times in a row by a timer interrupt without being able to complete the loop are pretty low. Otherwise it would mean that there are plenty of timer interrupts. There is another more important issue. Currently, the timer code is designed around the idea that the parameters used for conversion between cpu time and wall clock time do not change often, namely only when xnpod_set_time is called. If we start using dynamic parameters for the conversion, it will break the assumption, and break the current timer code (namely, real-time timers expiry date will be approximate). So, we will have to rework the real-time timers handling code. So, I think we should focus on the ABI issue and find a way to keep these changes for later. -- Gilles. ___ Xenomai-core mailing list Xenomai-core@gna.org https://mail.gna.org/listinfo/xenomai-core