Hi Philippe,

thanks a lot for your comments

First to the overall positioning: My answer is yes and no

Yes you can view it that way, although I'm not quite sure what you exactly mean 
with "POSIX co-kernel that does not look like a co-kernel". But no with respect 
to the "mixing consequences". With other words, the changes we did to the 
library are not necessary because of the two kernel approach, they are 
necessary if you want to provide for realtime capability in general. For e.g. 
the implementation of the pthread_spinlock function in 2.3.6 cannot be used for 
static priorities > 0. (Since they are not protected by some kind of 
"scheduling lock", a system could get locked up). But using static priorities 
is a must for realtime.(But on the other side regular prio 0 processes by no 
means are allowed to get control over the CPU).  So an RT_PREEMPT solution 
faces the same problem.
The changes we did to the sigevent handler notification are also completely 
independent of that. You simply cannot afford having a thread creation inside a 
time critical notification path. I admit the implementation of the changes 
itself has to consider the two kernel approach( the protection function for the 
spinlock has span two domains,  for the sigevent handler we need two default 
thread_attribute objects).

For the kernel changes it is mixed. Yes we need three additional hooks 
(futex.c, mqueue., posix-timers.c) They are necessary for transparent 
communication and for tracking time across clock_setime(). But no with respect 
to a function where a device can provide a tick for a new clock type. POSIX 
allows for additional clocks but posix-timers.c doesn't export a function for 
safely allocating a new clock. An no with respect to the implementation of 
mq_send()/mq_receive(). The POSIX definition allows for a completely 
deterministic implemention, but the Linux implementation does
dynamic allocation from a slab pool. Again RT_PREEMPT faces the same problem.

Sharing the changed FPU routines is also not caused by the two kernel approach, 
it was basically a matter of saving work by using existing code an by not 
figuring out what it takes to have FPU save/restore interruptible.

The ISR stuff:
I haven't looked into the ipipe for genirq. In our pre genirq version we didn't 
find a place where to put end_ack and we cannot have it placed in the ISR 
I'll give it a try.

I don't know how to explain best the requirement that a thread as soon as
it is in user mode must be in its "home" domain. If that is not guaranteed our 
approach would fall apart. 
One example:
Lets assume the last system call of a thread running in the realtime domain 
would be a mq_open() (end of a setup phase) and the next system call would be a 
sigtimedwait(). To route the system call to the realtime domain the thread must 
already execute in the realtime domain. Does this make sense?
If not then I have to dig deeper listing the other reasons dealing with 
decoupling of signalling of the two domains.

More RT processes:

Futexes have been surely the initial reason to restrict to one process. 
Now in the light of RT-PREEMPT we see even less need as we may had seen 
initially. (the better the realtime capability of the base system,the more
we need to serve the hardcore realtime only).
The fact that the Linux kernel developers consider the introduction of local 
futexes encourages us to stay with that.

Testsuite (Jan):

We use the POSIX testsuite adapitestsuite-1.4.3. The sourc we loaded from 
sourceforge. Besides using only the part which deals with the calls, we 
additionally had to do some subsetting which is in line with our "subset API 
definition". For example: Since signal handlers for asynchronous events are an 
indeterministic construct and not usable for realtime, we deleted them.
Besides subsetting we added a few scripts for easier handling and a prolog 
which gets linked to a test. This prolog brings main to the right scheduling 
policy and priority. Of cources there have been a few tests which implicitly 
assumed prio 0 with SCHED_OTHER.
To figure out possible synergy I doesn't know enough about your build and 
runtime environment. Just let me know. 

Don't be surprised about radio silence, I'll be in vacation for a week.



-----Urspr√ľngliche Nachricht-----
Von: Philippe Gerum [mailto:[EMAIL PROTECTED] Im Auftrag von Philippe Gerum
Gesendet: Donnerstag, 19. April 2007 20:14
An: Krause, Karl-Heinz
Cc: [EMAIL PROTECTED]; xenomai-core@gna.org
Betreff: Re: [Xenomai-core] Ipipe and Siemens A&D Realtime

On Thu, 2007-04-19 at 18:09 +0200, Krause, Karl-Heinz wrote:
> Yes, it contains the full A&D RT Kernel.

There seems to be lot of work there. I had a look at this patch, and
basically, it looks like an effort to create an integrated POSIX
co-kernel that does not look like a co-kernel anymore, while still
exhibiting rt and non-rt domain separation though.  This seems to have
required adding new real-time support in vanilla code, and ironing other
parts of the original code in order to allow mixed uses between both
domains (hence the glibc patching too).

> If it were only the path to ISR entry, there is not much difference to the 
> wired optimization (the main difference is a stripped down ack-Routine).

Perhaps a bit too much stripped down; I see you don't handle the per-IRQ
locking bit anymore, which means that RT drivers are not allowed to
solely rely on the PIC to mask interrupts without going for
local_irq_disable(). This said, only vanilla Linux drivers might rely on
this trick, so this is ok in the fast mode approach, I guess.

>  The 
> problem we saw with the wired-optimzation is, that -- if we want an interrupt 
> dedicated exclusively to the RT domain -- it doesn't provide for a final 
> ack-Routine.

It does since genirq is supported by the I-pipe; it's the ipipe_end
handler from the irq descriptor.

>  We always have to take the entire round trip.

It's due to the way this co-kernel is layered, but that's not a
requirement. For instance, Xenomai ISRs are in charge of ending the
interrupt by calling the ipipe_end hook for the processed IRQ channel;
this is no different than ending it upon return from your ISR to
__ipipe_handle_irq, except that the vanilla I-pipe never overrides any
decision the RTOS ISR could make regarding PIC manipulation.

> If you look at the comment for the fast handle_irq, you will see what we had 
> in  mind. 
> For the other part --the system call entry/exit -- we simply put it under the 
> same optimization decision, but  I guess we can separate it.

I recall we once discussed about the SYSCALL_EXIT event, but did not
catch the idea behind this requirement, i.e. forcing a relaxed thread
back to the real-time domain after a syscall. Is there any reason why
you don't want the lazy scheme to be used, i.e. migrate to whatever
domain may process any given syscall upon entry, run the handler, then
leave the caller running in the current domain until the next syscall

> The fpu stuff has two parts, first we borrowed somewhat from the 2.6.20 
> solution to avoid restore on exception, but had to disable the 
> "256-exception". Second we hacked meaning we replaced the 
> preempt_disable/enable in the three low level routines in i387.h, by local 
> interrupt disable/enable. There may be a way around, but since we call the 
> same __switch_to() routine, we had to avoid running into 
> preempt_disable/enable.
> Can understand, when I said I don't expect big surprises, then I assumed be 
> based on an existing ipipe-patch. Yes HRT timers are a problem but doesn't 
> cause us to much of a headache right now, since we are required to support 
> only features available on all architectures.

Generally speaking, this code has to know about a lot of kernel innards
through. I understand this is due to the effort to integrate this
co-kernel into the vanilla one, so that it does not look like a separate
extension anymore.

> This leads to the last point. No we haven't running RT on ARM and MIPS yet.
> The main work so far was to have common base for the tool chain and the 
> glibc. And having a working solution for TLS on ARM without MMU caused us 
> some trouble. But now we are through and I expect having RT on ARM very soon. 
> MIPS is scheduled for the third quarter.

How much much would it require to be able to run more than a single RT
process? Is this only related to the RT futex support, or are there
other issues to solve?

> As for the test suite, let me defer it to tomorrow.

> Karl-Heinz
> -----Urspr√ľngliche Nachricht-----
> Gesendet: Donnerstag, 19. April 2007 14:40
> An: Krause, Karl-Heinz
> Cc: xenomai-core@gna.org
> Betreff: Re: AW: [Xenomai-core] Ipipe and Siemens A&D Realtime
> Krause, Karl-Heinz wrote:
> > Hi Jan
> > 
> > Sorry for attaching the wrong patch. Here is the right one.
> Thanks. Am I right, it also contains the full A&D RT-kernel?
> > From all the ipipe patches/hooks we use only a few one.
> > The ipipe changes we did fall into two categories
> > - one additional hook in entry.s which checks at system call exit whether  
> >   the thread has to migrate to the realtime domain
> > - optimizations for a dual domain system concerning the dispatching of 
> >   events and interrupt handling.
> The latter makes me wonder if you already analysed the "wired" top-most
> domain optimisation in recent I-pipe and compared it to your approach.
> > So if just that kind of dual domain system is of interest, the patches may 
> > be of general interest and we are willing to support it.  
> > To allow for a possible integration we made it configurable as follows
> > [ ] interrupt pipeline
> > [ ]   rtx domain system
> > [ ]     fast rtx dual domain system
> A wired ipipe domain is known to stay the top-most one in the pipeline,
> but this optimisation still allows multiple lower domains, not just Linux.
> Maybe it is worth looking at the I-pipe abstractions again with the
> hard-wired scenario "dual domain system" in mind, checking for further
> optimisation possibilities that do not kill the common model and its
> clean realisation. Optimisations are not bad, but they also have to be
> balanced against potentially increased porting effort for other archs.
> > 
> > Common for both is the check for preemption which we have in the optimized 
> > handle_irq routine as well as in sync_stage.
> > 
> > More generic are the patches we made for the POSIX message queues and for 
> > saving/restoring the fpu context
> IIRC, sanitising xxx_fpu services in order to make them domain-agnostic
> without hacking preempt_enable/disable is something of generic interest
> for ipipe as well (Or are you patching fpu services for another
> reason?). I recall to stumble over this on my own when trying
> to detect invalid calls of Linux services from non-root domains
> automatically.
> > 
> > We are not specifically locked to and we don't expect any 
> > surprises if we would upgrade.
> Hmm, I wouldn't be surprised if things like genirq or upcoming
> clockevent/hrtimers _will_ cause a few surprises. The latter is actually
> also a to-do for vanilla ipipe in order to move to 2.6.21 and beyond.
> > (Of course we know that  we have to do some additions to hook into PI 
> > mutexes, despite the fact that for priority changes itself the domain 
> > boundary is already transparent). It is only caused by our general policy 
> > to keep the number of different versions minimal, since we cannot simply 
> > forget older versions, we have to maintain it.(It is not only  a kernel for 
> > x86, it is for ARM with and without MMU and for MIPS as well and it is the 
> > complete tool chain). So we always will do some skipping and we haven't 
> > decided yet what our next version will be.
> Does your RT-kernel run on ARM and MIPS as well already? Did you port
> ipipe (or a subset) to MIPS?
> > 
> > Concerning testing we took mainly the existing POSIX test suites which we
> > had to make suitable for static priorities > 0. We added a few tests for 
> > checking the transparency of the domain boundary for POSIX message queues, 
> > futexes and priority changes and we combined them with performance 
> > measurement that is all.
> Those changes may be of some interest for POSIX testing with Xenomai as
> well, that's why I was asking for a complete, instrumented testsuite. If
> you could share it with the community, that would be great.
> > 
> > Just let me know what else you need.
> > 
> > 
> > Karl-Heinz
> > 
> Thanks for making your code available. I think both sides can benefit
> from discussing new approaches based on public code.
> Jan
> _______________________________________________
> Xenomai-core mailing list
> Xenomai-core@gna.org
> https://mail.gna.org/listinfo/xenomai-core

Xenomai-core mailing list

Reply via email to