Philippe Gerum wrote:
Jan Kiszka wrote:

Philippe Gerum wrote:

Jan Kiszka wrote:


Philippe Gerum wrote:


Jan Kiszka wrote:



Philippe Gerum wrote:



Jan Kiszka wrote:




Hi,

I happened to stumble over this comment[1]. It made me curious,
especially as it is not totally correct (the loop is executed in
IRQ-off
context, thus it *is* timecritical).


Critical should be understood here in the sense that IRQs are off
while
the loop workload is high, which is fortunately not the case. Hence
the
comment.





Sure, there is not much to do inside the loop. But it does not scale
very well in case a significant number of elements are registered - and
they are scattered over a larger memory area so that cache missed
strike us.


Compared to what it costs to actually call Linux to release the system
memory which is an operation the syscall will do anyway, those cache
misses account for basically nothing.




I don't have the function caller's cost in mind here (which is likely
either starting up or on the way to termination anyway), I just worry
about the rest of the system which may want to continue it's operation
undisturbed.


Again, it's a matter of tradeoff: do we want to add more locking
complexity, which means more code and likely more data fetches in the
hot path, in order to be able to avoid a series of uninterruptible cache
misses when scanning a short heap descriptor queue? The queue we are
talking about links all the currently active heaps, which means 1
element for the system heap, plus 1 element for each of the user-defined
heaps.



I think I should rather come up with a patch to demonstrate the difference.

The point is that we are practicing such context-dependent locking in
RTnet for quite a while now: all operations that only take place in
non-RT (as here) use Linux locks. This simply reduces the amount of code
you have to consider when analysing the real-time system's worst-case
behaviour.


Ok, but do consider the following point too in your analysis: if you use Linux locking to protect a Xenomai section, in the contended case, a Linux task switch will occur. At that point, during a context switch, the memory context will be changed while _hw_ interrupts are locked. Vanilla Linux wants this on many if not most archs (except ARM which cannot afford this), which includes x86 and PPC. The I-pipe cannot even virtualize this locking, because it would be unsafe to allow preemption by a user-space Xenomai thread during the core operations of a Linux task switch (i.e. mm context update). So in that case, the penalty will be high, way higher than a few potential cache misses as it is now.


It's a bit theoretical, but I also think we can easily resolve it by
using Linux locks as soon as we can sanely sleep inside
xnheap_init/destroy_shared and xnheap_ioctl.




While thinking about the possibility to convert the hard IRQ lock
protection of kheapq into some Linux mutex or whatever, I analysed
the
contexts the users of this queue (__validate_heap_addr/xnheap_ioctl,
xnheap_init_shared, xnheap_destroy_shared) execute in. Basically,
it is
Linux/secondary mode, but there are unfortunate exceptions:

rt_heap_delete(): take nklock[2], then call
xnheap_destroy_shared()[3].
The latter will call __unreserve_and_free_heap()[4] which calls Linux functions like vfree()[5] or kfree()[6] -- I would say: not good! At
least on SMP we could easily get trapped by non-deterministic
waiting on
Linux spinlocks inside those functions.

The same applies to rt_queue_delete()[7].


Good spot. Better not calling the heap deletion routines under nklock
protection in the first place. The committed fix does just that for
both
rt_heap_delete and rt_queue_delete.





Ok, we no longer have IRQs locked over vfree/kfree, but task scheduling
is still suffering from potential delays. Wouldn't it be better to
defer
such operations to an asynchronous Linux call?


Do we really want heap creation/deletion to be short time bounded
operations at the expense of added complexity?



Again, the side effects on other real-time programs are my concern.
There are quite a lot of scenarios where only parts of the real-time
programs are started or stopped while others keep on working as usual.
The caller's cost is more or less irrelevant in that case.


What does an asynchronous Linux call for freeing the memory would buy us
for the rest of the real-time system, compared to the now fixed
situation where no real-time lock is being held? I don't see your point
about the potentially induced task scheduling delays in the current case.



You lock the real-time scheduler, doesn't this have global relevance? My
high prio task will still have to wait until some low prio task
completes its heap release?!


Got it now, my mistake, we were not looking at the same sources. I've already removed this sched lock in my tree, because we'd better have a per-task safe mutex to handle this kind of situation ala VxWorks (i.e. taskSafe/taskUnsafe). The registry provides rt_registry_put/get, but unfortunately, we need to make it work for registry-disabled configs too.


A safe/unsafe barrier mechanism for tasks is now available. This said, we don't need it anymore, after I discovered that we were lacking the "lostage" exec bit for the rt_heap_delete and rt_queue_delete syscalls. Since we must run those on the root thread, which cannot be deleted, we don't need any safe section here. Anyway, opportunities to use the safe section in place of raw interrupt masking may exist elsewhere.

--

Philippe.

Reply via email to