Philippe Gerum wrote:
> Jan Kiszka wrote:
> 
>> Philippe Gerum wrote:
>>
>>> Jan Kiszka wrote:
>>>
>>>
>>>> Philippe Gerum wrote:
>>>>
>>>>
>>>>> Jan Kiszka wrote:
>>>>>
>>>>>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I happened to stumble over this comment[1]. It made me curious,
>>>>>> especially as it is not totally correct (the loop is executed in
>>>>>> IRQ-off
>>>>>> context, thus it *is* timecritical).
>>>>>>
>>>>>
>>>>> Critical should be understood here in the sense that IRQs are off
>>>>> while
>>>>> the loop workload is high, which is fortunately not the case. Hence
>>>>> the
>>>>> comment.
>>>>
>>>>
>>>>
>>>>
>>>> Sure, there is not much to do inside the loop. But it does not scale
>>>> very well in case a significant number of elements are registered - and
>>>> they are scattered over a larger memory area so that cache missed
>>>> strike us.
>>>>
>>>
>>> Compared to what it costs to actually call Linux to release the system
>>> memory which is an operation the syscall will do anyway, those cache
>>> misses account for basically nothing.
>>
>>
>>
>> I don't have the function caller's cost in mind here (which is likely
>> either starting up or on the way to termination anyway), I just worry
>> about the rest of the system which may want to continue it's operation
>> undisturbed.
>>
> 
> Again, it's a matter of tradeoff: do we want to add more locking
> complexity, which means more code and likely more data fetches in the
> hot path, in order to be able to avoid a series of uninterruptible cache
> misses when scanning a short heap descriptor queue? The queue we are
> talking about links all the currently active heaps, which means 1
> element for the system heap, plus 1 element for each of the user-defined
> heaps.
> 

I think I should rather come up with a patch to demonstrate the difference.

The point is that we are practicing such context-dependent locking in
RTnet for quite a while now: all operations that only take place in
non-RT (as here) use Linux locks. This simply reduces the amount of code
you have to consider when analysing the real-time system's worst-case
behaviour.

>>
>>>> It's a bit theoretical, but I also think we can easily resolve it by
>>>> using Linux locks as soon as we can sanely sleep inside
>>>> xnheap_init/destroy_shared and xnheap_ioctl.
>>>>
>>>>
>>>>
>>>>>> While thinking about the possibility to convert the hard IRQ lock
>>>>>> protection of kheapq into some Linux mutex or whatever, I analysed
>>>>>> the
>>>>>> contexts the users of this queue (__validate_heap_addr/xnheap_ioctl,
>>>>>> xnheap_init_shared, xnheap_destroy_shared) execute in. Basically,
>>>>>> it is
>>>>>> Linux/secondary mode, but there are unfortunate exceptions:
>>>>>>
>>>>>> rt_heap_delete(): take nklock[2], then call
>>>>>> xnheap_destroy_shared()[3].
>>>>>> The latter will call __unreserve_and_free_heap()[4] which calls Linux
>>>>>> functions like vfree()[5] or kfree()[6] -- I would say: not good! At
>>>>>> least on SMP we could easily get trapped by non-deterministic
>>>>>> waiting on
>>>>>> Linux spinlocks inside those functions.
>>>>>>
>>>>>> The same applies to rt_queue_delete()[7].
>>>>>>
>>>>>
>>>>> Good spot. Better not calling the heap deletion routines under nklock
>>>>> protection in the first place. The committed fix does just that for
>>>>> both
>>>>> rt_heap_delete and rt_queue_delete.
>>>>
>>>>
>>>>
>>>>
>>>> Ok, we no longer have IRQs locked over vfree/kfree, but task scheduling
>>>> is still suffering from potential delays. Wouldn't it be better to
>>>> defer
>>>> such operations to an asynchronous Linux call?
>>>>
>>>
>>> Do we really want heap creation/deletion to be short time bounded
>>> operations at the expense of added complexity?
>>>
>>
>>
>> Again, the side effects on other real-time programs are my concern.
>> There are quite a lot of scenarios where only parts of the real-time
>> programs are started or stopped while others keep on working as usual.
>> The caller's cost is more or less irrelevant in that case.
>>
> 
> What does an asynchronous Linux call for freeing the memory would buy us
> for the rest of the real-time system, compared to the now fixed
> situation where no real-time lock is being held? I don't see your point
> about the potentially induced task scheduling delays in the current case.
> 

You lock the real-time scheduler, doesn't this have global relevance? My
high prio task will still have to wait until some low prio task
completes its heap release?!

Jan

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to