On Thu, May 22, 2008 at 4:51 PM, Fabien MAHOT
<[EMAIL PROTECTED]> wrote:
>> Hello,
>>
>> Thanks to a serial console, I obtained the kernel traces before the crash
>> when I executed Cyclictest.
>> However, I could not have the same error : kernel panic - not syncing:
>> Aiee, killing interrupt handler. The system crashed like before, without
>> error message.
> I do not understand what you mean. You tell us that you get no traces,
> but then shows some traces.
> Anyway, could you enable the Xenomai nucleus debugging ? In
> particular, I would be interested to know if you get the fatal "Error
> relaxing a kicked thread".
sorry, my english is bad. I would like to say that I had not the explicit
error message kernel panic - not syncing: Aiee, killing interrupt handler
> On Thu, May 22, 2008 at 4:51 PM, Fabien MAHOT
> <[EMAIL PROTECTED]> wrote:
>> I also tried without your patch, and when I stopped Cyclictest, there
>> was
>> no problem !
>
> Ah! You put interesting information in the middle of I-pipe tracer
> traces, how do you want me to discover them ? So, this is a bug caused
> by my patch. In my patch, do you get better results if you do not set
the XNKICKED bit ?
>
> --
> Gilles
>
yes, I cancelled XNKICKED in xnthread_set_info call. And, it's better,
there is no problem when I stop Cyclictest.
With my sem_wait test program, sem_wait returns an error when a signal of
time-out end unlocks it, so that's good.
And , I no longer have system crash after the sem_wait call.
However, When I executed my big application, the system crashed when a
thread seemed to call pthread_cond_wait. I m not sure, in this
application, there are a lot of threads. The kernel trace when the system
crashed is : Xenomai: fatal: Relaxing a kicked thread(thread=Dialogue
serie 1, mask=200)?!
That is what you planned. What does this message mean ?
I tried to reproduce this issue with a small test program but I didn't
succeed.
I ve got a test program with this issue, but it's the one in which I ve
got a lock in the time-out end routine. (pthread_cond_broadcast) I posted
it in the mailing list and you corrected it.
I replaced the mutex use by a semaphore.
Otherwise, I ve got an other test program in which there are 3 real-time
threads. Time-outs are created dynamically and have the same duration.
(5ms)
I ve a display function, to display traces on the console. (use write
function)
The threads :
threadTimeOut : creates and deletes time-outs
threadTimeOutEnd : waits the end of a time-out (sem_wait) and indicates
it to threadTimeOut
threadDisplay : calls display function in a loop.
When I execute this program, there is a system crash. the kernel trace is
: Xenomai: fatal: inserting element twice, holder=c7761c78, qslot=c7761160
at include/xenomai/nucleus/queue.h:321
I don't execute it without your patch.
this is the test program :
###########################################################################################
#include <sys/mman.h>
#include <pthread.h>
#include <unistd.h>
#include <sys/time.h>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#include <stdarg.h>
#include <time.h>
#include <math.h>
#include <signal.h>
#include <semaphore.h>
#include <errno.h>
#define NB_PTR_TEMPO 3 // 5 tempos maximum
#define STACKSIZE 350
// stack
static int Stack[STACKSIZE];
static unsigned short Write_ptr = 0;
static unsigned short Read_ptr = 0;
// Display
pthread_mutex_t lockDisplay;
unsigned char bufferDisplay[2048];
// Timer
struct stTimeOut {
timer_t timer_h;
struct sigaction sa;
struct sigevent sig_spec;
struct itimerspec tmr_setting;
int number;
}*timeOut0_ptr, *timeOut1_ptr, *timeOut2_ptr, *timeOut3_ptr, *timeOut4_ptr;
// TimeOut_ptr
struct stTimeOut* tabTimeOut_ptr[STACKSIZE];
static unsigned short Write_to_ptr = 0;
static unsigned short Read_to_ptr = 0;
// Thread start
pthread_cond_t start_signal;
pthread_mutex_t main_start_lock;
bool bMainStart = false;
// Time-out end
pthread_cond_t TimeOutEnd_signal;
pthread_mutex_t timeOutEnd_lock;
bool bTimeOutEnd = false;
sem_t TimeOutWait_sem;
// Stack mutex
pthread_mutex_t Stack_lock;
/************************ Stack functions *************************/
int StackCreation(void)
{
Write_ptr = 0;
Read_ptr = 0;
return 0;
}
void StackWrite(int number)
{
if (Write_ptr >= STACKSIZE)
{
Write_ptr = 0;
}
Stack[Write_ptr++] = number;
}
int StackRead(void)
{
int number;
if (Read_ptr >= STACKSIZE)
{
Read_ptr = 0;
}
number = Stack[Read_ptr++];
return number;
}
unsigned short GetWritePtr(void)
{
return Write_ptr;
}
unsigned short GetReadPtr(void)
{
return Read_ptr;
}
/************************ Functions ******************************/
void display(char * chaine,...)
{
pthread_mutex_lock(&lockDisplay);
va_list ArgDisplay;
va_start(ArgDisplay, chaine);
vsprintf((char *)bufferDisplay,chaine,ArgDisplay);
write(2, (char*)bufferDisplay, strlen((char *)bufferDisplay));
//printf(bufferDisplay);
pthread_mutex_unlock(&lockDisplay);
}
int func(volatile int* i)
{
return (*i)++;
}
void DeleteTimer(timer_t timer)
{
if (timer!=NULL)
{
timer_delete(timer);
}
}
void EndTimeOut (int signo,siginfo_t *info,void*context)
{
volatile int i, result = 0;
DeleteTimer(((struct stTimeOut*)(info->si_value.sival_ptr))->timer_h);
StackWrite(((struct stTimeOut*)(info->si_value.sival_ptr))->number);
if ((sem_post(&TimeOutWait_sem)) == -1)
{
printf("sem_post error - errno : %d -> %s\n",errno,
strerror(errno));
}
}
void StartTimeOut (int nb_Sec, int nb_nSec, struct stTimeOut* timeOut)
{
(timeOut->sa).sa_flags = SA_SIGINFO;
(timeOut->sa).sa_sigaction = EndTimeOut;
while(sigaction(SIGRTMIN, &(timeOut->sa), NULL) < 0)
{
display("sigaction error - errno : %d -> %s\n",errno,
strerror(errno));
}
(timeOut->sig_spec).sigev_notify = SIGEV_SIGNAL;
(timeOut->sig_spec).sigev_signo = SIGRTMIN;
(timeOut->sig_spec).sigev_value.sival_ptr = timeOut;
while (timer_create(CLOCK_REALTIME, &(timeOut->sig_spec),
&(timeOut->timer_h)) < 0)
{
display("timer_create error - errno : %d -> %s\n",errno,
strerror(errno));
}
(timeOut->tmr_setting).it_value.tv_sec = nb_Sec;
(timeOut->tmr_setting).it_value.tv_nsec = nb_nSec;
(timeOut->tmr_setting).it_interval.tv_sec = 0;
(timeOut->tmr_setting).it_interval.tv_nsec = 0;
while (timer_settime((timeOut->timer_h), 0,
&(timeOut->tmr_setting),NULL) < 0)
{
display("timer_settime error - errno : %d -> %s\n",errno,
strerror(errno));
}
}
/************************** Threads ********************************/
void* threadTimeOutEnd(void * arg) {
int NbSem = 0;
bool bSemWaitError;
display("TimeOutCreation thread\n");
pthread_mutex_lock(&main_start_lock);
while (!bMainStart)
{
pthread_cond_wait(&start_signal, &main_start_lock);
}
pthread_mutex_unlock(&main_start_lock);
display("TimeOutCreation thread\n");
while (1)
{
do
{
bSemWaitError = false;
if ((sem_wait(&TimeOutWait_sem)) < 0)
{
display("sem_wait error - errno : %d -> %s\n",errno,
strerror(errno));
if (errno == EINTR) // la tache appelant sem_wait a été
débloquée de son attente par un signal d'interruption
{
bSemWaitError = true;
}
}
}while (bSemWaitError);
pthread_mutex_lock(&timeOutEnd_lock);
if((pthread_cond_broadcast(&TimeOutEnd_signal)) < 0)
{
display("pthread_cond_broadcast error - errno : %d ->
%s\n",errno, strerror(errno));
exit(0);
}
bTimeOutEnd = true;
pthread_mutex_unlock(&timeOutEnd_lock);
}
}
void* threadTimeOut(void * arg) {
int i=1;
int j, k, NbTimeOut, numTimeOut;
timeOut0_ptr = timeOut1_ptr = timeOut2_ptr = timeOut3_ptr =
timeOut4_ptr = NULL;
display("TimeOut thread\n");
pthread_mutex_lock(&main_start_lock);
while (!bMainStart)
{
pthread_cond_wait(&start_signal, &main_start_lock);
}
pthread_mutex_unlock(&main_start_lock);
display("TimeOut thread\n");
while (i < 100)
{
// Malloc and start of time out
for (j=0 ; j < NB_PTR_TEMPO; j++)
{
switch(j)
{
case 0 : if (timeOut0_ptr == NULL)
{
timeOut0_ptr=(struct
stTimeOut*)malloc(sizeof(struct stTimeOut));
if (timeOut0_ptr == NULL) exit(1);
timeOut0_ptr->number = i;
i++;
display("Start of time out %d - 5ms\n",
timeOut0_ptr->number);
StartTimeOut(0,500000000,timeOut0_ptr);
}
break;
case 1 : if (timeOut1_ptr == NULL)
{
timeOut1_ptr=(struct
stTimeOut*)malloc(sizeof(struct stTimeOut));
if (timeOut1_ptr == NULL) exit(1);
timeOut1_ptr->number = i;
i++;
display("Start of time out %d - 5ms\n",
timeOut1_ptr->number);
StartTimeOut(0,500000000,timeOut1_ptr);
}
break;
case 2 : if (timeOut2_ptr == NULL)
{
timeOut2_ptr=(struct
stTimeOut*)malloc(sizeof(struct stTimeOut));
if (timeOut2_ptr == NULL) exit(1);
timeOut2_ptr->number = i;
i++;
display("Start of time out %d - 5ms\n",
timeOut2_ptr->number);
StartTimeOut(0,500000000,timeOut2_ptr);
}
break;
case 3 : if (timeOut3_ptr == NULL)
{
timeOut3_ptr=(struct
stTimeOut*)malloc(sizeof(struct stTimeOut));
if (timeOut3_ptr == NULL) exit(1);
timeOut3_ptr->number = i;
i++;
display("Start of time out %d - 5ms\n",
timeOut3_ptr->number);
StartTimeOut(0,500000000,timeOut3_ptr);
}
break;
case 4 : if (timeOut4_ptr == NULL)
{
timeOut4_ptr=(struct
stTimeOut*)malloc(sizeof(struct stTimeOut));
if (timeOut4_ptr == NULL) exit(1);
timeOut4_ptr->number = i;
i++;
display("Start of time out %d - 5ms\n",
timeOut4_ptr->number);
StartTimeOut(0,500000000,timeOut4_ptr);
}
break;
}
}
pthread_mutex_lock(&timeOutEnd_lock);
while (!bTimeOutEnd)
{
if
((pthread_cond_wait(&TimeOutEnd_signal,&timeOutEnd_lock))
< 0)
{
display("pthread_cond_wait error - errno : %d ->
%s\n",errno, strerror(errno));
exit(0);
}
}
bTimeOutEnd = false;
pthread_mutex_unlock(&timeOutEnd_lock);
NbTimeOut = GetWritePtr() - GetReadPtr();
display("Number of time-outs ends : %d\n", NbTimeOut);
for (j=0; j < NbTimeOut; j++)
{
numTimeOut = StackRead();
display("TimeOut%d ends\n", numTimeOut);
for (k=0; k < NB_PTR_TEMPO; k++)
{
switch(k)
{
case 0 : if (timeOut0_ptr != NULL)
{
if (timeOut0_ptr->number == numTimeOut)
{
free(timeOut0_ptr);
timeOut0_ptr = NULL;
}
}
break;
case 1 : if (timeOut1_ptr != NULL)
{
if (timeOut1_ptr->number == numTimeOut)
{
free(timeOut1_ptr);
timeOut1_ptr = NULL;
}
}
break;
case 2 : if (timeOut2_ptr != NULL)
{
if (timeOut2_ptr->number == numTimeOut)
{
free(timeOut2_ptr);
timeOut2_ptr = NULL;
}
}
break;
case 3 : if (timeOut3_ptr != NULL)
{
if (timeOut3_ptr->number == numTimeOut)
{
free(timeOut3_ptr);
timeOut3_ptr = NULL;
}
}
break;
case 4 : if (timeOut4_ptr != NULL)
{
if (timeOut4_ptr->number == numTimeOut)
{
free(timeOut4_ptr);
timeOut4_ptr = NULL;
}
}
break;
}
}
}
}
while(1)
{
sleep(10);
}
return NULL;
}
void* threadDisplay(void * arg) {
volatile int i=0;
volatile int result;
display("Display thread\n");
pthread_mutex_lock(&main_start_lock);
while (!bMainStart)
{
pthread_cond_wait(&start_signal, &main_start_lock);
}
pthread_mutex_unlock(&main_start_lock);
display("Display thread\n");
while (i <= 200000) {
result = func(&i);
display("Display thread :%d \r",result);
}
display("End of display thread\n");
return NULL;
}
/***********************************************************************/
void cleanup_upon_sig(int sig __attribute__((unused)))
{
DeleteTimer(timeOut0_ptr->timer_h);
DeleteTimer(timeOut1_ptr->timer_h);
DeleteTimer(timeOut2_ptr->timer_h);
exit(0);
}
int main(int argc, char** argv) {
pthread_attr_t attr;
pthread_mutexattr_t attr_proto;
pthread_t p0;
pthread_t p1;
pthread_t p2;
struct sched_param sch;
// Stack & TabTImeOutptr creation
StackCreation();
signal(SIGINT, cleanup_upon_sig);
signal(SIGTERM, cleanup_upon_sig);
mlockall(MCL_CURRENT|MCL_FUTURE);
// mutex and sem initialisation
pthread_cond_init(&start_signal, NULL);
pthread_mutex_init(&main_start_lock, NULL);
pthread_cond_init(&TimeOutEnd_signal, NULL);
pthread_mutex_init(&timeOutEnd_lock, NULL);
sem_init (&TimeOutWait_sem, 0,0);
pthread_mutexattr_init(&attr_proto);
pthread_mutexattr_setprotocol(&attr_proto,PTHREAD_PRIO_INHERIT);
pthread_mutex_init(&lockDisplay, &attr_proto);
pthread_attr_init(&attr);
pthread_attr_setinheritsched(&attr,PTHREAD_EXPLICIT_SCHED);
pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
// TimeOutCreation thread creation
sch.sched_priority = 85;
pthread_attr_setschedparam(&attr, &sch);
pthread_create(&p0, &attr, threadTimeOutEnd, NULL);
// TimeOut thread creation
sch.sched_priority = 80;
pthread_attr_setschedparam(&attr, &sch);
pthread_create(&p1, &attr, threadTimeOut, NULL);
// Display thread creation
sch.sched_priority = 70;
pthread_attr_setschedparam(&attr, &sch);
pthread_create(&p2, &attr, threadDisplay, NULL);
pthread_attr_destroy(&attr);
display("Main condition broadcast\n");
// Start of all threads
pthread_mutex_lock(&main_start_lock);
bMainStart = true;
pthread_cond_broadcast(&start_signal);
pthread_mutex_unlock(&main_start_lock);
while (1) {
sleep(5);
}
return 0;
}
##################################################################################
Results on the console :
TimeOutCreation thread
TimeOut thread
Display thread
Main condition broadcast
TimeOutCreation thread
TimeOut thread
Start of time out 1 - 5ms
Display thread
Start of time out 2 - 5ms
Start of time out 3 - 5ms
Display thread :8791 <---- the system crashed
I hope that you can help me. I will try to execute this program without
your patch to see what kernel traces I will have if the kernel crashes.
I wonder why my system is so unsteady...
Thanks a lot for your help.
_______________________________________________
Xenomai-help mailing list
[email protected]
https://mail.gna.org/listinfo/xenomai-help
