On 01/25/10 11:07, Qihua Wu wrote:
Hi Max/Jim,
Based on your script, I created the following script to print which
pid is interrupted how many times every 2 second. The result shows pid
0 is interrupted most: about 50k every 2 second, that's 25k per
second. How could it be interrupted so frequently? For the clock
interrupt, it only interrupts 100 times per second, which is much less
than 25k per second. Where could it come from?
In solaris, the basic schedule unit is 'thread', so when a user process
is forked, at lease one lwp/thread pair will be created in the kernel
for that process.(if the process is a multithread process, the kernel
would create multiple those pairs). Besides those user created process,
which would be scheduled on the CPU through its underlying thread, there
are also kernel threads running on the CPU, which have no corresponding
user process(PID). For simplicity, a PID 0 is defined for all of the
kernel threads. so all kernel thread( a kthread_t structure in the
kernel) has its t_procp as &p0(and pid_id as 0 of course).
For interrupt, in solaris, all of the level interrupt(NIC, HBA etc) as
handled as kernel thread. so for your dtrace script, when the
'interrult-start' probe fires, the curthread is the interrupt thread,
and curthread->t_intr is the thread which is interrupted by the hardware
interrupt. This hardware interrupt can occur at any time, when the CPU
may run a kernel thread, or a user thread, saying mysqld, or a special
kernel thread named 'idle'.(this thread has also t_procp as &p0, and has
a priority as 0, and can be preempted by any other thread).
Now let me explain your dtrace output,
10490-7: 4962
means the interrupt occurs when the CPU is running the mysqld process.
0-7: 49131
means the interrupt occurs when the CPU is idle or running a kernel thread.
You can know which CPU the interrupt is on with mpstat(1M), and know
which CPU is assigned to a hardware interupt by '::interrupts' in mdb.
-Brian
The box has lots of network traffic from mysql, I think the network
interrupt will cause the mysqld (pid 10490) to be interrupted instead
of pid 0.
#!/usr/sbin/dtrace -s
interrupt-start
{
self->intr = curthread->t_intr;
}
interrupt-start
/self->intr/
{
@inter[self->intr->t_procp->p_pidp->pid_id, cpu] = count();
}
interrupt-complete
/self->intr/
{
self->intr = 0;
}
profile:::tick-2sec
{
printf("report completed at time:%Y\n",walltimestamp);
printf("*******************************************\n");
printa("%d-%d: %...@d\n", @inter);
trunc(@inter,0);
}
0 89152 :tick-2sec report completed at
time:2010 Jan 24 02:48:56
*******************************************
0-0: 1
0-2: 1
0-8: 1
0-12: 1
0-14: 1
0-16: 1
0-20: 1
0-21: 1
0-22: 1
1095-7: 4
0-6: 5
0-5: 8
264-7: 8
10490-7: 4962
0-7: 49131
0 89152 :tick-2sec report completed at
time:2010 Jan 24 02:48:58
*******************************************
0-0: 1
0-5: 4
1095-7: 4
10490-7: 4200
0-7: 50305
0 89152 :tick-2sec report completed at
time:2010 Jan 24 02:49:00
*******************************************
0-3: 4
1095-7: 4
0-5: 8
25388-7: 818
10490-7: 1582
0-7: 51507
On Fri, Jan 22, 2010 at 4:21 PM, m...@bruningsystems.com
<mailto:m...@bruningsystems.com> <m...@bruningsystems.com
<mailto:m...@bruningsystems.com>> wrote:
Hi Qihua Wu,
Here is a script that shows process args for process that was
interrupted, and cpu. It is easily modified
to add pid.
#!/usr/sbin/dtrace -s
interrupt-start
{
self->intr = curthread->t_intr;
}
interrupt-start
/self->intr/
{
@[stringof(self->intr->t_procp->p_user.u_psargs), cpu] = count();
}
interrupt-complete
/self->intr/
{
self->intr = 0;
}
max
Qihua Wu wrote:
So do we have a way to know what causes the interrupt (this
may be got by
intrstat) and which pid/execname is interrupted?
Thanks
Daniel
On Thu, Jan 21, 2010 at 10:43 PM, Jim Mauro
<james.ma...@sun.com <mailto:james.ma...@sun.com>> wrote:
"sched" is the execname of the PID 0 process (run "ps -e").
The string "sched" gets plugged into the DTrace execname
variable
if the CPU is in an interrupt handler when a probe fires.
CPU 0 is very likely taking the clock interrupts, which by
default
occur every 10 milliseconds.
HTH,
/jim
Qihua Wu wrote:
Why sched is often interrupt on cpu 0 instead of
distribution evenly on
all cpu?
dtrace -n 'sdt:::interrupt-start { @num[execname,cpu]
= count(); }'
dtrace: description 'sdt:::interrupt-start ' matched 1
probe
^C
sched
4
355
sched
18
358
sched
8
369
sched
10
370
sched
11
370
sched
12
408
sched
13
408
sched
15
408
sched
19
408
sched
20
408
sched
21
408
sched
22
408
sched
23
408
sched
14
409
sched
9
410
sched
2
411
sched
16
411
sched
6
422
sched
17
456
sched
3
469
sched
5
510
sched
7
564
sched
1
3079
sched
0
535071
Thanks,
Daniel
------------------------------------------------------------------------
_______________________________________________
dtrace-discuss mailing list
dtrace-discuss@opensolaris.org
<mailto:dtrace-discuss@opensolaris.org>
------------------------------------------------------------------------
_______________________________________________
dtrace-discuss mailing list
dtrace-discuss@opensolaris.org
<mailto:dtrace-discuss@opensolaris.org>
------------------------------------------------------------------------
_______________________________________________
dtrace-discuss mailing list
dtrace-discuss@opensolaris.org
_______________________________________________
dtrace-discuss mailing list
dtrace-discuss@opensolaris.org
