Nikos
"Thanks God" it was "just" my laptop motherboard that died, not the server.
In this moment I have no traffic and bearerbox still at 6-8%
[r...@srvcom1 ~]# top
top - 15:50:54 up 26 days, 15:23, 1 user, load average: 0.08, 0.27, 0.24
Tasks: 218 total, 1 running, 204 sleeping, 0 stopped, 13 zombie
Cpu(s): 0.0%us, 0.1%sy, 0.0%ni, 99.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2074712k total, 1989044k used, 85668k free, 42872k buffers
Swap: 2031608k total, 344k used, 2031264k free, 859020k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2337 root 20 0 420m 13m 2448 S 7 0.6 0:25.80 bearerbox
2716 root 20 0 2332 1088 800 R 0 0.1 0:00.07 top
1 root 20 0 2044 652 560 S 0 0.0 0:10.44 init
2 root 15 -5 0 0 0 S 0 0.0 0:00.00 kthreadd
3 root RT -5 0 0 0 S 0 0.0 0:00.95 migration/0
4 root 15 -5 0 0 0 S 0 0.0 0:02.18 ksoftirqd/0
[r...@srvcom1 ~]# mpstat
Linux 2.6.22.14-72.fc6 (srvcom1.smsglobal.com.mx) 03/26/2010
03:55:49 PM CPU %user %nice %sys %iowait %irq %soft
%steal %idle intr/s
03:55:49 PM all 2.37 0.00 0.84 0.67 0.02 0.03
0.00 96.07 37.07
[r...@srvcom1 ~]# iostat 1 10
Linux 2.6.22.14-72.fc6 (srvcom1.smsglobal.com.mx) 03/26/2010
avg-cpu: %user %nice %system %iowait %steal %idle
2.37 0.00 0.89 0.67 0.00 96.07
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 6.33 79.19 176.27 182289095 405742931
dm-0 23.24 79.19 176.26 182280322 405722192
dm-1 0.00 0.00 0.01 1632 19936
avg-cpu: %user %nice %system %iowait %steal %idle
11.00 0.00 9.50 0.00 0.00 79.50
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
dm-0 0.00 0.00 0.00 0 0
dm-1 0.00 0.00 0.00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
14.29 0.00 11.28 0.00 0.00 74.44
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
dm-0 0.00 0.00 0.00 0 0
dm-1 0.00 0.00 0.00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
12.47 0.00 12.47 0.00 0.00 75.06
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
dm-0 0.00 0.00 0.00 0 0
dm-1 0.00 0.00 0.00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
11.97 0.00 11.97 3.24 0.00 72.82
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 21.00 0.00 496.00 0 496
dm-0 62.00 0.00 496.00 0 496
dm-1 0.00 0.00 0.00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 0.00 0.00 0.00 100.00
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
dm-0 0.00 0.00 0.00 0 0
dm-1 0.00 0.00 0.00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 0.00 0.00 0.00 100.00
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
dm-0 0.00 0.00 0.00 0 0
dm-1 0.00 0.00 0.00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 0.00 0.00 0.00 100.00
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
dm-0 0.00 0.00 0.00 0 0
dm-1 0.00 0.00 0.00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
0.50 0.00 0.50 0.00 0.00 99.00
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
dm-0 0.00 0.00 0.00 0 0
dm-1 0.00 0.00 0.00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 0.25 4.00 0.00 95.75
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 131.00 0.00 1680.00 0 1680
dm-0 210.00 0.00 1680.00 0 1680
dm-1 0.00 0.00 0.00 0 0
[r...@srvcom1 ~]#
About debugging... have no idea what that looks like ;=(( ... should I
eat it with a salad? :=)
I do use the CNMI to receive MO in kannel but sometimes it fails and
messages start to clog the modems so I do also use sim-buffering in
order to keep modems free of messages. Sim-buffering looks for the
messages once per minute. I do have only about 20 modems.
[r...@srvcom1 ~]# tail -f /var/log/kannel/com1_t1.log
2010-03-26 16:05:54 [2337] [7] DEBUG: AT2[t1]: <-- OK
2010-03-26 16:05:58 [2337] [7] DEBUG: AT2[t1]: --> AT+CSQ^M
2010-03-26 16:05:58 [2337] [7] DEBUG: AT2[t1]: <-- +CSQ: 26,0
2010-03-26 16:05:58 [2337] [7] DEBUG: AT2[t1]: <-- OK
2010-03-26 16:06:56 [2337] [7] DEBUG: AT2[t1]: --> AT+CPMS?^M
2010-03-26 16:06:56 [2337] [7] DEBUG: AT2[t1]: <-- +CPMS:
"SM",0,20,"SM",0,20,"SM",0,20
2010-03-26 16:06:56 [2337] [7] DEBUG: AT2[tl1]: <-- OK
2010-03-26 16:07:00 [2337] [7] DEBUG: AT2[t1]: --> AT+CSQ"^M
2010-03-26 16:07:00 [2337] [7] DEBUG: AT2[t1]: <-- +CSQ: 26,0
2010-03-26 16:07:00 [2337] [7] DEBUG: AT2[t1]: <-- OK
2010-03-26 16:07:58 [2337] [7] DEBUG: AT2[t1]: --> AT+CPMS?^M
2010-03-26 16:07:58 [2337] [7] DEBUG: AT2[t1]: <-- +CPMS:
"SM",0,20,"SM",0,20,"SM",0,20
2010-03-26 16:07:58 [2337] [7] DEBUG: AT2[t1]: <-- OK
2010-03-26 16:08:02 [2337] [7] DEBUG: AT2[t1]: --> AT+CSQ"^M
2010-03-26 16:08:02 [2337] [7] DEBUG: AT2[t1]: <-- +CSQ: 26,0
2010-03-26 16:08:02 [2337] [7] DEBUG: AT2[t1]: <-- OK
2010-03-26 16:09:00 [2337] [7] DEBUG: AT2[t1]: --> AT+CPMS?^M
2010-03-26 16:09:00 [2337] [7] DEBUG: AT2[t1]: <-- +CPMS:
"SM",0,20,"SM",0,20,"SM",0,20
2010-03-26 16:09:00 [2337] [7] DEBUG: AT2[t1]: <-- OK
Regards
Alvaro
|-----------------------------------------------------------------------------------------------------------------|
Envíe y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier
celular y Nextel
en el Perú, México y en mas de 180 paises. Use aplicaciones 2 vias via
SMS y GPRS online
Visitenos en www.perusms.NET www.smsglobal.com.mx y
www.pravcom.com
2010/3/26 Nikos Balkanas <[email protected]>:
> Pretty descent system. I hope it isn't the system with the fried motherboard
> :-)
>
> Can you post again the output of top, or better mpstat or sar? Seems someone
> deleted this from the thread.
>
> 8% is reasonably low under traffic. However, when there is none, it is a
> cause for concern. I presume you have maximum level logs and nothing shows
> there either. Something strange is happening with your setup. In my case,
> starting up bearerbox without *any* traffic is @0% CPU as expected. Try that
> on another server with no traffic.
>
> You should run a process debugger, truss in Solaris, to see what system
> calls it makes during that 8%. Maybe the periodic polling of your modem
> bank? Are you using SIM buffering?
>
> BR,
> Nikos
> ----- Original Message ----- From: "Alvaro Cornejo"
> <[email protected]>
> To: "Nikos Balkanas" <[email protected]>
> Cc: <[email protected]>; "users" <[email protected]>
> Sent: Friday, March 26, 2010 11:11 PM
> Subject: Re: bearebox eating cpu time
>
>
> Hi Nikkos
>
> I do have a dual cpu dual core xeon with 4GB Ram
>
> [r...@srvcom1 ~]# dmesg | grep -i "hz"
> Detected 1596.575 MHz processor.
> hpet0: 3 64-bit timers, 14318180 Hz
> CPU0: Intel(R) Xeon(R) CPU 5110 @ 1.60GHz stepping 06
> CPU1: Intel(R) Xeon(R) CPU 5110 @ 1.60GHz stepping 06
> CPU2: Intel(R) Xeon(R) CPU 5110 @ 1.60GHz stepping 06
> CPU3: Intel(R) Xeon(R) CPU 5110 @ 1.60GHz stepping 06
> ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
> eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B1) PCI-X 64-bit 133MHz
> found at mem f8000000, IRQ 16, node addr 0015c5e7e0a1
> eth1: Broadcom NetXtreme II BCM5708 1000Base-T (B1) PCI-X 64-bit 133MHz
> found at mem f4000000, IRQ 16, node addr 0015c5e7e0a3
>
>
> Maybe I expressed the issue wrong.
>
> By eating cpu time I mean that even if there is no traffic MO/MT
> bearerbox process uses the same amount of cpu time.
>
> Regards
>
> Alvaro
>
> |-----------------------------------------------------------------------------------------------------------------|
> EnvΓe y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier
> celular y Nextel
> en el PerΓΊ, MΓ©xico y en mas de 180 paises. Use aplicaciones 2 vias via
> SMS y GPRS online
> Visitenos en www.perusms.NET www.smsglobal.com.mx y
> www.pravcom.com
>
>
>
> 2010/3/26 Nikos Balkanas <[email protected]>:
>>
>> Well, if you had a store filesystem problem, you would get waiting I/O CPU
>> in your top, which you don't. You can verify with:
>>
>> iostat 1 10
>>
>> Anyway, why do you say that 8% CPU is excessive? It seems normal to me.
>> How
>> much did it use to be? What is your CPU hardware?
>>
>> BR,
>> Nikos
>> ----- Original Message ----- From: "Alvaro Cornejo"
>> <[email protected]>
>> To: <[email protected]>
>> Cc: "users" <[email protected]>
>> Sent: Friday, March 26, 2010 4:43 AM
>> Subject: Re: bearebox eating cpu time
>>
>>
>> I monitor kannel queue in order to manage the amount of kannel queued
>> messages and be able to loadbalance the messages from different
>> customers. Therefore I never have more that 1000 messages in kannel
>> queue. Usually arround 400. I do have a pool of modems so that is
>> aprox a 5 min queue.
>>
>> Therefore I don't think the problem comes from there.
>>
>> Any other idea?
>>
>> Thanks
>>
>> Alvaro
>>
>>
>>
>> |-----------------------------------------------------------------------------------------------------------------|
>> EnvΞ½e y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier
>> celular y Nextel
>> en el PerΟ , MΞΉxico y en mas de 180 paises. Use aplicaciones 2 vias via
>> SMS y GPRS online
>> Β Β Β Β Β Β Visitenos en www.perusms.NET www.smsglobal.com.mx y
>> www.pravcom.com
>>
>>
>>
>> On Thu, Mar 25, 2010 at 9:57 PM, Β <[email protected]> wrote:
>>>
>>> You get the dubious benefit of losing your store if the server crashes ;)
>>>
>>> However, if you're using ext3 for the store and it grows too big, this is
>>> known to cause io problems which could result into high load. Using a
>>> separate ext2 partition for the store is actually a very good idea,
>>> though
>>> I'd check the rest of the setup for other possible bottlenecks as well.
>>>
>>> Regards,
>>>
>>> Alex
>>>
>>> BlackBerry de movistar, allΞ½ donde estΞΉs estΞ± tu oficin@
>>>
>>> ________________________________
>>> From: Alvaro Cornejo <[email protected]>
>>> Date: Thu, 25 Mar 2010 21:37:39 -0400
>>> To: Benaiad<[email protected]>
>>> Cc: users<[email protected]>
>>> Subject: Re: bearebox eating cpu time
>>> Hi Benaiad
>>>
>>> I don't get it. Why should I use ramdisk for MT?
>>>
>>> Do you mean move kannel to run from the ramdisk? What advantage might I
>>> get
>>> if the rest of the information (mysql) and my app is still running from
>>> HD?
>>>
>>> Regards
>>>
>>> Alvaro
>>>
>>>
>>>
>>>
>>> |-----------------------------------------------------------------------------------------------------------------|
>>> EnvΞ½e y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier
>>> celular y Nextel
>>> en el PerΟ , MΞΉxico y en mas de 180 paises. Use aplicaciones 2 vias via
>>> SMS
>>> y
>>> GPRS online
>>> Visitenos en www.perusms.NET www.smsglobal.com.mx y
>>> www.pravcom.com
>>>
>>>
>>> On Thu, Mar 25, 2010 at 9:05 PM, Benaiad <[email protected]> wrote:
>>>>
>>>> Hi Alvaro,
>>>> I suggest you to try ramdisk for MT files.
>>>> Regards.
>>>> --
>>>> Benaiad
>>>
>>
>>
>
>
