My MythTV box recently started showing odd behavior during recordings, at certain times the load of the box would spike to 10+ and recordings would start losing frames and become unwatchable. TOP would show mythbackend as using 90+% SYS CPU usage, which under normal circumstances it normally uses about 5% USR.
So I finally got around to profiling mythbackend when the load starts to
spike. To my surprise it appears that once I have less then 10% (30GB)
free on the drive reiserfs can't up, even just writing at 1mb/sec is too
much for it.
Is there something that can be done to fix this, 30gb seems like a lot
of wasted space.
#opreport
CPU: CPU with timer interrupt, speed 0 MHz (estimated)
Profiling through timer interrupt
TIMER:0|
samples| %|
------------------
77863 78.7856 reiserfs
18183 18.3984 vmlinux
695 0.7032 mysqld
452 0.4574 libc-2.4.so
360 0.3643 libmythtv-0.19.so.0.19.0
324 0.3278 ivtv
323 0.3268 nvidia
242 0.2449 libqt-mt.so.3.3.6
110 0.1113 libpthread-2.4.so
53 0.0536 libstdc++.so.6.0.8
35 0.0354 ld-2.4.so
23 0.0233 libperl.so
22 0.0223 libz.so.1.2.3
<snip>
#opreport -l /usr/src/linux/vmlinux
CPU: CPU with timer interrupt, speed 0 MHz (estimated)
Profiling through timer interrupt
samples % symbol name
9607 52.8351 default_idle
7694 42.3142 find_next_zero_bit
183 1.0064 __copy_from_user_ll
57 0.3135 handle_IRQ_event
37 0.2035 __copy_to_user_ll
34 0.1870 ide_outb
30 0.1650 ide_end_request
22 0.1210 ioread8
22 0.1210 schedule
21 0.1155 get_page_from_freelist
17 0.0935 mmx_clear_page
<snip>
System Details:
-----------------------------------------------
Kernel v2.6.16.21 (custom compiled)
- This issue also happened with 2.6.14 too though.
Filesystem Size Used Avail Use% Mounted on
/dev/hda1 280G 269G 12G 97% /
[EMAIL PROTECTED] cat /proc/mounts
rootfs / rootfs rw 0 0
/dev /dev tmpfs rw 0 0
/dev/root / reiserfs rw,noatime,nodiratime 0 0
[EMAIL PROTECTED] cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 6
model : 6
model name : AMD Athlon(tm) XP 2100+
stepping : 2
cpu MHz : 1759.680
cache size : 256 KB
[EMAIL PROTECTED] free
total used free shared buffers
cached
Mem: 515992 496256 19736 0 36256
271728
-/+ buffers/cache: 188272 327720
Swap: 262136 408 261728
[EMAIL PROTECTED] ~]# hdparm -i /dev/hda
/dev/hda:
Model=ST3300622A, FwRev=3.AND, SerialNo=3NF1GAGW
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
BuffType=unknown, BuffSize=16384kB, MaxMultSect=16, MultSect=16
CurCHS=4047/16/255, CurSects=16511760, LBA=yes, LBAsects=268435455
IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5
AdvancedPM=no WriteCache=enabled
Drive conforms to: Unspecified: ATA/ATAPI-1 ATA/ATAPI-2 ATA/ATAPI-3
ATA/ATAPI-4 ATA/ATAPI-5 ATA/ATAPI-6 ATA/ATAPI-7
* signifies the current active mode
[EMAIL PROTECTED] ~]# hdparm -tT /dev/hda
/dev/hda:
Timing cached reads: 1296 MB in 2.00 seconds = 646.99 MB/sec
Timing buffered disk reads: 166 MB in 3.02 seconds = 55.05 MB/sec
vmstat 1 output:
--------------------------------------------------------------
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us sy
id wa
8 0 408 5800 29308 248604 0 0 0 1036 406 132 2 98
0 0
4 0 408 5644 29396 248608 0 0 0 1128 437 184 2 92
0 6
7 0 408 6316 29428 248020 0 0 0 1316 539 287 0 86
0 14
5 0 408 6104 29480 248180 0 0 0 588 415 187 0 99
0 1
4 0 408 5764 29536 248364 0 0 0 1092 421 172 2 97
1 0
6 0 408 6528 29592 247684 0 0 0 1092 425 161 2 98
0 1
2 1 408 6372 29676 247724 0 0 0 2304 385 170 2 97
1 0
5 0 408 6400 29676 247616 0 0 0 48 383 122 0
100 0 0
7 0 408 6192 29704 247872 0 0 0 1080 409 162 1 98
0 1
6 0 408 5720 29732 248304 0 0 0 1076 414 178 1 98
0 1
7 0 408 6348 29800 247552 0 0 0 1656 460 300 2 87
1 11
5 0 408 6628 29848 247248 0 0 0 1164 407 207 1 94
0 5
5 0 408 5884 29896 247996 0 0 4 1116 453 353 1 76
0 23
6 0 408 5640 29868 248204 0 0 0 1052 416 132 1 99
0 0
4 0 408 5772 29940 248104 0 0 0 648 490 314 1 84
1 14
6 1 408 6328 30036 247464 0 0 0 1928 488 305 2 85
0 13
4 0 408 6184 30076 247472 0 0 4 860 404 201 1 94
0 5
4 0 408 6332 30044 247328 0 0 0 1312 429 156 1 99
0 0
9 0 408 6120 30100 247580 0 0 0 604 494 305 3 81
1 16
2 1 408 6460 30140 247116 0 0 0 1372 436 315 1 79
0 20
10 0 408 6252 30176 247372 0 0 0 456 412 126 1 99
0 0
6 0 408 6432 30164 247276 0 0 4 1268 425 255 1 88
1 10
3 0 408 5688 30220 247948 0 0 0 1332 454 352 0 78
0 22
2 1 408 6352 30284 247124 0 0 0 1140 362 156 2 96
1 1
5 0 408 6564 30284 246908 0 0 0 92 472 316 2 83
0 15
5 0 408 6348 30352 247056 0 0 0 1168 506 350 0 83
0 17
4 0 408 5604 30404 247828 0 0 4 1124 448 262 2 87
0 11
3 0 408 5880 30444 247500 0 0 0 1104 426 315 2 77
1 20
2 1 408 5916 30496 247352 0 0 0 1064 365 152 1 97
0 2
7 0 408 6072 30496 247204 0 0 0 440 489 307 1 82
0 17
6 0 408 5936 30528 247288 0 0 0 816 434 130 2 98
0 0
4 0 408 5944 30588 247300 0 0 0 1108 359 172 0 98
0 2
4 0 408 5664 30624 247508 0 0 0 1444 426 161 0 99
1 0
5 0 408 6656 30608 246572 0 0 0 1220 425 163 2 98
0 1
6 0 408 6316 30656 246848 0 0 0 1552 441 180 1 98
0 1
4 0 408 6408 30632 246776 0 0 0 644 403 140 1 99
0 0
9 0 408 6072 30696 247060 0 0 4 744 496 351 2 82
1 16
5 0 408 5864 30708 247240 0 0 0 1680 509 335 1 83
1 15
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us sy
id wa
3 1 408 6284 30768 246768 0 0 0 1132 434 328 1 76
1 22
6 0 408 6352 30772 246692 0 0 0 576 373 170 0 93
0 7
4 0 408 6008 30820 246932 0 0 4 612 496 322 1 83
0 16
4 0 408 6288 30836 246600 0 0 0 1484 480 304 1 85
0 14
4 0 408 6064 30896 246844 0 0 0 1136 504 337 1 84
1 15
5 0 408 5728 30900 247116 0 0 4 1188 426 156 1 99
0 0
6 0 408 5696 30968 247144 0 0 0 1104 367 123 3 97
0 0
4 0 408 5608 31016 247144 0 0 0 1152 445 378 2 74
1 23
7 0 408 5576 31008 247088 0 0 0 964 402 115 1 99
0 0
4 0 408 6328 31052 246396 0 0 0 628 355 152 1 98
0 1
5 0 408 6116 31112 246524 0 0 0 1620 472 299 2 85
1 12
2 1 408 6336 31204 246176 0 0 0 1112 367 156 2 96
0 2
7 0 408 6388 31176 246192 0 0 0 76 457 272 0 86
0 14
5 0 408 6268 31232 246284 0 0 0 1136 466 267 1 85
1 13
2 1 408 5932 31304 246616 0 0 4 2068 374 173 1 99
0 0
6 0 408 5960 31224 246564 0 0 0 104 472 273 1 84
0 15
6 0 408 5692 31308 246716 0 0 0 1160 412 206 2 94
0 4
5 0 408 5600 31336 246892 0 0 4 1660 480 289 2 86
0 12
7 0 408 6400 31336 245964 0 0 0 1052 418 160 3 97
0 0
6 0 408 6316 31292 246136 0 0 0 512 432 127 1 99
0 0
5 0 408 5856 31372 246528 0 0 0 1824 404 159 2 96
0 2
3 0 408 5880 31424 246412 0 0 0 1156 454 174 1 97
1 1
3 0 408 6024 31372 246336 0 0 0 896 399 130 0
100 0 0
5 0 408 5812 31432 246492 0 0 0 708 413 160 1 97
0 2
5 0 408 6396 31424 246024 0 0 0 1604 436 163 1 97
1 1
6 1 408 6276 31492 245924 0 0 216 1176 511 409 3 82
0 15
4 0 408 6312 31528 245944 0 0 0 1116 468 263 1 86
0 13
1 2 408 6592 31576 245628 0 0 56 1044 343 126 0 97
0 3
5 0 408 6312 31576 245904 0 0 32 48 427 155 0 97
0 3
1 0 408 5816 31624 246360 0 0 72 1796 590 834 2 40
35 24
1 1 408 16872 31704 247564 0 0 1232 248 513 1185 28 4
11 57
1 1 408 31240 31768 248520 0 0 932 92 403 996 32 4
10 54
1 0 408 29576 31880 248704 0 0 188 248 372 997 7 6
61 26
1 1 408 28284 31952 249852 0 0 316 344 402 842 20 21
45 13
0 1 408 27188 32008 250940 0 0 112 976 393 465 33 58
0 9
5 1 408 24748 32100 253228 0 0 1212 1424 571 949 31 31
0 37
2 0 408 23052 32156 255032 0 0 544 1036 415 351 16 80
0 4
0 1 408 21148 32232 256808 0 0 516 1480 454 692 33 41
0 25
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us sy
id wa
2 1 408 19616 32288 258308 0 0 576 1352 414 478 33 59
0 8
4 0 408 18084 32348 259816 0 0 496 1344 423 524 29 56
0 15
5 0 408 17016 32428 260844 0 0 192 812 518 574 24 63
0 13
2 0 408 15348 32488 262444 0 0 208 1064 416 295 14 85
0 1
5 0 408 13616 32552 264104 0 0 84 1684 497 615 32 66
0 2
5 1 408 13496 32612 263992 0 0 92 1148 530 526 14 71
0 14
0 1 408 13000 32784 264556 0 0 80 1240 506 504 1 59
0 40
3 1 408 12132 32864 265324 0 0 36 612 431 438 2 65
0 34
1 1 408 10196 33048 266960 0 0 216 4 440 565 1 60
0 39
1 1 408 9252 33284 267768 0 0 168 2444 463 617 1 56
0 43
0 3 408 7208 33376 269680 0 0 32 3460 459 497 1 59
0 40
2 1 408 6416 33444 270392 0 0 24 748 448 423 0 71
0 29
0 1 408 5976 33664 270568 0 0 220 1436 481 654 2 55
0 43
1 0 408 6100 33700 270356 0 0 8 844 406 389 9 70
16 5
0 0 408 5848 33732 270568 0 0 0 1128 435 401 0 72
27 1
1 0 408 5720 33772 270664 0 0 0 852 398 350 1 73
25 1
1 0 408 6100 33780 270320 0 0 0 1216 446 522 0 54
45 1
3 0 408 5736 33780 270644 0 0 0 1092 475 736 0 32
67 1
1 0 408 6372 33952 269720 0 0 0 1040 462 522 4 69
26 1
2 0 408 6436 33944 269592 0 0 0 864 433 287 0 83
16 1
0 0 408 5848 34024 270140 0 0 4 1232 480 701 3 39
53 5
2 0 408 9196 33936 266612 0 0 104 212 596 1035 10 43
40 8
3 0 408 8824 33936 267380 0 0 0 512 388 90 0
100 0 0
4 0 408 7956 33968 268148 0 0 0 548 400 114 1 98
0 1
2 0 408 6492 34000 269604 0 0 0 892 432 629 0 38
61 1
2 0 408 6416 34084 269648 0 0 0 1712 403 591 0 40
58 2
5 0 408 6612 34120 269376 0 0 0 844 447 557 1 49
49 1
4 0 408 6424 34148 269548 0 0 0 880 465 493 0 65
35 0
1 0 408 6336 34196 269596 0 0 0 1112 475 552 3 59
36 2
4 1 408 6304 34340 269404 0 0 0 1668 378 316 0 78
22 0
3 0 408 6096 34368 269608 0 0 0 308 411 625 1 38
59 2
3 0 408 6268 34412 269372 0 0 0 1148 398 583 0 39
60 1
5 0 408 6400 34444 269264 0 0 0 824 431 414 0 67
33 0
--
Mike Benoit <[EMAIL PROTECTED]>
signature.asc
Description: This is a digitally signed message part
