Hi, Nigel
It turned out that pronlem was with I/OAT DMA.
My server IBM x3650 has no any I/OAT DMA settings, so I went other way and
googled for I/OAT stuff in OpenSolaris. Finally I came to that post:
http://www.alekz.net/archives/123
And disabled ioat in the /etc/system.
It really helped!
So, back to tests you've asked:
Here is what scanpci said:
pci bus 0x0005 cardnum 0x00 function 0x00: vendor 0x1166 device 0x0103
Broadcom EPB PCI-Express to PCI-X Bridge
pci bus 0x0006 cardnum 0x00 function 0x00: vendor 0x14e4 device 0x164c
Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet
Now procsystime:
Good case:
bash-3.2# ./procsystime -a -n iscsitgtd
Hit Ctrl-C to stop sampling...
^C
Elapsed Times for processes iscsitgtd,
SYSCALL TIME (ns)
brk 31746
fcntl 20165107
write 103080806
pread 309323470
yield 497170160
kaio 5352818217
pollsys 7547104605
pwrite 9221540368
recv 10827185376
lwp_park 371822753313
TOTAL: 405701173168
CPU Times for processes iscsitgtd,
SYSCALL TIME (ns)
brk 29784
pread 4961359
fcntl 7349732
yield 9507158
pollsys 38460877
kaio 61915421
write 84095636
lwp_park 513784651
recv 1280704673
pwrite 1843377361
TOTAL: 3844186652
Syscall Counts for processes iscsitgtd,
SYSCALL COUNT
brk 1
pread 117
yield 1432
fcntl 6014
pwrite 6014
pollsys 8039
write 8156
kaio 12098
recv 14053
lwp_park 83459
TOTAL: 139383
bash-3.2#
With enabled ioat I even couldn't copy os200805.iso to iscsi lun, and here is
output:
bash-3.2# ./procsystime -a -n iscsitgtd
Hit Ctrl-C to stop sampling...
^C
Elapsed Times for processes iscsitgtd,
SYSCALL TIME (ns)
lwp_mutex_timedlock 7887
getsockname 29730
lwp_kill 47971
ioctl 70967
setcontext 129278
lwp_sigmask 130555
read 162712
munmap 181442
schedctl 201043
lwp_continue 230005
close 244764
setsockopt 386370
brk 412561
mmap 891651
lwp_wait 1060983
open 1298203
lwp_create 1701625
fcntl 14821973
yield 30768089
write 67366053
gtime 944859766
pread 1408797954
recv 6590864698
kaio 8197202654
pwrite 36215168090
pollsys 312898353322
accept 407648188282
nanosleep 4179563291902
lwp_park 6863882243838
TOTAL: 11817469114368
CPU Times for processes iscsitgtd,
SYSCALL TIME (ns)
lwp_mutex_timedlock 5745
getsockname 12968
lwp_kill 16511
setcontext 35740
lwp_sigmask 45508
ioctl 55125
schedctl 109515
lwp_wait 128850
read 133898
lwp_continue 138737
munmap 173188
close 227311
brk 306202
setsockopt 320206
mmap 682719
accept 728868
open 1263713
lwp_create 1596839
yield 4738553
fcntl 6395223
pollsys 17947253
kaio 44237731
pread 51813946
write 56149372
gtime 223258209
recv 723804628
lwp_park 1102259910
pwrite 2337317015
nanosleep 2525019005
TOTAL: 7098922488
After removing ioat driver everything works nice again.
Great thanks to you and Tano for helping!
Eugene
--
This message posted from opensolaris.org
_______________________________________________
storage-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
