On Tue, Feb 11, 2003 at 10:08:29AM -0800, Aaron Bannert wrote: > A couple questions and a couple observations: > > 1) How many network cards are in the server? (How many unique > interrupt handlers for the network?)
There are two network cards in the server, but I don't think they are used in my experiment. I am running the server/client on the same machine. So I don't think the real network interface, like hme0 is used at all. > 2) How much context switching was going on, and how impacted > were the mutexes (see mpstat)? for 100 threads and 3us sleep time, here is a typical output for 'mpstat 5'. CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 33 17 6543 748 300 1694 357 289 679 24 4744 14 33 21 33 1 37 16 6164 623 100 2162 498 356 688 29 5122 10 34 19 37 4 32 18 5723 700 100 2449 556 361 681 21 5431 9 33 17 42 5 24 14 5648 721 100 2648 581 364 663 18 5597 12 32 18 39 12 23 11 5802 756 102 2827 612 419 663 18 5629 11 24 15 50 13 21 15 3507 791 100 2923 646 416 657 20 5708 8 29 20 43 16 25 16 2268 803 100 3025 662 434 651 22 5794 9 31 18 42 17 23 11 5944 1186 571 2646 590 421 562 20 5207 11 31 18 41 20 30 17 4773 637 100 2212 515 402 660 27 5125 8 33 21 39 21 31 17 4582 615 100 2196 495 369 700 24 5281 11 30 18 41 24 31 16 3647 647 100 2359 514 355 709 24 5359 10 28 20 42 25 25 13 3780 706 100 2655 574 362 695 18 5536 13 27 21 39 28 27 14 3800 711 100 2686 582 391 666 19 5524 10 27 17 46 29 26 14 3344 821 100 2980 675 423 649 19 5768 9 30 21 40 CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl 0 34 17 4135 701 299 1596 315 265 670 24 4681 7 21 22 50 1 37 16 3130 523 100 1989 404 331 658 23 5220 6 20 20 54 4 29 12 6719 563 100 2236 448 355 662 21 5212 5 17 21 56 5 26 13 2100 639 100 2498 520 355 606 20 5495 8 15 18 60 12 26 8 3041 614 102 2593 486 372 621 20 5603 4 18 14 64 13 21 12 3256 664 100 2808 541 411 620 18 5561 6 15 18 61 16 19 11 3042 616 100 2768 491 439 556 13 5542 4 17 16 63 17 22 6 1554 1021 480 2777 519 455 570 18 5521 4 17 8 71 20 31 11 5914 552 100 2157 433 414 644 16 5290 5 20 15 60 21 31 17 3986 513 100 2012 396 343 682 20 5221 5 22 21 51 24 33 12 4072 520 100 2105 404 349 683 20 5118 4 19 20 57 25 27 14 2756 606 100 2478 488 366 632 20 5459 4 19 20 57 28 26 13 6575 646 100 2659 521 380 620 14 5353 5 16 23 56 29 24 10 3818 652 100 2708 520 403 615 18 5552 5 19 15 61 The context switch is a bit high. smtx have been reduced, it use to have more smtx due to client synchronizations. Totally two httpd is started: xu 26557 13.5 0.4 9904 7760 ? O 13:25:32 1:44 /local.pbr.2/static-web/apache/bin/httpd -k start xu 26587 0.0 0.1 1080 784 pts/3 S 13:26:12 0:00 grep httpd xu 26512 0.0 0.2 4568 3080 ? S 13:19:22 0:00 /local.pbr.2/static-web/apache/bin/httpd -k start xu 26513 0.0 0.1 4344 1456 ? S 13:19:22 0:00 /local.pbr.2/static-web/apache/bin/httpd -k start server config: <IfModule worker.c> StartServers 5 ServerLimit 16 MinSpareThreads 1 MaxSpareThreads 1 ThreadsPerChild 64 # MaxClients = ServerLimit * ThreadsPerChild MaxClients 1024 MaxRequestsPerChild 100000 </IfModule> > 3) Was the workload uniformly distributed across the CPUs? I think so. (see mpstat) > I've seen large MP systems completely fail to distribute the > workload, and suffer because of it. My current theory for why > this occurs is that the interrupt load is overwhelming the CPU > where that interrupt is being serviced. This combined with the > relatively small amount of userspace work that must be done to > push a small static file out is wreaking havoc on the scheduler > (and it's probably more dramatic if your system enabled sendfile > support). I think sendfile is enabled (is it default?) in apache too. -Min
