Re: Performance problems with pagedaemon
On Mon, Jan 02, 2012 at 10:17:11AM -0800, Artem Belevich wrote: On Mon, Jan 2, 2012 at 5:41 AM, Victor Balada Diaz vic...@bsdes.net wrote: Mysql uses more than 20G of RAM. You may want to tune it down a bit so that there is a bit of free RAM around. Page daemon is trying to maintain v_free_target + v_cache_min. vm.v_free_target: 161771 vm.v_cache_min: 161771 In your case that would be about 1.2GB. If 'v_free_count + v_cache_count' are below that page daemon will periodically wake up and will start scanning active/inactive lists trying to find the pages it could use. In your case, when most of the memory is in active use, page daemon's job will be almost pointless and would just waste CPU time. On large memory systems default tuning for v_free/cache_min/target is probably somewhat conservative. You may try setting them somewhat lower via sysctl and see if you can find an equilibrium with mysql happy, pagedaemon sleeping and the system up and running. The danger of tuning these parameters too low is that is you don't have enough memory available for allocation without having to sleep, things will start falling apart and will eventually hang or crash your box. That was the problem. Thanks a lot! Is there any place or documentation about how to properly tune the VM? None of the sysctls you mentioned have description or appear to be in any man page. -- La prueba más fehaciente de que existe vida inteligente en otros planetas, es que no han intentado contactar con nosotros. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance problems with pagedaemon
On Tue, Jan 3, 2012 at 12:05 AM, Victor Balada Diaz vic...@bsdes.net wrote: On Mon, Jan 02, 2012 at 10:17:11AM -0800, Artem Belevich wrote: On Mon, Jan 2, 2012 at 5:41 AM, Victor Balada Diaz vic...@bsdes.net wrote: Mysql uses more than 20G of RAM. You may want to tune it down a bit so that there is a bit of free RAM around. Page daemon is trying to maintain v_free_target + v_cache_min. vm.v_free_target: 161771 vm.v_cache_min: 161771 In your case that would be about 1.2GB. If 'v_free_count + v_cache_count' are below that page daemon will periodically wake up and will start scanning active/inactive lists trying to find the pages it could use. In your case, when most of the memory is in active use, page daemon's job will be almost pointless and would just waste CPU time. On large memory systems default tuning for v_free/cache_min/target is probably somewhat conservative. You may try setting them somewhat lower via sysctl and see if you can find an equilibrium with mysql happy, pagedaemon sleeping and the system up and running. The danger of tuning these parameters too low is that is you don't have enough memory available for allocation without having to sleep, things will start falling apart and will eventually hang or crash your box. That was the problem. Thanks a lot! Is there any place or documentation about how to properly tune the VM? None of the sysctls you mentioned have description or appear to be in any man page. They probably should be mentioned in tuning(7). Thanks, -Garrett ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance problems with pagedaemon
Hi, there's some effort to improve tuning(7). If somone could go over to http://wiki.freebsd.org/SystemTuning and add some sentences, it will make the life of other people maybe a little bit more easy Bye, Alexander. -- Send via an Android device, please forgive brevity and typographic and spelling errors. Garrett Cooper yaneg...@gmail.com hat geschrieben:On Tue, Jan 3, 2012 at 12:05 AM, Victor Balada Diaz vic...@bsdes.net wrote: On Mon, Jan 02, 2012 at 10:17:11AM -0800, Artem Belevich wrote: On Mon, Jan 2, 2012 at 5:41 AM, Victor Balada Diaz vic...@bsdes.net wrote: Mysql uses more than 20G of RAM. You may want to tune it down a bit so that there is a bit of free RAM around. Page daemon is trying to maintain v_free_target + v_cache_min. vm.v_free_target: 161771 vm.v_cache_min: 161771 In your case that would be about 1.2GB. If 'v_free_count + v_cache_count' are below that page daemon will periodically wake up and will start scanning active/inactive lists trying to find the pages it could use. In your case, when most of the memory is in active use, page daemon's job will be almost pointless and would just waste CPU time. On large memory systems default tuning for v_free/cache_min/target is probably somewhat conservative. You may try setting them somewhat lower via sysctl and see if you can find an equilibrium with mysql happy, pagedaemon sleeping and the system up and running. The danger of tuning these parameters too low is that is you don't have enough memory available for allocation without having to sleep, things will start falling apart and will eventually hang or crash your box. That was the problem. Thanks a lot! Is there any place or documentation about how to properly tune the VM? None of the sysctls you mentioned have description or appear to be in any man page. They probably should be mentioned in tuning(7). Thanks, -Garrett ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance problems with pagedaemon
On Mon, Jan 02, 2012 at 01:25:49PM +0100, Victor Balada Diaz wrote: Hello, First: Happy new year to everybody! Last year i had a problem with pagedaemon that i reported here[1] but i had no replies. The problem went away and now it's back to beat me again. Seems that mysql creates some bad interactions with pagedaemon and system starts to get slow for disk access. Eg, accessing a man page takes a few seconds. If i stop mysql and start it again, it recovers again. As this is a problem that have appeared just twice (on October and now), it's not easy to reproduce and test. Disk I/O as per gstat is not high, not many context switches, etc. Bottleneck seems to be CPU. I've reported more information on october's mail[1]. What can i do to debug it? Seems that maybe the most interesting thing would be enabling lock profiling and wait for the system to fail again, but is there anything that i can do/report that would be helpful to track the problem down now that's failing? Thanks a lot. Regards. [1]: http://lists.freebsd.org/pipermail/freebsd-stable/2011-October/064248.html The only things I can think of are output from the following: * vmstat -z * sysctl -a vm * top (mainly for mysqld) * top -H -S Is this machine in question still running 8.1-RELEASE? If so, have you tried upgrading it to 8.2-STABLE? I believe there have been some VM adjustments, but I'd have to go back and examine commits. Taken from our mysql server. Note that the mysql storage filesystem is stored on ZFS as well, which has added benefits/complexities yet we see no issue. FreeBSD X 8.2-STABLE FreeBSD 8.2-STABLE #0: Tue Dec 13 00:00:16 PST 2011 root@X:/usr/obj/usr/src/sys/X7SBI_RELENG_8_amd64 amd64 real memory = 8589934592 (8192 MB) avail memory = 8250040320 (7867 MB) Mem: 539M Active, 2559M Inact, 3218M Wired, 4K Cache, 828M Buf, 1603M Free Swap: 16G Total, 16G Free PID USERNAME THR PRI NICE SIZERES STATE C TIME WCPU COMMAND 1940 mysql 16 200 781M 401M sigwai 1 158:40 0.00% [mysqld] top -H -S shows no indication pagedaemon is going crazy. loader.conf tunings: # Increase maximum allocatable memory on a process to 2.5GB. # We don't choose 8GB (our amount of RAM) since that would # exhaust all memory, and result in a kernel panic. Maximum # stack size is set to 256MB. One can view these settings # using limits(1). # # dfldsiz = Initial data size limit (bytes) # maxdsiz = Maximum data size limit (bytes) # dflssiz = Initial stack size limit (bytes) # maxssiz = Maximum stack size limit (bytes) # kern.maxdsiz=2560M kern.dfldsiz=2560M kern.maxssiz=256M # Limit ZFS ARC maximum. # NOTE #1: In 8.2-RELEASE and onward, vm.kmem_size_scale defaults to 1, # which means vm.kmem_size should match the amount of RAM installed # in the system. If using an earlier FreeBSD release, be sure to set # vm.kmem_size manually to the amount of RAM you have. # NOTE #2: Do not set vm.kmem_size to 2x that of physical RAM, otherwise # vfs.zfs.arc_max effectively becomes halved. # http://lists.freebsd.org/pipermail/freebsd-fs/2011-March/010875.html vfs.zfs.arc_max=5120M my.cnf tunings: [mysqld] ; http://dev.mysql.com/doc/refman/5.5/en/table-cache.html table_open_cache= 2048 max_connections = 512 tmp_table_size = 64M max_allowed_packet = 32M key_buffer_size = 64M join_buffer_size= 16M sort_buffer_size= 16M read_buffer_size= 16M query_cache_size= 64M query_cache_limit = 32M innodb_file_per_table innodb_buffer_pool_size = 512M innodb_log_file_size= 128M innodb_log_buffer_size = 8M innodb_additional_mem_pool_size = 20M sysctl -a vm output: vm.vmtotal: System wide totals computed every five seconds: (values in kilobytes) === Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 Sleep: 59) Virtual Memory: (Total: 1075323828K Active: 1483408K) Real Memory:(Total: 2970948K Active: 585264K) Shared Virtual Memory: (Total: 17884K Active: 6756K) Shared Real Memory: (Total: 7500K Active: 6564K) Free Memory Pages: 1641708K vm.loadavg: { 0.00 0.00 0.00 } vm.v_free_min: 12824 vm.v_free_target: 53985 vm.v_free_reserved: 2689 vm.v_inactive_target: 80977 vm.v_cache_min: 53985 vm.v_cache_max: 107970 vm.v_pageout_free_min: 34 vm.pageout_algorithm: 0 vm.swap_enabled: 1 vm.md_malloc_wait: 0 vm.kmem_map_free: 2897080320 vm.kmem_map_size: 2349465600 vm.kmem_size_scale: 1 vm.kmem_size_max: 329853485875 vm.kmem_size_min: 0 vm.kmem_size: 8303726592 vm.nswapdev: 1 vm.dmmax: 32 vm.swap_async_max: 4 vm.overcommit: 0 vm.swap_reserved: 11426652160 vm.swap_total: 17179869184 vm.zone_count: 195 vm.swap_idle_threshold2: 10 vm.swap_idle_threshold1: 2 vm.kstacks: 475 vm.kstack_cache_size: 128 vm.exec_map_entries: 16 vm.stats.misc.zero_page_count: 158 vm.stats.misc.cnt_prezero: 0
Re: Performance problems with pagedaemon
Thanks a lot for your fast reply Jeremy! On Mon, Jan 02, 2012 at 04:59:29AM -0800, Jeremy Chadwick wrote: On Mon, Jan 02, 2012 at 01:25:49PM +0100, Victor Balada Diaz wrote: Hello, First: Happy new year to everybody! Last year i had a problem with pagedaemon that i reported here[1] but i had no replies. The problem went away and now it's back to beat me again. Seems that mysql creates some bad interactions with pagedaemon and system starts to get slow for disk access. Eg, accessing a man page takes a few seconds. If i stop mysql and start it again, it recovers again. As this is a problem that have appeared just twice (on October and now), it's not easy to reproduce and test. Disk I/O as per gstat is not high, not many context switches, etc. Bottleneck seems to be CPU. I've reported more information on october's mail[1]. What can i do to debug it? Seems that maybe the most interesting thing would be enabling lock profiling and wait for the system to fail again, but is there anything that i can do/report that would be helpful to track the problem down now that's failing? Thanks a lot. Regards. [1]: http://lists.freebsd.org/pipermail/freebsd-stable/2011-October/064248.html The only things I can think of are output from the following: * vmstat -z * sysctl -a vm * top (mainly for mysqld) * top -H -S Attached at the end. Is this machine in question still running 8.1-RELEASE? If so, have you tried upgrading it to 8.2-STABLE? I believe there have been some VM adjustments, but I'd have to go back and examine commits. 8.1-RELEASE-p5, yes. I didn't try updating to 8.2, but i'll try upgrading to 9.0 once it's out. Once i upgrade i think it will take a few months until i can say if it's fixed or not. I must say that this problem just appeared twice on all of our servers. The rest have been rock solid with 8.1 RELEASE-p* and no problems in a long time. Taken from our mysql server. Note that the mysql storage filesystem is stored on ZFS as well, which has added benefits/complexities yet we see no issue. We're using UFS for filesystem with RAID 10 in HW, controller is Adaptec 5405. So far we've just tested ZFS on FreeBSD for our backup servers and don't feel it's a good choice for our database needs as it wastes much memory and don't get us enough benefits. Out of curiosity: why did you prefer ZFS over UFS for big databases? The biggest reason i can think of, is snapshots and backups, but with xtrabackup available and in ports, it's not that useful. vmstat -z: ITEM SIZE LIMIT USED FREE REQUESTS FAILURES UMA Kegs: 208,0, 104, 15, 104,0 UMA Zones:576,0, 104,4, 104,0 UMA Slabs:568,0, 3880, 495, 463422,0 UMA RCntSlabs:568,0, 3006, 592, 682012,0 UMA Hash: 256,0,0,0,3,0 16 Bucket:152,0, 47, 153, 204,0 32 Bucket:280,0, 130, 122, 340,2 64 Bucket:536,0, 94, 95, 583, 86 128 Bucket: 1048,0, 894, 15, 153173, 287797 VM OBJECT:216,0,33316,50834, 86207077,0 MAP: 232,0,7, 25,7,0 KMAP ENTRY: 120, 786315, 80, 1346, 6366241,0 MAP ENTRY:120,0, 1590, 2347, 212747051,0 DP fakepg:120,0,0, 124, 21,0 SG fakepg:120,0,0,0,0,0 mt_zone: 2056,0, 260,3, 260,0 16:16,0, 2336, 1696, 6042441,0 32:32,0, , 2424, 426544,0 64:64,0, 8137, 1271, 986811990,0 128: 128,0,15488, 3826, 25899478,0 256: 256,0, 581, 2059, 47822165,0 512: 512,0, 2972, 1949, 9053615,0 1024:1024,0, 54, 358,24855,0 2048:2048,0, 56, 266,41473,0 4096:4096,0, 396, 2047, 3139352,0 Files: 80,0, 487, 2258, 54800616,0 TURNSTILE:136,0, 4045, 435, 4413,0 umtx pi: 96,0,0,0,0,0 MAC labels:40,0,0,0,
Re: Performance problems with pagedaemon
On Mon, Jan 2, 2012 at 5:41 AM, Victor Balada Diaz vic...@bsdes.net wrote: ... System wide totals computed every five seconds: (values in kilobytes) === Processes: (RUNQ: 2 Disk Wait: 0 Page Wait: 0 Sleep: 51) Virtual Memory: (Total: 1098017100K, Active 24065448K) Real Memory: (Total: 21157668K Active 20971144K) Shared Virtual Memory: (Total: 27740K Active: 8632K) Shared Real Memory: (Total: 6128K Active: 4928K) Free Memory Pages: 315636K On a system with 24GB of RAM you seem to have almost all of it active. It appears that you're simply on the edge of running out of memory and thus page daemon wakes up constantly trying to find more pages... ... Top: last pid: 24777; load averages: 3.26, 4.07, 4.49 up 34+19:43:58 14:32:05 66 processes: 5 running, 61 sleeping CPU: 0.2% user, 0.0% nice, 37.6% system, 0.0% interrupt, 62.2% idle Mem: 18G Active, 1908M Inact, 3008M Wired, 73M Cache, 2465M Buf, 232M Free Swap: 4096M Total, 4096M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 1059 mysql 22 44 0 21004M 19741M ucond 3 71.2H 124.51% mysqld Mysql uses more than 20G of RAM. You may want to tune it down a bit so that there is a bit of free RAM around. Page daemon is trying to maintain v_free_target + v_cache_min. vm.v_free_target: 161771 vm.v_cache_min: 161771 In your case that would be about 1.2GB. If 'v_free_count + v_cache_count' are below that page daemon will periodically wake up and will start scanning active/inactive lists trying to find the pages it could use. In your case, when most of the memory is in active use, page daemon's job will be almost pointless and would just waste CPU time. On large memory systems default tuning for v_free/cache_min/target is probably somewhat conservative. You may try setting them somewhat lower via sysctl and see if you can find an equilibrium with mysql happy, pagedaemon sleeping and the system up and running. The danger of tuning these parameters too low is that is you don't have enough memory available for allocation without having to sleep, things will start falling apart and will eventually hang or crash your box. --Artem ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance AMD Phenom II X6 1090T
Yes, I agree. That was due to human error and me not paying enough attention to what I was doing. Pretty embarrassing :) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance issues with 8.0 ZFS and sendfile/lighttpd
2009/11/7 Jeremy Chadwick free...@jdc.parodius.com: Yes, loader values are one year old when I installed this machine. But I think auto tuning was commited after 7.2-RELEASE by Kip Macy, so some of them are still needed or am I wrong? (this is 7.2-RELEASE). ... We don't know, because none of the individuals who are maintaining ZFS at this point in time have actually responded to this question. http://lists.freebsd.org/pipermail/freebsd-stable/2009-October/052256.html Only as a data point and not suggesting anything official: I have managed to panic 8-STABLE with ZFS kmem exhaustion, so... *shrug* I'm still scared of using it. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance issues with 8.0 ZFS and sendfile/lighttpd
Ivan Voras wrote: 2009/11/6 Miroslav Lachman000.f...@quip.cz: I do not understand why there are 10MB/s read from disks when network traffic dropped to around 1MB/s (8Mbps) r...@cage ~/# iostat -w 20 tty ad4 ad6 cpu tin tout KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 14 41.66 53 2.17 41.82 53 2.18 0 0 2 0 97 0 18 50.92 96 4.77 54.82 114 6.12 0 0 3 1 96 06 53.52 101 5.29 54.98 108 5.81 1 0 4 1 94 06 54.82 98 5.26 55.89 108 5.89 0 0 3 1 96 Yes, this could limit your IO if the requests are random enough. Unfortunately I don't know how would you track down what is really going on. Maybe some tracing with DTrace? I'd tell you to use top -m io to see if there is a process responsible, but apparently these statistics are not updated for ZFS, which in itself may be a bug (which is why I'm crossposting to freebsd-fs). DTrace is totally out of my skills ;( There is otput of top -m io sorted by VCSW displaying JID. last pid: 17724; load averages: 0.01, 0.07, 0.08 up 74+20:49:49 21:03:40 195 processes: 1 running, 193 sleeping, 1 zombie CPU: 0.0% user, 0.0% nice, 3.6% system, 0.4% interrupt, 96.1% idle Mem: 462M Active, 2385M Inact, 977M Wired, 21M Cache, 399M Buf, 100M Free Swap: 6144M Total, 2024K Used, 6142M Free PID JID USERNAME VCSW IVCSW READ WRITE FAULT TOTAL PERCENT COMMAND 17681 8 www 657 64 0 0 0 0 0.00% lighttpd 17683 8 www 379 41 0 0 0 0 0.00% lighttpd 17680 8 www 136 5 0 0 0 0 0.00% lighttpd 17682 8 www 85 0 0 0 0 0 0.00% lighttpd 4689 1 90 10 0 0 0 0 0 0.00% fb_inet_server 3403 1 90 10 0 0 0 0 0 0.00% fb_inet_server 2632 1 90 10 0 0 0 0 0 0.00% fb_inet_server All four top consumers is Lighttpd workers. And as you noted, read, write, fault, total and percent are not updated on machine with ZFS, so I can't compare it with UFS2 based machine. Is this bug in top fixed in 8.x? Will you file a PR? (you know more about FS related things than me :]) Miroslav Lachman ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance issues with 8.0 ZFS and sendfile/lighttpd
2009/11/7 Miroslav Lachman 000.f...@quip.cz: And as you noted, read, write, fault, total and percent are not updated on machine with ZFS, so I can't compare it with UFS2 based machine. Is this bug in top fixed in 8.x? Will you file a PR? (you know more about FS related things than me :]) Not much... it depends on from where the stats are collected - there is a fair bit of file system infrastructure that ZFS bypasses and if these stats come from it, they cannot be collected. The stat is apparently updated around sys/kern/vfs_cluster.c: 233 . I'm not very familiar with this layer but since it uses struct buf and the ZFS doesn't use bufcache, this is probably one of the things that is bypassed, though it would be nice if it weren't since this code also defines and uses the vfs.write_behind and vfs.read_max sysctls. Also, since ZFS uses its own threads for IO and the stats are for curthread, it looks like it would maybe need careful work to actually assign the IO stats to the correct thread; otherwise it may be sufficient to add it to vdev_disk.c in vdev_disk_physio(). I don't really know this code and this is mostly mechanical analisys - it might be wrong. At least I'd like to read someone's comment about what is curthread in this code path. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance issues with 8.0 ZFS and sendfile/lighttpd
On Fri, Nov 06, 2009 at 11:41:12PM +0100, Miroslav Lachman wrote: Thomas Backman wrote: On Nov 6, 2009, at 7:36 PM, Miroslav Lachman wrote: Ivan Voras wrote: Miroslav Lachman wrote: Ivan Voras wrote: Miroslav Lachman wrote: [..] I have more strange issue with Lighttpd in jail on top of ZFS. Lighttpd is serving static content (mp3 downloads thru flash player). Is runs fine for relatively small number of parallel clients with bandwidth about 30 Mbps, but after some number of clients is reached (about 50-60 parallel clients) the throughput drops down to 6 Mbps. I can server hundereds of clients on same HW using Lighttpd not in jail and UFS2 with gjournal instead of ZFS reaching 100 Mbps (maybe more). I don't know if it is ZFS or Jail issue. Do you have actual disk IO or is the vast majority of your data served from the caches? (actually - the same question to the OP) I had ZFS zpool as mirror of two SATA II drives (500GB) and in the peak iostat (or systat -vm or gstat) shows about 80 tps / 60% busy. In case of UFS, I am using gmirrored 1TB SATA II drives working nice with 160 or more tps. Both setups are using FreeBSD 7.x amd64 with GENERIC kernel, 4GB of RAM. As the ZFS + Lighttpd in jail was unreliable, I am no longer using it, but if you want some more info for debuging, I can set it up again. For what it's worth, I have just set up a little test on a production machine with 3 500 GB SATA drives in RAIDZ, FreeBSD 7.2-RELEASE. The total data set is some 2 GB in 5000 files but the machine has only 2 GB RAM total so there is some disk IO - about 40 IOPS per drive. I'm also using Apache-worker, not lighty, and siege to benchmark with 10 concurrent users. In this setup, the machine has no problems saturating a 100 Mbit/s link - it's not on a LAN but the latency is close enough and I get ~~ 11 MB/s. [...] /boot/loader.conf: ## eLOM support hw.bge.allow_asf=1 ## gmirror RAID1 geom_mirror_load=YES ## ZFS tuning vm.kmem_size=1280M vm.kmem_size_max=1280M kern.maxvnodes=40 vfs.zfs.prefetch_disable=1 vfs.zfs.arc_min=16M vfs.zfs.arc_max=128M I won't pretend to know much about this area, but your ZFS values here are very low. May I assume that they are remnants of the times when the ARC grew insanely large and caused a kernel panic? You're effectively forcing ZFS to not use more than 128MB cache, which doesn't sound like a great idea if you've got 2+ GB of RAM. I've had no trouble without any tuning whatsoever on 2GB for a long time now. The kmem lines can probably be omitted if you're on amd64, too (the default value for kmem_size_max is about 307GB on my machine). Yes, loader values are one year old when I installed this machine. But I think auto tuning was commited after 7.2-RELEASE by Kip Macy, so some of them are still needed or am I wrong? (this is 7.2-RELEASE). ... We don't know, because none of the individuals who are maintaining ZFS at this point in time have actually responded to this question. http://lists.freebsd.org/pipermail/freebsd-stable/2009-October/052256.html The community really needs an official answer to this question, and one from those familiar with the code. -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance issues with 8.0 ZFS and sendfile/lighttpd
gnu...@alltel.blackberry.com wrote: I can send in more documentation later but I am seeing severe zfs performance issues with lighttpd. Same machine using UFS will push 1gbit or more but same content and traffic load can not hit 200mbit. Ufs does around 3 megabytes/sec IO at 800mbit network but zfs pushes the disks into the ground with 50+ megabytes/sec dusk i/o. No compression no atime no checksums on zfs and still same IO levels. Ufs with soft updates and atime on. Orders of magnitude more disk IO... Like zfs isn't using cache or isn't coalescing disk reads or both. Has anyone else seen this or have any recommendations? Lighttpd config remains exactly the same as well FYI. Only difference is ufs vs zfs. Hi, I can confirm this on FreeBSD 7.2-STABLE. first run: :~# wget -O /dev/null http://192.168.1.1/1000m.bin --2009-11-04 15:36:15-- http://192.168.1.1/1000m.bin Connecting to 192.168.1.1:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1048576000 (1000M) [application/octet-stream] Saving to: `/dev/null' 100%[] 1,048,576,000 17.0M/s in 81s 2009-11-04 15:37:36 (12.3 MB/s) - `/dev/null' saved [1048576000/1048576000] second run is even slower, cannot wait till end: :~# wget -O /dev/null http://192.168.1.1/1000m.bin --2009-11-04 15:40:00-- http://192.168.1.1/1000m.bin Connecting to 192.168.1.1:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1048576000 (1000M) [application/octet-stream] Saving to: `/dev/null' 71% [==] 752,173,056 2.10M/s eta 2m 0s ^C After changing in lighttpd.conf server.network-backend = writev first run: :~# wget -O /dev/null http://192.168.1.1/1000m.bin --2009-11-04 15:47:51-- http://192.168.1.1/1000m.bin Connecting to 192.168.1.1:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1048576000 (1000M) [application/octet-stream] Saving to: `/dev/null' 100%[] 1,048,576,000 44.1M/s in 27s 2009-11-04 15:48:18 (37.2 MB/s) - `/dev/null' saved [1048576000/1048576000] second third run: :~# wget -O /dev/null http://192.168.1.1/1000m.bin --2009-11-04 15:48:20-- http://192.168.1.1/1000m.bin Connecting to 192.168.1.1:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1048576000 (1000M) [application/octet-stream] Saving to: `/dev/null' 100%[] 1,048,576,000 788M/s in 1.3s 2009-11-04 15:48:21 (788 MB/s) - `/dev/null' saved [1048576000/1048576000] :~# wget -O /dev/null http://192.168.1.1/1000m.bin --2009-11-04 15:48:24-- http://192.168.1.1/1000m.bin Connecting to 192.168.1.1:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1048576000 (1000M) [application/octet-stream] Saving to: `/dev/null' 100%[] 1,048,576,000 910M/s in 1.1s 2009-11-04 15:48:25 (910 MB/s) - `/dev/null' saved [1048576000/1048576000] I have Intel 10GbE PCI-Express directly connected between two servers. -- Urmas Lett Tel:+(372) 7 302 110 Fax:+(372) 7 302 111 E-Mail: urmas.l...@eenet.ee ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance issues with 8.0 ZFS and sendfile/lighttpd
gnu...@alltel.blackberry.com wrote: I can send in more documentation later but I am seeing severe zfs performance issues with lighttpd. Same machine using UFS will push 1gbit or more but same content and traffic load can not hit 200mbit. Ufs does around 3 megabytes/sec IO at 800mbit network but zfs pushes the disks into the ground with 50+ megabytes/sec dusk i/o. No compression no atime no checksums on zfs and still same IO levels. Ufs with soft updates and atime on. Orders of magnitude more disk IO... Like zfs isn't using cache or isn't coalescing disk reads or both. Has anyone else seen this or have any recommendations? Lighttpd config remains exactly the same as well FYI. Only difference is ufs vs zfs. AFAIK, ZFS is incompatible (currently) with some advanced VM operations (like mmap, and I think sendfile relies on the same mechanism as mmap), so that could be a cause of the slowdown. Though I'm surprised you can only get 200 MBit/s - that's 25 MB/s and I think that even with multiple memcpy-ing data around the kernel you should be able to get hundreds of MB/s on newer hardware (which normally really can achieve tens of gigabytes/s of sustained memory access). What else can you observe from your system? Do you have exceedingly high sys times and load numbers? I'm also interested in what does 10 seconds of running 'vmstat 1' looks like on your system. Is it a bare machine or a virtual machine? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance issues with 8.0 ZFS and sendfile/lighttpd
Ivan Voras wrote: gnu...@alltel.blackberry.com wrote: I can send in more documentation later but I am seeing severe zfs performance issues with lighttpd. Same machine using UFS will push 1gbit or more but same content and traffic load can not hit 200mbit. Ufs does around 3 megabytes/sec IO at 800mbit network but zfs pushes the disks into the ground with 50+ megabytes/sec dusk i/o. No compression no atime no checksums on zfs and still same IO levels. Ufs with soft updates and atime on. Orders of magnitude more disk IO... Like zfs isn't using cache or isn't coalescing disk reads or both. Has anyone else seen this or have any recommendations? Lighttpd config remains exactly the same as well FYI. Only difference is ufs vs zfs. AFAIK, ZFS is incompatible (currently) with some advanced VM operations (like mmap, and I think sendfile relies on the same mechanism as mmap), so that could be a cause of the slowdown. Though I'm surprised you can only get 200 MBit/s - that's 25 MB/s and I think that even with multiple memcpy-ing data around the kernel you should be able to get hundreds of MB/s on newer hardware (which normally really can achieve tens of gigabytes/s of sustained memory access). I have more strange issue with Lighttpd in jail on top of ZFS. Lighttpd is serving static content (mp3 downloads thru flash player). Is runs fine for relatively small number of parallel clients with bandwidth about 30 Mbps, but after some number of clients is reached (about 50-60 parallel clients) the throughput drops down to 6 Mbps. I can server hundereds of clients on same HW using Lighttpd not in jail and UFS2 with gjournal instead of ZFS reaching 100 Mbps (maybe more). I don't know if it is ZFS or Jail issue. Miroslav Lachman ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance issues with 8.0 ZFS and sendfile/lighttpd
Miroslav Lachman wrote: Ivan Voras wrote: gnu...@alltel.blackberry.com wrote: I can send in more documentation later but I am seeing severe zfs performance issues with lighttpd. Same machine using UFS will push 1gbit or more but same content and traffic load can not hit 200mbit. Ufs does around 3 megabytes/sec IO at 800mbit network but zfs pushes the disks into the ground with 50+ megabytes/sec dusk i/o. No compression no atime no checksums on zfs and still same IO levels. Ufs with soft updates and atime on. Orders of magnitude more disk IO... Like zfs isn't using cache or isn't coalescing disk reads or both. Has anyone else seen this or have any recommendations? Lighttpd config remains exactly the same as well FYI. Only difference is ufs vs zfs. AFAIK, ZFS is incompatible (currently) with some advanced VM operations (like mmap, and I think sendfile relies on the same mechanism as mmap), so that could be a cause of the slowdown. Though I'm surprised you can only get 200 MBit/s - that's 25 MB/s and I think that even with multiple memcpy-ing data around the kernel you should be able to get hundreds of MB/s on newer hardware (which normally really can achieve tens of gigabytes/s of sustained memory access). I have more strange issue with Lighttpd in jail on top of ZFS. Lighttpd is serving static content (mp3 downloads thru flash player). Is runs fine for relatively small number of parallel clients with bandwidth about 30 Mbps, but after some number of clients is reached (about 50-60 parallel clients) the throughput drops down to 6 Mbps. I can server hundereds of clients on same HW using Lighttpd not in jail and UFS2 with gjournal instead of ZFS reaching 100 Mbps (maybe more). I don't know if it is ZFS or Jail issue. Do you have actual disk IO or is the vast majority of your data served from the caches? (actually - the same question to the OP) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance issues with 8.0 ZFS and sendfile/lighttpd
Ivan Voras wrote: Miroslav Lachman wrote: [..] I have more strange issue with Lighttpd in jail on top of ZFS. Lighttpd is serving static content (mp3 downloads thru flash player). Is runs fine for relatively small number of parallel clients with bandwidth about 30 Mbps, but after some number of clients is reached (about 50-60 parallel clients) the throughput drops down to 6 Mbps. I can server hundereds of clients on same HW using Lighttpd not in jail and UFS2 with gjournal instead of ZFS reaching 100 Mbps (maybe more). I don't know if it is ZFS or Jail issue. Do you have actual disk IO or is the vast majority of your data served from the caches? (actually - the same question to the OP) I had ZFS zpool as mirror of two SATA II drives (500GB) and in the peak iostat (or systat -vm or gstat) shows about 80 tps / 60% busy. In case of UFS, I am using gmirrored 1TB SATA II drives working nice with 160 or more tps. Both setups are using FreeBSD 7.x amd64 with GENERIC kernel, 4GB of RAM. As the ZFS + Lighttpd in jail was unreliable, I am no longer using it, but if you want some more info for debuging, I can set it up again. Miroslav Lachman ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance issues with 8.0 ZFS and sendfile/lighttpd
Miroslav Lachman wrote: Ivan Voras wrote: Miroslav Lachman wrote: [..] I have more strange issue with Lighttpd in jail on top of ZFS. Lighttpd is serving static content (mp3 downloads thru flash player). Is runs fine for relatively small number of parallel clients with bandwidth about 30 Mbps, but after some number of clients is reached (about 50-60 parallel clients) the throughput drops down to 6 Mbps. I can server hundereds of clients on same HW using Lighttpd not in jail and UFS2 with gjournal instead of ZFS reaching 100 Mbps (maybe more). I don't know if it is ZFS or Jail issue. Do you have actual disk IO or is the vast majority of your data served from the caches? (actually - the same question to the OP) I had ZFS zpool as mirror of two SATA II drives (500GB) and in the peak iostat (or systat -vm or gstat) shows about 80 tps / 60% busy. In case of UFS, I am using gmirrored 1TB SATA II drives working nice with 160 or more tps. Both setups are using FreeBSD 7.x amd64 with GENERIC kernel, 4GB of RAM. As the ZFS + Lighttpd in jail was unreliable, I am no longer using it, but if you want some more info for debuging, I can set it up again. For what it's worth, I have just set up a little test on a production machine with 3 500 GB SATA drives in RAIDZ, FreeBSD 7.2-RELEASE. The total data set is some 2 GB in 5000 files but the machine has only 2 GB RAM total so there is some disk IO - about 40 IOPS per drive. I'm also using Apache-worker, not lighty, and siege to benchmark with 10 concurrent users. In this setup, the machine has no problems saturating a 100 Mbit/s link - it's not on a LAN but the latency is close enough and I get ~~ 11 MB/s. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance with hundreds of nullfs mounts?
Russell Jackson wrote: Ivan Voras wrote: hi, I seem to remember hearing an anecdote somewhere that using hundreds (or thousands?) nullfs mounts for jails results in unreasonably bad file system access performance. Does somebody have this kind of setup / is it true? I was doing this with jails --before we moved to VMware ESX (for better or worse)-- and didn't see any noticeable performance degradation at the time (6.x series). Thanks, everyone. I've tracked it down and I heard it from a collegue, only he was talking about unionfs not nullfs. For those interested, the biggest plus for going to the ESX model is that it decoupled low utilization Windows boxes from over-spec'ed hardware and made it available for FreeBSD to use ;-). The downsides are that it's proprietary, it's expensive, it's inefficient (e.g. duplicated files and kernel instances everywhere), and you need freak'in Windows boxes to manage it. Yes, ESX is nice. I've also tried XenServer (the official Xen) and it's been terrible - both slow and clunky. Any other experiences? signature.asc Description: OpenPGP digital signature
Re: Performance with hundreds of nullfs mounts?
2009/3/10 Ivan Voras ivo...@freebsd.org: hi, I seem to remember hearing an anecdote somewhere that using hundreds (or thousands?) nullfs mounts for jails results in unreasonably bad file system access performance. Does somebody have this kind of setup / is it true? ~600-700 null mount points (without using jails) on 6.2. No visible fs slowdowns. -- wbr, pluknet ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance with hundreds of nullfs mounts?
Ivan Voras wrote: hi, I seem to remember hearing an anecdote somewhere that using hundreds (or thousands?) nullfs mounts for jails results in unreasonably bad file system access performance. Does somebody have this kind of setup / is it true? I was doing this with jails --before we moved to VMware ESX (for better or worse)-- and didn't see any noticeable performance degradation at the time (6.x series). For those interested, the biggest plus for going to the ESX model is that it decoupled low utilization Windows boxes from over-spec'ed hardware and made it available for FreeBSD to use ;-). The downsides are that it's proprietary, it's expensive, it's inefficient (e.g. duplicated files and kernel instances everywhere), and you need freak'in Windows boxes to manage it. -- Russell A. Jackson r...@csub.edu Network Analyst California State University, Bakersfield The first thing I do in the morning is brush my teeth and sharpen my tongue. -- Dorothy Parker signature.asc Description: OpenPGP digital signature
Re: Performance with hundreds of nullfs mounts?
Ivan Voras wrote: I seem to remember hearing an anecdote somewhere that using hundreds (or thousands?) nullfs mounts for jails results in unreasonably bad file system access performance. Does somebody have this kind of setup / is it true? I'm using about several readonly nullfs mounts per jail: usr, bin sbin, lib, libexec, with ~20 jails per machine, and the speed is just fine, on 7.0-STABLE. - Andrew ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Performance of madvise / msync
:With madvise() and without msync(), there are high numbers of :faults, which matches the number of disk io operations. It :goes through cycles, every once in a while stalling while about :60MB of data is dumped to disk at 20MB/s or so (buffers flushing?) :At the beginning of each cycle it's fast, with 140 faults/s or so, :and slows as the number of faults climbs to 180/s or so before :stalling and flusing again. It never gets _really_ slow though. Yah, without the msync() the dirty pages build up in the kernel's VM page cache. A flush should happen automatically every 30-60 seconds, or sooner if the buffer cache builds up too many dirty pages. The activity you are seeing sounds like the 30-60 second filesystem sync the kernel does periodically. Either NetBSD or OpenBSD, I forget which, implemented a partial sync feature to prevent long stalls when the filesystem syncer hits a file with a lot of dirty pages. FreeBSD could borrow that optimization if they want to reduce stalls from the filesytem sync. I ported it to DFly a while back myself. :With msync() and without madvise(), things are very slow, and :there are no faults, just writes. :... : The size_t argument to msync() (0x453b7618) is highly questionable. : It could be ktrace reporting the wrong value, but maybe not. : :That's the size of rg2.rrd. It's 1161524760 bytes long. :... :Looks like the source of my problem is very slow msync() on the :file when the file is over a certain size. It's still fastest :without either madvise or msync. : :Thanks for your time, : :Marcus The msync() is clearly the problem. There are numerous optimizations in the kernel but msync() is frankly a rather nasty critter even with the optimizations work. Nobody using msync() in real life ever tries to run it over the entirety of such a large mapping... usually it is just run on explicit sub-ranges that the program wishes to sync. One reason why msync() is so nasty is that the kernel must physically check the page table(s) to determine whether a page has been marked dirty by the MMU, so it can't just iterate the pages it knows are dirty in the VM object. It's nasty whether it scans the VM object and iterates the page tables, or scans the page tables and looks up the related VM pages. The only way to optimize this is to force write-faults by mapping clean pages read-only, in order to track whether a page is actually dirty in real time instead of lazily. Then msync() would only have to do a ranged-scan of the VM object's dirty-page list and would not have to actually check the page tables for clean pages. A secondary effect of the msync() is that it is initiating asynchronous I/O for what sounds like hundreds of VM pages, or even more. All those pages are locked and busied from the point they are queued to the point the I/O finishes, which for some of the pages can be a very, very long time (into the multiples of seconds). Pages locked that long will interfere with madvise() calls made after the msync(), and probably even interfere with the follow msync(). It used to be that msync() only synced VM pages to the underlying file, making them consistent with read()'s and write()'s against the underlying file. Since FreeBSD uses a unified VM page cache this is always true. However, the Open Group specification now requires that the dirty pages actually be written out to the underlying media... i.e. issue real I/O. So msync() can't be a NOP if you go by the OpenGroup specification. -Matt Matthew Dillon [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance of madvise / msync
: 65074 python 0.06 CALL madvise(0x287c5000,0x70,_MADV_WILLNEED) : 65074 python 0.027455 RET madvise 0 : 65074 python 0.58 CALL madvise(0x287c5000,0x1c20,_MADV_WILLNEED) : 65074 python 0.016904 RET madvise 0 : 65074 python 0.000179 CALL madvise(0x287c6000,0x1950,_MADV_WILLNEED) : 65074 python 0.008629 RET madvise 0 : 65074 python 0.40 CALL madvise(0x287c8000,0x8,_MADV_WILLNEED) : 65074 python 0.004173 RET madvise 0 :... : 65074 python 0.006084 CALL msync(0x287c5000,0x453b7618,MS_ASYNC) : 65074 python 0.106284 RET msync 0 :... :As you can see, it's quite a bit faster. : :I know that msync is necessary under Linux but obsolete under FreeBSD, but :it's still funny that it takes a tenth of a second to return even with :MS_ASYNC specified. : :Also, why is it that the madvise() calls take so much longer when the :program does a couple of its own madvise() calls? Was madvise() never :intended to be run so frequently and is therefore a little slower than :it could be? : :Here's the diff between the code for the first kdump above and the :second one. Those times are way way too large, even with other running threads in the system. madvise() should not take that long unless it is being forced to wait on a busied page, and neither should msync(). madvise() doesn't even do any I/O (or shouldn't anyhow). Try removing just the msync() but keep the madvise() calls and see if the madvise() calls continue to take horrendous amounts of time. Then try the vise-versa. It kinda feels like a prior msync() is initiating physical I/O on pages and a later mmap/madvise or page fault is being forced to wait on the prior pages for the I/O to finish. The size_t argument to msync() (0x453b7618) is highly questionable. It could be ktrace reporting the wrong value, but maybe not. On any sort of random writing test, particularly if multiple threads are involved, specifying a size that large could result in very large latencies. -Matt ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance of madvise / msync
On Thu, Jun 26, 2008 at 05:48:13PM -0700, Matthew Dillon wrote: : 65074 python 0.06 CALL madvise(0x287c5000,0x70,_MADV_WILLNEED) : 65074 python 0.027455 RET madvise 0 : 65074 python 0.58 CALL madvise(0x287c5000,0x1c20,_MADV_WILLNEED) : 65074 python 0.016904 RET madvise 0 : 65074 python 0.000179 CALL madvise(0x287c6000,0x1950,_MADV_WILLNEED) : 65074 python 0.008629 RET madvise 0 : 65074 python 0.40 CALL madvise(0x287c8000,0x8,_MADV_WILLNEED) : 65074 python 0.004173 RET madvise 0 :... : 65074 python 0.006084 CALL msync(0x287c5000,0x453b7618,MS_ASYNC) : 65074 python 0.106284 RET msync 0 :... :As you can see, it's quite a bit faster. : :I know that msync is necessary under Linux but obsolete under FreeBSD, but :it's still funny that it takes a tenth of a second to return even with :MS_ASYNC specified. : :Also, why is it that the madvise() calls take so much longer when the :program does a couple of its own madvise() calls? Was madvise() never :intended to be run so frequently and is therefore a little slower than :it could be? : :Here's the diff between the code for the first kdump above and the :second one. Those times are way way too large, even with other running threads in the system. madvise() should not take that long unless it is being forced to wait on a busied page, and neither should msync(). madvise() doesn't even do any I/O (or shouldn't anyhow). Try removing just the msync() but keep the madvise() calls and see if the madvise() calls continue to take horrendous amounts of time. Then try the vise-versa. Ok, first off, I'm noticing that of the 4 other files that this is doing the same operations on, sized 569, 940, 116 and 116mB, all of the msync() and madvise() calls are nice and fast. It's only with the 1161524760 byte file that msync is much, much slower. It's not linear -- it hits a wall somewhere between 940 and 1161 million bytes. With madvise() and without msync(), there are high numbers of faults, which matches the number of disk io operations. It goes through cycles, every once in a while stalling while about 60MB of data is dumped to disk at 20MB/s or so (buffers flushing?) At the beginning of each cycle it's fast, with 140 faults/s or so, and slows as the number of faults climbs to 180/s or so before stalling and flusing again. It never gets _really_ slow though. 36286 python 0.16 NAMI rg2.rrd 36286 python 0.25 RET open 7 36286 python 0.09 CALL fstat(0x7,0xbfbfe428) 36286 python 0.14 RET fstat 0 36286 python 0.10 CALL mmap(0,0x453b7618,PROT_READ|PROT_WRITE,MAP_SHARED,0x7,0,0) 36286 python 0.20 RET mmap 679235584/0x287c5000 36286 python 0.10 CALL madvise(0x287c5000,0x453b7618,_MADV_RANDOM) 36286 python 0.10 RET madvise 0 36286 python 0.09 CALL madvise(0x287c5000,0x70,_MADV_WILLNEED) 36286 python 0.67 RET madvise 0 36286 python 0.16 CALL madvise(0x287c5000,0x1c20,_MADV_WILLNEED) 36286 python 0.15 RET madvise 0 36286 python 0.19 CALL madvise(0x287c6000,0x1950,_MADV_WILLNEED) 36286 python 0.13 RET madvise 0 36286 python 0.10 CALL madvise(0x287c8000,0x8,_MADV_WILLNEED) 36286 python 0.10 RET madvise 0 36286 python 0.12 CALL gettimeofday(0xbfbfe554,0) 36286 python 0.10 RET gettimeofday 0 36286 python 0.14 CALL fcntl(0x7,invalid=12,0xbfbfe478) 36286 python 0.21 RET fcntl 0 36286 python 0.040061 CALL munmap(0x287c5000,0x453b7618) 36286 python 0.000324 RET munmap 0 36286 python 0.16 CALL close(0x7) 36286 python 0.44 RET close 0 36286 python 0.000113 CALL __sysctl(0xbfbfe388,0x2,0xbfbfe394,0xbfbfe398,0,0) 36286 python 0.18 RET __sysctl 0 With msync() and without madvise(), things are very slow, and there are no faults, just writes. 61609 python 0.16 NAMI rg2.rrd 61609 python 0.24 RET open 7 61609 python 0.10 CALL fstat(0x7,0xbfbfe428) 61609 python 0.13 RET fstat 0 61609 python 0.10 CALL mmap(0,0x453b7618,PROT_READ|PROT_WRITE,MAP_SHARED,0x7,0,0) 61609 python 0.21 RET mmap 679235584/0x287c5000 61609 python 0.066603 CALL madvise(0x287c5000,0x1c20,_MADV_WILLNEED) 61609 python 0.57 RET madvise 0 61609 python 0.09 CALL madvise(0x287c6000,0x1950,_MADV_WILLNEED) 61609 python 0.10 RET madvise 0 61609 python 0.09 CALL madvise(0x287c8000,0x8,_MADV_WILLNEED) 61609 python 0.09 RET madvise 0 61609 python 0.10 CALL gettimeofday(0xbfbfe554,0) 61609 python 0.18 RET gettimeofday 0 61609 python 0.14 CALL fcntl(0x7,invalid=12,0xbfbfe478) 61609 python 0.26 RET fcntl 0 61609 python 0.004044 CALL msync(0x287c5000,0x453b7618,MS_ASYNC) 61609 python
Re: Performance! [SOLVED]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Phillip N. wrote: El vie, 11-01-2008 a las 18:15 +0100, Kris Kennaway escribió: Krassimir reports that with these two fixes, the standard 7.0 kernel has performance: #threadstransactions/sec 1 755 8 7129 40 6580 100 6768 Hi. May i ask what kind of hardware is running this test? Just wanted to compare with this AMDX2 4600+ 2G DDR800 box wich gives: 1 500 2 887 4 880 6 834 8 791 thanks! HP ProLiant DL380G5, 2 x X5450 CPUs, 15k SAS HDD, 8G RAM, see the top of the thread. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHlFRexJBWvpalMpkRAlC6AKCZmWSF+TEiIfOTIbC2S6/Ak3DlsACgoZ6i 2NLhlPSCDoAz+74CuWerLw0= =Q8Z1 -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance! [SOLVED]
El vie, 11-01-2008 a las 18:15 +0100, Kris Kennaway escribió: Krassimir reports that with these two fixes, the standard 7.0 kernel has performance: #threadstransactions/sec 1 755 8 7129 40 6580 100 6768 Hi. May i ask what kind of hardware is running this test? Just wanted to compare with this AMDX2 4600+ 2G DDR800 box wich gives: 1 500 2 887 4 880 6 834 8 791 thanks! ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Here are lock profiling results with select patch applied. OK, you are doing I/O over TCP. Are you sure you are using TCP on both systems? Linux may not be defaulting to TCP transport for local queries. Add --pgsql-host= to your sysbench command line to make it communicate over a local domain socket, which is much more efficient. Kris Hmm, Yes linux uses local domain sockets! Here are results using local domain sockets on FreeBSD too: #threads#tranzactions/sec 1 728 5 2996 10 5301 20 3931 40 2466 60 1852 80 1424 100 1216 Just to remember: Linux (2.6.18) #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 I have results using Fedora 8 on the same hardware: Linux (2.6.23) #threads#transactions/sec 1 740 5 2675 10 6486 20 6893 40 6623 60 6623 80 6522 100 6417 If we look at the results with up to 10 threads the performance of FreeBSD is very good. May be something can be tuned for number of threads number of CPUs? Are you interested in lock profiling statistics with more threads than the number of CPUs? Yes, it's still performing anomalously. Glad we're making progress though :) Kris Okay, but how many threads will be more useful to test with? Let's start with 8 and say 40. The two problems seem to be slightly lower peak and poor scaling above peak. They might be the same, or they might be different. kris I sent to you the results with 8 and 40 threads. The lower peak may be caused by differences in installations like different partitions I use ... The most important is keeping performance above this peak. Best Regards -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHh06CxJBWvpalMpkRAjosAJ0djbiU/foRiI+/ne1YyuqnVjKOaACfRCpV fI3vb0m1I+fIZ1M7cHCBjR8= =jiPg -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance! [SOLVED]
Krassimir Slavchev wrote: I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. Just to summarize some discussion we had off-list, this problem is now resolved. It turned out to have two causes: 1) sysbench on linux was defaulting to using a unix domain socket to communicate with pgsql, but FreeBSD was using TCP to 127.0.0.1. TCP has much more overhead so performance was lower. Using --pgsql-host= in sysbench is the fix for this. 2) pgsql was not compiled with thread support enabled, which caused lots of bad interactions with the threaded sysbench client. This probably would have caused data corruption too (silent in this case because nothing was checking the query responses). The fix was to recompile the pgsql client and server with the THREADSAFE option enabled. Krassimir reports that with these two fixes, the standard 7.0 kernel has performance: #threadstransactions/sec 1 755 8 7129 40 6580 100 6768 compared to Linux: Linux (2.6.18) #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 Linux (2.6.23) #threads#transactions/sec 1 740 5 2675 10 6486 20 6893 40 6623 60 6623 80 6522 100 6417 I think this is a satisfactory resolution :) Kris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance! [SOLVED]
I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. Just to summarize some discussion we had off-list, this problem is now resolved. It turned out to have two causes: 1) sysbench on linux was defaulting to using a unix domain socket to communicate with pgsql, but FreeBSD was using TCP to 127.0.0.1. TCP has much more overhead so performance was lower. Using --pgsql-host= in sysbench is the fix for this. 2) pgsql was not compiled with thread support enabled, which caused lots of bad interactions with the threaded sysbench client. This probably would have caused data corruption too (silent in this case because nothing was checking the query responses). The fix was to recompile the pgsql client and server with the THREADSAFE option enabled. Krassimir reports that with these two fixes, the standard 7.0 kernel has performance: #threadstransactions/sec 1 755 8 7129 40 6580 100 6768 compared to Linux: Linux (2.6.18) #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 Linux (2.6.23) #threads#transactions/sec 1 740 5 2675 10 6486 20 6893 40 6623 60 6623 80 6522 100 6417 I think this is a satisfactory resolution :) Thank you so much! (both of you and those who helped along the way) :-) On monday I will start testing a setup where we are moving from four 10K rpm sas disk in raid 1+0 to eight 15K rpm sas disks in raid 1+0 as well. The former is a ciss p400 controller with 512 MB bbc and the latter is a p800 with 512 MB bbc and an external msa-box. I will test with 7.0 and postgresql 8.2 from ports. Should I test with stable or release? Are there any patches I can apply? Our current db-server is a DL380 G5 woodcrest at 3 Ghz and my test-server will be a 8-core DL360 G5 at 2.4 Ghz (I think). I will log the queries from our live-server and repeat them on the test-server, use pgbench on FreeBSD, Solaris and Linux. I was getting a bit worried when the number varied so much between Linux and FreeBSD. -- regards Claus When lenity and cruelty play for a kingdom, the gentlest gamester is the soonest winner. Shakespeare ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance! [SOLVED]
On Fri, 11 Jan 2008 18:15:08 +0100 Kris Kennaway [EMAIL PROTECTED] wrote: Krassimir reports that with these two fixes, the standard 7.0 kernel has performance: #threads transactions/sec 1 755 8 7129 406580 100 6768 compared to Linux: Linux (2.6.18) #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 Linux (2.6.23) #threads#transactions/sec 1 740 5 2675 10 6486 20 6893 40 6623 60 6623 80 6522 100 6417 I think this is a satisfactory resolution :) That's great news! Even greater if you consider that not a single line of code inside FreeBSD had to be changed. -- Ricardo Nabinger Sanchez [EMAIL PROTECTED] Powered by FreeBSD http://rnsanchez.wait4.org Left to themselves, things tend to go from bad to worse. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance! [SOLVED]
At 12:15 PM 1/11/2008, Kris Kennaway wrote: Just to summarize some discussion we had off-list, this problem is now resolved. It turned out to have two causes: 1) sysbench on linux was defaulting to using a unix domain socket to communicate with pgsql, but FreeBSD was using TCP to 127.0.0.1. TCP has much more overhead so performance was lower. Using --pgsql-host= in sysbench is the fix for this. 2) pgsql was not compiled with thread support enabled, which caused lots of bad interactions with the threaded sysbench client. This probably would have caused data corruption too (silent in this case because nothing was checking the query responses). The fix was to recompile the pgsql client and server with the THREADSAFE option enabled. Thats great news! Just for the record, for build options are there any other options that need to be set other than # diff -u Makefile.default Makefile --- Makefile.default2008-01-11 20:13:06.0 -0500 +++ Makefile2008-01-11 20:16:02.0 -0500 @@ -87,7 +87,7 @@ OPTIONS+= HEIMDAL_KRB5 Builds with Heimdal kerberos support off OPTIONS+= OPTIMIZED_CFLAGS Builds with compiler optimizations (-O3) off OPTIONS+= LIBC_R Link w/ libc_r, used by plpython (server) off -OPTIONS+= THREADSAFE make libpq thread safe off +OPTIONS+= THREADSAFE make libpq thread safe on # to run regression tests: OPTIONS+= TESTS Allows the use of a \check\ target (server) off OPTIONS+= DEBUG Builds with debugging symbols off ? In terms of the kernel was this with ULE or SCHED_4BSD ? ---Mike ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance! [SOLVED]
Mike Tancsa wrote: At 12:15 PM 1/11/2008, Kris Kennaway wrote: Just to summarize some discussion we had off-list, this problem is now resolved. It turned out to have two causes: 1) sysbench on linux was defaulting to using a unix domain socket to communicate with pgsql, but FreeBSD was using TCP to 127.0.0.1. TCP has much more overhead so performance was lower. Using --pgsql-host= in sysbench is the fix for this. 2) pgsql was not compiled with thread support enabled, which caused lots of bad interactions with the threaded sysbench client. This probably would have caused data corruption too (silent in this case because nothing was checking the query responses). The fix was to recompile the pgsql client and server with the THREADSAFE option enabled. Thats great news! Just for the record, for build options are there any other options that need to be set other than # diff -u Makefile.default Makefile --- Makefile.default2008-01-11 20:13:06.0 -0500 +++ Makefile2008-01-11 20:16:02.0 -0500 @@ -87,7 +87,7 @@ OPTIONS+= HEIMDAL_KRB5 Builds with Heimdal kerberos support off OPTIONS+= OPTIMIZED_CFLAGS Builds with compiler optimizations (-O3) off OPTIONS+= LIBC_R Link w/ libc_r, used by plpython (server) off -OPTIONS+= THREADSAFE make libpq thread safe off +OPTIONS+= THREADSAFE make libpq thread safe on # to run regression tests: OPTIONS+= TESTS Allows the use of a \check\ target (server) off OPTIONS+= DEBUG Builds with debugging symbols off That is necessary for the sysbench client, which is threaded. I don't know if there are any implications for non-threaded applications, but I would hope not. This was with the ULE scheduler, which is basically mandatory for good performance on SMP systems. Kris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Here are lock profiling results with select patch applied. OK, you are doing I/O over TCP. Are you sure you are using TCP on both systems? Linux may not be defaulting to TCP transport for local queries. Add --pgsql-host= to your sysbench command line to make it communicate over a local domain socket, which is much more efficient. Kris Hmm, Yes linux uses local domain sockets! Here are results using local domain sockets on FreeBSD too: #threads#tranzactions/sec 1 728 5 2996 10 5301 20 3931 40 2466 60 1852 80 1424 100 1216 Just to remember: Linux (2.6.18) #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 I have results using Fedora 8 on the same hardware: Linux (2.6.23) #threads#transactions/sec 1 740 5 2675 10 6486 20 6893 40 6623 60 6623 80 6522 100 6417 If we look at the results with up to 10 threads the performance of FreeBSD is very good. May be something can be tuned for number of threads number of CPUs? Are you interested in lock profiling statistics with more threads than the number of CPUs? Yes, it's still performing anomalously. Glad we're making progress though :) Kris Okay, but how many threads will be more useful to test with? Best Regards -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHhdMIxJBWvpalMpkRAowhAKCOLh6eUptQhVQlUA2O2vSS4fi6XACaA8Bx BVgsYLIpqdKZpMdaz4mj9gg= =MhN8 -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Here are lock profiling results with select patch applied. OK, you are doing I/O over TCP. Are you sure you are using TCP on both systems? Linux may not be defaulting to TCP transport for local queries. Add --pgsql-host= to your sysbench command line to make it communicate over a local domain socket, which is much more efficient. Kris Hmm, Yes linux uses local domain sockets! Here are results using local domain sockets on FreeBSD too: #threads#tranzactions/sec 1 728 5 2996 10 5301 20 3931 40 2466 60 1852 80 1424 100 1216 Just to remember: Linux (2.6.18) #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 I have results using Fedora 8 on the same hardware: Linux (2.6.23) #threads#transactions/sec 1 740 5 2675 10 6486 20 6893 40 6623 60 6623 80 6522 100 6417 If we look at the results with up to 10 threads the performance of FreeBSD is very good. May be something can be tuned for number of threads number of CPUs? Are you interested in lock profiling statistics with more threads than the number of CPUs? Yes, it's still performing anomalously. Glad we're making progress though :) Kris Okay, but how many threads will be more useful to test with? Let's start with 8 and say 40. The two problems seem to be slightly lower peak and poor scaling above peak. They might be the same, or they might be different. kris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Results with lock_manager and select patches and kern.hz=100 #threads#transactions/sec 1 582 5 2083 10 2030 20 2421 40 1739 60 1409 80 1124 100 1034 This is with the BETA4. May be I should upgrade to RC1? I will collect lock profiling statistics again. Best Regards Kris Kennaway wrote: Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello Kris, Here is the lock profiling results, see the attachment. Please, let me know if you want ssh access to this machine? Thanks, this is very interesting. The problem is already fixed in 8.0 but we were not seeing it being this much of a factor on our test hardware. Possibly it is due to your CPUs being moderately faster and causing a timing difference. Anyway, try this patch. If there is still a performance deficit then repeating the lock profiling will be useful. Kris Now with actual patch. Kris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHhKCcxJBWvpalMpkRAi4WAJ4/lbXTq/7/rGnURw5DMICRqKnFvQCeJKWc KxGflXbc3KfvyuyTRVzStEk= =aBqO -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Here are lock profiling results with select patch applied. OK, you are doing I/O over TCP. Are you sure you are using TCP on both systems? Linux may not be defaulting to TCP transport for local queries. Add --pgsql-host= to your sysbench command line to make it communicate over a local domain socket, which is much more efficient. Kris Hmm, Yes linux uses local domain sockets! Here are results using local domain sockets on FreeBSD too: #threads#tranzactions/sec 1 728 5 2996 10 5301 20 3931 40 2466 60 1852 80 1424 100 1216 Just to remember: Linux (2.6.18) #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 I have results using Fedora 8 on the same hardware: Linux (2.6.23) #threads#transactions/sec 1 740 5 2675 10 6486 20 6893 40 6623 60 6623 80 6522 100 6417 If we look at the results with up to 10 threads the performance of FreeBSD is very good. May be something can be tuned for number of threads number of CPUs? Are you interested in lock profiling statistics with more threads than the number of CPUs? Best Regards -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHhMCxxJBWvpalMpkRAvuFAJ4+ab5o4DoLZXrJ3hrr6vbJG9veTgCfdjn5 d8kf8v17QhiYSn20yDgWl3E= =Tsub -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, Here are lock profiling results with select patch applied. OK, you are doing I/O over TCP. Are you sure you are using TCP on both systems? Linux may not be defaulting to TCP transport for local queries. Add --pgsql-host= to your sysbench command line to make it communicate over a local domain socket, which is much more efficient. Kris Hmm, Yes linux uses local domain sockets! Here are results using local domain sockets on FreeBSD too: #threads#tranzactions/sec 1 728 5 2996 10 5301 20 3931 40 2466 60 1852 80 1424 100 1216 Just to remember: Linux (2.6.18) #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 I have results using Fedora 8 on the same hardware: Linux (2.6.23) #threads#transactions/sec 1 740 5 2675 10 6486 20 6893 40 6623 60 6623 80 6522 100 6417 If we look at the results with up to 10 threads the performance of FreeBSD is very good. May be something can be tuned for number of threads number of CPUs? Are you interested in lock profiling statistics with more threads than the number of CPUs? Yes, it's still performing anomalously. Glad we're making progress though :) Kris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello Kris, Here is the lock profiling results, see the attachment. Please, let me know if you want ssh access to this machine? Thanks, this is very interesting. The problem is already fixed in 8.0 but we were not seeing it being this much of a factor on our test hardware. Possibly it is due to your CPUs being moderately faster and causing a timing difference. Anyway, try this patch. If there is still a performance deficit then repeating the lock profiling will be useful. Kris Now with actual patch. Kris FreeBSD src repository Modified files: sys/kern kern_event.c kern_thread.c sys_generic.c sys_pipe.c uipc_sockbuf.c sys/netncp ncp_rq.c ncp_sock.c ncp_sock.h sys/netsmb smb_trantcp.c sys/sys proc.h selinfo.h socketvar.h systm.h Log: Refactor select to reduce contention and hide internal implementation details from consumers. - Track individual selecters on a per-descriptor basis such that there are no longer collisions and after sleeping for events only those descriptors which triggered events must be rescaned. - Protect the selinfo (per descriptor) structure with a mtx pool mutex. mtx pool mutexes were chosen to preserve api compatibility with existing code which does nothing but bzero() to setup selinfo structures. - Use a per-thread wait channel rather than a global wait channel. - Hide select implementation details in a seltd structure which is opaque to the rest of the kernel. - Provide a 'selsocket' interface for those kernel consumers who wish to select on a socket when they have no fd so they no longer have to be aware of select implementation details. Tested by: kris Reviewed on:arch Revision ChangesPath 1.114 +6 -3 src/sys/kern/kern_event.c 1.264 +2 -0 src/sys/kern/kern_thread.c 1.160 +414 -168 src/sys/kern/sys_generic.c 1.194 +6 -3 src/sys/kern/sys_pipe.c 1.173 +2 -1 src/sys/kern/uipc_sockbuf.c 1.16 +9 -3 src/sys/netncp/ncp_rq.c 1.20 +0 -105src/sys/netncp/ncp_sock.c 1.7 +0 -3 src/sys/netncp/ncp_sock.h 1.27 +1 -79 src/sys/netsmb/smb_trantcp.c 1.498 +2 -3 src/sys/sys/proc.h 1.19 +8 -8 src/sys/sys/selinfo.h 1.159 +2 -0 src/sys/sys/socketvar.h 1.263 +0 -4 src/sys/sys/systm.h http://cvsweb.FreeBSD.org/src/sys/kern/kern_event.c.diff?r1=1.113r2=1.114 --- src/sys/kern/kern_event.c 2007/07/14 21:23:30 1.113 +++ src/sys/kern/kern_event.c 2007/12/16 06:21:19 1.114 @@ -1400,7 +1400,8 @@ kqueue_poll(struct file *fp, int events, revents |= events (POLLIN | POLLRDNORM); } else { selrecord(td, kq-kq_sel); - kq-kq_state |= KQ_SEL; + if (SEL_WAITING(kq-kq_sel)) + kq-kq_state |= KQ_SEL; } } kqueue_release(kq, 1); @@ -1486,8 +1487,9 @@ kqueue_close(struct file *fp, struct thr } if ((kq-kq_state KQ_SEL) == KQ_SEL) { - kq-kq_state = ~KQ_SEL; selwakeuppri(kq-kq_sel, PSOCK); + if (!SEL_WAITING(kq-kq_sel)) + kq-kq_state = ~KQ_SEL; } KQ_UNLOCK(kq); @@ -1522,8 +1524,9 @@ kqueue_wakeup(struct kqueue *kq) wakeup(kq); } if ((kq-kq_state KQ_SEL) == KQ_SEL) { - kq-kq_state = ~KQ_SEL; selwakeuppri(kq-kq_sel, PSOCK); + if (!SEL_WAITING(kq-kq_sel)) + kq-kq_state = ~KQ_SEL; } if (!knlist_empty(kq-kq_sel.si_note)) kqueue_schedtask(kq); http://cvsweb.FreeBSD.org/src/sys/kern/kern_thread.c.diff?r1=1.263r2=1.264 diff -u -r1.255.2.1 kern_thread.c --- src/sys/kern/kern_thread.c 14 Dec 2007 13:41:09 - 1.255.2.1 +++ src/sys/kern/kern_thread.c 8 Jan 2008 23:47:00 - @@ -40,6 +40,7 @@ #include sys/sysctl.h #include sys/sched.h #include sys/sleepqueue.h +#include sys/selinfo.h #include sys/turnstile.h #include sys/ktr.h #include sys/umtx.h @@ -207,6 +208,7 @@ turnstile_free(td-td_turnstile); sleepq_free(td-td_sleepqueue); umtx_thread_fini(td); + seltdfini(td); vm_thread_dispose(td); } http://cvsweb.FreeBSD.org/src/sys/kern/sys_generic.c.diff?r1=1.159r2=1.160 --- src/sys/kern/sys_generic.c 2007/11/14 06:21:23 1.159 +++ src/sys/kern/sys_generic.c 2007/12/16 06:21:19 1.160 @@ -69,17 +69,59 @@ __FBSDID($FreeBSD: /usr/local/www/cvsro #include sys/ktrace.h #endif +#include sys/ktr.h + static MALLOC_DEFINE(M_IOCTLOPS, ioctlops, ioctl data buffer); static MALLOC_DEFINE(M_SELECT, select, select() buffer); MALLOC_DEFINE(M_IOV,
Re: Performance!
Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello Kris, Here is the lock profiling results, see the attachment. Please, let me know if you want ssh access to this machine? Thanks, this is very interesting. The problem is already fixed in 8.0 but we were not seeing it being this much of a factor on our test hardware. Possibly it is due to your CPUs being moderately faster and causing a timing difference. Anyway, try this patch. If there is still a performance deficit then repeating the lock profiling will be useful. Kris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I can't see the patch? Best Regards Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello Kris, Here is the lock profiling results, see the attachment. Please, let me know if you want ssh access to this machine? Thanks, this is very interesting. The problem is already fixed in 8.0 but we were not seeing it being this much of a factor on our test hardware. Possibly it is due to your CPUs being moderately faster and causing a timing difference. Anyway, try this patch. If there is still a performance deficit then repeating the lock profiling will be useful. Kris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHhHaAxJBWvpalMpkRAuZ0AKCCmeeAMv5VadvnSYmND+8MwOx0IACghPHm JSurVau6L0resFWqNu15iKo= =NTTA -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kris Kennaway wrote: Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kris Kennaway wrote: Krassimir Slavchev wrote: Hello, I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. dmesg: ... CPU: Intel(R) Xeon(R) CPU X5450 @ 3.00GHz (3000.02-MHz K8-class CPU) Origin = GenuineIntel Id = 0x10676 Stepping = 6 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0xce3bdSSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,b19 AMD Features=0x2800SYSCALL,LM AMD Features2=0x1LAHF Cores per package: 4 usable memory = 8575655936 (8178 MB) avail memory = 8288337920 (7904 MB) ACPI APIC Table: HP ProLiant FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs ... test: sysbench --num-threads=${i} --test=oltp --pgsql-user=bench --pgsql-db=bench --db-driver=pgsql --max-time=60 --max-requests=0 --oltp-read-only=on run tuning: kern.ipc.shmmax=2147483647 kern.ipc.shmall=524288 kern.ipc.semmsl=512 kern.ipc.semmap=256 kern.ipc.somaxconn=2048 kern.maxfiles=65536 vfs.read_max=32 kern.ipc.semmni=256 kern.ipc.semmns=2048 results: FreeBSD 7.0-BETA4 amd64 (cvsup on 20.12) GENERIC with SCHED_ULE #threads#transactions/sec user/system 1 500 7.4%,5.3% 5 199030.9%,23.4% 10 251039.9%,35.0% 20 254944.5%,43.5% 40 192129.8%,59.4% 60 158022.7%,70.6% 80 134118.9%,75.9% 100 122716.5%,79.3% Linux #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 What can be done to improve these results? Best Regards ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] . postgresql has some poor default settings on FreeBSD. You need to add: stats_command_string = off update_process_title = off Kris I use a copy of postgresql.conf file from linux. Only 'stats_command_string = on' was commented. Here are results with these settings and lock_manager patch: #threads#transactions/sec 1 582 5 2154 10 2253 20 2705 40 2215 60 1713 80 1574 100 1256 Please enable LOCK_PROFILING in your kernel and then do sysctl debug.lock.prof.enable=1 run the test with 8 threads sysctl debug.lock.prof.enable=0 and send me the output of sysctl debug.lock.prof.stats Kris I have sent the results to you. If someone else interested in these results I will put them somewhere. Are you using postgresql 8.1 or older? It didn't have the update_process_title option to disable the setproctitle() calls that have a large performance penalty on FreeBSD. Try with 8.2 or hack the source to disable it. It is 8.2 from ports. pkg_info: postgresql-server-8.2.5_2 The most advanced open-source database available anywhere Kris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHfMkvxJBWvpalMpkRAhe+AJ9Bsl0ciZ6kPHtJT9RVjDo4E4fNxgCcC7qH 5veO734c+yDgEY945dwINAg= =cdYT -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kris Kennaway wrote: Krassimir Slavchev wrote: Hello, I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. dmesg: ... CPU: Intel(R) Xeon(R) CPU X5450 @ 3.00GHz (3000.02-MHz K8-class CPU) Origin = GenuineIntel Id = 0x10676 Stepping = 6 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0xce3bdSSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,b19 AMD Features=0x2800SYSCALL,LM AMD Features2=0x1LAHF Cores per package: 4 usable memory = 8575655936 (8178 MB) avail memory = 8288337920 (7904 MB) ACPI APIC Table: HP ProLiant FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs ... test: sysbench --num-threads=${i} --test=oltp --pgsql-user=bench --pgsql-db=bench --db-driver=pgsql --max-time=60 --max-requests=0 --oltp-read-only=on run tuning: kern.ipc.shmmax=2147483647 kern.ipc.shmall=524288 kern.ipc.semmsl=512 kern.ipc.semmap=256 kern.ipc.somaxconn=2048 kern.maxfiles=65536 vfs.read_max=32 kern.ipc.semmni=256 kern.ipc.semmns=2048 results: FreeBSD 7.0-BETA4 amd64 (cvsup on 20.12) GENERIC with SCHED_ULE #threads#transactions/sec user/system 1 500 7.4%,5.3% 5 199030.9%,23.4% 10 251039.9%,35.0% 20 254944.5%,43.5% 40 192129.8%,59.4% 60 158022.7%,70.6% 80 134118.9%,75.9% 100 122716.5%,79.3% Linux #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 What can be done to improve these results? Best Regards ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] . postgresql has some poor default settings on FreeBSD. You need to add: stats_command_string = off update_process_title = off Kris I use a copy of postgresql.conf file from linux. Only 'stats_command_string = on' was commented. Here are results with these settings and lock_manager patch: #threads#transactions/sec 1 582 5 2154 10 2253 20 2705 40 2215 60 1713 80 1574 100 1256 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHe7fAxJBWvpalMpkRAipaAJ4uNYByfRxOnPFf4HwG4MqV/zFDIACcC3Pj W8uGwpdL0oBG0OKHJ/4b/PQ= =1jGZ -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kris Kennaway wrote: Krassimir Slavchev wrote: Hello, I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. dmesg: ... CPU: Intel(R) Xeon(R) CPU X5450 @ 3.00GHz (3000.02-MHz K8-class CPU) Origin = GenuineIntel Id = 0x10676 Stepping = 6 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0xce3bdSSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,b19 AMD Features=0x2800SYSCALL,LM AMD Features2=0x1LAHF Cores per package: 4 usable memory = 8575655936 (8178 MB) avail memory = 8288337920 (7904 MB) ACPI APIC Table: HP ProLiant FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs ... test: sysbench --num-threads=${i} --test=oltp --pgsql-user=bench --pgsql-db=bench --db-driver=pgsql --max-time=60 --max-requests=0 --oltp-read-only=on run tuning: kern.ipc.shmmax=2147483647 kern.ipc.shmall=524288 kern.ipc.semmsl=512 kern.ipc.semmap=256 kern.ipc.somaxconn=2048 kern.maxfiles=65536 vfs.read_max=32 kern.ipc.semmni=256 kern.ipc.semmns=2048 results: FreeBSD 7.0-BETA4 amd64 (cvsup on 20.12) GENERIC with SCHED_ULE #threads#transactions/sec user/system 1 500 7.4%,5.3% 5 199030.9%,23.4% 10 251039.9%,35.0% 20 254944.5%,43.5% 40 192129.8%,59.4% 60 158022.7%,70.6% 80 134118.9%,75.9% 100 122716.5%,79.3% Linux #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 What can be done to improve these results? Best Regards ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] . postgresql has some poor default settings on FreeBSD. You need to add: stats_command_string = off update_process_title = off Kris I use a copy of postgresql.conf file from linux. Only 'stats_command_string = on' was commented. Here are results with these settings and lock_manager patch: #threads#transactions/sec 1 582 5 2154 10 2253 20 2705 40 2215 60 1713 80 1574 100 1256 Please enable LOCK_PROFILING in your kernel and then do sysctl debug.lock.prof.enable=1 run the test with 8 threads sysctl debug.lock.prof.enable=0 and send me the output of sysctl debug.lock.prof.stats Kris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
Kris Kennaway wrote: Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kris Kennaway wrote: Krassimir Slavchev wrote: Hello, I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. dmesg: ... CPU: Intel(R) Xeon(R) CPU X5450 @ 3.00GHz (3000.02-MHz K8-class CPU) Origin = GenuineIntel Id = 0x10676 Stepping = 6 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0xce3bdSSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,b19 AMD Features=0x2800SYSCALL,LM AMD Features2=0x1LAHF Cores per package: 4 usable memory = 8575655936 (8178 MB) avail memory = 8288337920 (7904 MB) ACPI APIC Table: HP ProLiant FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs ... test: sysbench --num-threads=${i} --test=oltp --pgsql-user=bench --pgsql-db=bench --db-driver=pgsql --max-time=60 --max-requests=0 --oltp-read-only=on run tuning: kern.ipc.shmmax=2147483647 kern.ipc.shmall=524288 kern.ipc.semmsl=512 kern.ipc.semmap=256 kern.ipc.somaxconn=2048 kern.maxfiles=65536 vfs.read_max=32 kern.ipc.semmni=256 kern.ipc.semmns=2048 results: FreeBSD 7.0-BETA4 amd64 (cvsup on 20.12) GENERIC with SCHED_ULE #threads#transactions/sec user/system 1 500 7.4%,5.3% 5 199030.9%,23.4% 10 251039.9%,35.0% 20 254944.5%,43.5% 40 192129.8%,59.4% 60 158022.7%,70.6% 80 134118.9%,75.9% 100 122716.5%,79.3% Linux #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 What can be done to improve these results? Best Regards ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] . postgresql has some poor default settings on FreeBSD. You need to add: stats_command_string = off update_process_title = off Kris I use a copy of postgresql.conf file from linux. Only 'stats_command_string = on' was commented. Here are results with these settings and lock_manager patch: #threads#transactions/sec 1 582 5 2154 10 2253 20 2705 40 2215 60 1713 80 1574 100 1256 Please enable LOCK_PROFILING in your kernel and then do sysctl debug.lock.prof.enable=1 run the test with 8 threads sysctl debug.lock.prof.enable=0 and send me the output of sysctl debug.lock.prof.stats Kris Are you using postgresql 8.1 or older? It didn't have the update_process_title option to disable the setproctitle() calls that have a large performance penalty on FreeBSD. Try with 8.2 or hack the source to disable it. Kris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kip Macy wrote: Are you sure that the settitle call is disabled on FreeBSD? I don't know anything about this. Could you explain? On 12/20/07, Krassimir Slavchev [EMAIL PROTECTED] wrote: Hello, I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. dmesg: ... CPU: Intel(R) Xeon(R) CPU X5450 @ 3.00GHz (3000.02-MHz K8-class CPU) Origin = GenuineIntel Id = 0x10676 Stepping = 6 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0xce3bdSSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,b19 AMD Features=0x2800SYSCALL,LM AMD Features2=0x1LAHF Cores per package: 4 usable memory = 8575655936 (8178 MB) avail memory = 8288337920 (7904 MB) ACPI APIC Table: HP ProLiant FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs ... test: sysbench --num-threads=${i} --test=oltp --pgsql-user=bench --pgsql-db=bench --db-driver=pgsql --max-time=60 --max-requests=0 --oltp-read-only=on run tuning: kern.ipc.shmmax=2147483647 kern.ipc.shmall=524288 kern.ipc.semmsl=512 kern.ipc.semmap=256 kern.ipc.somaxconn=2048 kern.maxfiles=65536 vfs.read_max=32 kern.ipc.semmni=256 kern.ipc.semmns=2048 results: FreeBSD 7.0-BETA4 amd64 (cvsup on 20.12) GENERIC with SCHED_ULE #threads#transactions/sec user/system 1 500 7.4%,5.3% 5 199030.9%,23.4% 10 251039.9%,35.0% 20 254944.5%,43.5% 40 192129.8%,59.4% 60 158022.7%,70.6% 80 134118.9%,75.9% 100 122716.5%,79.3% Linux #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 What can be done to improve these results? Best Regards ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHb29axJBWvpalMpkRAszAAJ9bBbHr5lskjC9fD3zHrZNvoRQL6QCeP0Sh B1YvfmvSuyXkIG15HUZ1Hfw= =ZdzN -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
Krassimir Slavchev wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. dmesg: ... CPU: Intel(R) Xeon(R) CPU X5450 @ 3.00GHz (3000.02-MHz K8-class CPU) Origin = GenuineIntel Id = 0x10676 Stepping = 6 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0xce3bdSSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,b19 AMD Features=0x2800SYSCALL,LM AMD Features2=0x1LAHF Cores per package: 4 usable memory = 8575655936 (8178 MB) avail memory = 8288337920 (7904 MB) ACPI APIC Table: HP ProLiant FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs ... test: sysbench --num-threads=${i} --test=oltp --pgsql-user=bench - --pgsql-db=bench --db-driver=pgsql --max-time=60 --max-requests=0 - --oltp-read-only=on run tuning: kern.ipc.shmmax=2147483647 kern.ipc.shmall=524288 kern.ipc.semmsl=512 kern.ipc.semmap=256 kern.ipc.somaxconn=2048 kern.maxfiles=65536 vfs.read_max=32 kern.ipc.semmni=256 kern.ipc.semmns=2048 results: FreeBSD 7.0-BETA4 amd64 (cvsup on 20.12) GENERIC with SCHED_ULE #threads#transactions/sec user/system 1 500 7.4%,5.3% 5 199030.9%,23.4% 10 251039.9%,35.0% 20 254944.5%,43.5% 40 192129.8%,59.4% 60 158022.7%,70.6% 80 134118.9%,75.9% 100 122716.5%,79.3% Linux #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 What can be done to improve these results? Best Regards -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHal7hxJBWvpalMpkRAsaiAJ9G3ZoTv6cUbR4yix07TEMf7PKm7gCgoroM +xvcXkcaFjSTxYfjk5rqMko= =Tpsq -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] . postgresql has some poor default settings on FreeBSD. You need to add: stats_command_string = off update_process_title = off Kris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Claus Guttesen wrote: I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. results: FreeBSD 7.0-BETA4 amd64 (cvsup on 20.12) GENERIC with SCHED_ULE #threads#transactions/sec user/system 1 500 7.4%,5.3% 5 199030.9%,23.4% 10 251039.9%,35.0% 20 254944.5%,43.5% 40 192129.8%,59.4% 60 158022.7%,70.6% 80 134118.9%,75.9% 100 122716.5%,79.3% Linux #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 What can be done to improve these results? What postgres-version did you use for this benchmark? Eventhough this is a synthetic benchmark the difference in performance may indicate some penalties on 8-core servers on FreeBSD. 8.2.5 from ports According to http://people.freebsd.org/~kris/scaling/mysql.html mysql scale the same until until 8 clients on both Linux and FreeBSD. This is an older test though and Linux has probably done some optimizations. Could be interesting so see whether the results differ if you disable one of the cpu's and rerun the tests. 2 cores: #threads#transactions/sec user/system 1 541 27.8%,19.0% 5 106857.1%,28.4% 40 108655.2%,32.1% 100 997 57.5%,28.9% 4 cores: #threads#transactions/sec user/system 1 512 12.3%,8.8% 5 153150.3%,31.5% 10 165151.6%,33.7% 40 149047.3%,37.8% 60 132044.0%,44.8% 100 111737.5%,51.8% -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHa4t6xJBWvpalMpkRAgKOAKCgZiNdHgL5zyIcrjKp9W8/2gW2JACfVcWz HdAOjT+DS6ysVohTUtNg9Fg= =pGDK -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Krassimir Slavchev wrote: Hello, I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. dmesg: ... CPU: Intel(R) Xeon(R) CPU X5450 @ 3.00GHz (3000.02-MHz K8-class CPU) Origin = GenuineIntel Id = 0x10676 Stepping = 6 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0xce3bdSSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,b19 AMD Features=0x2800SYSCALL,LM AMD Features2=0x1LAHF Cores per package: 4 usable memory = 8575655936 (8178 MB) avail memory = 8288337920 (7904 MB) ACPI APIC Table: HP ProLiant FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs ... test: sysbench --num-threads=${i} --test=oltp --pgsql-user=bench --pgsql-db=bench --db-driver=pgsql --max-time=60 --max-requests=0 --oltp-read-only=on run tuning: kern.ipc.shmmax=2147483647 kern.ipc.shmall=524288 kern.ipc.semmsl=512 kern.ipc.semmap=256 kern.ipc.somaxconn=2048 kern.maxfiles=65536 vfs.read_max=32 kern.ipc.semmni=256 kern.ipc.semmns=2048 results: FreeBSD 7.0-BETA4 amd64 (cvsup on 20.12) GENERIC with SCHED_ULE #threads#transactions/sec user/system 1 500 7.4%,5.3% 5 199030.9%,23.4% 10 251039.9%,35.0% 20 254944.5%,43.5% 40 192129.8%,59.4% 60 158022.7%,70.6% 80 134118.9%,75.9% 100 122716.5%,79.3% With kern.hz=100 #threads#transactions/sec 1 553 5 2133 10 2168 20 2567 40 2279 60 1773 80 1529 100 1463 What can be done to improve these results? Best Regards ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHa5gaxJBWvpalMpkRAgjFAJ9Yqz7Yj6ee/z7FUQkDncRuetAwTwCfSBXY h66FOS97LRGr/VmAmdQcJKQ= =jEDa -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
Are you sure that the settitle call is disabled on FreeBSD? On 12/20/07, Krassimir Slavchev [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. dmesg: ... CPU: Intel(R) Xeon(R) CPU X5450 @ 3.00GHz (3000.02-MHz K8-class CPU) Origin = GenuineIntel Id = 0x10676 Stepping = 6 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0xce3bdSSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,b19 AMD Features=0x2800SYSCALL,LM AMD Features2=0x1LAHF Cores per package: 4 usable memory = 8575655936 (8178 MB) avail memory = 8288337920 (7904 MB) ACPI APIC Table: HP ProLiant FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs ... test: sysbench --num-threads=${i} --test=oltp --pgsql-user=bench - --pgsql-db=bench --db-driver=pgsql --max-time=60 --max-requests=0 - --oltp-read-only=on run tuning: kern.ipc.shmmax=2147483647 kern.ipc.shmall=524288 kern.ipc.semmsl=512 kern.ipc.semmap=256 kern.ipc.somaxconn=2048 kern.maxfiles=65536 vfs.read_max=32 kern.ipc.semmni=256 kern.ipc.semmns=2048 results: FreeBSD 7.0-BETA4 amd64 (cvsup on 20.12) GENERIC with SCHED_ULE #threads#transactions/sec user/system 1 500 7.4%,5.3% 5 199030.9%,23.4% 10 251039.9%,35.0% 20 254944.5%,43.5% 40 192129.8%,59.4% 60 158022.7%,70.6% 80 134118.9%,75.9% 100 122716.5%,79.3% Linux #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 What can be done to improve these results? Best Regards -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHal7hxJBWvpalMpkRAsaiAJ9G3ZoTv6cUbR4yix07TEMf7PKm7gCgoroM +xvcXkcaFjSTxYfjk5rqMko= =Tpsq -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
I have read all related threads about performance problems with multi core systems but still have no idea what to do to make thinks better. Below are results of testing postgresql on HP DL380G5 using sysbench. The results are comparable to: http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0 but the same tests running on the same hardware using Linux (kernel 2.6.18-53.1.4.el5 SMP x86_64) are very different. PostgreSQL is tuned equal. results: FreeBSD 7.0-BETA4 amd64 (cvsup on 20.12) GENERIC with SCHED_ULE #threads#transactions/sec user/system 1 500 7.4%,5.3% 5 199030.9%,23.4% 10 251039.9%,35.0% 20 254944.5%,43.5% 40 192129.8%,59.4% 60 158022.7%,70.6% 80 134118.9%,75.9% 100 122716.5%,79.3% Linux #threads#transactions/sec 1 693 5 3539 10 5789 20 5791 40 5661 60 5517 80 5401 100 5319 What can be done to improve these results? What postgres-version did you use for this benchmark? Eventhough this is a synthetic benchmark the difference in performance may indicate some penalties on 8-core servers on FreeBSD. According to http://people.freebsd.org/~kris/scaling/mysql.html mysql scale the same until until 8 clients on both Linux and FreeBSD. This is an older test though and Linux has probably done some optimizations. Could be interesting so see whether the results differ if you disable one of the cpu's and rerun the tests. -- regards Claus When lenity and cruelty play for a kingdom, the gentlest gamester is the soonest winner. Shakespeare ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance!
On Thu, 20 Dec 2007 15:35:52 +0100 Claus Guttesen [EMAIL PROTECTED] wrote: What postgres-version did you use for this benchmark? Eventhough this is a synthetic benchmark the difference in performance may indicate some penalties on 8-core servers on FreeBSD. According to http://people.freebsd.org/~kris/scaling/mysql.html mysql scale the same until until 8 clients on both Linux and FreeBSD. This is an older test though and Linux has probably done some optimizations. Could be interesting so see whether the results differ if you disable one of the cpu's and rerun the tests. I would try asking in the pgsql's performance mailing list , [EMAIL PROTECTED] (u'll need to subscribe first). there was a bit of discussion on this subject recently on a somewhat unrelated thread ( http://archives.postgresql.org/pgsql-performance/2007-12/msg00276.php )... B _ {Beto|Norberto|Numard} Meijome If you're not part of the solution, you're part of the precipitate. I speak for myself, not my employer. Contents may be hot. Slippery when wet. Reading disclaimers makes you go blind. Writing them is worse. You have been Warned. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x
Just an interesting thing to note ... Danial Thom is accusing the FreeBSD team of lying and being dishonest. He's saying that FreeBSD is going to die and DragonFly BSD will take its place in one year, and that Matt Dillon had more IQ than the whole FreeBSD team together. Not very long ago, the very same person said exactly the opposite. He wrote that DragonFly was going in the wrong direction, that the project would die and never would catch up with FreeBSD, and he insulted Matt Dillon and other developers up to the point he had to be banned from the mailing list in order to get the S/N ration back to a usable level. (I'm not wasting my time with quotes; you can easily find all of that with Google if you care.) Now look who is lying and being dishonest. Best regards Oliver PS: Redirected to -chat. PPS: I've switched several machines from 4.x to RELENG_6, skipping 5.x alltogether, and so far I'm very pleased with the new features and the performance (except for the NFS performance which is not as good as it used to be, but I assume it's already being worked on, since NFS is such an important thing). -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. I started using PostgreSQL around a month ago, and the feeling is similar to the switch from Linux to FreeBSD in '96 -- 'wow!'. -- Oddbjorn Steffensen ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x
Quoting Chris [EMAIL PROTECTED] (from Mon, 16 Oct 2006 19:00:54 +0100): On 16/10/06, Mark Kirkwood [EMAIL PROTECTED] wrote: He might have got further by volunteering to create and supply profiles for those specific workloads that were faster in 4.x than 6.x on UP machinery etc... i.e. help make 6.x better rather than discourage the development team (whose efforts are much appreciated by the rest of us that are happily using 6.x...) I recently ordered some servers from a datacentre on lease, specs were UP p4 2.8ghz gig of ddr2 ram and sata hd, intel lan card. None of the servers would boot in freebsd 6.x, they booted in freebsd 4.x but needed a pata controller, they only worked properly in freebsd 5.x. What's the Problem Report number of your bug report? Does it contain the error message of the problem (if there's one), a detailed description of the hardware, and anything else what may be interesting to know about this situation? It seems their are 2 major problems with freebsd at the moment (1) is the hardware support is still way behind both linux and windows and its very frustrating in the amount of datacentres that dont support freebsd. and (2) the uniprocessor performance remains below par. What's the PR number of your report where you describe under which specific load (and how to produce/simulate this load) it doesn't perform as fast as other operating systems? Bye, Alexander. -- MSDOS is not dead, it just smells that way. -- Henry Spencer http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x
On 16/10/06, Mark Kirkwood [EMAIL PROTECTED] wrote: Mark Linimon wrote: On Sun, Oct 15, 2006 at 02:01:08PM -0400, Michael Butler wrote: For everyone's benefit then, please feel free to submit your patches along with your technical analysis. I think his best bet is a fork, instead. Then he can tell all the people that volunteer to work on _his_ project exactly what to do, and see how far he gets with that approach. He might have got further by volunteering to create and supply profiles for those specific workloads that were faster in 4.x than 6.x on UP machinery etc... i.e. help make 6.x better rather than discourage the development team (whose efforts are much appreciated by the rest of us that are happily using 6.x...) regards Mark ___ I recently ordered some servers from a datacentre on lease, specs were UP p4 2.8ghz gig of ddr2 ram and sata hd, intel lan card. None of the servers would boot in freebsd 6.x, they booted in freebsd 4.x but needed a pata controller, they only worked properly in freebsd 5.x. It seems their are 2 major problems with freebsd at the moment (1) is the hardware support is still way behind both linux and windows and its very frustrating in the amount of datacentres that dont support freebsd. and (2) the uniprocessor performance remains below par. The freebsd team it would help to realise not everyone can pick and choose their hardware and not everyone has the budget for state of the art hardware, certianly if you go around browsing datacentre websites for dedicated servers the dominant spec is x86 single processor, dual cpu is growing but still not dominent and I think 32bit UP wont be dead for at least half a decade. freebsd 4.x their is nothing to dispute, its leaner and smoother on pretty much every UP setup and I think it would do freebsd's reputation some good if a 4.12 was to come about. just my 2 pence worth. Chris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x
--- Mark Linimon [EMAIL PROTECTED] wrote: On Sun, Oct 15, 2006 at 02:01:08PM -0400, Michael Butler wrote: For everyone's benefit then, please feel free to submit your patches along with your technical analysis. I think his best bet is a fork, instead. Then he can tell all the people that volunteer to work on _his_ project exactly what to do, and see how far he gets with that approach. As an extra-special bonus, since it's the BSD license, he can start with whatever version of FreeBSD he finds most meets his needs. Even better, with his own project, he can then redirect all his postings there and leave the rest of us in peace. Until then, I think I'll watch out for any flying monkeys. I consider their existance equally probable. mcl Why do I need to start a project? Matt Dillon is already doing it. One thing that Matt has proved is that IQ isn't cumulative. Because hes doing on his own what an entire team of FreeBSD engineers can't do. But hey, you're not getting paid, so I guess we shouldn't expect anything good. Bravo for trying guys. We appreciate your wasted efforts. I'm not nearly as concerned about the project at this point. Dfly will be usable before freebsd, and at least we know there's someone that knows what they're doing over there. What concerns me is the lying to all of the small businessman out there. People wasting their money on hardware that freebsd can't utilize. And you clowns telling them how great it is. Its just plain dishonest. DT __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x
If you see/grep Danial Thom in FreeBSD related, consider this: http://www.netfunny.com/rhf/jokes/88q1/13785.8.html http://amasci.com/weird/flamer.html http://en.wikipedia.org/wiki/Flame_war My personal fav' is the first link... How do we know that 'DT' even exists? Hmmm. DT - S, go away for you do not exist. RB. On 10/16/06, Danial Thom [EMAIL PROTECTED] wrote: --- Mark Linimon [EMAIL PROTECTED] wrote: On Sun, Oct 15, 2006 at 02:01:08PM -0400, Michael Butler wrote: For everyone's benefit then, please feel free to submit your patches along with your technical analysis. I think his best bet is a fork, instead. Then he can tell all the people that volunteer to work on _his_ project exactly what to do, and see how far he gets with that approach. As an extra-special bonus, since it's the BSD license, he can start with whatever version of FreeBSD he finds most meets his needs. Even better, with his own project, he can then redirect all his postings there and leave the rest of us in peace. Until then, I think I'll watch out for any flying monkeys. I consider their existance equally probable. mcl Why do I need to start a project? Matt Dillon is already doing it. One thing that Matt has proved is that IQ isn't cumulative. Because hes doing on his own what an entire team of FreeBSD engineers can't do. But hey, you're not getting paid, so I guess we shouldn't expect anything good. Bravo for trying guys. We appreciate your wasted efforts. I'm not nearly as concerned about the project at this point. Dfly will be usable before freebsd, and at least we know there's someone that knows what they're doing over there. What concerns me is the lying to all of the small businessman out there. People wasting their money on hardware that freebsd can't utilize. And you clowns telling them how great it is. Its just plain dishonest. DT __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ freebsd-performance@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to [EMAIL PROTECTED] -- Why use Gmail? Cause HOTMAIL SUCKS! If you *STILL* are using HOTMAIL you only have to ask yourself Why? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x
On Mon, 16 Oct 2006 16:13:13 -0700 (PDT) Danial Thom [EMAIL PROTECTED] wrote: Why do I need to start a project? Matt Dillon is already doing it. One thing that Matt has proved is that IQ isn't cumulative. Because hes doing on his own what an entire team of FreeBSD engineers can't do. But hey, you're not getting paid, so I guess we shouldn't expect anything good. Bravo for trying guys. We appreciate your wasted efforts. Sorry, but I don't get your point. Why aren't you using Dragonfly or Linux or any other OS that suits your needs already? I'm not nearly as concerned about the project at this point. Dfly will be usable before freebsd, and at least we know there's someone that knows what they're doing over there. What concerns me is the lying to all of the small businessman out there. People wasting their money on hardware that freebsd can't utilize. And you clowns telling them how great it is. Its just plain dishonest. And another: have you read (and understood) the copyright message? Specifically this part (deCAPSed for your comfort): This software is provided by the regents and contributors ``as is'' and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. I don't have the IQ to understand why do you keep using FreeBSD if it makes you unhappy, doesn't support the hardware you bought/have, perform poorly on most situations you have to deal with, and you think its developers don't have a clue of what they're doing. -- Ricardo Nabinger Sanchez [EMAIL PROTECTED],wait4.org} Powered by FreeBSD Left to themselves, things tend to go from bad to worse. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
Hi Kip, Where you a troll when you outlined how your port of FreeBSD 6 to Solaris was so bad that it was virtually unusable? Stating facts is not trolling. The fact that you may not want to hear it is your own problem. I'm fairly certain that you know that every single thing I'm saying is true, but you have some agenda that it doesn't suit. You can't keep promoting this junk they're putting out. You can't just keep kicking the Matt Dillons out of the camp because they think that your design is a piece of crap. At some point you have to come to terms with the fact that your kernel design stinks, or its never going to get fixed. DT --- Kip Macy [EMAIL PROTECTED] wrote: Please do not feed the trolls. -Kip On Thu, 12 Oct 2006, Danial Thom wrote: --- Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting Dan Lukes [EMAIL PROTECTED] (from Thu, 12 Oct 2006 09:43:20 +0200): [moved from security@ to [EMAIL PROTECTED] The main problem is - 6.x is still not competitive replacement for 4.x. I'm NOT speaking about old unsupported hardware - I speaked about performance in some situation and believe in it's stability. You can't be sure that a committer has the resources to setup an environment where he is able to reproduce your performance problems. You on the other hand have hands-on experience with the performance problem. If you are able to setup a -current system (because there are changes which may affect performance already, and it is the place where the nuw stuff will be developt) which exposes the bad behavior, you could make yourself familiar with the pmc framework (http://wiki.freebsd.org/PmcTools, I'm sure jkoshy@ will help if you have questions) and point out the bottlenecks on current@ and/or performance@ (something similar happened for MySQL, and now we have a webpage in the wiki about it). Without such reports, we can't handle the issue. Further discussion about this should happen in performance@ or [EMAIL PROTECTED] Bye, Alexander. Maybe its just time for the entire FreeBSD team to come out of its world of delusion and come to terms with what every real-life user of FreeBSD knows: In how ever many years of development, there is still no good reason to use anything other than FreeBSD 4.x except that 4.x doesn't support a lot of newer harder. There is no performance advantage in real world applications with multiple processors, and the performance is far worse with 1 processor. The right thing to do is to port the SATA support and new NIC support back to 4.x and support both. 4.x is far superior on a Uniprocessor system and FreeBSD-5+ may be an entire re-write away from ever being any good at MP. Come to terms with it, PLEASE, because it is the case and saying otherwise won't change it. My prediction is that a year from now we'll all be using DragonflyBSD and you guys will be looking for a new bunch of beta-test guinea pigs. DT __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ freebsd-performance@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-performance@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to [EMAIL PROTECTED] __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
On Sun, Oct 15, 2006 at 07:57:32AM -0700, Danial Thom wrote: Hi Kip, Where you a troll when you outlined how your port of FreeBSD 6 to Solaris was so bad that it was virtually unusable? Stating facts is not trolling. And you crossposted this to performance...why? Kip might be right, you could be a troll. I am not on freebsd-stable. And I do miss Mr. Dillon but I don't think he needs your support, unless you are going to run DragonFly BSD - which I kind of doubt you are since it isn't 64bit and does not fit your latest must be greated mentality. -- Mike Horwath, reachable via [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
On Sun, Oct 15, 2006 at 07:57:32AM -0700, Danial Thom wrote: Stating facts is not trolling. true, but ... The fact that you may not want to hear it is your own problem [...] You can't keep promoting this junk they're putting out. You can't just keep kicking the Matt Dillons out of the camp because they think that your design is a piece of crap. At some point you have to come to terms with the fact that your kernel design stinks [...] ... *is*. I think there are valid points to be made about 4.X vs 5.X vs 6.X (which is why, for the sake of being informative, I wrote an article about it). Performance was also discussed extensively at BSDCan, and a lot of work was done on improvements and ideas were discussed for the next steps to make (e.g.: actual work, not just talk). This work is continuing. But as long as you keep the above tone, I and everyone else who is actually doing the work to advance the project will just ignore you; because, frankly, there's too much work to do and in any case, life is just too short. (I intend to do just that from now on, so I will not be adding any more to this thread. You may have the last word, if that kind of thing is important to you.) Finally, if you think Matt's design and/or ability to accept criticism is better than ours, then DragonFly is clearly a better choice for you. mcl ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
--- Mark Linimon [EMAIL PROTECTED] wrote: On Sun, Oct 15, 2006 at 07:57:32AM -0700, Danial Thom wrote: Stating facts is not trolling. true, but ... The fact that you may not want to hear it is your own problem [...] You can't keep promoting this junk they're putting out. You can't just keep kicking the Matt Dillons out of the camp because they think that your design is a piece of crap. At some point you have to come to terms with the fact that your kernel design stinks [...] ... *is*. I think there are valid points to be made about 4.X vs 5.X vs 6.X (which is why, for the sake of being informative, I wrote an article about it). Performance was also discussed extensively at BSDCan, and a lot of work was done on improvements and ideas were discussed for the next steps to make (e.g.: actual work, not just talk). This work is continuing. But as long as you keep the above tone, I and everyone else who is actually doing the work to advance the project will just ignore you; because, frankly, there's too much work to do and in any case, life is just too short. (I intend to do just that from now on, so I will not be adding any more to this thread. You may have the last word, if that kind of thing is important to you.) Finally, if you think Matt's design and/or ability to accept criticism is better than ours, then DragonFly is clearly a better choice for you. Dragonfly will be a better choice for everyone in a year or so, but as of now there are no choices. The *tone* is caused by the entire user base being lied to over and over, performance is addressed in the next release, yada, yada, yada. Those of us in the know are tired of it. You're wasting a lot of our time testing this crap over and over. So stop lying. Stop allowing Kris Kennaway to BS everyone on the questions list over and over and calling anyone who speaks the truth a troll. Fess up to your debacle publically because you're affecting people's businesses with your lying, trying to cover your asses and refusing to admit that you are just rambling cluelessly about. There isn't one person on that team that knows how to fix what's wrong, and you're using thousand of people trying to run businesses as guinea pigs. Thats why some of us have a *tone* DT __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Danial Thom wrote: There isn't one person on that team that knows how to fix what's wrong .. For everyone's benefit then, please feel free to submit your patches along with your technical analysis, Michael -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFFMndkQv9rrgRC1JIRAnQjAKCg2/LAVNAmezwLPixJADE2D618SQCeKc+R SqrDsGNiBWAn8b4JfhTsWAw= =xWRp -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
Danial Thom wrote: --- Mark Linimon [EMAIL PROTECTED] wrote: On Sun, Oct 15, 2006 at 07:57:32AM -0700, Danial Thom wrote: Stating facts is not trolling. true, but ... The fact that you may not want to hear it is your own problem [...] You can't keep promoting this junk they're putting out. You can't just keep kicking the Matt Dillons out of the camp because they think that your design is a piece of crap. At some point you have to come to terms with the fact that your kernel design stinks [...] ... *is*. I think there are valid points to be made about 4.X vs 5.X vs 6.X (which is why, for the sake of being informative, I wrote an article about it). Performance was also discussed extensively at BSDCan, and a lot of work was done on improvements and ideas were discussed for the next steps to make (e.g.: actual work, not just talk). This work is continuing. But as long as you keep the above tone, I and everyone else who is actually doing the work to advance the project will just ignore you; because, frankly, there's too much work to do and in any case, life is just too short. (I intend to do just that from now on, so I will not be adding any more to this thread. You may have the last word, if that kind of thing is important to you.) Finally, if you think Matt's design and/or ability to accept criticism is better than ours, then DragonFly is clearly a better choice for you. Dragonfly will be a better choice for everyone in a year or so, but as of now there are no choices. The *tone* is caused by the entire user base being lied to over and over, performance is addressed in the next release, yada, yada, yada. Those of us in the know are tired of it. You're wasting a lot of our time testing this crap over and over. So stop lying. Stop allowing Kris Kennaway to BS everyone on the questions list over and over and calling anyone who speaks the truth a troll. Fess up to your debacle publically because you're affecting people's businesses with your lying, trying to cover your asses and refusing to admit that you are just rambling cluelessly about. There isn't one person on that team that knows how to fix what's wrong, and you're using thousand of people trying to run businesses as guinea pigs. Thats why some of us have a *tone* DT Congrats, you are the second entry in my permanent kill file. Scott ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x
On Sun, Oct 15, 2006 at 02:01:08PM -0400, Michael Butler wrote: For everyone's benefit then, please feel free to submit your patches along with your technical analysis. I think his best bet is a fork, instead. Then he can tell all the people that volunteer to work on _his_ project exactly what to do, and see how far he gets with that approach. As an extra-special bonus, since it's the BSD license, he can start with whatever version of FreeBSD he finds most meets his needs. Even better, with his own project, he can then redirect all his postings there and leave the rest of us in peace. Until then, I think I'll watch out for any flying monkeys. I consider their existance equally probable. mcl ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x
Mark Linimon wrote: On Sun, Oct 15, 2006 at 02:01:08PM -0400, Michael Butler wrote: For everyone's benefit then, please feel free to submit your patches along with your technical analysis. I think his best bet is a fork, instead. Then he can tell all the people that volunteer to work on _his_ project exactly what to do, and see how far he gets with that approach. He might have got further by volunteering to create and supply profiles for those specific workloads that were faster in 4.x than 6.x on UP machinery etc... i.e. help make 6.x better rather than discourage the development team (whose efforts are much appreciated by the rest of us that are happily using 6.x...) regards Mark ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
--- Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting Dan Lukes [EMAIL PROTECTED] (from Thu, 12 Oct 2006 09:43:20 +0200): [moved from security@ to [EMAIL PROTECTED] The main problem is - 6.x is still not competitive replacement for 4.x. I'm NOT speaking about old unsupported hardware - I speaked about performance in some situation and believe in it's stability. You can't be sure that a committer has the resources to setup an environment where he is able to reproduce your performance problems. You on the other hand have hands-on experience with the performance problem. If you are able to setup a -current system (because there are changes which may affect performance already, and it is the place where the nuw stuff will be developt) which exposes the bad behavior, you could make yourself familiar with the pmc framework (http://wiki.freebsd.org/PmcTools, I'm sure jkoshy@ will help if you have questions) and point out the bottlenecks on current@ and/or performance@ (something similar happened for MySQL, and now we have a webpage in the wiki about it). Without such reports, we can't handle the issue. Further discussion about this should happen in performance@ or [EMAIL PROTECTED] Bye, Alexander. Maybe its just time for the entire FreeBSD team to come out of its world of delusion and come to terms with what every real-life user of FreeBSD knows: In how ever many years of development, there is still no good reason to use anything other than FreeBSD 4.x except that 4.x doesn't support a lot of newer harder. There is no performance advantage in real world applications with multiple processors, and the performance is far worse with 1 processor. The right thing to do is to port the SATA support and new NIC support back to 4.x and support both. 4.x is far superior on a Uniprocessor system and FreeBSD-5+ may be an entire re-write away from ever being any good at MP. Come to terms with it, PLEASE, because it is the case and saying otherwise won't change it. My prediction is that a year from now we'll all be using DragonflyBSD and you guys will be looking for a new bunch of beta-test guinea pigs. DT __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
On Thu 12 Oct 07:19, Danial Thom wrote: [...] Maybe its just time for the entire FreeBSD team to come out of its world of delusion and come to terms with what every real-life user of FreeBSD knows: In how ever many years of development, there is still no good reason to use anything other than FreeBSD 4.x except that 4.x doesn't support a lot of newer harder. I am a _real-life_ user of FreeBSD and I am *very* happy with FreeBSD 6.2-PRERELEASE and before that 6.1-STABLE. This particular machine is quite old - it's a 350MHz K6 - and I from a subjective point of view, FreeBSD 6.1 has been the fastest OS which has run on it. YMMV of course. Cheers, Nick. -- We're predicting third stage shutdown at 11 minutes 42 seconds. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
Danial Thom wrote: The right thing to do is to port the SATA support and new NIC support back to 4.x and support both. 4.x is far superior on a Uniprocessor system and FreeBSD-5+ may be an entire re-write away from ever being any good at MP. Come to terms with it, PLEASE, because it is the case and saying otherwise won't change it. Despite I'm initiator of this way of discussion (in security list), I can't agree with you. No way. You are not allowed to tell to someone working as volunteer several months on something that the best way is rollback all work and start from scratch. Despite of your complaints are competent or not. You totally miss the right time for this type of complain. It's too late now. 6.x is not crap in any way. It has some problem, even after many months of development, but it can be resolved if volunteers decide to use it's power to polish previously implemented code. Current 6.x is better in many parameters than 4.x. Well, some important parameters are worse, but correct decision is improve them, not rollback all work. I voted against premature EOLing of 4.x, but returning to FreeBSD 4.x is not acceptable way in any way - at least because it's the DragonBSD's nest now. Dan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
On 10/12/06, Dan Lukes [EMAIL PROTECTED] wrote: Danial Thom wrote: The right thing to do is to port the SATA support and new NIC support back to 4.x and support both. 4.x is far superior on a Uniprocessor system and FreeBSD-5+ may be an entire re-write away from ever being any good at MP. Come to terms with it, PLEASE, because it is the case and saying otherwise won't change it. Despite I'm initiator of this way of discussion (in security list), I can't agree with you. No way. You are not allowed to tell to someone working as volunteer several months on something that the best way is rollback all work and start from scratch. Despite of your complaints are competent or not. You totally miss the right time for this type of complain. It's too late now. 6.x is not crap in any way. It has some problem, even after many months of development, but it can be resolved if volunteers decide to use it's power to polish previously implemented code. Current 6.x is better in many parameters than 4.x. Well, some important parameters are worse, but correct decision is improve them, not rollback all work. I voted against premature EOLing of 4.x, but returning to FreeBSD 4.x is not acceptable way in any way - at least because it's the DragonBSD's nest now. Dan Don't go with the flow, he's a known troll. -- If it's there, and you can see it, it's real. If it's not there, and you can see it, it's virtual. If it's there, and you can't see it, it's transparent. If it's not there, and you can't see it, you erased it. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
On Thu, 12 Oct 2006 07:19:30 -0700 (PDT) Danial Thom [EMAIL PROTECTED] wrote: Maybe its just time for the entire FreeBSD team to come out of its world of delusion and come to terms with what every real-life user of FreeBSD knows: In how ever many years of development, there is still no good reason to use anything other than FreeBSD 4.x except that 4.x doesn't This statement is way too broad, I as a FreeBSD user can vouch for FreeBSD 6.x - it works in most situations. The right thing to do is to port the SATA support and new NIC support back to 4.x and support both. Go ahead - there is nothing stopping you. Come to terms with it, PLEASE, because it is the case and saying otherwise won't change it. From wher I'm sitting (and I'm just a FreeBSD user), it is you who have to come to terms with the annouced EOL for 4.x My prediction is that a year from now we'll all be using DragonflyBSD and you guys will be looking for a new bunch of beta-test guinea pigs. We'll see. -- Regards, Torfinn Ingolfsen, Norway ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
Please do not feed the trolls. -Kip On Thu, 12 Oct 2006, Danial Thom wrote: --- Alexander Leidinger [EMAIL PROTECTED] wrote: Quoting Dan Lukes [EMAIL PROTECTED] (from Thu, 12 Oct 2006 09:43:20 +0200): [moved from security@ to [EMAIL PROTECTED] The main problem is - 6.x is still not competitive replacement for 4.x. I'm NOT speaking about old unsupported hardware - I speaked about performance in some situation and believe in it's stability. You can't be sure that a committer has the resources to setup an environment where he is able to reproduce your performance problems. You on the other hand have hands-on experience with the performance problem. If you are able to setup a -current system (because there are changes which may affect performance already, and it is the place where the nuw stuff will be developt) which exposes the bad behavior, you could make yourself familiar with the pmc framework (http://wiki.freebsd.org/PmcTools, I'm sure jkoshy@ will help if you have questions) and point out the bottlenecks on current@ and/or performance@ (something similar happened for MySQL, and now we have a webpage in the wiki about it). Without such reports, we can't handle the issue. Further discussion about this should happen in performance@ or [EMAIL PROTECTED] Bye, Alexander. Maybe its just time for the entire FreeBSD team to come out of its world of delusion and come to terms with what every real-life user of FreeBSD knows: In how ever many years of development, there is still no good reason to use anything other than FreeBSD 4.x except that 4.x doesn't support a lot of newer harder. There is no performance advantage in real world applications with multiple processors, and the performance is far worse with 1 processor. The right thing to do is to port the SATA support and new NIC support back to 4.x and support both. 4.x is far superior on a Uniprocessor system and FreeBSD-5+ may be an entire re-write away from ever being any good at MP. Come to terms with it, PLEASE, because it is the case and saying otherwise won't change it. My prediction is that a year from now we'll all be using DragonflyBSD and you guys will be looking for a new bunch of beta-test guinea pigs. DT __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ freebsd-performance@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance 4.x vs. 6.x (was: e: [fbsd] HEADS UP: FreeBSD 5.3, 5.4, 6.0 EoLs coming soon)
--- Dan Lukes [EMAIL PROTECTED] wrote: Danial Thom wrote: The right thing to do is to port the SATA support and new NIC support back to 4.x and support both. 4.x is far superior on a Uniprocessor system and FreeBSD-5+ may be an entire re-write away from ever being any good at MP. Come to terms with it, PLEASE, because it is the case and saying otherwise won't change it. Despite I'm initiator of this way of discussion (in security list), I can't agree with you. No way. You are not allowed to tell to someone working as volunteer several months on something that the best way is rollback all work and start from scratch. Despite of your complaints are competent or not. You totally miss the right time for this type of complain. It's too late now. 6.x is not crap in any way. It has some problem, even after many months of development, but it can be resolved if volunteers decide to use it's power to polish previously implemented code. Current 6.x is better in many parameters than 4.x. Well, some important parameters are worse, but correct decision is improve them, not rollback all work. I voted against premature EOLing of 4.x, but returning to FreeBSD 4.x is not acceptable way in any way - at least because it's the DragonBSD's nest now. I didn't say to roll back all of the work. I said to support 4.x as a UP solution, and 6,7 or whatever as what it is now. 5+ will never be as good as 4.x UP, and many networking applications such as firewalls and routers simply will never be able to scale to utilize MP anyway. You had the best damn UP OS in the world, why not continue to support it as so while you try to figure out MP? DT __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hajimu UMEMOTO wrote: On Tue, 22 Nov 2005 12:06:12 -0800 Nate Lawson [EMAIL PROTECTED] said: nate Thank you for tracking this down. It is interesting that BIF is nate heavyweight while BST is not. I guess that is expected behavior by OEMs nate which only test on Windows and so not everyone makes BIF simple. On my nate laptops, BIF is as fast as BST. You are welcome. My laptops are also fast enough for BIF. I remembered that iwasaki-san grouched at the heavyweight of BIF when he was writing cmbat support. nate I don't like the patch approach (changing the API), however. Let me nate look at it and commit a fix that doesn't change the API. Yes, I didn't feel satisfaction with my patch, too. So, I anticipated that you say so. :-) My patch has been committed, tested, and MFCd. Thank you for your debugging help, Umemoto-san. -- Nate ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hi, On Thu, 17 Nov 2005 11:24:16 -0500 Pierre-Luc Drouin [EMAIL PROTECTED] said: Ok, there is new development. I realized by playing with debug.acpi.disabled=smbat, debug.acpi.disabled=smbat cmbat and debug.acpi.disabled=cmbat, that my laptop battery is not a smbat, but a cmbat. When I played with hw.acpi.battery.info_expire after to have applied the patch for acpi_smbat.c, it was freezing less often because that sysctl variable was shared by both cmbat and smbat. So I can only get battery status from cmbat (disabling cmbat disables the use of acpiconf -i loop). To get the status of my battery via cmbat was working fine up to 6.0-RELEASE (included), but makes my laptop to freeze since I upgraded to 6.0-stable with Nov 10th sources. What change related to cmbat between 6.0-release and 6.0-stable could be causing this? pldrouin Has someone found how to fix this problem in -stable? Perhaps, I found the cause. acpi_cmbat_get_bif() is heavy process, and it was called only when ACPIIO_CMBAT_GET_BIF ioctl was issued explicitly, until smbat stuff was committed. However, acpiio_cmbat_get_bif() is called from every acpi_battery_get_battinfo() call, now. The attached patch will bring back to former behavior. Please try it and let me know the result. It is against 7-CURRENT as of today. If you want to try it on 6-STABLE, you need to apply following diff before applying it: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/dev/acpica/acpi_smbat.c.diff?r1=1.1r2=1.2 Index: sys/dev/acpica/acpi_battery.c diff -u -p sys/dev/acpica/acpi_battery.c.orig sys/dev/acpica/acpi_battery.c --- sys/dev/acpica/acpi_battery.c.orig Mon Nov 7 01:12:11 2005 +++ sys/dev/acpica/acpi_battery.c Tue Nov 22 16:57:11 2005 @@ -178,7 +178,7 @@ acpi_battery_get_battinfo(device_t dev, * return that the device is present. */ if (ACPI_BATT_GET_STATUS(batt_dev, bst[i]) != 0 || - ACPI_BATT_GET_INFO(batt_dev, bif) != 0) + ACPI_BATT_GET_INFO(batt_dev, bif, TRUE) != 0) continue; /* If a battery is not installed, we sometimes get strange values. */ @@ -382,7 +382,7 @@ acpi_battery_ioctl(u_long cmd, caddr_t a case ACPIIO_BATT_GET_BIF: if (dev != NULL) { bzero(ioctl_arg-bif, sizeof(ioctl_arg-bif)); - error = ACPI_BATT_GET_INFO(dev, ioctl_arg-bif); + error = ACPI_BATT_GET_INFO(dev, ioctl_arg-bif, FALSE); /* * Remove invalid characters. Perhaps this should be done Index: sys/dev/acpica/acpi_cmbat.c diff -u -p sys/dev/acpica/acpi_cmbat.c.orig sys/dev/acpica/acpi_cmbat.c --- sys/dev/acpica/acpi_cmbat.c.origSun Nov 20 12:23:21 2005 +++ sys/dev/acpica/acpi_cmbat.c Tue Nov 22 16:23:30 2005 @@ -83,7 +83,8 @@ static void acpi_cmbat_info_updated(str static voidacpi_cmbat_get_bst(device_t dev); static voidacpi_cmbat_get_bif(device_t dev); static int acpi_cmbat_bst(device_t dev, struct acpi_bst *bstp); -static int acpi_cmbat_bif(device_t dev, struct acpi_bif *bifp); +static int acpi_cmbat_bif(device_t dev, struct acpi_bif *bifp, + int cache); static voidacpi_cmbat_init_battery(void *arg); static device_method_t acpi_cmbat_methods[] = { @@ -354,14 +355,15 @@ end: } static int -acpi_cmbat_bif(device_t dev, struct acpi_bif *bifp) +acpi_cmbat_bif(device_t dev, struct acpi_bif *bifp, int cache) { struct acpi_cmbat_softc *sc; sc = device_get_softc(dev); ACPI_SERIAL_BEGIN(cmbat); -acpi_cmbat_get_bif(dev); +if (!cache) + acpi_cmbat_get_bif(dev); bifp-units = sc-bif.units; bifp-dcap = sc-bif.dcap; bifp-lfcap = sc-bif.lfcap; Index: sys/dev/acpica/acpi_if.m diff -u sys/dev/acpica/acpi_if.m.orig sys/dev/acpica/acpi_if.m --- sys/dev/acpica/acpi_if.m.orig Sun Nov 20 12:23:21 2005 +++ sys/dev/acpica/acpi_if.mTue Nov 22 16:58:09 2005 @@ -206,10 +206,12 @@ # # device_t dev: Battery device # struct acpi_bif *bif: Pointer to storage for _BIF results +# int cache: if set, read from cache without any I/O to ACPI BIOS # METHOD int batt_get_info { device_tdev; struct acpi_bif *bif; + int cache; }; # Index: sys/dev/acpica/acpi_smbat.c diff -u -p sys/dev/acpica/acpi_smbat.c.orig sys/dev/acpica/acpi_smbat.c --- sys/dev/acpica/acpi_smbat.c.origTue Nov 22 16:07:38 2005 +++ sys/dev/acpica/acpi_smbat.c Tue Nov 22 18:37:13 2005 @@ -56,7 +56,8 @@ static intacpi_smbat_attach(device_t de static int acpi_smbat_shutdown(device_t dev); static int acpi_smbat_info_expired(struct timespec *lastupdated); static voidacpi_smbat_info_updated(struct timespec *lastupdated); -static int acpi_smbat_get_bif(device_t dev, struct acpi_bif *bif); +static int acpi_smbat_get_bif(device_t dev, struct acpi_bif *bif, + int cache); static int
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hajimu UMEMOTO wrote: Hi, On Thu, 17 Nov 2005 11:24:16 -0500 Pierre-Luc Drouin [EMAIL PROTECTED] said: Ok, there is new development. I realized by playing with debug.acpi.disabled=smbat, debug.acpi.disabled=smbat cmbat and debug.acpi.disabled=cmbat, that my laptop battery is not a smbat, but a cmbat. When I played with hw.acpi.battery.info_expire after to have applied the patch for acpi_smbat.c, it was freezing less often because that sysctl variable was shared by both cmbat and smbat. So I can only get battery status from cmbat (disabling cmbat disables the use of acpiconf -i loop). To get the status of my battery via cmbat was working fine up to 6.0-RELEASE (included), but makes my laptop to freeze since I upgraded to 6.0-stable with Nov 10th sources. What change related to cmbat between 6.0-release and 6.0-stable could be causing this? pldrouin Has someone found how to fix this problem in -stable? Perhaps, I found the cause. acpi_cmbat_get_bif() is heavy process, and it was called only when ACPIIO_CMBAT_GET_BIF ioctl was issued explicitly, until smbat stuff was committed. However, acpiio_cmbat_get_bif() is called from every acpi_battery_get_battinfo() call, now. The attached patch will bring back to former behavior. Please try it and let me know the result. It is against 7-CURRENT as of today. If you want to try it on 6-STABLE, you need to apply following diff before applying it: Wonderful! It fixes it!! I applied both patches and everything seams to be working fine now! ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hajimu UMEMOTO wrote: Hi, On Thu, 17 Nov 2005 11:24:16 -0500 Pierre-Luc Drouin [EMAIL PROTECTED] said: Ok, there is new development. I realized by playing with debug.acpi.disabled=smbat, debug.acpi.disabled=smbat cmbat and debug.acpi.disabled=cmbat, that my laptop battery is not a smbat, but a cmbat. When I played with hw.acpi.battery.info_expire after to have applied the patch for acpi_smbat.c, it was freezing less often because that sysctl variable was shared by both cmbat and smbat. So I can only get battery status from cmbat (disabling cmbat disables the use of acpiconf -i loop). To get the status of my battery via cmbat was working fine up to 6.0-RELEASE (included), but makes my laptop to freeze since I upgraded to 6.0-stable with Nov 10th sources. What change related to cmbat between 6.0-release and 6.0-stable could be causing this? pldrouin Has someone found how to fix this problem in -stable? Perhaps, I found the cause. acpi_cmbat_get_bif() is heavy process, and it was called only when ACPIIO_CMBAT_GET_BIF ioctl was issued explicitly, until smbat stuff was committed. However, acpiio_cmbat_get_bif() is called from every acpi_battery_get_battinfo() call, now. The attached patch will bring back to former behavior. Please try it and let me know the result. It is against 7-CURRENT as of today. If you want to try it on 6-STABLE, you need to apply following diff before applying it: Thank you for tracking this down. It is interesting that BIF is heavyweight while BST is not. I guess that is expected behavior by OEMs which only test on Windows and so not everyone makes BIF simple. On my laptops, BIF is as fast as BST. I don't like the patch approach (changing the API), however. Let me look at it and commit a fix that doesn't change the API. Thanks, -- Nate ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hi, On Tue, 22 Nov 2005 12:06:12 -0800 Nate Lawson [EMAIL PROTECTED] said: nate Thank you for tracking this down. It is interesting that BIF is nate heavyweight while BST is not. I guess that is expected behavior by OEMs nate which only test on Windows and so not everyone makes BIF simple. On my nate laptops, BIF is as fast as BST. You are welcome. My laptops are also fast enough for BIF. I remembered that iwasaki-san grouched at the heavyweight of BIF when he was writing cmbat support. nate I don't like the patch approach (changing the API), however. Let me nate look at it and commit a fix that doesn't change the API. Yes, I didn't feel satisfaction with my patch, too. So, I anticipated that you say so. :-) Sincerely, -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan [EMAIL PROTECTED] [EMAIL PROTECTED],jp.}FreeBSD.org http://www.imasy.org/~ume/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hajimu UMEMOTO wrote: Hi, On Tue, 22 Nov 2005 12:06:12 -0800 Nate Lawson [EMAIL PROTECTED] said: nate Thank you for tracking this down. It is interesting that BIF is nate heavyweight while BST is not. I guess that is expected behavior by OEMs nate which only test on Windows and so not everyone makes BIF simple. On my nate laptops, BIF is as fast as BST. You are welcome. My laptops are also fast enough for BIF. I remembered that iwasaki-san grouched at the heavyweight of BIF when he was writing cmbat support. nate I don't like the patch approach (changing the API), however. Let me nate look at it and commit a fix that doesn't change the API. Yes, I didn't feel satisfaction with my patch, too. So, I anticipated that you say so. :-) Sincerely, -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan [EMAIL PROTECTED] [EMAIL PROTECTED],jp.}FreeBSD.org http://www.imasy.org/~ume/ Is it that much related to laptop performance? Because the M70 I have is currently the fastest laptop made by Dell and cmbat makes it to hang on FreeBSD... ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hajimu UMEMOTO wrote: Hi, On Mon, 14 Nov 2005 12:40:36 -0500 Pierre-Luc Drouin [EMAIL PROTECTED] said: pldrouin Yep, smart battery is definately the problem. The performance of my pldrouin laptop is back to normal when I remove the xfce4-battery-plugin. pldrouin acpiconf -i loop reproduces the problem for me too. So it looks like pldrouin there is something wrong in smart battery. The cmbat has similar issue on some laptops. So, acpi_cmbat.c uses cache for retrieval to reduce its influence, and its expiration time is set by hw.acpi.battery.info_expire. However, acpi_smbat.c doesn't use cache. So, I made a patch. Since I don't have a laptop which has smbat, I cannot test it by myself. Please test it and let me know the result. The patch looks good to me, you should commit please. -- Nate ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hi, On Mon, 21 Nov 2005 10:37:25 -0800 Nate Lawson [EMAIL PROTECTED] said: The cmbat has similar issue on some laptops. So, acpi_cmbat.c uses cache for retrieval to reduce its influence, and its expiration time is set by hw.acpi.battery.info_expire. However, acpi_smbat.c doesn't use cache. So, I made a patch. Since I don't have a laptop which has smbat, I cannot test it by myself. Please test it and let me know the result. nate The patch looks good to me, you should commit please. Thanks for your review. I've just committed it into HEAD. Sincerely, -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan [EMAIL PROTECTED] [EMAIL PROTECTED],jp.}FreeBSD.org http://www.imasy.org/~ume/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Pierre-Luc Drouin wrote: Nate Lawson wrote: Pierre-Luc Drouin wrote: Hajimu UMEMOTO wrote: On Mon, 14 Nov 2005 12:40:36 -0500 Pierre-Luc Drouin [EMAIL PROTECTED] said: pldrouin Yep, smart battery is definately the problem. The performance of my pldrouin laptop is back to normal when I remove the xfce4-battery-plugin. pldrouin acpiconf -i loop reproduces the problem for me too. So it looks like pldrouin there is something wrong in smart battery. The cmbat has similar issue on some laptops. So, acpi_cmbat.c uses cache for retrieval to reduce its influence, and its expiration time is set by hw.acpi.battery.info_expire. However, acpi_smbat.c doesn't use cache. So, I made a patch. Since I don't have a laptop which has smbat, I cannot test it by myself. Please test it and let me know the result. The patch seams to do its job correctly, but it is still very annoying to have the whole computer to freeze for 1 second when the cache expires. What does make the whole system to freeze? Before the code was changed in 6.0-stable, FreeBSD was able to read the battery status without freezing my laptop... I have been running 3 OSes (FreeBSD, Ubuntu and Win XP) on my laptop for a while and never experienced that kind of problem with either Linux or Win XP. I guess there is something wrong in the new code added after 6.0-release. If you have both smbat and cmbat, just disable smbat if you don't like it. Add this to /boot/loader.conf: debug.acpi.disabled=smbat The pause is related to a buggy or slow EC. Caching the values happens to hide it. Do you get pauses with smbat disabled (see above for how to do this)? Can you still get battery status from cmbat? Ok, there is new development. I realized by playing with debug.acpi.disabled=smbat, debug.acpi.disabled=smbat cmbat and debug.acpi.disabled=cmbat, that my laptop battery is not a smbat, but a cmbat. When I played with hw.acpi.battery.info_expire after to have applied the patch for acpi_smbat.c, it was freezing less often because that sysctl variable was shared by both cmbat and smbat. So I can only get battery status from cmbat (disabling cmbat disables the use of acpiconf -i loop). To get the status of my battery via cmbat was working fine up to 6.0-RELEASE (included), but makes my laptop to freeze since I upgraded to 6.0-stable with Nov 10th sources. What change related to cmbat between 6.0-release and 6.0-stable could be causing this? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] Has someone found how to fix this problem in -stable? Thank you ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Nate Lawson wrote: Pierre-Luc Drouin wrote: Hajimu UMEMOTO wrote: On Mon, 14 Nov 2005 12:40:36 -0500 Pierre-Luc Drouin [EMAIL PROTECTED] said: pldrouin Yep, smart battery is definately the problem. The performance of my pldrouin laptop is back to normal when I remove the xfce4-battery-plugin. pldrouin acpiconf -i loop reproduces the problem for me too. So it looks like pldrouin there is something wrong in smart battery. The cmbat has similar issue on some laptops. So, acpi_cmbat.c uses cache for retrieval to reduce its influence, and its expiration time is set by hw.acpi.battery.info_expire. However, acpi_smbat.c doesn't use cache. So, I made a patch. Since I don't have a laptop which has smbat, I cannot test it by myself. Please test it and let me know the result. The patch seams to do its job correctly, but it is still very annoying to have the whole computer to freeze for 1 second when the cache expires. What does make the whole system to freeze? Before the code was changed in 6.0-stable, FreeBSD was able to read the battery status without freezing my laptop... I have been running 3 OSes (FreeBSD, Ubuntu and Win XP) on my laptop for a while and never experienced that kind of problem with either Linux or Win XP. I guess there is something wrong in the new code added after 6.0-release. If you have both smbat and cmbat, just disable smbat if you don't like it. Add this to /boot/loader.conf: debug.acpi.disabled=smbat The pause is related to a buggy or slow EC. Caching the values happens to hide it. Do you get pauses with smbat disabled (see above for how to do this)? Can you still get battery status from cmbat? Ok, there is new development. I realized by playing with debug.acpi.disabled=smbat, debug.acpi.disabled=smbat cmbat and debug.acpi.disabled=cmbat, that my laptop battery is not a smbat, but a cmbat. When I played with hw.acpi.battery.info_expire after to have applied the patch for acpi_smbat.c, it was freezing less often because that sysctl variable was shared by both cmbat and smbat. So I can only get battery status from cmbat (disabling cmbat disables the use of acpiconf -i loop). To get the status of my battery via cmbat was working fine up to 6.0-RELEASE (included), but makes my laptop to freeze since I upgraded to 6.0-stable with Nov 10th sources. What change related to cmbat between 6.0-release and 6.0-stable could be causing this? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hi, On Mon, 14 Nov 2005 12:40:36 -0500 Pierre-Luc Drouin [EMAIL PROTECTED] said: pldrouin Yep, smart battery is definately the problem. The performance of my pldrouin laptop is back to normal when I remove the xfce4-battery-plugin. pldrouin acpiconf -i loop reproduces the problem for me too. So it looks like pldrouin there is something wrong in smart battery. The cmbat has similar issue on some laptops. So, acpi_cmbat.c uses cache for retrieval to reduce its influence, and its expiration time is set by hw.acpi.battery.info_expire. However, acpi_smbat.c doesn't use cache. So, I made a patch. Since I don't have a laptop which has smbat, I cannot test it by myself. Please test it and let me know the result. Index: sys/dev/acpica/acpi_smbat.c diff -u -p sys/dev/acpica/acpi_smbat.c.orig sys/dev/acpica/acpi_smbat.c --- sys/dev/acpica/acpi_smbat.c.origSun Nov 6 08:55:56 2005 +++ sys/dev/acpica/acpi_smbat.c Tue Nov 15 16:41:00 2005 @@ -44,11 +44,18 @@ __FBSDID($FreeBSD: src/sys/dev/acpica/a struct acpi_smbat_softc { uint8_t sb_base_addr; device_tec_dev; + + struct acpi_bif bif; + struct acpi_bst bst; + struct timespec bif_lastupdated; + struct timespec bst_lastupdated; }; static int acpi_smbat_probe(device_t dev); static int acpi_smbat_attach(device_t dev); static int acpi_smbat_shutdown(device_t dev); +static int acpi_smbat_info_expired(struct timespec *lastupdated); +static voidacpi_smbat_info_updated(struct timespec *lastupdated); static int acpi_smbat_get_bif(device_t dev, struct acpi_bif *bif); static int acpi_smbat_get_bst(device_t dev, struct acpi_bst *bst); @@ -114,6 +121,9 @@ acpi_smbat_attach(device_t dev) return (ENXIO); } + timespecclear(sc-bif_lastupdated); + timespecclear(sc-bst_lastupdated); + if (acpi_battery_register(dev) != 0) { device_printf(dev, cannot register battery\n); return (ENXIO); @@ -132,6 +142,34 @@ acpi_smbat_shutdown(device_t dev) } static int +acpi_smbat_info_expired(struct timespec *lastupdated) +{ + struct timespec curtime; + + ACPI_SERIAL_ASSERT(smbat); + + if (lastupdated == NULL) + return (TRUE); + if (!timespecisset(lastupdated)) + return (TRUE); + + getnanotime(curtime); + timespecsub(curtime, lastupdated); + return (curtime.tv_sec 0 || + curtime.tv_sec acpi_battery_get_info_expire()); +} + +static void +acpi_smbat_info_updated(struct timespec *lastupdated) +{ + + ACPI_SERIAL_ASSERT(smbat); + + if (lastupdated != NULL) + getnanotime(lastupdated); +} + +static int acpi_smbus_read_2(struct acpi_smbat_softc *sc, uint8_t addr, uint8_t cmd, uint16_t *ptr) { @@ -284,6 +322,11 @@ acpi_smbat_get_bst(device_t dev, struct error = ENXIO; sc = device_get_softc(dev); + if (!acpi_smbat_info_expired(sc-bst_lastupdated)) { + error = 0; + goto out; + } + if (acpi_smbus_read_2(sc, addr, SMBATT_CMD_BATTERY_MODE, val)) goto out; if (val SMBATT_BM_CAPACITY_MODE) { @@ -299,7 +342,7 @@ acpi_smbat_get_bst(device_t dev, struct goto out; if (val SMBATT_BS_DISCHARGING) { - bst-state |= ACPI_BATT_STAT_DISCHARG; + sc-bst.state |= ACPI_BATT_STAT_DISCHARG; /* * If the rate is negative, it is discharging. Otherwise, @@ -308,27 +351,31 @@ acpi_smbat_get_bst(device_t dev, struct if (acpi_smbus_read_2(sc, addr, SMBATT_CMD_AT_RATE, val)) goto out; if (val 0) - bst-rate = (-val) * factor; + sc-bst.rate = (-val) * factor; else - bst-rate = -1; + sc-bst.rate = -1; } else { - bst-state |= ACPI_BATT_STAT_CHARGING; - bst-rate = -1; + sc-bst.state |= ACPI_BATT_STAT_CHARGING; + sc-bst.rate = -1; } if (val SMBATT_BS_REMAINING_CAPACITY_ALARM) - bst-state |= ACPI_BATT_STAT_CRITICAL; + sc-bst.state |= ACPI_BATT_STAT_CRITICAL; if (acpi_smbus_read_2(sc, addr, SMBATT_CMD_REMAINING_CAPACITY, val)) goto out; - bst-cap = val * factor; + sc-bst.cap = val * factor; if (acpi_smbus_read_2(sc, addr, SMBATT_CMD_VOLTAGE, val)) goto out; - bst-volt = val; + sc-bst.volt = val; + + acpi_smbat_info_updated(sc-bst_lastupdated); + error = 0; out: + memcpy(bst, sc-bst, sizeof(sc-bst)); ACPI_SERIAL_END(smbat); return (error); } @@ -348,55 +395,63 @@ acpi_smbat_get_bif(device_t dev, struct error = ENXIO; sc = device_get_softc(dev); + if
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hajimu UMEMOTO wrote: Hi, On Mon, 14 Nov 2005 12:40:36 -0500 Pierre-Luc Drouin [EMAIL PROTECTED] said: pldrouin Yep, smart battery is definately the problem. The performance of my pldrouin laptop is back to normal when I remove the xfce4-battery-plugin. pldrouin acpiconf -i loop reproduces the problem for me too. So it looks like pldrouin there is something wrong in smart battery. The cmbat has similar issue on some laptops. So, acpi_cmbat.c uses cache for retrieval to reduce its influence, and its expiration time is set by hw.acpi.battery.info_expire. However, acpi_smbat.c doesn't use cache. So, I made a patch. Since I don't have a laptop which has smbat, I cannot test it by myself. Please test it and let me know the result. Index: sys/dev/acpica/acpi_smbat.c diff -u -p sys/dev/acpica/acpi_smbat.c.orig sys/dev/acpica/acpi_smbat.c --- sys/dev/acpica/acpi_smbat.c.origSun Nov 6 08:55:56 2005 +++ sys/dev/acpica/acpi_smbat.c Tue Nov 15 16:41:00 2005 @@ -44,11 +44,18 @@ __FBSDID($FreeBSD: src/sys/dev/acpica/a struct acpi_smbat_softc { uint8_t sb_base_addr; device_tec_dev; + + struct acpi_bif bif; + struct acpi_bst bst; + struct timespec bif_lastupdated; + struct timespec bst_lastupdated; }; static int acpi_smbat_probe(device_t dev); static int acpi_smbat_attach(device_t dev); static int acpi_smbat_shutdown(device_t dev); +static int acpi_smbat_info_expired(struct timespec *lastupdated); +static voidacpi_smbat_info_updated(struct timespec *lastupdated); static int acpi_smbat_get_bif(device_t dev, struct acpi_bif *bif); static int acpi_smbat_get_bst(device_t dev, struct acpi_bst *bst); @@ -114,6 +121,9 @@ acpi_smbat_attach(device_t dev) return (ENXIO); } + timespecclear(sc-bif_lastupdated); + timespecclear(sc-bst_lastupdated); + if (acpi_battery_register(dev) != 0) { device_printf(dev, cannot register battery\n); return (ENXIO); @@ -132,6 +142,34 @@ acpi_smbat_shutdown(device_t dev) } static int +acpi_smbat_info_expired(struct timespec *lastupdated) +{ + struct timespec curtime; + + ACPI_SERIAL_ASSERT(smbat); + + if (lastupdated == NULL) + return (TRUE); + if (!timespecisset(lastupdated)) + return (TRUE); + + getnanotime(curtime); + timespecsub(curtime, lastupdated); + return (curtime.tv_sec 0 || + curtime.tv_sec acpi_battery_get_info_expire()); +} + +static void +acpi_smbat_info_updated(struct timespec *lastupdated) +{ + + ACPI_SERIAL_ASSERT(smbat); + + if (lastupdated != NULL) + getnanotime(lastupdated); +} + +static int acpi_smbus_read_2(struct acpi_smbat_softc *sc, uint8_t addr, uint8_t cmd, uint16_t *ptr) { @@ -284,6 +322,11 @@ acpi_smbat_get_bst(device_t dev, struct error = ENXIO; sc = device_get_softc(dev); + if (!acpi_smbat_info_expired(sc-bst_lastupdated)) { + error = 0; + goto out; + } + if (acpi_smbus_read_2(sc, addr, SMBATT_CMD_BATTERY_MODE, val)) goto out; if (val SMBATT_BM_CAPACITY_MODE) { @@ -299,7 +342,7 @@ acpi_smbat_get_bst(device_t dev, struct goto out; if (val SMBATT_BS_DISCHARGING) { - bst-state |= ACPI_BATT_STAT_DISCHARG; + sc-bst.state |= ACPI_BATT_STAT_DISCHARG; /* * If the rate is negative, it is discharging. Otherwise, @@ -308,27 +351,31 @@ acpi_smbat_get_bst(device_t dev, struct if (acpi_smbus_read_2(sc, addr, SMBATT_CMD_AT_RATE, val)) goto out; if (val 0) - bst-rate = (-val) * factor; + sc-bst.rate = (-val) * factor; else - bst-rate = -1; + sc-bst.rate = -1; } else { - bst-state |= ACPI_BATT_STAT_CHARGING; - bst-rate = -1; + sc-bst.state |= ACPI_BATT_STAT_CHARGING; + sc-bst.rate = -1; } if (val SMBATT_BS_REMAINING_CAPACITY_ALARM) - bst-state |= ACPI_BATT_STAT_CRITICAL; + sc-bst.state |= ACPI_BATT_STAT_CRITICAL; if (acpi_smbus_read_2(sc, addr, SMBATT_CMD_REMAINING_CAPACITY, val)) goto out; - bst-cap = val * factor; + sc-bst.cap = val * factor; if (acpi_smbus_read_2(sc, addr, SMBATT_CMD_VOLTAGE, val)) goto out; - bst-volt = val; + sc-bst.volt = val; + + acpi_smbat_info_updated(sc-bst_lastupdated); + error = 0; out: + memcpy(bst, sc-bst, sizeof(sc-bst)); ACPI_SERIAL_END(smbat); return (error); } @@ -348,55 +395,63 @@ acpi_smbat_get_bif(device_t dev, struct error = ENXIO; sc = device_get_softc(dev); + if
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Pierre-Luc Drouin wrote: Hajimu UMEMOTO wrote: On Mon, 14 Nov 2005 12:40:36 -0500 Pierre-Luc Drouin [EMAIL PROTECTED] said: pldrouin Yep, smart battery is definately the problem. The performance of my pldrouin laptop is back to normal when I remove the xfce4-battery-plugin. pldrouin acpiconf -i loop reproduces the problem for me too. So it looks like pldrouin there is something wrong in smart battery. The cmbat has similar issue on some laptops. So, acpi_cmbat.c uses cache for retrieval to reduce its influence, and its expiration time is set by hw.acpi.battery.info_expire. However, acpi_smbat.c doesn't use cache. So, I made a patch. Since I don't have a laptop which has smbat, I cannot test it by myself. Please test it and let me know the result. The patch seams to do its job correctly, but it is still very annoying to have the whole computer to freeze for 1 second when the cache expires. What does make the whole system to freeze? Before the code was changed in 6.0-stable, FreeBSD was able to read the battery status without freezing my laptop... I have been running 3 OSes (FreeBSD, Ubuntu and Win XP) on my laptop for a while and never experienced that kind of problem with either Linux or Win XP. I guess there is something wrong in the new code added after 6.0-release. If you have both smbat and cmbat, just disable smbat if you don't like it. Add this to /boot/loader.conf: debug.acpi.disabled=smbat The pause is related to a buggy or slow EC. Caching the values happens to hide it. Do you get pauses with smbat disabled (see above for how to do this)? Can you still get battery status from cmbat? -- Nate ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Nate Lawson wrote: Pierre-Luc Drouin wrote: Hajimu UMEMOTO wrote: On Mon, 14 Nov 2005 12:40:36 -0500 Pierre-Luc Drouin [EMAIL PROTECTED] said: pldrouin Yep, smart battery is definately the problem. The performance of my pldrouin laptop is back to normal when I remove the xfce4-battery-plugin. pldrouin acpiconf -i loop reproduces the problem for me too. So it looks like pldrouin there is something wrong in smart battery. The cmbat has similar issue on some laptops. So, acpi_cmbat.c uses cache for retrieval to reduce its influence, and its expiration time is set by hw.acpi.battery.info_expire. However, acpi_smbat.c doesn't use cache. So, I made a patch. Since I don't have a laptop which has smbat, I cannot test it by myself. Please test it and let me know the result. The patch seams to do its job correctly, but it is still very annoying to have the whole computer to freeze for 1 second when the cache expires. What does make the whole system to freeze? Before the code was changed in 6.0-stable, FreeBSD was able to read the battery status without freezing my laptop... I have been running 3 OSes (FreeBSD, Ubuntu and Win XP) on my laptop for a while and never experienced that kind of problem with either Linux or Win XP. I guess there is something wrong in the new code added after 6.0-release. If you have both smbat and cmbat, just disable smbat if you don't like it. Add this to /boot/loader.conf: debug.acpi.disabled=smbat The pause is related to a buggy or slow EC. Caching the values happens to hide it. Do you get pauses with smbat disabled (see above for how to do this)? Can you still get battery status from cmbat? Ok, there is new development. I realized by playing with debug.acpi.disabled=smbat, debug.acpi.disabled=smbat cmbat and debug.acpi.disabled=cmbat, that my laptop battery is not a smbat, but a cmbat. When I played with hw.acpi.battery.info_expire after to have applied the patch for acpi_smbat.c, it was freezing less often because that sysctl variable was shared by both cmbat and smbat. So I can only get battery status from cmbat (disabling cmbat disables the use of acpiconf -i loop). To get the status of my battery via cmbat was working fine up to 6.0-RELEASE (included), but makes my laptop to freeze since I upgraded to 6.0-stable with Nov 10th sources. What change related to cmbat between 6.0-release and 6.0-stable could be causing this? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Pierre-Luc Drouin wrote: Last friday I did a cvsup src/buildworld/buildkernel/installkernel/installworld/mergemaster on my laptop from 6.0-RELEASE to 6.0-STABLE and since the performance of my laptop (Dell Precision M70 with a Pentium M 2GHz) is not good. It behaves like if the bus was saturated. I noticed that sound skips when I play a mp3 with xmms and it also skips when I play dvds (with mplayer, ogle or vlc). I have tried to recompile a kernel with apic disabled since I had some performance issues with this enabled before 6.0-stable, but it didn't fix anything. Does somebody have an idea of what change in the FreeBSD sources could be causing this? /me too, and I switched back to 6.0-RELEASE because of that. The problem is the smart battery support that was merged shortly after 6.0-RELEASE. Running wmbsdbatt the system freezes for 0.5s every 10s or so, this is because wmbsdbatt is polling for the battery status. Disabling wmbsdbatt everything was back to normal. This can also be reproduced here with a acpiconf -i loop. Other sources say, it's the thermal stuff thats blocking the OS, but for me it's battery status. (Dell Inspiron 8600c, Pentium-M) Ulrich Spoerlein -- PGP Key ID: F0DB9F44 Encrypted mail welcome! Fingerprint: F1CE D062 0CA9 ADE3 349B 2FE8 980A C6B5 F0DB 9F44 Ok, which part of Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn. didn't you understand? pgpHc6608jrQS.pgp Description: PGP signature
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Ulrich Spoerlein wrote: Pierre-Luc Drouin wrote: Last friday I did a cvsup src/buildworld/buildkernel/installkernel/installworld/mergemaster on my laptop from 6.0-RELEASE to 6.0-STABLE and since the performance of my laptop (Dell Precision M70 with a Pentium M 2GHz) is not good. It behaves like if the bus was saturated. I noticed that sound skips when I play a mp3 with xmms and it also skips when I play dvds (with mplayer, ogle or vlc). I have tried to recompile a kernel with apic disabled since I had some performance issues with this enabled before 6.0-stable, but it didn't fix anything. Does somebody have an idea of what change in the FreeBSD sources could be causing this? /me too, and I switched back to 6.0-RELEASE because of that. The problem is the smart battery support that was merged shortly after 6.0-RELEASE. Running wmbsdbatt the system freezes for 0.5s every 10s or so, this is because wmbsdbatt is polling for the battery status. Disabling wmbsdbatt everything was back to normal. This can also be reproduced here with a acpiconf -i loop. Other sources say, it's the thermal stuff thats blocking the OS, but for me it's battery status. (Dell Inspiron 8600c, Pentium-M) Ulrich Spoerlein Yep, smart battery is definately the problem. The performance of my laptop is back to normal when I remove the xfce4-battery-plugin. acpiconf -i loop reproduces the problem for me too. So it looks like there is something wrong in smart battery. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
On Mon, Nov 14, 2005 at 12:40:36PM -0500, Pierre-Luc Drouin wrote: Yep, smart battery is definately the problem. The performance of my laptop is back to normal when I remove the xfce4-battery-plugin. acpiconf -i loop reproduces the problem for me too. So it looks like there is something wrong in smart battery. Glad to hear you tracked it down. Please file a PR so this bug can be tracked. Kris pgpOb3Dpjb4go.pgp Description: PGP signature
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Kris Kennaway wrote: On Mon, Nov 14, 2005 at 12:40:36PM -0500, Pierre-Luc Drouin wrote: Yep, smart battery is definately the problem. The performance of my laptop is back to normal when I remove the xfce4-battery-plugin. acpiconf -i loop reproduces the problem for me too. So it looks like there is something wrong in smart battery. Glad to hear you tracked it down. Please file a PR so this bug can be tracked. Kris Done ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Hi. On Sun, Nov 13, 2005 at 12:40:53PM -0500, Pierre-Luc Drouin wrote: I wanted also to add that my mouse pointer hangs on X since the upgrade. So the computer behaves as if it was under heavy load even if the CPU usage is almost 0% cpufreq/powerd? Did you check, your laptop runs at reasonable speed? - Olli -- | Oliver Brandmueller | Offenbacher Str. 1 | Germany D-14197 Berlin | | Fon +49-172-3130856 | Fax +49-172-3145027 | WWW: http://the.addict.de/ | | Ich bin das Internet. Sowahr ich Gott helfe. | | Eine gewerbliche Nutzung aller enthaltenen Adressen ist nicht gestattet! | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Oliver Brandmueller wrote: Hi. On Sun, Nov 13, 2005 at 12:40:53PM -0500, Pierre-Luc Drouin wrote: I wanted also to add that my mouse pointer hangs on X since the upgrade. So the computer behaves as if it was under heavy load even if the CPU usage is almost 0% cpufreq/powerd? Did you check, your laptop runs at reasonable speed? - Olli I am using the highest frequency all the time dev.cpu.0.freq: 1997 ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
On Sun, Nov 13, 2005 at 02:35:23PM -0500, Pierre-Luc Drouin wrote: Oliver Brandmueller wrote: Hi. On Sun, Nov 13, 2005 at 12:40:53PM -0500, Pierre-Luc Drouin wrote: I wanted also to add that my mouse pointer hangs on X since the upgrade. So the computer behaves as if it was under heavy load even if the CPU usage is almost 0% cpufreq/powerd? Did you check, your laptop runs at reasonable speed? - Olli I am using the highest frequency all the time dev.cpu.0.freq: 1997 Check for high interrupt rates with vmstat -i. Kris pgpFH1Sw9cbHO.pgp Description: PGP signature
Re: Performance problem since updating from 6.0-RELEASE to 6.0-STABLE last friday
Kris Kennaway wrote: On Sun, Nov 13, 2005 at 02:35:23PM -0500, Pierre-Luc Drouin wrote: Oliver Brandmueller wrote: Hi. On Sun, Nov 13, 2005 at 12:40:53PM -0500, Pierre-Luc Drouin wrote: I wanted also to add that my mouse pointer hangs on X since the upgrade. So the computer behaves as if it was under heavy load even if the CPU usage is almost 0% cpufreq/powerd? Did you check, your laptop runs at reasonable speed? - Olli I am using the highest frequency all the time dev.cpu.0.freq: 1997 Check for high interrupt rates with vmstat -i. Kris vmstat -i interrupt total rate irq0: clk 342175953 irq1: atkbd0 832 2 irq4: sio0 1 0 irq7: 3 0 stray irq7 3 0 irq8: rtc 43826122 irq9: iwi0 uhci1++ 124 0 irq11: nvidia0 pcm+18037 50 irq12: psm0 4164 11 irq13: npx01 0 irq14: ata0 5209 14 irq15: ata1 82 0 Total 414457 1154 I have noticed this error message on the console when I run X, I don't know if it can be related to the problem: Error in ictl (sockfd): Device not configured ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance of 4.x vs 5.x (Re: Lifetime of FreeBSD branches)
I wrote: Kris Kennaway wrote: http://www.chesapeake.net/~jroberson/flushbuf.diff Does it work for you on 5.4? The patch seems to work. Cool, that makes a difference like between BTW., is that change being included in 5-STABLE or just for 6-CURRENT? mkb. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance of 4.x vs 5.x (Re: Lifetime of FreeBSD branches)
On Sat, Jun 11, 2005 at 09:52:13PM +0200, Matthias Buelow wrote: I wrote: Kris Kennaway wrote: http://www.chesapeake.net/~jroberson/flushbuf.diff Does it work for you on 5.4? The patch seems to work. Cool, that makes a difference like between BTW., is that change being included in 5-STABLE or just for 6-CURRENT? mkb. Jeff said he'll merge it in a week or two after it's been well-tested. Kris pgpkOYLjgWhdk.pgp Description: PGP signature
Re: Performance of 4.x vs 5.x (Re: Lifetime of FreeBSD branches)
- Original Message - From: Kris Kennaway [EMAIL PROTECTED] Jeff said he'll merge it in a week or two after it's been well-tested. Been running it here on our ftp which was getting major issues with disk access spiking system usage to 90+% making the server totally unresponsive for 5 - 10 seconds at a time. With the patch things are MUCH better. No problems to report and the server is under major load including some heavy disk access as I clean up some dirs with 1.4 million files in a single dir /me slaps users :p Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone (023) 8024 3137 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance of 4.x vs 5.x (Re: Lifetime of FreeBSD branches)
Steven Hartland wrote: With the patch things are MUCH better. No problems to report and the server is under major load including some heavy disk access as Yeah, no problems here either, so far. mkb. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Performance of 4.x vs 5.x (Re: Lifetime of FreeBSD branches)
On Thu, May 26, 2005 at 02:15:54AM +0200, Matthias Buelow wrote: Others don't see this though, and in other cases it was *definitively proven* to be caused by the issue I mentioned. I'll have to think more about what to try next..thanks for running the tests. Perhaps it's something SATA-related? Before restoring my 5.4 dumps after testing -current, I installed fedora3 linux just to verify it isn't somehow the hardware itself. Ok, plain installation from CDs, kernel 2.6.9-1.667smp (default installation kernel). There was absolutely zero noticable lag or any effect on response time on X11 while untarring the same firefox source. So there really seems to be something foul in FreeBSD in that regard. And now for the dumps.. *sigh*. In case you didn't see it, Jeff identified the probable cause of this and developed the followng patch: http://www.chesapeake.net/~jroberson/flushbuf.diff Does it work for you on 5.4? Kris pgplKQmUvEU5U.pgp Description: PGP signature
Re: Performance of 4.x vs 5.x (Re: Lifetime of FreeBSD branches)
Kris Kennaway wrote: http://www.chesapeake.net/~jroberson/flushbuf.diff Does it work for you on 5.4? The patch seems to work. Cool, that makes a difference like between night and day. I can't determine any observable effect of untarring the firefox source anymore to interactive response time (when it's non-disk bound, of course), where before applying the patch, the mouse cursor would jump, audio stutter and I could barely type in xterms. Seems to work perfectly again. Thanks! mkb. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]