Re: SMP Version of tar
On Mon, 1 Oct 2012 22:16:53 -0700 Tim Kientzle t...@kientzle.com wrote: There are a few different parallel command-line compressors and decompressors in ports; experiment a lot (with large files being read from and/or written to disk) and see what the real effect is. In particular, some decompression algorithms are actually faster than memcpy() when run on a single processor. Parallelizing such algorithms is not likely to help much in the real world. The two popular algorithms I would expect to benefit most are bzip2 compression and lzma compression (targeting xz or lzip format). For decompression, bzip2 is block-oriented so fits SMP pretty naturally. Other popular algorithms are stream-oriented and less amenable to parallelization. Take a careful look at pbzip2, which is a parallelized bzip2/bunzip2 implementation that's already under a BSD license. You should be able to get a lot of ideas about how to implement a parallel compression algorithm. Better yet, you might be able to reuse a lot of the existing pbzip2 code. Mark Adler's pigz is also worth studying. It's also license-friendly, and is built on top of regular zlib, which is a nice technique when it's feasible. Just a small note: There's a parallel implementation of xz called pixz. It's build atop of liblzma and libarchiv and stands under a BSD style license. See: https://github.com/vasi/pixz Maybe it's possible to reuse most of the code. -- Homepage: www.yamagi.org XMPP: yam...@yamagi.org GnuPG/GPG: 0xEFBCCBCB pgp4AZtefgufA.pgp Description: PGP signature
Re: SMP Version of tar
.. please keep in mind that embedded platforms (a) don't necessarily benefit from it, and (b) have a very small footprint. Bloating out the compression/archival tools for the sake of possible SMP support will make me very, very sad. Adrian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: SMP Version of tar
Don't worry. I'm well known to over-optimize for both size and speed. I have an old Pentium 3 800MHz single core that I can use to simulate an embedded device (well, a decently powered one), to verify that I'm not killing the single-core performance (I could add CPU capability detection to help with that). Also, I still need to find time to do all this research and development. -Brandon On 10/2/2012 3:06 AM, Adrian Chadd wrote: .. please keep in mind that embedded platforms (a) don't necessarily benefit from it, and (b) have a very small footprint. Bloating out the compression/archival tools for the sake of possible SMP support will make me very, very sad. Adrian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: SMP Version of tar
On Mon, Oct 01, 2012 at 10:16:53PM -0700, Tim Kientzle wrote: * Implement within libarchive directly. This would benefit tar and a handful of other programs that use libarchive, but may not be worth the complexity. The complexity shouldn't actually be that bad. Basically, use a large internal buffer in the compression filter and make the queueing for distribution to threads an implementation detail. Alternatively, just provide a compatible implementation of libz's event interface. Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: NFS server bottlenecks
Garrett Wollman wrote: I had an email conversation with Rick Macklem about six months ago about NFS server bottlenecks. I'm now in a position to observe my large-scale NFS server under an actual production load, so I thought I would update folks on what it looks like. This is a 9.1 prerelease kernel (I hope 9.1 will be released soon as I have four moe of these servers to deploy!). When under nearly 100% load on an 8-core (16-thread) Quanta QSSC-S99Q storage server, with a 10G network interface, pmcstat tells me this: PMC: [INST_RETIRED.ANY_P] Samples: 2727105 (100.0%) , 27 unresolved Key: q = exiting... %SAMP IMAGE FUNCTION CALLERS 29.3 kernel _mtx_lock_sleep nfsrvd_updatecache:10.0 nfsrvd_getcache:7.4 ... 9.5 kernel cpu_search_highest cpu_search_highest:8.1 sched_idletd:1.4 7.4 zfs.ko lzjb_decompress zio_decompress 4.3 kernel _mtx_lock_spin turnstile_trywait:2.2 pmclog_reserve:1.0 ... 4.0 zfs.ko fletcher_4_native zio_checksum_error:3.1 zio_checksum_compute:0.8 3.6 kernel cpu_search_lowest cpu_search_lowest 3.3 kernel nfsrc_trimcache nfsrvd_getcache:1.6 nfsrvd_updatecache:1.6 2.3 kernel ipfw_chk ipfw_check_hook 2.1 pmcstat _init 1.1 kernel _sx_xunlock 0.9 kernel _sx_xlock 0.9 kernel spinlock_exit This does seem to confirm my original impression that the NFS replay cache is quite expensive. Running a gprof(1) analysis on the same PMC data reveals a bit more detail (I've removed some uninteresting parts of the call graph): I can't remember (I am early retired now;-) if I mentioned this patch before: http://people.freebsd.org/~rmacklem/drc.patch It adds tunables vfs.nfsd.tcphighwater and vfs.nfsd.udphighwater that can be twiddled so that the drc is trimmed less frequently. By making these values larger, the trim will only happen once/sec until the high water mark is reached, instead of on every RPC. The tradeoff is that the DRC will become larger, but given memory sizes these days, that may be fine for you. jwd@ was going to test it, but he moved to a different job away from NFS, so the patch has just been collecting dust. If you could test it, that would be nice, rick ps: Also, the current patch still locks before checking if it needs to do the trim. I think that could safely be changed so that it doesn't lock/unlock when it isn't doing the trim, if that makes a significant difference. called/total parents index %time self descendents called+self name index called/total children 4881.00 2004642.70 932627/932627 svc_run_internal [2] [4] 45.1 4881.00 2004642.70 932627 nfssvc_program [4] 13199.00 504436.33 584319/584319 nfsrvd_updatecache [9] 23075.00 403396.18 468009/468009 nfsrvd_getcache [14] 1032.25 416249.44 2239/2284 svc_sendreply_mbuf [15] 6168.00 381770.44 11618/11618 nfsrvd_dorpc [24] 3526.87 86869.88 112478/112514 nfsrvd_sentcache [74] 890.00 50540.89 4252/4252 svc_getcred [101] 14876.60 32394.26 4177/24500 crfree cycle 3 [263] 11550.11 25150.73 3243/24500 free cycle 3 [102] 1348.88 15451.66 2716/16831 m_freem [59] 4066.61 216.81 1434/1456 svc_freereq [321] 2342.15 677.40 557/1459 malloc_type_freed [265] 59.14 1916.84 134/2941 crget [113] 1602.25 0.00 322/9682 bzero [105] 690.93 0.00 43/44 getmicrotime [571] 287.22 7.33 138/1205 prison_free [384] 233.61 0.00 60/798 PHYS_TO_VM_PAGE [358] 203.12 0.00 94/230 nfsrv_mallocmget_limit [632] 151.76 0.00 51/1723 pmap_kextract [309] 0.78 70.28 9/3281 _mtx_unlock_sleep [154] 19.22 16.88 38/400403 nfsrc_trimcache [26] 11.05 21.74 7/197 crsetgroups [532] 30.37 0.00 11/6592 critical_enter [190] 25.50 0.00 9/36 turnstile_chain_unlock [844] 24.86 0.00 3/7 nfsd_errmap [913] 12.36 8.57 8/2145 in_cksum_skip [298] 9.10 3.59 5/12455 mb_free_ext [140] 1.84 4.85 2/2202 VOP_UNLOCK_APV [269] --- 0.49 0.15 1/1129009 uhub_explore [1581] 0.49 0.15 1/1129009 tcp_output [10] 0.49 0.15 1/1129009 pmap_remove_all [1141] 0.49 0.15 1/1129009 vm_map_insert [236] 0.49 0.15 1/1129009 vnode_create_vobject [281] 0.49 0.15 1/1129009 biodone [351] 0.49 0.15 1/1129009 vm_object_madvise [670] 0.49 0.15 1/1129009 xpt_done [483] 0.49 0.15 1/1129009 vputx [80] 0.49 0.15 1/1129009 vm_map_delete cycle 3 [49] 0.49 0.15 1/1129009 vm_object_deallocate cycle 3 [356] 0.49 0.15 1/1129009 vm_page_unwire [338] 0.49 0.15 1/1129009 pmap_change_wiring [318] 0.98 0.31 2/1129009 getnewvnode [227] 0.98 0.31 2/1129009 pmap_clear_reference [1004] 0.98 0.31 2/1129009 usbd_do_request_flags [1282] 0.98 0.31 2/1129009 vm_object_collapse cycle 3 [587] 0.98 0.31 2/1129009 vm_object_page_remove [122] 1.48 0.46 3/1129009 mpt_pci_intr [487] 1.48 0.46 3/1129009 pmap_extract [355] 1.48 0.46 3/1129009 vm_fault_unwire [171] 1.97 0.62 4/1129009 vgonel [270] 1.97 0.62 4/1129009 vm_object_shadow [926] 1.97 0.62 4/1129009 zone_alloc_item [434] 2.46 0.77 5/1129009 vnlru_free [235] 2.46 0.77 5/1129009 insmntque1 [737] 2.95 0.93 6/1129009 zone_free_item [409] 3.94 1.24
Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Monday, October 01, 2012 6:31:00 pm Simon J. Gerraty wrote: Hi Garrett, From: Garrett Cooper yaneg...@gmail.com Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple = programs instead of a singular program Date: September 2, 2012 11:01:09 PM PDT To: freebsd-hackers@freebsd.org Cc: freebsd-a...@freebsd.org Arch freebsd-a...@freebsd.org =20 Hello, I've been a bit busy working on porting over ATF from NetBSD, and Thanks, we've been using ATF in Junos for a while and glad to see it being imported to FreeBSD. one of the pieces that's currently not available in FreeBSD that's available in NetBSD is the ability to understand and compile multiple programs. In order to do this I had to refactor bsd.prog.mk (a lot). A change like this to bsd.prog.mk can have considerable fallout. Eg. any makefile that tweaks OBJS is suddenly out of luck. Not to mention the fact that bsd.prog.mk goes from being relatively simple, to unspeakably hard to read, and all for rather limited return. Apart from ATF, is there any huge demand for building multiple progs in the same directory? FWIW we use progs.mk (as bsd.progs.mk) from ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz It isn't ideal, but it certainly avoids a lot of churn and complexity for what is essentially a corner case. This sounds like a superior approach. It doesn't break any current use cases while giving the ability to build multiple programs in the few places that need it. It sounds like there are a few places under gnu/ from Garrett's reply that might be able to make use of this as well. BTW, one general comment. There seem to be two completely independent groups of folks working on ATF (e.g. there have been two different imports of ATF into the tree in two different locations IIRC, and now we have two different sets of patches to our system makefiles). Are these two groups talking to each other at all? I know in May that many folks (certainly multiple vendors) are interested in ATF, and it seems that both Juniper and Isilon have ported ATF internally. It seems that it might be good for the two groups to work together to avoid stomping on each other's toes. It seems there are some differences in the two approaches that merit working out to avoid a lot of wasted effort on both sides. Do we already have a freebsd-atf@ mailing list? If not, perhaps we should create one and start these discussions there? -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Mon, Oct 1, 2012 at 5:00 PM, Simon J. Gerraty s...@juniper.net wrote: Not to mention the fact that bsd.prog.mk goes from being relatively simple, to unspeakably hard to read, and all for rather limited = return. This btw I think is the more important issue. I was looking at bsd.prog.mk in netbsd the other day. It has no business being that complex. Ok. Even though I originally thought that my changes were a bit hacky for not modifying bsd.man.mk, bsd.nls.mk, etc, looking back the way they were resolved was a little elegant in its simplicity (albeit not optimized). ... It order to do this, I need to be able to build multiple programs so = things are as transparent. On the flipside, bsd.prog[s].mk can live on = We put the test cases in a subdir of the lib/prog This has multiple benefits, and eliminates any impact on the normal build of said libs/progs. Hmmm... that's one of the 3 approaches I provided, but it turned out to be annoying to make this work at first inspection because of how things were done in our build system. Just for review (even though it's OT), the three approaches I presented were as follows... Note: in all 3 examples, clock.c is the source code and t_clock.c is the relevant unittest code. 1. Test programs live with the sources (this was the requested approach), e.g. lib/libc/gen/... .../clock.c .../t_clock.c 2. Test programs live in subdirs: lib/libc/gen/... .../clock.c .../tests/... .../t_clock.c 3. Test programs completely decoupled from the source tree: lib/libc/gen/... .../clock.c tests/lib/libc/gen/... .../t_clock.c A hybrid approach (1.+3. / 2.+3.) should probably be used anyhow for stress tests and the like that really have no business living in one particular directory in the source tree (sort of achieving what tools/regression does today, but hopefully in a less messy manner as tools/regression appears to have grown organically minus any single sane order). Also a number of our teams find it necessary to create mock classes etc to adequately test their libs. Keeping all this in a tests/ subdir makes it easy, to keep things neat up to date, and ensure that the tests pass. Trying to do all that in the same dir as the lib would be ugly. This (consolidating everything under a single directory) is the way that was requested by the beforementioned parties. FWIW we use progs.mk (as bsd.progs.mk) from ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20 It isn't ideal, but it certainly avoids a lot of churn and complexity for what is essentially a corner case. This requires pulling bmake into FreeBSD proper in order for things to = function the last I checked as bsd.prog.mk depends upon bmake directives = This is already happening ;-) I wish I had known that. I pinged marcel/obrien about this some weeks ago and didn't receive a response. Originally bsd.progs.mk was going to be a home grown effort that was going to be killed off and replaced with NetBSD's build infrastructure, but I didn't get a reply and had to put this (my work to replace bsd.prog.mk) plan in motion given requests I was provided above for resolving the source/unittest code consolidation effort. Ideally however, I would like doing this compared to running a custom = build infrastructure, but (as you probably know) this requires = rototilling the current FreeBSD build system to a large degree = Actually building FreeBSD-current using bmake requires only modest changes. Yes for the most part, and I agree that bmake is definitely more advanced than FreeBSD pmake so I consider it a very welcome change. What are the plans for importing bmake into FreeBSD (this should probably be a different thread)? ... I know, but it is a very useful thing to be able to do. You can add knobs to control such things of course. I agree that it's a good thing (in theory) -- it'd just help to discuss this with Julio first because I know of a couple cases where this would be unusable given how bsd.test.mk is coded: 1. The it works for me on my machine! certification: It encourages environment tainting, which could be a really, really bad thing; I've seen developers take interesting shortcuts when testing their code (me included sometimes :)..); I've seen hardcoded paths, harcoded paths for named semaphores, things that just work because of someone's host environment, feature specific assumptions (developer X was doing testing on feature Y and things broke when he/she committed the testcase to head), etc where the it works for me on my machine! certification would be true more often than not. These same individuals would more likely than not execute things taking shortcuts, but I want to avoid creating a system where developers cut corners and commit too early because working within the confines of the system is not conducive to getting work done. 2. Failure
Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Tue, Oct 2, 2012 at 4:50 AM, John Baldwin j...@freebsd.org wrote: ... This sounds like a superior approach. It doesn't break any current use cases while giving the ability to build multiple programs in the few places that need it. It sounds like there are a few places under gnu/ from Garrett's reply that might be able to make use of this as well. For the record, gnu/cc/cc_tools/Makefile is where I first spotted a potential bsd.progs.mk candidate. Most of the other code doesn't care given how things are organized in our source tree. BTW, one general comment. There seem to be two completely independent groups of folks working on ATF (e.g. there have been two different imports of ATF into the tree in two different locations IIRC, and now we have two different sets of patches to our system makefiles). Are these two groups talking to each other at all? I know in May that many folks (certainly multiple vendors) are interested in ATF, and it seems that both Juniper and Isilon have ported ATF internally. It seems that it might be good for the two groups to work together to avoid stomping on each other's toes. It seems there are some differences in the two approaches that merit working out to avoid a lot of wasted effort on both sides. Both parties (Isilon/Juniper) are converging on the ATF porting work that Giorgos/myself have done after talking at the FreeBSD Foundation meet-n-greet. I have contributed all of the patches that I have other to marcel for feedback. Do we already have a freebsd-atf@ mailing list? If not, perhaps we should create one and start these discussions there? Probably wouldn't be a bad idea as I'm currently suspended a bit waiting on feedback for how to proceed; too bad freebsd-test is being used for other things :).. Thanks! -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Tue, 2 Oct 2012 07:50:23 -0400, John Baldwin writes: BTW, one general comment. There seem to be two completely independent groups of folks working on ATF (e.g. there have been two different imports of ATF into the tree in two different locations IIRC, and now we have two different sets of patches to our system makefiles). Yes, and no. We (Juniper) have been using ATF for some time, and were going to do import etc, but for various reasons haven't done it yet. In part I guess because having bmake in tree would have made things much simpler - avoiding re-inventing wheels. Garrett, has forged ahead, and we are fine with that - Marcel did the import for him. Obviously though (as I've probably just made clear) we don't want ATF to unnecessarily complicate the build. We hope to get the initial bmake commit done this week, and then we can help Garrett get ATF going with minimal fuss. Are these two groups talking to each other at all? Yes, but I don't think Garrett was aware of the bmake work. It seems there are some differences in the two approaches that merit working out to avoid a lot of wasted effort on both sides. The differences are probably very minor, and hopefully easily sorted out. The most significant being how to build the multiple test apps in one directory. Related to that is the exact location. I believe we are all agreed that tests should be co-located with the code they exercise - to facilitate testing as you make changes. We use a tests/ subdir per lib and prog that has unit-tests and I would recommend doing the same. This is consistent with the above goal, and the separate directory is very useful for keeping the build simple - eg. libfoo/Makefile can continue to just use bsd.lib.mk while libfoo/tests/Makefile can use any of bsd.prog.mk, bsd.progs.mk, bsd.test.mk or atf.test.mk Also, these separate dirs become even more important when using meta mode because you want the all/default target to just do the right thing, and do it the same way every time, to avoid churn in dependencies. Do we already have a freebsd-atf@ mailing list? If not, perhaps we should create one and start these discussions there? Don't know, but fine either way. --sjg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Tue, 2 Oct 2012 07:19:55 -0700, Garrett Cooper writes: We put the test cases in a subdir of the lib/prog This has multiple benefits, and eliminates any impact on the normal build of said libs/progs. Hmmm... that's one of the 3 approaches I provided, but it turned out to be annoying to make this work at first inspection because of how things were done in our build system. Just for review (even though it's OT), the three approaches I presented were as follows... Note: in all 3 examples, clock.c is the source code and t_clock.c is the relevant unittest code. There is actually another: lib/libc/Makefile lib/libc/gen/... .../clock.c .../t_clock.c lib/libc/tests/Makefile that is the makefile for building/running the tests lives in the subdir, there are advantages to this, but the test code itself can be anywhere you like. Either next to the code that it tests, or in the tests/ subdir, another subdir, or whatever combination makes the most sense. Most of our ATF users put their test code in the subdir fwiw. Also a number of our teams find it necessary to create mock classes etc to adequately test their libs. Keeping all this in a tests/ subdir makes it easy, to keep things neat up to date, and ensure that the tests pass. Trying to do all that in the same dir as the lib would be ugly. This (consolidating everything under a single directory) is the way that was requested by the beforementioned parties. Sorry for not participating in that discussion ;-) John - perhaps we do need that ATF list? For example building a library and test apps in the one directory either requires turning bsd.lib.mk into an even more complex thing that a multi-prog bsd.pog.mk, or having the makefile behave completely differently depending on the target requested. That later is a bad idea. It precludes being able to seamlessly integrate unit-tests into the build, and would need to be fixed if/when attempting to introduce meta mode. Typically the unit-tests depend on the library they are testing, a separate subdir for the tests/Makefile makes capturing that dependency easy - otherwise you are back to the unspeakably complex bsd.lib.mk Far better to get these things right first up. I wish I had known that. I pinged marcel/obrien about this some weeks ago and didn't receive a response. Many appologies. What are the plans for importing bmake into FreeBSD (this should probably be a different thread)? The import has been done - just needs to be merged to contrib, and there's a small patch to be committed. We hope to get that done this week. I agree that it's a good thing (in theory) -- it'd just help to discuss this with Julio first because I know of a couple cases where this would be unusable given how bsd.test.mk is coded: I do talk to Julio about ATF (I'm s...@netbsd.org ;-) though not specifically this - beyond (I think) mentioning that I did it differently. 1. The it works for me on my machine! certification: It encourages environment tainting, which could be a really, really bad thing; I've seen developers take interesting shortcuts when True. In our (Junos) build we cleanse the environment rather carefully and have a very clear distinction between building for the host (which may just happend to be i386 based) and the i386 target for example. I can provide build smarts to do all that - but didn't want to shove it down anyone's throat ;-) What works for us doesn't necessarily work for everyone. 2. Failure in a subdirectory results in lost coverage: a/... .../b/... - tests live here .../c/d/... - tests live here If I execute make test from a/ (one of my goals), then it should iterate down a/b, then a/c/d and run the tests in a DFS style (BFS if This gets back to the bmake topic. We actually avoid walking the tree at all, eg. in the Junos build I have a target atf-tests which builds all ATF tests dirs regardless of where they are (and any pre-requisits they have). This is handy to include as a dependency of the build target we use for branch syncs etc, to ensure no unit-tests atrophy. But the most common case - and the one to optimize for, is the person making changes to libfoo, testing that he hasn't broken anything. could disguise other bugs. This would be fixed by changing bsd.test.mk to use atf-run properly, but that's work (not hard) that still needs to be done. Yes, atf.test.mk already does that. H... My goal was to make ATF a one ring to rule them all infrastructure so that way everything could be unified under ATF in ATF is a pretty good framework - which is why we (Juniper) use it. We want the flexibility to have more than one framework, but the number should be very small. So far I have persuaded teams that wanted to adopt alternate frameworks, that ATF could easily meet their needs. one way, shape or form (even if it's just reporting), because lord knows we don't need a lua, unittest/nose, etc wrapper for reporting results. All of the
Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Tuesday, October 02, 2012 10:29:49 am Garrett Cooper wrote: On Tue, Oct 2, 2012 at 4:50 AM, John Baldwin j...@freebsd.org wrote: ... This sounds like a superior approach. It doesn't break any current use cases while giving the ability to build multiple programs in the few places that need it. It sounds like there are a few places under gnu/ from Garrett's reply that might be able to make use of this as well. For the record, gnu/cc/cc_tools/Makefile is where I first spotted a potential bsd.progs.mk candidate. Most of the other code doesn't care given how things are organized in our source tree. BTW, one general comment. There seem to be two completely independent groups of folks working on ATF (e.g. there have been two different imports of ATF into the tree in two different locations IIRC, and now we have two different sets of patches to our system makefiles). Are these two groups talking to each other at all? I know in May that many folks (certainly multiple vendors) are interested in ATF, and it seems that both Juniper and Isilon have ported ATF internally. It seems that it might be good for the two groups to work together to avoid stomping on each other's toes. It seems there are some differences in the two approaches that merit working out to avoid a lot of wasted effort on both sides. Both parties (Isilon/Juniper) are converging on the ATF porting work that Giorgos/myself have done after talking at the FreeBSD Foundation meet-n-greet. I have contributed all of the patches that I have other to marcel for feedback. This is very non-obvious to the public at large (e.g. there was no public response to one group's inquiry about the second ATF import for example). Also, given that you had no idea that sgf@ and obrien@ were working on importing NetBSD's bmake as a prerequisite for ATF, it seems that whatever discussions were held were not very detailed at best. I think it would be good to have the various folks working on ATF to at least summarize the current state of things and sketch out some sort of plan or roadmap for future work in a public forum (such as atf@, though a summary mail would be quite appropriate for arch@). -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: NFS server bottlenecks
[Adding freebsd-fs@ to the Cc list, which I neglected the first time around...] On Tue, 2 Oct 2012 08:28:29 -0400 (EDT), Rick Macklem rmack...@uoguelph.ca said: I can't remember (I am early retired now;-) if I mentioned this patch before: http://people.freebsd.org/~rmacklem/drc.patch It adds tunables vfs.nfsd.tcphighwater and vfs.nfsd.udphighwater that can be twiddled so that the drc is trimmed less frequently. By making these values larger, the trim will only happen once/sec until the high water mark is reached, instead of on every RPC. The tradeoff is that the DRC will become larger, but given memory sizes these days, that may be fine for you. It will be a while before I have another server that isn't in production (it's on my deployment plan, but getting the production servers going is taking first priority). The approaches that I was going to look at: Simplest: only do the cache trim once every N requests (for some reasonable value of N, e.g., 1000). Maybe keep track of the number of entries in each hash bucket and ignore those buckets that only have one entry even if is stale. Simple: just use a sepatate mutex for each list that a cache entry is on, rather than a global lock for everything. This would reduce the mutex contention, but I'm not sure how significantly since I don't have the means to measure it yet. Moderately complicated: figure out if a different synchronization type can safely be used (e.g., rmlock instead of mutex) and do so. More complicated: move all cache trimming to a separate thread and just have the rest of the code wake it up when the cache is getting too big (or just once a second since that's easy to implement). Maybe just move all cache processing to a separate thread. It's pretty clear from the profile that the cache mutex is heavily contended, so anything that reduces the length of time it's held is probably a win. That URL again, for the benefit of people on freebsd-fs who didn't see it on hackers, is: http://people.csail.mit.edu/wollman/nfs-server.unhalted-core-cycles.png. (This graph is slightly modified from my previous post as I removed some spurious edges to make the formatting look better. Still looking for a way to get a profile that includes all kernel modules with the kernel.) -GAWollman ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
