Re: [RFT][patch] Scheduling for HTT and not only
On 02/15/12 21:54, Jeff Roberson wrote: On Wed, 15 Feb 2012, Alexander Motin wrote: As before I've tested this on Core i7-870 with 4 physical and 8 logical cores and Atom D525 with 2 physical and 4 logical cores. On Core i7 I've got speedup up to 10-15% in super-smack MySQL and PostgreSQL indexed select for 2-8 threads and no penalty in other cases. pbzip2 shows up to 13% performance increase for 2-5 threads and no penalty in other cases. Can you also test buildworld or buildkernel with a -j value twice the number of cores? This is an interesting case because it gets little benefit from from affinity and really wants the best balancing possible. It's also the first thing people will complain about if it slows. All night long buildworld run on Core i7-2600K (4/8 cores) with 8GB RAM and RAID0 of two fast SSDs found no bad surprises: old new % 1 4242.33 4239.69 -0.0622299538 2 2376.4433 2340.47 -1.5137453521 4 1581.3033 1430.1733 -9.5573063055 6 1394.8033 1348.0533 -3.3517270858 8 1365.8067 1315.87 -3.6562055231 10 1312.8533 12 1350.23 1313.2667 -2.7375558238 16 1346.2267 1306.0733 -2.9826625783 20 1313.31 Each point there averaged of 3 runs. -- Alexander Motin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [RFT][patch] Scheduling for HTT and not only
On 02/16/12 10:48, Alexander Motin wrote: On 02/15/12 21:54, Jeff Roberson wrote: On Wed, 15 Feb 2012, Alexander Motin wrote: As before I've tested this on Core i7-870 with 4 physical and 8 logical cores and Atom D525 with 2 physical and 4 logical cores. On Core i7 I've got speedup up to 10-15% in super-smack MySQL and PostgreSQL indexed select for 2-8 threads and no penalty in other cases. pbzip2 shows up to 13% performance increase for 2-5 threads and no penalty in other cases. Can you also test buildworld or buildkernel with a -j value twice the number of cores? This is an interesting case because it gets little benefit from from affinity and really wants the best balancing possible. It's also the first thing people will complain about if it slows. All night long buildworld run on Core i7-2600K (4/8 cores) with 8GB RAM and RAID0 of two fast SSDs found no bad surprises: old new % 1 4242.33 4239.69 -0.0622299538 2 2376.4433 2340.47 -1.5137453521 4 1581.3033 1430.1733 -9.5573063055 6 1394.8033 1348.0533 -3.3517270858 8 1365.8067 1315.87 -3.6562055231 10 1312.8533 12 1350.23 1313.2667 -2.7375558238 16 1346.2267 1306.0733 -2.9826625783 20 1313.31 Each point there averaged of 3 runs. Ah! Just recalled, this kernel included patch from avg@ that removed extra equal topology level from the tree. That could improve balancer behavior in this case of single-socket system. -- Alexander Motin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
8 to 9: A longer wait early in the boot of a (damaged) Compaq Presario
About three years ago, my Compaq Presario F700 notebook got damaged in BIOS: it carried Windows Vista then, and that OS could not be recovered from the system image disks I had created for a brand-new machine. The damage was somewhere around BIOS/firmware area -- the way the console looked on a bootup looked differently (simpler) now that after several reboots trying to recover Vista, it got fried. Some googling told me then that the irreversible loss of Windows was not unusual for these Compaq machines -- the damaged systems didn't give one a chance to use the recovery disks. OK, I made the system dual bootable to Debian Linux and FreeBSD 8 then; with that, it booted all right, but in both cases the 'nfe0' interface Ethernet address was being set to 0. No big deal: I used an Ethernet address from my older laptop destined to be destroyed and gave it to 'nfe0' when setting the network interface properties at the system initialization. Works great, both in Debian and FreeBSD. There was one other odd thing that I noticed then: while Debian booted without a delay, FreeBSD 8 made a long pause after passing the boot menu: it would display the '/' character and sit there for some non-trivial amount of seconds. I assumed that it was doing some BIOS querying, and with BIOS (firmware?) being damaged, it took the system some time to figure things out... perhaps it was re-querying BIOS, seeing the insane value of 0 for an interface's Ethernet address (I have many machines running FreeBSD, including multiple laptops, and this machine is unique in the long bootup pause). About a week ago, I made a jump and upgraded the system's FreeBSD from version 8 to 9. Everything is great (I am typing this message on that machine now) but the boot pause after the (looking new in 9) boot menu is *much* longer now -- it will show the '\' character and wait for, subjectively, half a minute before putting anything else on the screen. This is not of any practical importance for me, I feel very good about what I got in FreeBSD 9 but I am puzzled and earn for the knowledge. Can anybody educate me on: * What might have happened with this notebook three years ago, when some layer over BIOS burned out? What are these layers? Where are the interface Ethernet addresses set up? Interesting, it was only this interface that lost its factory-assigned address: nfe0@pci0:0:10:0: class=0x02 card=0x30ea103c chip=0x054c10derev=0xa2 hdr=0x00 vendor = 'nVidia Corporation' device = 'MCP67 Ethernet' but not this one: ath0@pci0:3:0:0:class=0x02 card=0x137a103c chip=0x001c168crev=0x01 hdr=0x00 vendor = 'Atheros Communications Inc.' device = 'AR242x / AR542x Wireless Network Adapter (PCI-Express)' * What is the boot process doing, hanging out there after passing the boot menu stage? * Why does it hang there longer in FreeBSD 9, compared to 8? (And why doesn't it hang there at all in Debian?) * Is there any loader.conf variable or some such that would tell the system to safely skip things leading to this pause? Thanks, -- Alex -- alex-goncha...@comcast.net -- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: 8 to 9: A longer wait early in the boot of a (damaged) Compaq Presario
,--- I/Alex (Thu, 16 Feb 2012 12:34:36 -0500) * | There was one other odd thing that I noticed then: while Debian booted | without a delay, FreeBSD 8 made a long pause after passing the boot | menu: it would display the '/' character and sit there for some | non-trivial amount of seconds. I assumed that it was doing some BIOS | querying, and with BIOS (firmware?) being damaged, it took the system | some time to figure things out... perhaps it was re-querying BIOS, | seeing the insane value of 0 for an interface's Ethernet address (I | have many machines running FreeBSD, including multiple laptops, and | this machine is unique in the long bootup pause). `-* ,--- Rares Aioanei (Thu, 16 Feb 2012 20:08:14 +0200) * | I get the same on my HP Pavilion dv9750 laptop, but with an intact BIOS, | afaict. And that happens regardless of the wi-fi card's state (eg | disabled or enabled from the hardware button). Maybe this helps. `* To add to the fact base: I don't have any of this with my HP Pavilion DV6-1334US: neither with FreeBSD 8 nor 9 (upgraded that laptop two days ago, too.) -- Alex -- alex-goncha...@comcast.net -- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: 8 to 9: A longer wait early in the boot of a (damaged) Compaq Presario
On 02/16/2012 07:34 PM, Alex Goncharov wrote: About three years ago, my Compaq Presario F700 notebook got damaged in BIOS: it carried Windows Vista then, and that OS could not be recovered from the system image disks I had created for a brand-new machine. The damage was somewhere around BIOS/firmware area -- the way the console looked on a bootup looked differently (simpler) now that after several reboots trying to recover Vista, it got fried. Some googling told me then that the irreversible loss of Windows was not unusual for these Compaq machines -- the damaged systems didn't give one a chance to use the recovery disks. OK, I made the system dual bootable to Debian Linux and FreeBSD 8 then; with that, it booted all right, but in both cases the 'nfe0' interface Ethernet address was being set to 0. No big deal: I used an Ethernet address from my older laptop destined to be destroyed and gave it to 'nfe0' when setting the network interface properties at the system initialization. Works great, both in Debian and FreeBSD. There was one other odd thing that I noticed then: while Debian booted without a delay, FreeBSD 8 made a long pause after passing the boot menu: it would display the '/' character and sit there for some non-trivial amount of seconds. I assumed that it was doing some BIOS querying, and with BIOS (firmware?) being damaged, it took the system some time to figure things out... perhaps it was re-querying BIOS, seeing the insane value of 0 for an interface's Ethernet address (I have many machines running FreeBSD, including multiple laptops, and I get the same on my HP Pavilion dv9750 laptop, but with an intact BIOS, afaict. And that happens regardless of the wi-fi card's state (eg disabled or enabled from the hardware button). Maybe this helps. -- Rares Aioanei ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: 8 to 9: A longer wait early in the boot of a (damaged) CompaqPresario
- Original Message -
From: Alex Goncharov alex-goncha...@comcast.net
About a week ago, I made a jump and upgraded the system's FreeBSD from
version 8 to 9. Everything is great (I am typing this message on that
machine now) but the boot pause after the (looking new in 9) boot menu
is *much* longer now -- it will show the '\' character and wait for,
subjectively, half a minute before putting anything else on the
screen.
Two things spring to mind which could help:
1. The reduce the slice sampling size in sys/boot/zfs/zfs.c which was
increased recently
2. disable boot time mem tests using the attached patch
Patches for both from 8.2-RELEASE attached. For #2 you also need the
following in /boot/loader.conf
hw.memtest.tests=0
Regards
Steve
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.
In the event of misdirection, illegible or incomplete transmission please
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.
--- sys/amd64/amd64/machdep.c 2011/06/08 08:12:15 222853
+++ sys/amd64/amd64/machdep.c 2011/07/30 13:33:05 224516
@@ -1309,7 +1309,7 @@
{
int i, physmap_idx, pa_indx, da_indx;
vm_paddr_t pa, physmap[PHYSMAP_SIZE];
- u_long physmem_tunable;
+ u_long physmem_tunable, memtest, tmpul;
pt_entry_t *pte;
struct bios_smap *smapbase, *smap, *smapend;
u_int32_t smapsize;
@@ -1372,6 +1372,14 @@
Maxmem = atop(physmem_tunable);
/*
+* By default keep the memtest enabled. Use a general name so that
+* one could eventually do more with the code than just disable it.
+*/
+ memtest = 1;
+ if (TUNABLE_ULONG_FETCH(hw.memtest.tests, tmpul))
+ memtest = tmpul;
+
+ /*
* Don't allow MAXMEM or hw.physmem to extend the amount of memory
* in the system.
*/
@@ -1433,6 +1441,8 @@
goto do_dump_avail;
page_bad = FALSE;
+ if (memtest == 0)
+ goto skip_memtest;
/*
* map page into kernel: valid, read/write,non-cacheable
@@ -1470,6 +1480,7 @@
*/
*(int *)ptr = tmp;
+skip_memtest:
/*
* Adjust array of valid/good pages.
*/
--- sys/boot/zfs/zfs.c.orig 2011-10-20 18:15:29.966685430 +
+++ sys/boot/zfs/zfs.c 2011-10-20 18:18:22.291033636 +
@@ -45,6 +45,12 @@
#include zfsimpl.c
+/*
+ * For GPT this should be 128 but leads to 50+ second delay in BTX loader so
+ * we use the original 4 pre r198420 by default for the boot process
+ */
+#define ZFS_MAX_SLICES 4
+
static int zfs_open(const char *path, struct open_file *f);
static int zfs_write(struct open_file *f, void *buf, size_t size, size_t
*resid);
static int zfs_close(struct open_file *f);
@@ -415,7 +421,7 @@
if (vdev_probe(vdev_read, (void*) (uintptr_t) fd, 0))
close(fd);
- for (slice = 1; slice = 128; slice++) {
+ for (slice = 1; slice = ZFS_MAX_SLICES; slice++) {
sprintf(devname, disk%dp%d:, unit, slice);
fd = open(devname, O_RDONLY);
if (fd == -1) {
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [RFT][patch] Scheduling for HTT and not only
On 15.02.12 20:47, Alexander Motin wrote: On 02/14/12 00:38, Alexander Motin wrote: I see no much point in committing them sequentially, as they are quite orthogonal. I need to make one decision. I am going on small vacation next week. It will give time for thoughts to settle. May be I indeed just clean previous patch a bit and commit it when I get back. I've spent too much time trying to make these things formal and so far results are not bad, but also not so brilliant as I would like. May be it is indeed time to step back and try some more simple solution. I've decided to stop those cache black magic practices and focus on things that really exist in this world -- SMT and CPU load. I've dropped most of cache related things from the patch and made the rest of things more strict and predictable: http://people.freebsd.org/~mav/sched.htt34.patch This patch adds check to skip fast previous CPU selection if it's SMT neighbor is in use, not just if no SMT present as in previous patches. I've took affinity/preference algorithm from the first patch and improved it. That makes pickcpu() to prefer previous core or it's neighbors in case of equal load. That is very simple to keep it, but still should give cache hits. I've changed the general algorithm of topology tree processing. First I am looking for idle core on the same last-level cache as before, with affinity to previous core or it's neighbors on higher level caches. Original code could put additional thread on already busy core, while next socket is completely idle. Now if there is no idle core on this cache, then all other CPUs are checked. CPU groups comparison now done in two steps: first, same as before, compared summary load of all cores; but now, if it is equal, I am comparing load of the less/most loaded cores. That should allow to differentiate whether load 2 really means 1+1 or 2+0. In that case group with 2+0 will be taken as more loaded than one with 1+1, making group choice more grounded and predictable. I've added randomization in case if all above factors are equal. As before I've tested this on Core i7-870 with 4 physical and 8 logical cores and Atom D525 with 2 physical and 4 logical cores. On Core i7 I've got speedup up to 10-15% in super-smack MySQL and PostgreSQL indexed select for 2-8 threads and no penalty in other cases. pbzip2 shows up to 13% performance increase for 2-5 threads and no penalty in other cases. Tests on Atom show mostly about the same performance as before in database benchmarks: faster for 1 thread, slower for 2-3 and about the same for other cases. Single stream network performance improved same as for the first patch. That CPU is quite difficult to handle as with mix of effective SMT and lack of L3 cache different scheduling approaches give different results in different situations. Specific performance numbers can be found here: http://people.freebsd.org/~mav/bench.ods Every point there includes at least 5 samples and except pbzip2 test that is quite unstable with previous sources all are statistically valid. Florian is now running alternative set of benchmarks on dual-socket hardware without SMT. I have updated my PostgreSQL [1] and pbzip2 [2] benchmarks. You should be looking for ULE+mav-htt33. On a system without HTT this patch is at least as good as stock ULE and in some cases it's a nice improvement. Florian [1] https://docs.google.com/spreadsheet/ccc?key=0Ai0N1xDe3uNAdDRxcVFiYjNMSnJWOTZhUWVWWlBlemchl=en_USpli=1#gid=4 [2]https://docs.google.com/spreadsheet/ccc?key=0Ai0N1xDe3uNAdDRxcVFiYjNMSnJWOTZhUWVWWlBlemchl=en_USpli=1#gid=2 signature.asc Description: OpenPGP digital signature
Re: 8 to 9: A longer wait early in the boot of a (damaged) CompaqPresario
,--- You/Steven (Thu, 16 Feb 2012 18:23:15 -) *
| Two things spring to mind which could help:
| 1. The reduce the slice sampling size in sys/boot/zfs/zfs.c which was
| increased recently
Your mentioning of ZFS made me realize that I was building
RELENG_{8,9} with /etc/src.conf not excluding it; I'll try to rebuild
9 with WITHOUT_ZFS = 1.
| 2. disable boot time mem tests using the attached patch
|
| Patches for both from 8.2-RELEASE attached. For #2 you also need the
| following in /boot/loader.conf
| hw.memtest.tests=0
I'll try this later, hopefully within a week, but I doubt the
likelihood of good effects from either: after all, the same builds
used on my HP Pavilion (4G of memory) have always led to non-pausing
bootups, as opposed to the pausing ones in my (damaged) Compaq
Presario (2G of memory.)
Thanks!
-- Alex -- alex-goncha...@comcast.net --
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Kerberos and FreeBSD
On Wed, 15 Feb 2012, Ansar Mohammed wrote: Going back on this topic, it seems that there are alot of things that are being shipped with FreeBSD that I am not sure we need in the base distribution. Does anyone use portalfs? Maybe not ... per http://wiki.freebsd.org/NONMPSAFE_DEORBIT_VFS it is scheduled for removal from the tree in September if no one gives it some love. For things like that, which are perhaps useful to someone, somewhere, I see no harm in keeping them around if there is minimal effort in doing so. But, when there comes a point where substantial effort is required for upkeep, if there is not a visible use, removing them (with advance warning on the order of a year) seems like the right thing to do. We don't really have the manpower to make gratuitous changes to what's in the tree just for the sake of making changes, and removing anything has a chance of sparking a big flamewar. -Ben Kaduk ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: nologin size
From the Makefile... # It is important that nologin be statically linked for security # reasons. A dynamic non-setuid binary can be linked against a trojan # libc by setting LD_LIBRARY_PATH appropriately. Both sshd(8) and # login(1) make it possible to log in with an unsanitized environment, # rendering a dynamic nologin binary virtually useless. NO_SHARED= YES On Wed, Feb 15, 2012 at 02:28:54PM -0500, Ansar Mohammed wrote: Hello all, I am trying to build a minimal size FreeBSD 9 installation and I noticed that the size of nologin is almost 200k. I built FreeBSD from source so I checked the distribution, and it's also 200k. So I went back to the source and just compiled nologin.c and it came up to 5k. Can anyone explain why this executable is so large? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org -- ;s =; pgpyxfFSEGY5u.pgp Description: PGP signature
BUG: 9.0 stage 2 boot (/boot/boot)
Anyway, after upgrading to 9.0, my USB stick, when created, started to hang at stage 2 boot. I have a custom setup, where BSD label 'a', has a content of /boot/* So when 'a' is being hit by stage 2 boot, there is boot.config waiting for it. After it reads it and displays it's content, it echos 'No' and hangs. I stare at it and can't believe as boot.config's information is correct! I hit '?' and it list all files in 'a'. Then I simply RE-type what is displayed on screen (content of boot.config - path to loader) And loader kicks in! I do this a few times more and EACH time I have to RE-type correct info! Tested on other machine, same thing. However, this same custom layout works for HDD's, but NOT for USB stick. I've extracted binary installs of 8.2 and 9.0 R: MD5 (8_boot) = adb1e84e96bd434e51cafaaa0ef22584 MD5 (9_boot) = 40f3f6403ebd5e131259d1336b4b50ad Then: # gpart bootcode -b 8_boot da0s2 And sudenly that USB stick boots, without ANY other change! Just an old stage 2 boot code, from R8 was enough. Domagoj Smolčić ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
