Re: build failures after stdlib update
On 2010-03-21 22:20, Garrett Cooper wrote: From gcc(1): -s Remove all symbol table and relocation information from the exe- cutable. This is more or less the same as running strip(1) over the produced executables. Usually one uses it for non-debug builds. That seems a bit harsh (especially because that makes certain libraries uses kind of moot, like *_p.a, right?). No, since -s only applies to the linking stage, so for executables or shared libraries. It does not apply to object files or libraries. It could be argued that -s really belongs in LDFLAGS... :) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: build failures after stdlib update
on 22/03/2010 00:12 Alexander Best said the following: Andriy Gapon schrieb am 2010-03-21: on 21/03/2010 23:11 Alexander Best said the following: *hehe* that makes more sense. well i already sent you lp. unfortunately str is not available to gdb: (gdb) print str Variable str is not available. In cases like this sometimes the argument is still available for examination as a variable further down the stack (i.e. in one of the calling functions). hmmm... str might be -m then, because of: #1 0x00416782 in concat (first=0x417618 -m) at /usr/src/gnu/usr.bin/cc/libiberty/../../../../contrib/gcclibs/libiberty/concat.c:76 Perhaps, but doesn't look like it. Could you please do 'print *lp'? -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [head tinderbox] failure on i386/i386
on 21/03/2010 08:25 Garrett Cooper said the following: On Sat, Mar 20, 2010 at 9:58 PM, FreeBSD Tinderbox tinder...@freebsd.org wrote: TB --- 2010-03-21 04:53:25 - /usr/bin/make -B buildkernel KERNCONF=PAE Kernel build for PAE started on Sun Mar 21 04:53:25 UTC 2010 stage 1: configuring the kernel stage 2.1: cleaning up the object tree stage 2.2: rebuilding the object tree stage 2.3: build tools stage 3.1: making dependencies stage 3.2: building everything [...] cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/libkern/qdivrem.c cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/libkern/ucmpdi2.c cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/libkern/udivdi3.c cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/libkern/umoddi3.c cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/compat/x86bios/x86bios.c cc1: warnings being treated as errors /src/sys/compat/x86bios/x86bios.c: In function 'x86bios_map_mem': /src/sys/compat/x86bios/x86bios.c:558: warning: cast to pointer from integer of different size *** Error code 1 Stop in /obj/i386/src/sys/PAE. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2010-03-21 04:58:08 - WARNING: /usr/bin/make returned exit code 1 TB --- 2010-03-21 04:58:08 - ERROR: failed to build PAE kernel TB --- 2010-03-21 04:58:08 - 5080.43 user 936.95 system 6788.14 real Hi Jung, Could you please resolve this error? Thanks, It would have been nice to actually CC Jung-uk :-) The problem seems to be that with PAE type of x86bios_rom_phys, vm_paddr_t, is 64-bit. I am not sure what values x86bios_rom_phys can have, but most likely it can be simply cast to a 32-bit value. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
In message 4ba633a0.2090...@icyb.net.ua, Andriy Gapon writes: on 21/03/2010 16:05 Alexander Motin said the following: Ivan Voras wrote: Hmm, it looks like it could be easy to spawn more g_* threads (and, barring specific class behaviour, it has a fair chance of working out of the box) but the incoming queue will need to also be broken up for greater effect. According to notes, looks there is a good chance to obtain races, as some places expect only one up and one down thread. I haven't given any deep thought to this issue, but I remember us discussing them over beer :-) The easiest way to obtain more parallelism, is to divide the mesh into multiple independent meshes. This will do you no good if you have five disks in a RAID-5 config, but if you have two disks each mounted on its own filesystem, you can run a g_up g_down for each of them. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: build failures after stdlib update
Andriy Gapon schrieb am 2010-03-22: on 22/03/2010 00:12 Alexander Best said the following: Andriy Gapon schrieb am 2010-03-21: on 21/03/2010 23:11 Alexander Best said the following: *hehe* that makes more sense. well i already sent you lp. unfortunately str is not available to gdb: (gdb) print str Variable str is not available. In cases like this sometimes the argument is still available for examination as a variable further down the stack (i.e. in one of the calling functions). hmmm... str might be -m then, because of: #1 0x00416782 in concat (first=0x417618 -m) at /usr/src/gnu/usr.bin/cc/libiberty/../../../../contrib/gcclibs/libiberty/concat.c:76 Perhaps, but doesn't look like it. Could you please do 'print *lp'? (gdb) print *lp Cannot access memory at address 0xc092f0 -- Alexander Best ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
On Sun, 21 Mar 2010 19:03:56 +0200 Alexander Motin m...@freebsd.org wrote: Scott Long wrote: Are there non-CAM drivers that look at MAXPHYS, or that silently assume that MAXPHYS will never be more than 128k? That is a question. I only did a quickdirty grep looking for MAXPHYS in /sys. Some drivers redefine MAXPHYS to be 512KiB. Some use their own local MAXPHYS which is usually 128KiB. Some look at MAXPHYS to figure out other things; the details escape me. There's one driver which actually uses 100*MAXPHYS for something, but I didn't check the details. Lots of them were non-CAM drivers AFAICT. -- Gary Jennejohn ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: csup/svn/ldd make host unresponsive [WAS: Re: ldd leaves the machine unresponsive]
On Sun, 21 Mar 2010 14:22, Anton Shterenlikht wrote:
In Message-Id: 20100321182214.gk84...@mech-cluster241.men.bris.ac.uk
On Sat, Mar 20, 2010 at 08:53:37PM +, Anton Shterenlikht wrote:
On Sat, Mar 20, 2010 at 03:44:46PM +, Anton Shterenlikht wrote:
On Sat, Mar 20, 2010 at 07:27:43AM -0400, jhell wrote:
On Fri, 19 Mar 2010 17:15, Anton Shterenlikht wrote:
In Message-Id: 20100319211535.ga76...@mech-cluster241.men.bris.ac.uk
On Thu, Mar 18, 2010 at 11:29:36AM -0400, jhell wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Wed, 17 Mar 2010 12:32, Anton Shterenlikht wrote:
In Message-Id: 20100317163230.gj87...@mech-cluster241.men.bris.ac.uk
Just updated to ia64 r205248
If my problem is due to my mis-configuration,
I apologise in advance.
I run this shell script after each upgrade
and 'make delete-old-libs' to check
if any shared objects need to be rebuilt:
start script
#!/bin/sh
for file in `find /bin /sbin /usr/bin /usr/sbin /usr/lib /usr/libexec /usr/local -name
*`
do
echo $file
ldd $file /root/ldd_results 2 /dev/zero
done
end script
This will probably do closer to what you actually would want to look for.
Writing to /dev/zero ... I don't know never tried it since /dev/null is
usually the standard place to throw trash.
#!/bin/sh
for file in `find /*bin /usr/*bin /usr/lib* /usr/local/*bin -type f` do
echo $file
ldd $file /root/ldd_results 2/dev/null
done
The problem with your script is that it finds most files that it can not
or is not useful to run ldd on and leaves you junk in return.
It might be more useful if you searched for dynamically linked ELF
binaries to run ldd against like the following.
=== Script starts here ===
#!/bin/sh
SEARCHPATH=/*bin /usr/*bin /usr/lib* /usr/local/*bin
trap 'exit 1' 2
check_libs() {
for spath in $SEARCHPATH; do
for ifelf in `find $spath -type f`; do
ldd `file $ifelf | grep dynamically | cut -f1 -d:`
done
done
}
check_libs 2/dev/null
=== Script ends here ===
The above will find all type ELF * that are dynamically linked within the
SEARCHPATH variable and run ldd on them and print the results to stdout.
Obviously since you are going to have thousands of files being questioned,
stdout is not going to be useful.
So with the about stated:
save the script to: checklibs.sh
run with: sh checklibs.sh /root/checklibs_output
or: script /root/checklibs_output checklibs.sh
After the upgrade to r205248, the script
freezes at seemingly random points.
Unneeded disk usage execution.
I can still ssh to the machine (using keys), i.e.
I see the welcome message, but cannot get to the console prompt.
Of course... to many open files or processes in wait. SSH already has the
information it needs loaded into memory, that's why you can get sort-of-in
ZFS file-system perhaps ?
I've no ZFS.
I'm seeing very similar behaviour now with csup:
( I do csup -L2 /root/ports-supfile, where
# cat /root/ports-supfile
*default host=cvsup.uk.FreeBSD.org
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=. delete use-rel-suffix compress
ports-all
# )
top(1) shows:
last pid: 1160; load averages: 0.00, 0.06, 0.07
up 0+00:10:53 15:05:52
81 processes: 3 running, 61 sleeping, 17 waiting
CPU 0: 0.0% user, 0.0% nice, 0.2% system, 0.0% interrupt, 99.8% idle
CPU 1: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle
Mem: 23M Active, 19M Inact, 75M Wired, 136K Cache, 34M Buf, 5900M Free
Swap: 2780M Total, 2780M Free
PIDUIDTHR PRI NICE SIZERES STATE C TIME WCPU COMMAND
10 0 2 171 ki31 0K64K RUN 0 20:18 198.00% idle
11 0 17 -48- 0K 544K WAIT0 0:01 0.00% intr
1118 1001 1 960 12800K 3920K CPU00 0:00 0.00% top
4 0 1 -8- 0K32K - 1 0:00 0.00% g_down
1158 0 4 -80 43672K 6296K biowr 0 0:00 0.00% csup
which stays in biowr state indefinitely.
I can issue kill -9 or kill -HUP from top(1),
which makes csup change state to STOP, but
nothing else happens.
As before, I can't log in from other terminals
and have to do a cold reset. I've reinstalled
on another disk, so not sure what's going on.
I think rm(1) is also extremely slow, but
maybe I'm imagining things.
many thanks
anton
I would post up the contents of your make.conf your kernel config your
dmesg somewhere so it can be evaluated.
When I reinstalled 8.0 from a CD,
I updated source with csup, that worked.
However, after upgrading to current, I can't get
any luck with csup. The important bit is that
I don't really know what revision this is.
I've no /etc/make.conf
kernel config:
http://seis.bris.ac.uk/~mexas/freebsd/ia64/rx2600/uzi/UZI
dmesg:
http://seis.bris.ac.uk/~mexas/freebsd/ia64/rx2600/uzi/dmesg.boot
Marcel, this might be of some interest.
I
[head tinderbox] failure on i386/i386
TB --- 2010-03-22 10:55:00 - tinderbox 2.6 running on freebsd-current.sentex.ca TB --- 2010-03-22 10:55:00 - starting HEAD tinderbox run for i386/i386 TB --- 2010-03-22 10:55:00 - cleaning the object tree TB --- 2010-03-22 10:55:21 - cvsupping the source tree TB --- 2010-03-22 10:55:21 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/HEAD/i386/i386/supfile TB --- 2010-03-22 10:57:06 - building world TB --- 2010-03-22 10:57:06 - MAKEOBJDIRPREFIX=/obj TB --- 2010-03-22 10:57:06 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2010-03-22 10:57:06 - TARGET=i386 TB --- 2010-03-22 10:57:06 - TARGET_ARCH=i386 TB --- 2010-03-22 10:57:06 - TZ=UTC TB --- 2010-03-22 10:57:06 - __MAKE_CONF=/dev/null TB --- 2010-03-22 10:57:06 - cd /src TB --- 2010-03-22 10:57:06 - /usr/bin/make -B buildworld World build started on Mon Mar 22 10:57:07 UTC 2010 Rebuilding the temporary build tree stage 1.1: legacy release compatibility shims stage 1.2: bootstrap tools stage 2.1: cleaning up the object tree stage 2.2: rebuilding the object tree stage 2.3: build tools stage 3: cross tools stage 4.1: building includes stage 4.2: building libraries stage 4.3: make dependencies stage 4.4: building everything World build completed on Mon Mar 22 11:56:20 UTC 2010 TB --- 2010-03-22 11:56:20 - generating LINT kernel config TB --- 2010-03-22 11:56:20 - cd /src/sys/i386/conf TB --- 2010-03-22 11:56:20 - /usr/bin/make -B LINT TB --- 2010-03-22 11:56:20 - building LINT kernel TB --- 2010-03-22 11:56:20 - MAKEOBJDIRPREFIX=/obj TB --- 2010-03-22 11:56:20 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2010-03-22 11:56:20 - TARGET=i386 TB --- 2010-03-22 11:56:20 - TARGET_ARCH=i386 TB --- 2010-03-22 11:56:20 - TZ=UTC TB --- 2010-03-22 11:56:20 - __MAKE_CONF=/dev/null TB --- 2010-03-22 11:56:20 - cd /src TB --- 2010-03-22 11:56:20 - /usr/bin/make -B buildkernel KERNCONF=LINT Kernel build for LINT started on Mon Mar 22 11:56:20 UTC 2010 stage 1: configuring the kernel stage 2.1: cleaning up the object tree stage 2.2: rebuilding the object tree stage 2.3: build tools stage 3.1: making dependencies stage 3.2: building everything Kernel build for LINT completed on Mon Mar 22 12:21:18 UTC 2010 TB --- 2010-03-22 12:21:18 - building GENERIC kernel TB --- 2010-03-22 12:21:18 - MAKEOBJDIRPREFIX=/obj TB --- 2010-03-22 12:21:18 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2010-03-22 12:21:18 - TARGET=i386 TB --- 2010-03-22 12:21:18 - TARGET_ARCH=i386 TB --- 2010-03-22 12:21:18 - TZ=UTC TB --- 2010-03-22 12:21:18 - __MAKE_CONF=/dev/null TB --- 2010-03-22 12:21:18 - cd /src TB --- 2010-03-22 12:21:18 - /usr/bin/make -B buildkernel KERNCONF=GENERIC Kernel build for GENERIC started on Mon Mar 22 12:21:18 UTC 2010 stage 1: configuring the kernel stage 2.1: cleaning up the object tree stage 2.2: rebuilding the object tree stage 2.3: build tools stage 3.1: making dependencies stage 3.2: building everything Kernel build for GENERIC completed on Mon Mar 22 12:40:30 UTC 2010 TB --- 2010-03-22 12:40:30 - building PAE kernel TB --- 2010-03-22 12:40:30 - MAKEOBJDIRPREFIX=/obj TB --- 2010-03-22 12:40:30 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2010-03-22 12:40:30 - TARGET=i386 TB --- 2010-03-22 12:40:30 - TARGET_ARCH=i386 TB --- 2010-03-22 12:40:30 - TZ=UTC TB --- 2010-03-22 12:40:30 - __MAKE_CONF=/dev/null TB --- 2010-03-22 12:40:30 - cd /src TB --- 2010-03-22 12:40:30 - /usr/bin/make -B buildkernel KERNCONF=PAE Kernel build for PAE started on Mon Mar 22 12:40:30 UTC 2010 stage 1: configuring the kernel stage 2.1: cleaning up the object tree stage 2.2: rebuilding the object tree stage 2.3: build tools stage 3.1: making dependencies stage 3.2: building everything [...] cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/libkern/qdivrem.c cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/libkern/ucmpdi2.c cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes
Re: Increasing MAXPHYS
Quoting Scott Long sco...@samsco.org (from Sat, 20 Mar 2010 12:17:33 -0600): code was actually taking advantage of the larger I/O's. The improvement really depends on the workload, of course, and I wouldn't expect it to be noticeable for most people unless they're running something like a media server. I don't think this is limited to media servers, think about situations where you process a large amount of data seuqntially... (seuqntial access case in a big data-warehouse scenario or a 3D render farm which get's the huge amount of data from a shared resource (how many render-clients can I support at the same time with my disk infrastructure-scenario) or some of the bigtable/nosql stuff which seems to be more and more popular at some sites). There are enough situations where sequential file access is the key performance metric so that I wouldn't say that only media servers depend upon large sequential I/O's. Bye, Alexander. -- That's life. What's life? A magazine. How much does it cost? Two-fifty. I only have a dollar. That's life. http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
On Monday 22 March 2010 7:40:18 am Gary Jennejohn wrote: On Sun, 21 Mar 2010 19:03:56 +0200 Alexander Motin m...@freebsd.org wrote: Scott Long wrote: Are there non-CAM drivers that look at MAXPHYS, or that silently assume that MAXPHYS will never be more than 128k? That is a question. I only did a quickdirty grep looking for MAXPHYS in /sys. Some drivers redefine MAXPHYS to be 512KiB. Some use their own local MAXPHYS which is usually 128KiB. Some look at MAXPHYS to figure out other things; the details escape me. There's one driver which actually uses 100*MAXPHYS for something, but I didn't check the details. Lots of them were non-CAM drivers AFAICT. The problem is the drivers that _don't_ reference MAXPHYS. The driver author at the time knew that MAXPHYS was 128k, so he did the MAXPHYS-dependent calculation and just put the result in the driver (e.g. only supporting up to 32 segments (32 4k pages == 128k) in a bus dma tag as a magic number to bus_dma_tag_create() w/o documenting that the '32' was derived from 128k or what the actual hardware limit on nsegments is). These cannot be found by a simple grep, they require manually inspecting each driver. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
On Mon, 22 Mar 2010 01:53, Alexander Motin wrote: In Message-Id: 4ba705cb.9090...@freebsd.org jhell wrote: On Sun, 21 Mar 2010 20:54, jhell@ wrote: I played with it on one re-compile of a kernel and for the sake of it DFLTPHYS=128 MAXPHYS=256 and found out that I could not cause a crash dump to be performed upon request (reboot -d) due to the boundary being hit for DMA which is 65536. Obviously this would have to be adjusted in ata-dma.c. I suppose that there would have to be a better way to get the real allowable boundary from the running system instead of setting it statically. Other then the above I do not see a reason why not... It is HEAD and this is the type of experimental stuff it was meant for. I should have also said that I also repeated the above without setting DFLTPHYS and setting MAXPHYS to 256. It was bad idea to increase DFLTPHYS. It is not intended to be increased. I just wanted to see what I could break; when I increased DFLTPHYS it was just for that purpose. It booted and everything was running after. Wasn't long enough to do any damage. About DMA boundary, I do not very understand the problem. Yes, legacy ATA has DMA boundary of 64K, but there is no problem to submit S/G list of several segments. How long ago have you tried it, on which controller and which diagnostics do you have? atap...@pci0:0:31:1: class=0x01018a card=0x01271028 chip=0x24cb8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801DB/DBL (ICH4/ICH4-L) UltraATA/100 EIDE Controller' class = mass storage subclass = ATA I do not have any diagnostics but if any are requested I do have the kernel's that I have tuned to the above values readily available to run again. The first time I tuned MAXPHYS was roughly about 7 weeks ago. That was until I noticed I could not get a crash dump for a problem I was having a week later and had to revert back to its default setting of 128. The problem I had a week later was unrelated. Two days ago when I saw this thread I recalled having modified MAXPHYS but could not remember the problem it caused so I re-enabled it again to reproduce the problem for sureness. Anything else you need please address, Regards, -- jhell ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
On Mon, Mar 22, 2010 at 8:39 AM, John Baldwin j...@freebsd.org wrote: On Monday 22 March 2010 7:40:18 am Gary Jennejohn wrote: On Sun, 21 Mar 2010 19:03:56 +0200 Alexander Motin m...@freebsd.org wrote: Scott Long wrote: Are there non-CAM drivers that look at MAXPHYS, or that silently assume that MAXPHYS will never be more than 128k? That is a question. I only did a quickdirty grep looking for MAXPHYS in /sys. Some drivers redefine MAXPHYS to be 512KiB. Some use their own local MAXPHYS which is usually 128KiB. Some look at MAXPHYS to figure out other things; the details escape me. There's one driver which actually uses 100*MAXPHYS for something, but I didn't check the details. Lots of them were non-CAM drivers AFAICT. The problem is the drivers that _don't_ reference MAXPHYS. The driver author at the time knew that MAXPHYS was 128k, so he did the MAXPHYS-dependent calculation and just put the result in the driver (e.g. only supporting up to 32 segments (32 4k pages == 128k) in a bus dma tag as a magic number to bus_dma_tag_create() w/o documenting that the '32' was derived from 128k or what the actual hardware limit on nsegments is). These cannot be found by a simple grep, they require manually inspecting each driver. 100% awesome comment. On another kernel, I myself was guilty of this crime (I did have a nice comment though above the def). This has been a great thread since our application really needs some of the optimizations that are being thrown around here. We have found in real live performance testing that we are almost always either controller bound (i.e. adding more disks to spread IOPs has little to no effect in large array configurations on throughput, we suspect that is hitting the RAID controller's firmware limitations) or tps bound, i.e. I never thought going from 128k - 256k per transaction would have a dramatic effect on throughput (but I never verified). Back to HBAs, AFAIK, every modern iteration of the most popular HBAs can easily do way more than a 128k scatter/gather I/O. Do you guys know of any *modern* (circa within the last 3-4 years) that can not do more than 128k at a shot? In other words, I've always thought the limit was kernel imposed and not what the memory controller on the card can do (I certainly never got the impression talking with some of the IHVs over the years that they were designing their hardware for a 128k limit - I sure hope not!). -aps ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [head tinderbox] failure on i386/i386
On Monday 22 March 2010 04:19 am, Andriy Gapon wrote: on 21/03/2010 08:25 Garrett Cooper said the following: On Sat, Mar 20, 2010 at 9:58 PM, FreeBSD Tinderbox tinder...@freebsd.org wrote: TB --- 2010-03-21 04:53:25 - /usr/bin/make -B buildkernel KERNCONF=PAE Kernel build for PAE started on Sun Mar 21 04:53:25 UTC 2010 stage 1: configuring the kernel stage 2.1: cleaning up the object tree stage 2.2: rebuilding the object tree stage 2.3: build tools stage 3.1: making dependencies stage 3.2: building everything [...] cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/libkern/qdivrem.c cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/libkern/ucmpdi2.c cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/libkern/udivdi3.c cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/libkern/umoddi3.c cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /src/sys/compat/x86bios/x86bios.c cc1: warnings being treated as errors /src/sys/compat/x86bios/x86bios.c: In function 'x86bios_map_mem': /src/sys/compat/x86bios/x86bios.c:558: warning: cast to pointer from integer of different size *** Error code 1 Stop in /obj/i386/src/sys/PAE. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2010-03-21 04:58:08 - WARNING: /usr/bin/make returned exit code 1 TB --- 2010-03-21 04:58:08 - ERROR: failed to build PAE kernel TB --- 2010-03-21 04:58:08 - 5080.43 user 936.95 system 6788.14 real Hi Jung, Could you please resolve this error? Thanks, It would have been nice to actually CC Jung-uk :-) The problem seems to be that with PAE type of x86bios_rom_phys, vm_paddr_t, is 64-bit. I am not sure what values x86bios_rom_phys can have, but most likely it can be simply cast to a 32-bit value. Oops, sorry. It will be fixed soon. I am testing an i386/PAE kernel now. Jung-uk Kim ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: csup/svn/ldd make host unresponsive [WAS: Re: ldd leaves the machine unresponsive]
On Mar 22, 2010, at 5:04 AM, jhell wrote: Not that this has anything to do with your problem but might turn into a problem if it is not your intent. But the above command will update your src to 9.0-CURRENT That is his intend. Anton: I fixed a bunch of things yesterday. You should be able to upgrade to HEAD. I have one more fix pending that's needed for preemption, but you should not hold off an upgrade for that... FYI, -- Marcel Moolenaar xcl...@mac.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
On Mar 22, 2010, at 9:52 AM, Alexander Sack wrote: On Mon, Mar 22, 2010 at 8:39 AM, John Baldwin j...@freebsd.org wrote: On Monday 22 March 2010 7:40:18 am Gary Jennejohn wrote: On Sun, 21 Mar 2010 19:03:56 +0200 Alexander Motin m...@freebsd.org wrote: Scott Long wrote: Are there non-CAM drivers that look at MAXPHYS, or that silently assume that MAXPHYS will never be more than 128k? That is a question. I only did a quickdirty grep looking for MAXPHYS in /sys. Some drivers redefine MAXPHYS to be 512KiB. Some use their own local MAXPHYS which is usually 128KiB. Some look at MAXPHYS to figure out other things; the details escape me. There's one driver which actually uses 100*MAXPHYS for something, but I didn't check the details. Lots of them were non-CAM drivers AFAICT. The problem is the drivers that _don't_ reference MAXPHYS. The driver author at the time knew that MAXPHYS was 128k, so he did the MAXPHYS-dependent calculation and just put the result in the driver (e.g. only supporting up to 32 segments (32 4k pages == 128k) in a bus dma tag as a magic number to bus_dma_tag_create() w/o documenting that the '32' was derived from 128k or what the actual hardware limit on nsegments is). These cannot be found by a simple grep, they require manually inspecting each driver. 100% awesome comment. On another kernel, I myself was guilty of this crime (I did have a nice comment though above the def). This has been a great thread since our application really needs some of the optimizations that are being thrown around here. We have found in real live performance testing that we are almost always either controller bound (i.e. adding more disks to spread IOPs has little to no effect in large array configurations on throughput, we suspect that is hitting the RAID controller's firmware limitations) or tps bound, i.e. I never thought going from 128k - 256k per transaction would have a dramatic effect on throughput (but I never verified). Back to HBAs, AFAIK, every modern iteration of the most popular HBAs can easily do way more than a 128k scatter/gather I/O. Do you guys know of any *modern* (circa within the last 3-4 years) that can not do more than 128k at a shot? 64K broken in MPT at the moment. The hardware can do it, the driver thinks it can do it, but it fails. AAC hardware traditionally cannot, but maybe the firmware has been improved in the past few years. I know that there are other low-performance devices that can't do more than 64 or 128K, but none are coming to mind at the moment. Still, it shouldn't be a universal assumption that all hardware can do big I/O's. Another consideration is that some hardware can do big I/O's, but not very efficiently. Not all DMA engines are created equal, and moving to compound commands and excessively long S/G lists can be a pessimization. For example, MFI hardware does a hinted prefetch on the segment list, but once you exceed a certain limit, that prefetch doesn't work anymore and the firmware has to take the slow path to execute the i/o. I haven't quantified this penalty yet, but it's something that should be thought about. In other words, I've always thought the limit was kernel imposed and not what the memory controller on the card can do (I certainly never got the impression talking with some of the IHVs over the years that they were designing their hardware for a 128k limit - I sure hope not!). You'd be surprised at the engineering compromises and handicaps that are committed at IHVs because of misguided marketters. Scott ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
[PATCH] rename COMPAT_FREEBSD32
On Mon, Mar 15, 2010 at 09:00:18AM -0700, Julian Elischer wrote: I certainly agree.. can it be changed please? I've waited a while to see what other opinions would be expressed on this topic. I believe there is sufficient support to rename COMPAT_FREEBSD32 to something else based on responses in the mailing lists. I am sorry if some may wish to label this a bikeshead. But we seem to have many folks disliking COMPAT_FREEBSD32. Based on responses to the topic of COMPAT_FREEBSD32, the following were the suggestions offered: COMPAT_ARCH32, COMPAT_ARCH_32BIT, COMPAT_32BIT_ARCH, COMPAT_32BIT, COMPAT_FREEBSD32BIT I went with the first as it seemed the most direct to me, but I find all but the last are suitable as well. (The last is probably too close to COMPAT_FREEBSDn given we do have other suggestions.) This patch was produced by checking out a fresh tree and running this command from the top level: wcfind sys -type f | xargs fgrep -l COMPAT_FREEBSD32 | xargs \ sed -i '' \ -e 's/COMPAT_FREEBSD32/COMPAT_ARCH32/g' \ -e 's/compat_freebsd32/compat_arch32/g' svn revert -R sys/compat/freebsd32 Thoughts? -- -- David (obr...@freebsd.org) Index: conf/options.amd64 === --- conf/options.amd64 (revision 205451) +++ conf/options.amd64 (working copy) @@ -11,7 +11,7 @@ MP_WATCHDOG # Options for emulators. These should only be used at config time, so # they are handled like options for static filesystems # (see src/sys/conf/options), except for broken debugging options. -COMPAT_FREEBSD32 opt_compat.h +COMPAT_ARCH32 opt_compat.h #IBCS2 opt_dontuse.h #COMPAT_LINUX opt_dontuse.h COMPAT_LINUX32 opt_compat.h Index: conf/options.ia64 === --- conf/options.ia64 (revision 205451) +++ conf/options.ia64 (working copy) @@ -9,7 +9,7 @@ LOG2_PAGE_SIZE opt_global.h UWX_TRACE_ENABLE opt_global.h -COMPAT_FREEBSD32 opt_compat.h +COMPAT_ARCH32 opt_compat.h EXCEPTION_TRACING opt_xtrace.h Index: conf/files.ia64 === --- conf/files.ia64 (revision 205451) +++ conf/files.ia64 (working copy) @@ -28,11 +28,11 @@ ukbdmap.h optional ukbd_dflt_keymap\ no-obj no-implicit-rule before-depend \ clean ukbdmap.h # -compat/freebsd32/freebsd32_ioctl.c optionalcompat_freebsd32 -compat/freebsd32/freebsd32_misc.c optionalcompat_freebsd32 -compat/freebsd32/freebsd32_syscalls.c optionalcompat_freebsd32 -compat/freebsd32/freebsd32_sysent.coptionalcompat_freebsd32 -compat/ia32/ia32_sysvec.c optionalcompat_freebsd32 +compat/freebsd32/freebsd32_ioctl.c optionalcompat_arch32 +compat/freebsd32/freebsd32_misc.c optionalcompat_arch32 +compat/freebsd32/freebsd32_syscalls.c optionalcompat_arch32 +compat/freebsd32/freebsd32_sysent.coptionalcompat_arch32 +compat/ia32/ia32_sysvec.c optionalcompat_arch32 contrib/ia64/libuwx/src/uwx_bstream.c standard contrib/ia64/libuwx/src/uwx_context.c standard contrib/ia64/libuwx/src/uwx_env.c standard @@ -68,10 +68,10 @@ ia64/acpica/madt.c optionalacpi ia64/disasm/disasm_decode.cstandard ia64/disasm/disasm_extract.c standard ia64/disasm/disasm_format.cstandard -ia64/ia32/ia32_misc.c optionalcompat_freebsd32 -ia64/ia32/ia32_reg.c optionalcompat_freebsd32 -ia64/ia32/ia32_signal.coptionalcompat_freebsd32 -ia64/ia32/ia32_trap.c optionalcompat_freebsd32 +ia64/ia32/ia32_misc.c optionalcompat_arch32 +ia64/ia32/ia32_reg.c optionalcompat_arch32 +ia64/ia32/ia32_signal.coptionalcompat_arch32 +ia64/ia32/ia32_trap.c optionalcompat_arch32 ia64/ia64/autoconf.c standard ia64/ia64/bus_machdep.cstandard ia64/ia64/busdma_machdep.c standard @@ -117,7 +117,7 @@ ia64/isa/isa_dma.c optionalisa ia64/pci/pci_cfgreg.c optionalpci isa/syscons_isa.c optionalsc isa/vga_isa.c optionalvga -kern/imgact_elf32.coptionalcompat_freebsd32 +kern/imgact_elf32.coptionalcompat_arch32 libkern/bcmp.c standard libkern/ffsl.c standard libkern/fls.c standard Index: conf/files.amd64 === --- conf/files.amd64(revision 205451) +++ conf/files.amd64(working copy) @@ -227,20 +227,20 @@ kern/link_elf_obj.c standard # # IA32
Re: hang in rpccon from interrupting NFS operations (Re: pointyhat panic)
That's strange, after recompiling the lastest 8_0 that contain the patch ( http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/rpc/clnt_vc.c.diff?r1=1.8.2.2.2.1;r2=1.8.2.2.2.2) after 5 days it stuck again with same symptoms, I've also got some in the nfs state: FreeBSD .. 8.0-RELEASE-p2 FreeBSD 8.0-RELEASE-p2 #0: Tue Mar 16 22:56:51 EET 2010 @..:/usr/obj/usr/src/sys/MYGEN amd64 When attaching the debugger for an rpccon process, It stuck in here #0 0x00080124051c in stat () from /lib/libc.so.7 http://img705.imageshack.us/img705/741/10032219218.png Can I do the online debug of the kernel, or how can I can help you to solve the problem ? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
In message: d9d66012-16fd-4fb6-ab6a-9a8d17727...@samsco.org Scott Long sco...@samsco.org writes: : I'd like to go in the opposite direction. The queue-dispatch-queue : model of GEOM is elegant and easy to extend, but very wasteful for : the simple case, where the simple case is one or two simple : partition transforms (mbr, bsdlabel) and/or a simple stripe/mirror : transform. None of these need a dedicated dispatch context in order : to operate. What I'd like to explore is compiling the GEOM stack at : creation time into a linear array of operations that happen without : a g_down/g_up context switch. As providers and consumers taste each : other and build a stack, that stack gets compiled into a graph, and : that graph gets executed directly from the calling context, both : from the dev_strategy() side on the top and the bio_done() on the : bottom. GEOM classes that need a detached context can mark : themselves as such, doing so will prevent a graph from being : created, and the current dispatch model will be retained. I have a few things to say on this. First, I've done similar things at past companies for systems that are similar to geom's queueing environment. It is possible to convert the queueing nodes in the graph to filtering nodes in the graph. Another way to look at this is to say you're implementing direct dispatch into geom's stack. This can be both good and bad, but should reduce latency a lot. One problem that I see is that you are calling into the driver from a different set of contexts. The queueing stuff was there to protect the driver from LoRs due to its routines being called from many different contexts, sometimes with other locks held (fact of life often in the kernel). So this certainly is something worth exploring, especially if we have optimized paths for up/down for certain geom classes while still allowing the current robust, but slow, paths for the more complicated nodes in the tree. It remains to be see if there's going to be issues around locking order, but we've hit that with both geom and ifnet in the past, so caution (eg, running with WITNESS turned on early and often) is advised. Warner ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: HEADS UP: COMPAT_IA32 renamed COMPAT_FREEBSD32
On Fri, Mar 12, 2010 at 12:50:32PM -0700, M. Warner Losh wrote: : On Thu, Mar 11, 2010 at 07:24:23PM -0700, M. Warner Losh wrote: So the issue isn't as cut and dried as you might think. There's multiple different conventions used here in addition to your simple example. I guess we'd have to take a poll to find out. Seems pretty cut and dried to me. COMPAT_FREEBSDn has an established context that does not match this new usage. That is - same bit'ness, compatibility with an older FreeBSD API for the same architecture. All the other COMPAT_* are for foreign ABI compatibility. COMPAT_LINUX32 possibly should have been COMPAT_LINUX_X86_64. (or is it MI and is usable as-is for PowerPC and MIPS? I haven't looked that deeply at the code.) Users of 64-bit systems that will be using COMPAT_FREEBSD32 are likely to find this a natural extension of the COMPAT_LINUX32 that they are likely already using. You know I am such a user - and I don't think it is so clear given the existence (and purpose) of COMPAT_FREEBSDn for the past many years. While it does match the directory name of 'sys/compat/freebsd32', it may be that freebsd32 was a poor choice for that directory's name. But given the recent discussion in another thread - I won't even suggest we rename it. -- -- David (obr...@freebsd.org) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
On Mon, Mar 22, 2010 at 2:45 PM, M. Warner Losh i...@bsdimp.com wrote: In message: d9d66012-16fd-4fb6-ab6a-9a8d17727...@samsco.org Scott Long sco...@samsco.org writes: : I'd like to go in the opposite direction. The queue-dispatch-queue : model of GEOM is elegant and easy to extend, but very wasteful for : the simple case, where the simple case is one or two simple : partition transforms (mbr, bsdlabel) and/or a simple stripe/mirror : transform. None of these need a dedicated dispatch context in order : to operate. What I'd like to explore is compiling the GEOM stack at : creation time into a linear array of operations that happen without : a g_down/g_up context switch. As providers and consumers taste each : other and build a stack, that stack gets compiled into a graph, and : that graph gets executed directly from the calling context, both : from the dev_strategy() side on the top and the bio_done() on the : bottom. GEOM classes that need a detached context can mark : themselves as such, doing so will prevent a graph from being : created, and the current dispatch model will be retained. I have a few things to say on this. First, I've done similar things at past companies for systems that are similar to geom's queueing environment. It is possible to convert the queueing nodes in the graph to filtering nodes in the graph. Another way to look at this is to say you're implementing direct dispatch into geom's stack. This can be both good and bad, but should reduce latency a lot. One problem that I see is that you are calling into the driver from a different set of contexts. The queueing stuff was there to protect the driver from LoRs due to its routines being called from many different contexts, sometimes with other locks held (fact of life often in the kernel). So this certainly is something worth exploring, especially if we have optimized paths for up/down for certain geom classes while still allowing the current robust, but slow, paths for the more complicated nodes in the tree. It remains to be see if there's going to be issues around locking order, but we've hit that with both geom and ifnet in the past, so caution (eg, running with WITNESS turned on early and often) is advised. Am I going crazy or does this sound a lot like Sun/SVR's stream based network stack? (design and problems, stream stack locking was notoriously tricky for the exact issue mentioned above, different running contexts with different locking granularity/requirements). -aps ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: HEADS UP: COMPAT_IA32 renamed COMPAT_FREEBSD32
In message: 20100322185331.ga88...@dragon.nuxi.org David O'Brien obr...@freebsd.org writes: : On Fri, Mar 12, 2010 at 12:50:32PM -0700, M. Warner Losh wrote: : : On Thu, Mar 11, 2010 at 07:24:23PM -0700, M. Warner Losh wrote: : So the issue isn't as cut and dried as you might think. There's : multiple different conventions used here in addition to your simple : example. : : I guess we'd have to take a poll to find out. Seems pretty cut and dried : to me. COMPAT_FREEBSDn has an established context that does not match : this new usage. That is - same bit'ness, compatibility with an older : FreeBSD API for the same architecture. All the other COMPAT_* are for : foreign ABI compatibility. COMPAT_LINUX32 possibly should have been : COMPAT_LINUX_X86_64. (or is it MI and is usable as-is for PowerPC : and MIPS? I haven't looked that deeply at the code.) no, COMPAT_LINUX32 is the right name. While we don't have PowerPC or MIPS linux emulation bits in the kernel, the code if for dealing with running 32-bit binaries on 64-bit machines. There may be a little leakage of x86 specific goo here, but not a lot. Warner ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: HEADS UP: COMPAT_IA32 renamed COMPAT_FREEBSD32
P.S. I think that there's much traction to the idea of moving from COMPAT_FREEBSDx to some other variable called, for example, COMPAT_FREEBSD_BACK_TO=x, which will give compatibility for binaries as old as FreeBSD x.0, and have all the other magic handled behind the scenes. This would render the inconsistency with COMPAT_FREEBSDx part of the debate completely moot. Warner ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: HEADS UP: COMPAT_IA32 renamed COMPAT_FREEBSD32
[ Some CC's stripped ] On Mon, 22 Mar 2010, M. Warner Losh wrote: P.S. I think that there's much traction to the idea of moving from COMPAT_FREEBSDx to some other variable called, for example, COMPAT_FREEBSD_BACK_TO=x, which will give compatibility for binaries as old as FreeBSD x.0, and have all the other magic handled behind the scenes. This would render the inconsistency with COMPAT_FREEBSDx part of the debate completely moot. Doesn't matter. We're still use to COMPAT_FREEBSDx since it's been here so long. So regardless if you rename them to COMPAT_FREEBSD_BACK_TO=x, it is still potentially confusing. COMPAT_ARCH32 and all other choices David mentions seem like much better names - even if there wasn't any existing COMPAT_FREEBSDx knobs. My $0.02. -- DE ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
In message 3c0b01821003221207p4e4eecabqb4f448813bf5a...@mail.gmail.com, Alexa nder Sack writes: Am I going crazy or does this sound a lot like Sun/SVR's stream based network stack? That is a good and pertinent observation. I did investigate a number of optimizations to the g_up/g_down scheme I eventually adopted, but found none that gained anything justifying the complexity they brought. In some cases, the optimizations used more CPU cycles than the straight g_up/g_down path, but obviously, the circumstances are vastly different with CPUs having 10 times higher clock, multiple cores and SSD disks, so a fresh look at this tradeoff is in order. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [PATCH] rename COMPAT_FREEBSD32
On Mar 22, 2010, at 10:52 AM, David O'Brien wrote: On Mon, Mar 15, 2010 at 09:00:18AM -0700, Julian Elischer wrote: I certainly agree.. can it be changed please? I've waited a while to see what other opinions would be expressed on this topic. I believe there is sufficient support to rename COMPAT_FREEBSD32 to something else based on responses in the mailing lists. I am sorry if some may wish to label this a bikeshead. But we seem to have many folks disliking COMPAT_FREEBSD32. Based on responses to the topic of COMPAT_FREEBSD32, the following were the suggestions offered: COMPAT_ARCH32, COMPAT_ARCH_32BIT, COMPAT_32BIT_ARCH, COMPAT_32BIT, COMPAT_FREEBSD32BIT There's probably a bigger problem than just how we name it. The option really encodes 2 independent aspects: 1. Support for a 32-bit ABI (i.e. ILP32) 2. Support for a particular OS in ILP32. Of course 2 implies 1. For example: COMPAT_IA32 in sys/ia64/ia64/machdep.c enabled code to save and restore IA32 registers as part of cpu_switch(). In this context COMPAT_IA32 was perfectly named. It's now called COMPAT_FREEBSD32, which doesn't make a lot of sense because what if I only want to support Linux/ia32 and not FreeBSD/ia32 (or vice-versa if you club them under a single COMPAT_*32)? -- Marcel Moolenaar xcl...@mac.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
HEADSUP: zlib updated [svn commit: r205471 - in head: . lib/libz lib/libz/contrib lib/libz/doc sys/sys]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Just a heads-up that zlib in base system (libz) has been updated to 1.2.4. We tried to keep -HEAD as close as possible to the vendor version, but there is some changes in its internal data structure, and we did not use versioned symbols in the past, making shared library version bump unavoidable. A MFC of this update is planned, but we will have to make some rather aggressive changes against the library and more testing. Please make sure that you have at least libxml2-2.7.6_2 in your ports tree before even thinking about updating your ports tree. Older libxml2 uses some knowledge of zlib internals that has been changed in this update which is known to cause problem. - Original Message Subject: svn commit: r205471 - in head: . lib/libz lib/libz/contrib lib/libz/doc sys/sys Date: Mon, 22 Mar 2010 21:11:55 + (UTC) From: Xin LI delp...@freebsd.org To: src-committ...@freebsd.org, svn-src-...@freebsd.org, svn-src-h...@freebsd.org Author: delphij Date: Mon Mar 22 21:11:55 2010 New Revision: 205471 URL: http://svn.freebsd.org/changeset/base/205471 Log: Update to zlib 1.2.4 and add versioned symbols to the library. Sponsored by: iXsystems, Inc. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLp+C4AAoJEATO+BI/yjfBL8QH/12Mu9ihnH6JcySY7LiKukfl 448LMbxneP/JHwYuhFGhvbffd0N64HWzLKEF0Cp0HCH8ULuMIuoAjRsUjZSgF+et AaBXN0Su0Pw4kkcWIJ0Ruza5oDbVNVHB8XAHs/kIbWz0QS2v35FUCnFnvw1IVgIj 8NBe1Z+nR+dFWamc7ddFwxKnmSEIWzebykpvMrJoweAxzBKfcslNaCbZhfFe+j/D 1LrlRq97RM95SVUTnaFXPWr6zZCxFLhcFaGOOWxPV1g5TOgYaFO72fICXNgKgsWH TCwOzKV97vXfRx87yVWF/5CwOmlXiVAL5CmL1O3H6+PcOa2gWOZTZ48dorfol9w= =jrhi -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
On Mon, Mar 22, 2010 at 08:23:43AM +, Poul-Henning Kamp wrote: In message 4ba633a0.2090...@icyb.net.ua, Andriy Gapon writes: on 21/03/2010 16:05 Alexander Motin said the following: Ivan Voras wrote: Hmm, it looks like it could be easy to spawn more g_* threads (and, barring specific class behaviour, it has a fair chance of working out of the box) but the incoming queue will need to also be broken up for greater effect. According to notes, looks there is a good chance to obtain races, as some places expect only one up and one down thread. I haven't given any deep thought to this issue, but I remember us discussing them over beer :-) The easiest way to obtain more parallelism, is to divide the mesh into multiple independent meshes. This will do you no good if you have five disks in a RAID-5 config, but if you have two disks each mounted on its own filesystem, you can run a g_up g_down for each of them. A class is suppose to interact with other classes only via GEOM, so I think it should be safe to choose g_up/g_down threads for each class individually, for example: /dev/ad0s1a (DEV) | g_up_0 + g_down_0 | ad0s1a (BSD) | g_up_1 + g_down_1 | ad0s1 (MBR) | g_up_2 + g_down_2 | ad0 (DISK) We could easly calculate g_down thread based on bio_to-geom-class and g_up thread based on bio_from-geom-class, so we know I/O requests for our class are always coming from the same threads. If we could make the same assumption for geoms it would allow for even better distribution. -- Pawel Jakub Dawidek http://www.wheelsystems.com p...@freebsd.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgpFAxWFcI5ds.pgp Description: PGP signature
Re: Increasing MAXPHYS
On Mar 22, 2010, at 5:36 PM, Pawel Jakub Dawidek wrote: On Mon, Mar 22, 2010 at 08:23:43AM +, Poul-Henning Kamp wrote: In message 4ba633a0.2090...@icyb.net.ua, Andriy Gapon writes: on 21/03/2010 16:05 Alexander Motin said the following: Ivan Voras wrote: Hmm, it looks like it could be easy to spawn more g_* threads (and, barring specific class behaviour, it has a fair chance of working out of the box) but the incoming queue will need to also be broken up for greater effect. According to notes, looks there is a good chance to obtain races, as some places expect only one up and one down thread. I haven't given any deep thought to this issue, but I remember us discussing them over beer :-) The easiest way to obtain more parallelism, is to divide the mesh into multiple independent meshes. This will do you no good if you have five disks in a RAID-5 config, but if you have two disks each mounted on its own filesystem, you can run a g_up g_down for each of them. A class is suppose to interact with other classes only via GEOM, so I think it should be safe to choose g_up/g_down threads for each class individually, for example: /dev/ad0s1a (DEV) | g_up_0 + g_down_0 | ad0s1a (BSD) | g_up_1 + g_down_1 | ad0s1 (MBR) | g_up_2 + g_down_2 | ad0 (DISK) We could easly calculate g_down thread based on bio_to-geom-class and g_up thread based on bio_from-geom-class, so we know I/O requests for our class are always coming from the same threads. If we could make the same assumption for geoms it would allow for even better distribution. The whole point of the discussion, sans PHK's interlude, is to reduce the context switches and indirection, not to increase it. But if you can show decreased latency/higher-iops benefits of increasing it, more power to you. I would think that the results of DFly's experiment with parallelism-via-more-queues would serve as a good warning, though. Scott ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Increasing MAXPHYS
Pawel Jakub Dawidek wrote: On Mon, Mar 22, 2010 at 08:23:43AM +, Poul-Henning Kamp wrote: In message 4ba633a0.2090...@icyb.net.ua, Andriy Gapon writes: on 21/03/2010 16:05 Alexander Motin said the following: Ivan Voras wrote: Hmm, it looks like it could be easy to spawn more g_* threads (and, barring specific class behaviour, it has a fair chance of working out of the box) but the incoming queue will need to also be broken up for greater effect. According to notes, looks there is a good chance to obtain races, as some places expect only one up and one down thread. I haven't given any deep thought to this issue, but I remember us discussing them over beer :-) The easiest way to obtain more parallelism, is to divide the mesh into multiple independent meshes. This will do you no good if you have five disks in a RAID-5 config, but if you have two disks each mounted on its own filesystem, you can run a g_up g_down for each of them. A class is suppose to interact with other classes only via GEOM, so I think it should be safe to choose g_up/g_down threads for each class individually, for example: /dev/ad0s1a (DEV) | g_up_0 + g_down_0 | ad0s1a (BSD) | g_up_1 + g_down_1 | ad0s1 (MBR) | g_up_2 + g_down_2 | ad0 (DISK) We could easly calculate g_down thread based on bio_to-geom-class and g_up thread based on bio_from-geom-class, so we know I/O requests for our class are always coming from the same threads. If we could make the same assumption for geoms it would allow for even better distribution. doesn't really help my problem however.. I just want to access the base provider directly with no geom thread involved. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: build failures after stdlib update
On Sun, Mar 21, 2010 at 7:29 PM, jhell jh...@dataix.net wrote: Native is equal to CPUTYPE not being defined right ? Built into GCC is the ability to auto-detect the CPUTYPE when -mtune=native, if you run this command GCC will tell ouput your processor type: gcc -v -x c -E -mtune=native /dev/null -o /dev/null 21 | grep mtune | sed -e 's/.*mtune=//' So if the above is true why would you set CPUTYPE to native in the first place ? when you can just leave it unset and its equal to native. Native allows one to build the sources to the current build machines cputype. There is one bug when setting the CPUTYPE to native, the FreeBSD build system fails to correctly set the MACHINE_CPU to the correct value. I had discovered this problem back in May 2007: http://www.freebsd.org/cgi/query-pr.cgi?pr=112997 Just apply the last patch to share/mk/bsd.cpu.mk and that will fix the problem. Scot ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
MACHINE_CPU not being set correctly when CPUTYPE=native.
Could someone commit the last patch in PR 112997 to share/mk/bsd.cpu.mk, as it fixes the bug where MACHINE_CPU is not set correctly when CPUTYPE is set to 'native'. On Tue, Aug 28, 2007 at 1:54 AM, Scot Hetzel swhet...@gmail.com wrote: Gcc 4.2 has a new cpu_type (native) for x86 and amd64 systems. This cpu_type is to allow gcc to automatically detect the processor type that gcc is running on. The problem is that setting CPUTYPE?=native in either src.conf or make.conf will cause MACHINE_CPU to be set to the wrong value for the native cpu. For example on a system where the processor is a k8, setting CPUTYPE to k8, shows that MACHINE_CPU is set as follows: hp010# make -V MACHINE_CPU -DCPUTYPE=k8 k8 3dnow amd64 sse2 sse mmx But setting CPUTYPE to native on a k8 system sets MACHINE_CPU to this value: hp010# make -V MACHINE_CPU -DCPUTYPE=native unknown amd64 sse2 sse mmx After patching share/mk/bsd.cpu.mk (see attachment) or the last patch to PR 112997: http://www.freebsd.org/cgi/query-pr.cgi?pr=112997 setting CPUTYPE to native now works correctly when setting the value for MACHINE_CPU: hp010# make -V MACHINE_CPU -V CPUTYPE -DCPUTYPE=native k8 3dnow amd64 sse2 sse mmx k8 Could this get committed before -CURRENT is branched. Thanks, Scot ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
