Re: [RFC][CFT] GEOM direct dispatch and fine-grained CAM locking
Alexander Motin m...@freebsd.org wrote: I would like to invite more people to review and test my patches for improving CAM and GEOM scalability, that for last six months you could see developing in project/camlock SVN branch. Full diff of that branch against present head (r255131) can be found here: http://people.freebsd.org/~mav/camlock_patches/camlock_20130902.patch So far I haven't noticed any issues with this patch (or later on with camlock_20130906.patch) using glabel, ggatec, geli and ZFS on amd64. cdda2wav continued to work as expected as well. Fabian signature.asc Description: PGP signature
Re: Threaded 6.4 code compiled under 9.0 uses a lot more memory?..
Karl Pielorz kpielorz_...@tdx.co.uk wrote: Can anyone think of any quick pointers as to why some code originally written under 6.4 amd64 - when re-compiled under 9.0-stable amd64 takes up a *lot* more memory when running? 6.4 comes with phkmalloc while 9.0 uses jemalloc. Maybe you are allocating memory in a way that is less space-efficiently handled by jemalloc's default configuration. Fabian signature.asc Description: PGP signature
Re: FreeBSD in Google Code-In 2012? You can help too!
dweimer dwei...@dweimer.net wrote: On 2012-10-22 12:44, Wojciech A. Koszek wrote: Update: It looks pretty bad so far. Page: http://wiki.freebsd.org/GoogleCodeIn/2012Tasks Has 38 tasks so far out of which: ~30 would qualify. Consider this e-mail to be the last call for action. Otherwise we'll have to pull back and concentrate our efforts on GSOC instead. One thing I can think to add if it's not already been done and if its a practical idea for the Google project, would be to update the mount_udf command to support newer versions of UDF. It looks like as of FreeBSD 7 that 1.02 1.50 are supported, I( haven't been able to find any more recent documentation to support whether or not updates have been made since then. I only know that server I have running 9.0, for the purpose of hosting ISO images on the network so they are available to our ESX environment for mounting as a local CDROM/DVD within virtual machines, and also available as files over a network share, can't mount the some of the more recent DVDs in the UDF format, but I am unsure which format they are in, they could be 2.0, 2.5, or 2.6. There's work in progress to port the UDF code from NetBSD: http://wiki.freebsd.org/action/show/SummerOfCode2012/UDFImplementation Oleksandr posted a patch on freebsd-fs@ in July. Fabian signature.asc Description: PGP signature
Re: dtraceall.ko with old nfsclient
Sean Bruno sean...@yahoo-inc.com wrote:
On Thu, 2012-07-12 at 12:47 -0700, Andrew Boyer wrote:
On Jul 12, 2012, at 3:39 PM, Andriy Gapon wrote:
on 12/07/2012 22:36 Fabian Keil said the following:
Andriy Gapon a...@freebsd.org wrote:
on 12/07/2012 21:17 Fabian Keil said the following:
Benjamin Kaduk ka...@mit.edu wrote:
On Wed, 11 Jul 2012, Fabian Keil wrote:
I'm using the following modification of Sean's patch:
This way it seems to work as expected:
diff --git a/sys/modules/dtrace/dtraceall/Makefile
b/sys/modules/dtrace/dtraceall/Makefile index 456efd1..628583b 100644
--- a/sys/modules/dtrace/dtraceall/Makefile +++
b/sys/modules/dtrace/dtraceall/Makefile @@ -1,7 +1,7 @@ # $FreeBSD:
src/sys/modules/dtrace/dtraceall/Makefile,v 1.3 2011/04/09 09:07:31 uqs
Exp $
KMOD= dtraceall -SRCS= dtraceall.c opt_compat.h
+SRCS= dtraceall.c opt_compat.h opt_nfs.h
CFLAGS+= -I${.CURDIR}/../../..
If you do cd sys/modules/dtrace/dtraceall make [obj depend] all, does
it compile OK with the above change?
Depends on your expectations I guess. As neither NFS-related option gets
defined, no dependency on either NFS module is registered. The compiler
has
no complaints, though.
Interesting. Could you repeat after sufficient cleaning up?
I am not sure where from opt_nfs.h file could come.
Maybe related: check out sys/modules/ipfw/Makefile. It makes its own
option headers for INET and INET6.
I think guessing that INET and INET6 are available is a lot more
reasonable than doing the same for the external NFS modules.
I've pondered this on an off for a couple of weeks now. I can clearly
see that if we want compile time dependencies, that's fine, we can make
each nfclient object dependant on the #define. Both are valid cases to
have loaded though, so they shouldn't be conditional on each other as in
my patches.
If one does this however, the module makefile needs to be adjusted like
the ipfw makefile to create an opt_nfs.h with the NFS client defines,
else you will have neither dtrace object available.
There isn't a clear way that I can see to compile dtraceall.ko as a
module if you are doing so outside of the kernel compile though. The
module tree isn't really aware of what you are running, therefore it
would have to compile to load both regardless. Which, if your kernel
doesn't support one or both nfs objects, will cause a kldload failure.
In my opinion depending on neither nfs modules when the module
is build manually is preferable to depending on both.
For users without the nfs modules available, forcing the dependencies
renders the dtraceall module useless, while users who want to dtrace nfs
can still use dtraceall and simply load the nfs module manually afterwards
if the dependency is missing.
Fabian
signature.asc
Description: PGP signature
Re: dtraceall.ko with old nfsclient
Benjamin Kaduk ka...@mit.edu wrote:
On Wed, 11 Jul 2012, Fabian Keil wrote:
I'm using the following modification of Sean's patch:
This way it seems to work as expected:
diff --git a/sys/modules/dtrace/dtraceall/Makefile
b/sys/modules/dtrace/dtraceall/Makefile
index 456efd1..628583b 100644
--- a/sys/modules/dtrace/dtraceall/Makefile
+++ b/sys/modules/dtrace/dtraceall/Makefile
@@ -1,7 +1,7 @@
# $FreeBSD: src/sys/modules/dtrace/dtraceall/Makefile,v 1.3 2011/04/09
09:07:31 uqs Exp $
KMOD= dtraceall
-SRCS= dtraceall.c opt_compat.h
+SRCS= dtraceall.c opt_compat.h opt_nfs.h
CFLAGS+= -I${.CURDIR}/../../..
diff --git a/sys/modules/dtrace/dtraceall/dtraceall.c
b/sys/modules/dtrace/dtraceall/dtraceall.c
index d256489..0672556 100644
--- a/sys/modules/dtrace/dtraceall/dtraceall.c
+++ b/sys/modules/dtrace/dtraceall/dtraceall.c
@@ -33,6 +33,7 @@
#include sys/module.h
#include sys/errno.h
#include opt_compat.h
+#include opt_nfs.h
static int
dtraceall_modevent(module_t mod __unused, int type, void *data __unused)
@@ -67,8 +68,11 @@ MODULE_DEPEND(dtraceall, opensolaris, 1, 1, 1);
MODULE_DEPEND(dtraceall, dtrace, 1, 1, 1);
MODULE_DEPEND(dtraceall, dtio, 1, 1, 1);
MODULE_DEPEND(dtraceall, dtmalloc, 1, 1, 1);
+#if defined (NFSCL)
MODULE_DEPEND(dtraceall, dtnfscl, 1, 1, 1);
+#elif defined (NFSCLIENT)
MODULE_DEPEND(dtraceall, dtnfsclient, 1, 1, 1);
+#endif
#if defined(__amd64__) || defined(__i386__)
MODULE_DEPEND(dtraceall, fbt, 1, 1, 1);
MODULE_DEPEND(dtraceall, fasttrap, 1, 1, 1);
Note that dtnfscl.ko is not loaded even though loading
it manually works and I have NFSCL in the kernel.
I wasn't entirely clear here, what I meant was that
the KERNCONF used when compiling the module included
options NFSCL. I didn't expect run-time detection.
This is because dtraceall.c only #includes opt_compat.h, and the kernel
build system only passes -include opt_global.h, so the dtraceall module
build has no way of knowing about the NFSCL{IENT,} options defined in
opt_nfs.h. (As you noted earlier in the thread?)
Yes.
You would still need to address Andriy's comments in order to ensure that
the configuration seen by the module matches the kernel.
Sure.
Fabian
signature.asc
Description: PGP signature
Re: dtraceall.ko with old nfsclient
Andriy Gapon a...@freebsd.org wrote:
on 12/07/2012 21:17 Fabian Keil said the following:
Benjamin Kaduk ka...@mit.edu wrote:
On Wed, 11 Jul 2012, Fabian Keil wrote:
I'm using the following modification of Sean's patch:
This way it seems to work as expected:
diff --git a/sys/modules/dtrace/dtraceall/Makefile
b/sys/modules/dtrace/dtraceall/Makefile index 456efd1..628583b 100644 ---
a/sys/modules/dtrace/dtraceall/Makefile +++
b/sys/modules/dtrace/dtraceall/Makefile @@ -1,7 +1,7 @@ # $FreeBSD:
src/sys/modules/dtrace/dtraceall/Makefile,v 1.3 2011/04/09 09:07:31 uqs Exp
$
KMOD= dtraceall -SRCS= dtraceall.c opt_compat.h +SRCS=
dtraceall.c opt_compat.h opt_nfs.h
CFLAGS+= -I${.CURDIR}/../../..
If you do cd sys/modules/dtrace/dtraceall make [obj depend] all, does it
compile OK with the above change?
Depends on your expectations I guess. As neither NFS-related
option gets defined, no dependency on either NFS module is
registered. The compiler has no complaints, though.
Fabian
signature.asc
Description: PGP signature
Re: dtraceall.ko with old nfsclient
Andriy Gapon a...@freebsd.org wrote:
on 12/07/2012 22:36 Fabian Keil said the following:
Andriy Gapon a...@freebsd.org wrote:
on 12/07/2012 21:17 Fabian Keil said the following:
Benjamin Kaduk ka...@mit.edu wrote:
On Wed, 11 Jul 2012, Fabian Keil wrote:
I'm using the following modification of Sean's patch:
This way it seems to work as expected:
diff --git a/sys/modules/dtrace/dtraceall/Makefile
b/sys/modules/dtrace/dtraceall/Makefile index 456efd1..628583b 100644
--- a/sys/modules/dtrace/dtraceall/Makefile +++
b/sys/modules/dtrace/dtraceall/Makefile @@ -1,7 +1,7 @@ # $FreeBSD:
src/sys/modules/dtrace/dtraceall/Makefile,v 1.3 2011/04/09 09:07:31 uqs
Exp $
KMOD= dtraceall -SRCS= dtraceall.c opt_compat.h
+SRCS= dtraceall.c opt_compat.h opt_nfs.h
CFLAGS+= -I${.CURDIR}/../../..
If you do cd sys/modules/dtrace/dtraceall make [obj depend] all, does
it compile OK with the above change?
Depends on your expectations I guess. As neither NFS-related option gets
defined, no dependency on either NFS module is registered. The compiler has
no complaints, though.
Interesting. Could you repeat after sufficient cleaning up?
I am not sure where from opt_nfs.h file could come.
The Makefile seems to create an empty one:
fk@r500 /usr/src/sys/modules/dtrace/dtraceall $make clean
rm -f export_syms dtraceall.ko dtraceall.kld dtraceall.o dtraceall.ko.debug
dtraceall.ko.symbols opt_compat.h opt_nfs.h
fk@r500 /usr/src/sys/modules/dtrace/dtraceall $make
echo #define COMPAT_FREEBSD32 1 opt_compat.h
: opt_nfs.h
cc -O2 -pipe [...] -c /usr/src/sys/modules/dtrace/dtraceall/dtraceall.c
ld -d -warn-common -r -d -o dtraceall.ko.debug dtraceall.o
: export_syms
awk -f /usr/src/sys/modules/dtrace/dtraceall/../../../conf/kmod_syms.awk
dtraceall.ko.debug export_syms | xargs -J% objcopy % dtraceall.ko.debug
objcopy --only-keep-debug dtraceall.ko.debug dtraceall.ko.symbols
objcopy --strip-debug --add-gnu-debuglink=dtraceall.ko.symbols
dtraceall.ko.debug dtraceall.ko
Fabian
signature.asc
Description: PGP signature
Re: dtraceall.ko with old nfsclient
Andriy Gapon a...@freebsd.org wrote: on 10/07/2012 21:57 Fabian Keil said the following: I do not use a completely NFS-free kernel, but I don't build any NFS-related modules. Trying to load an unpatched dtraceall results in: Jul 9 21:58:48 r500 sudo: fk : TTY=pts/16 ; PWD=/home/fk ; USER=root ; COMMAND=/sbin/kldload dtraceall Jul 9 21:58:48 r500 kernel: [8922] KLD dtnfsclient.ko: depends on oldnfs - not available or version mismatch Jul 9 21:58:48 r500 kernel: [8922] linker_load_file: Unsupported file type Jul 9 21:58:48 r500 kernel: [8922] KLD dtraceall.ko: depends on dtnfsclient - not available or version mismatch Jul 9 21:58:48 r500 kernel: [8922] linker_load_file: Unsupported file type My assumption was that your patch and the #elif defined (NFSCLIENT) would fix that, and indeed it does, but I later on realized that I actually do have NFSCL in the kernel: fk@r500 /usr/src $sysctl kern.conftxt | grep NFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL My impression is that the patch is missing an opt_nfs.h inclusion combined with the Makefile voodoo to create the file. The dtraceall module already has an opt_compat.h, even though the Makefile logic to create it seems a bit dubious to me. It blindly assumes that COMPAT_FREEBSD32 is available on amd64. Not sure if I got correctly what the (perceived) problem actually is, but I have this to say: the proper way of building a module is either via buildkernel (MODULES_OVERRIDE, etc) or by using KERNBUILDDIR for independent module build if the module has any dependency on kernel options in effect. This could be a little bit sad (for third-party module providers), but this seems to be true. Sometimes Makefile-s fake kernel options for truly independent module build (no KERNBUILDDIR) to provide the widest and/or safest feature set. But this has to be done very carefully to ensure that module-kernel*s* compatibility actually occurs. I'm using the following modification of Sean's patch: diff --git a/sys/modules/dtrace/dtraceall/dtraceall.c b/sys/modules/dtrace/dtraceall/dtraceall.c index c57f590..d50b1e5 100644 --- a/sys/modules/dtrace/dtraceall/dtraceall.c +++ b/sys/modules/dtrace/dtraceall/dtraceall.c @@ -67,8 +67,11 @@ MODULE_DEPEND(dtraceall, opensolaris, 1, 1, 1); MODULE_DEPEND(dtraceall, dtrace, 1, 1, 1); MODULE_DEPEND(dtraceall, dtio, 1, 1, 1); MODULE_DEPEND(dtraceall, dtmalloc, 1, 1, 1); +#if defined (NFSCL) MODULE_DEPEND(dtraceall, dtnfscl, 1, 1, 1); +#elif defined (NFSCLIENT) MODULE_DEPEND(dtraceall, dtnfsclient, 1, 1, 1); +#endif #if defined(__amd64__) || defined(__i386__) MODULE_DEPEND(dtraceall, fbt, 1, 1, 1); MODULE_DEPEND(dtraceall, fasttrap, 1, 1, 1); The perceived problem is that after compiling dtraceall with make buildkernel, installing it with make installkernel and rebooting, loading it results in: fk@r500 ~ $kldstat Id Refs AddressSize Name 1 73 0x8020 e492c0 kernel 21 0x8104a000 226928 zfs.ko 3 14 0x81271000 82b8 opensolaris.ko 41 0x8127a000 23a48geom_eli.ko 52 0x8129e000 34380crypto.ko 71 0x812fe000 8640 acpi_video.ko 81 0x81307000 7d00 acpi_ibm.ko 94 0x81412000 29e31usb.ko 101 0x8143c000 32ec usb_quirk.ko 111 0x8144 bded ehci.ko 121 0x8144c000 8d02 umass.ko 131 0x81455000 5c2a nullfs.ko 141 0x8145b000 51ac fdescfs.ko 151 0x81461000 beb4 i915.ko 161 0x8146d000 173ccdrm.ko 171 0x81485000 b25 dtraceall.ko 181 0x81486000 4e00 profile.ko 193 0x8148b000 4073 cyclic.ko 20 10 0x8149 23b931 dtrace.ko 211 0x816cc000 125dasystrace_freebsd32.ko 221 0x816df000 13797systrace.ko 231 0x816f3000 44be sdt.ko 241 0x816f8000 484d lockstat.ko 251 0x816fd000 bce5 fasttrap.ko 261 0x81709000 6553 fbt.ko 271 0x8171 448b dtmalloc.ko 281 0x81715000 43d9 dtio.ko Note that dtnfscl.ko is not loaded even though loading it manually works and I have NFSCL in the kernel. Fabian signature.asc Description: PGP signature
Re: dtraceall.ko with old nfsclient
Andriy Gapon a...@freebsd.org wrote: on 09/07/2012 22:49 Sean Bruno said the following: Ran into some symbol errors with the dtraceall module when using the *old* nfs client. I think that this is more or less the right thing to do, but I'm not sure. --- //depot/yahoo/ybsd_9/src/sys/modules/dtrace/dtraceall/dtraceall.c 2011-11-02 23:46:55.0 +++ /home/seanbru/dtrace_9/src/sys/modules/dtrace/dtraceall/dtraceall.c 2011-11-02 23:46:55.0 @@ -66,8 +66,11 @@ MODULE_DEPEND(dtraceall, opensolaris, 1, 1, 1); MODULE_DEPEND(dtraceall, dtrace, 1, 1, 1); MODULE_DEPEND(dtraceall, dtmalloc, 1, 1, 1); +#if defined (NFSCL) MODULE_DEPEND(dtraceall, dtnfscl, 1, 1, 1); +#else /* defined (NFSCLIENT) */ Any objections to changing this to #elif defined (NFSCLIENT) ? MODULE_DEPEND(dtraceall, dtnfsclient, 1, 1, 1); +#endif #if defined(__amd64__) || defined(__i386__) MODULE_DEPEND(dtraceall, fbt, 1, 1, 1); MODULE_DEPEND(dtraceall, fasttrap, 1, 1, 1); Just to add some noise to the signal - my personal opinion is that nfs support doesn't have to be in dtraceall. Maybe in something all-er :-) I have no opinion on whether or not dtraceall should depend on nfs modules if they are available, but I would prefer it if the dependency was optional. I do not use any nfs modules and the hard-coded dependency made dtraceall useless for me in the past. Unlike Sean I worked around it with a shell function and was too lazy to investigate the cause, though. Fabian signature.asc Description: PGP signature
Re: dtraceall.ko with old nfsclient
Sean Bruno sean...@yahoo-inc.com wrote: On Tue, 2012-07-10 at 06:41 -0700, Fabian Keil wrote: --- //depot/yahoo/ybsd_9/src/sys/modules/dtrace/dtraceall/dtraceall.c 2011-11-02 23:46:55.0 +++ /home/seanbru/dtrace_9/src/sys/modules/dtrace/dtraceall/dtraceall.c 2011-11-02 23:46:55.0 @@ -66,8 +66,11 @@ MODULE_DEPEND(dtraceall, opensolaris, 1, 1, 1); MODULE_DEPEND(dtraceall, dtrace, 1, 1, 1); MODULE_DEPEND(dtraceall, dtmalloc, 1, 1, 1); +#if defined (NFSCL) MODULE_DEPEND(dtraceall, dtnfscl, 1, 1, 1); +#else /* defined (NFSCLIENT) */ Any objections to changing this to #elif defined (NFSCLIENT) ? No objections here. I suspect that this is the more correct thing regardless. I mean, it keeps the nfs dtrace objects loading in the event someone is running a non-nfs kernel... right? I do not use a completely NFS-free kernel, but I don't build any NFS-related modules. Trying to load an unpatched dtraceall results in: Jul 9 21:58:48 r500 sudo: fk : TTY=pts/16 ; PWD=/home/fk ; USER=root ; COMMAND=/sbin/kldload dtraceall Jul 9 21:58:48 r500 kernel: [8922] KLD dtnfsclient.ko: depends on oldnfs - not available or version mismatch Jul 9 21:58:48 r500 kernel: [8922] linker_load_file: Unsupported file type Jul 9 21:58:48 r500 kernel: [8922] KLD dtraceall.ko: depends on dtnfsclient - not available or version mismatch Jul 9 21:58:48 r500 kernel: [8922] linker_load_file: Unsupported file type My assumption was that your patch and the #elif defined (NFSCLIENT) would fix that, and indeed it does, but I later on realized that I actually do have NFSCL in the kernel: fk@r500 /usr/src $sysctl kern.conftxt | grep NFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL My impression is that the patch is missing an opt_nfs.h inclusion combined with the Makefile voodoo to create the file. The dtraceall module already has an opt_compat.h, even though the Makefile logic to create it seems a bit dubious to me. It blindly assumes that COMPAT_FREEBSD32 is available on amd64. Fabian signature.asc Description: PGP signature
Re: DTrace issue (infinite loop)
Chema García ch...@safetybits.net wrote: I've been trying to compile Python 2.7 with dtrace support from https://hg.jcea.es/cpython-2011/ (branch dtrace-issue13405_2.7 ) to test http://bugs.python.org/issue13405#msg164610. I've also recompiled the kernel to enable dtrace support by following http://www.freebsd.org/doc/en/books/handbook/dtrace-enable.html , but dtrace enters into an infinite loop when compiling, and using 100% of CPU usage. I'm using FreeBSD 9.0-RELEASE-p3, and used: ./configure --with-dtrace make And the issue occurs at: gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -o Include/pydtrace_offsets Include/pydtrace_offsets.c Include/pydtrace_offsets.sh OTHER Python/ceval.o Include/pydtrace_offsets Include/pydtrace_offsets.h if test dtrace != ; then dtrace -o Python/pydtrace.o -DPYDTRACE_STACK_HELPER -I. -IInclude -I./Include -32 -C -G -s Include/pydtrace.d Python/ceval.o Modules/gcmodule.o Objects/classobject.o Objects/typeobject.o; elsetouch Python/pydtrace.o ; fi; I've tried removing '-DPYDTRACE_STACK_HELPER' with same results. Does it make a difference if you remove the -O3 (and maybe -O2)? Fabian signature.asc Description: PGP signature
Re: decoding of multi-byte nops in dtrace
Andriy Gapon a...@freebsd.org wrote: It seems that the $subj is missing :-) In my environment that causes many functions to not have fbt return probe, because function body decoding fails before 'ret' is found. Here is my attempt at fixing the problem: http://people.freebsd.org/~avg/fbt-nop.patch Reviews and suggestions are welcome. The patch seems to reduce the number of missing fbt return probes by about 50% for me. Without the patch: fk@r500 /usr/src $sudo dtrace -ln fbt::: | grep -c entry 23395 fk@r500 /usr/src $sudo dtrace -ln fbt::: | grep -c return 16739 With the patch (and updated kernel sources): fk@r500 ~ $sudo dtrace -ln fbt::: | grep -c entry 23409 fk@r500 ~ $sudo dtrace -ln fbt::: | grep -c return 19879 Thanks Fabian signature.asc Description: PGP signature
Re: decoding of multi-byte nops in dtrace
Andriy Gapon a...@freebsd.org wrote: on 10/06/2012 16:27 Fabian Keil said the following: Andriy Gapon a...@freebsd.org wrote: It seems that the $subj is missing :-) In my environment that causes many functions to not have fbt return probe, because function body decoding fails before 'ret' is found. Here is my attempt at fixing the problem: http://people.freebsd.org/~avg/fbt-nop.patch Reviews and suggestions are welcome. The patch seems to reduce the number of missing fbt return probes by about 50% for me. Without the patch: fk@r500 /usr/src $sudo dtrace -ln fbt::: | grep -c entry 23395 fk@r500 /usr/src $sudo dtrace -ln fbt::: | grep -c return 16739 With the patch (and updated kernel sources): fk@r500 ~ $sudo dtrace -ln fbt::: | grep -c entry 23409 fk@r500 ~ $sudo dtrace -ln fbt::: | grep -c return 19879 Interesting observations, thank you. Do you use -O2 or higher optimization for kernel/modules build? Yes, I kept the default -O2. I use only -O1. With -O1 (and your patch) I get: fk@r500 ~ $sudo dtrace -ln fbt::: | grep -c entry 23421 fk@r500 ~ $sudo dtrace -ln fbt::: | grep -c return 22621 Here are some stats from my system: $ dtrace -ln fbt::: | fgrep -c entry 16876 $ dtrace -ln fbt::: | fgrep -c return 16729 So, 147 functions without return probe. From a quick look at them they all seem to really never return. Either they are noreturn type such panic, or functions that always call the functions of the first type, or functions with endless loops in them such as top level functions of many system threads. I looked at a couple of the functions that still lack return probes and the ones I looked at don't seem to belong into these categories. For example I get no return probes for g_eli_crypto_decrypt() and g_eli_crypto_encrypt(). Both return the return code of g_eli_crypto_cipher() for which I get a return probe. Fabian signature.asc Description: PGP signature
Re: decoding of multi-byte nops in dtrace
Andriy Gapon a...@freebsd.org wrote: on 10/06/2012 20:56 Fabian Keil said the following: Andriy Gapon a...@freebsd.org wrote: on 10/06/2012 16:27 Fabian Keil said the following: Andriy Gapon a...@freebsd.org wrote: It seems that the $subj is missing :-) In my environment that causes many functions to not have fbt return probe, because function body decoding fails before 'ret' is found. Here is my attempt at fixing the problem: http://people.freebsd.org/~avg/fbt-nop.patch Reviews and suggestions are welcome. The patch seems to reduce the number of missing fbt return probes by about 50% for me. Without the patch: fk@r500 /usr/src $sudo dtrace -ln fbt::: | grep -c entry 23395 fk@r500 /usr/src $sudo dtrace -ln fbt::: | grep -c return 16739 With the patch (and updated kernel sources): fk@r500 ~ $sudo dtrace -ln fbt::: | grep -c entry 23409 fk@r500 ~ $sudo dtrace -ln fbt::: | grep -c return 19879 Interesting observations, thank you. Do you use -O2 or higher optimization for kernel/modules build? Yes, I kept the default -O2. I use only -O1. With -O1 (and your patch) I get: fk@r500 ~ $sudo dtrace -ln fbt::: | grep -c entry 23421 fk@r500 ~ $sudo dtrace -ln fbt::: | grep -c return 22621 Here are some stats from my system: $ dtrace -ln fbt::: | fgrep -c entry 16876 $ dtrace -ln fbt::: | fgrep -c return 16729 So, 147 functions without return probe. From a quick look at them they all seem to really never return. Either they are noreturn type such panic, or functions that always call the functions of the first type, or functions with endless loops in them such as top level functions of many system threads. I looked at a couple of the functions that still lack return probes and the ones I looked at don't seem to belong into these categories. For example I get no return probes for g_eli_crypto_decrypt() and g_eli_crypto_encrypt(). Both return the return code of g_eli_crypto_cipher() for which I get a return probe. I don't have GELI in kernel, but it looks like an instance of well-known tail call optimization issue. Although I assumed that GCC wouldn't apply it at -O1. Perhaps you use a module that was built with -O2. That was it. I missed that COPTFLAGS aren't applied to the modules. After recompiling geom_eli manually with CFLAGS=-O1 the return codes show up as expected How did you set your -O1? Fabian signature.asc Description: PGP signature
Re: geli conversion
Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: is it possible to convert geli encrypted device using keyfile to password only? possibly i miss something in manual. thanks. The geli(8) action you are looking for is called setkey. Fabian signature.asc Description: PGP signature
Re: ULE/sched issues on stable/9 - why isn't preemption occuring?
Ryan Stone ryst...@gmail.com wrote: On Thu, May 31, 2012 at 8:33 AM, Andriy Gapon a...@freebsd.org wrote: In this vein it might make sense to enable KTR and KTR_SCHED in GENERIC. KTR_SCHED comes with a performance hit. Besides, with the DTrace sched provider that I committed this month (and MFC'ed yesterday) you can collect schedgraph data with a D script. Unfortunately DTrace's timestamp itself doesn't always work reliably on FreeBSD, and schedgraph.d depends on it. On my system DTrace's timestamp seems to work with kern.timecounter.hardware=HPET and dev.cpu.0.cx_lowest=C2 (or C1) but seems to tick much too slow with dev.cpu.0.cx_lowest=C3: http://lists.freebsd.org/pipermail/freebsd-current/2011-August/026710.html Fabian signature.asc Description: PGP signature
Re: [PATCH] Implementation of DTrace sched provider (with bonus schedgraph script)
Fabian Keil f...@fabiankeil.de wrote: Ryan Stone ryst...@gmail.com wrote: I've implemented the sched provider for FreeBSD. This provider provides probes that fire when various scheduling decisions are made. Thanks a lot. Works for me (so far) with HEAD on amd64. The following seems strange to me: runq add Xorg/Xorg tid 100463 on: cpu0 at: 3664816837528 attributes: duration: 126.85ms, prio: 104, linkedto: emacs/emacs tid 100463 running Xorg/Xorg tid 100463 on: cpu0 at: 3665070552032 attributes: duration: 49.00us, prio: 120 It seems to imply that the thread had to wait 126.85ms so it could run for 49.00us. While it waited for cpu time, both cpu cores spent a lot more than 49.00us being idle: running idle/idle: cpu0 tid 13 on: cpu0 at: 3665065545759 attributes: duration: 494.09us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665066533938 attributes: duration: 5.33us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665066544595 attributes: duration: 120.16us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665066797747 attributes: duration: 55.47us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665066908687 attributes: duration: 9.20us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665066933464 attributes: duration: 79.03us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665067091530 attributes: duration: 29.19us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665067560848 attributes: duration: 106.72us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665067774292 attributes: duration: 5.49us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665067785275 attributes: duration: 376.84us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665068538958 attributes: duration: 5.35us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665068585628 attributes: duration: 63.05us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665068711726 attributes: duration: 7.69us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665068727114 attributes: duration: 278.78us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665069297929 attributes: duration: 56.36us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665069410643 attributes: duration: 8.05us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665069435720 attributes: duration: 6.04us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665069447800 attributes: duration: 45.15us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665069538099 attributes: duration: 11.23us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665069560564 attributes: duration: 487.04us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665070534639 attributes: duration: 5.37us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665070549009 attributes: duration: 52.97us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665070654952 attributes: duration: 439.71us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665071545318 attributes: duration: 56.78us, prio: 255 running idle/idle: cpu0 tid 13 on: cpu0 at: 3665071672443 attributes: duration: 55.40us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665071783233 attributes: duration: 9.17us, prio: 255 idle idle/idle: cpu0 tid 13 on: cpu0 at: 3665071808741 attributes: duration: 83.01us, prio: 255 - running idle/idle: cpu1 tid 14 on: cpu1 at: 3665065783769 attributes: duration: 244.57us, prio: 255 running idle/idle: cpu1 tid 14 on: cpu1 at: 3665066283934 attributes: duration: 244.74us, prio: 255 running idle/idle: cpu1 tid 14 on: cpu1 at: 3665066790594 attributes: duration: 53.57us, prio: 255 running idle/idle: cpu1 tid 14 on: cpu1 at: 3665066914236 attributes: duration: 3.49us, prio: 255 idle idle/idle: cpu1 tid 14 on: cpu1 at: 3665066921213 attributes: duration: 22.79us, prio: 255 idle idle/idle: cpu1 tid 14 on: cpu1 at: 3665067019387 attributes: duration: 27.54us, prio: 255 idle idle/idle: cpu1 tid 14 on: cpu1 at: 3665067077581 attributes: duration: 820.16us, prio: 255 running idle/idle: cpu1 tid 14 on: cpu1 at: 3665068717906 attributes: duration: 27.57us, prio: 255 idle idle/idle: cpu1 tid 14 on: cpu1 at: 3665068773053 attributes: duration: 5.73us, prio: 255 running idle/idle: cpu1 tid 14 on: cpu1 at: 3665068784511 attributes: duration: 244.42us, prio: 255 running idle/idle: cpu1 tid 14 on: cpu1 at: 3665069290295 attributes: duration: 54.13us, prio: 255 idle idle/idle: cpu1 tid 14 on: cpu1 at: 3665069420973 attributes: duration: 10.37us, prio: 255 running idle/idle: cpu1 tid 14 on: cpu1 at: 3665069441715 attributes: duration: 40.54us, prio: 255 idle idle/idle: cpu1 tid 14 on: cpu1 at: 3665069522796 attributes: duration: 11.24us, prio: 255 running idle/idle: cpu1 tid 14 on: cpu1
Re: [PATCH] Implementation of DTrace sched provider (with bonus schedgraph script)
Ryan Stone ryst...@gmail.com wrote: I've implemented the sched provider for FreeBSD. This provider provides probes that fire when various scheduling decisions are made. Thanks a lot. Works for me (so far) with HEAD on amd64. Fabian signature.asc Description: PGP signature
Re: cdparanoia stuck in cbwait
Brandon Falk falk...@gamozo.org wrote: I was got some read errors when I was reading from my disk, which is fine. But then cdparanoia starting to no longer respond. Now it's zombied and sitting in cbwait. `kill -9 pid` does not take it down, it seems like it's stuck in the kernel (?). Is this an issue to look at cdparanoia for, or to look in the kernel? Perhaps cdparanoia or the kernel should have some sort of a timeout? Currently it looks like the only fix is a full reboot. I believe this is a kernel issue. Processes can become unresponsive and unkillable while waiting for blocks to be read and in case of read failures that can take quite some time. Note that the kernel itself has a somewhat high default retry count which multiplies the retries the firmware already does transparently, so a failed read operation can take minutes to complete while a successful one takes fractions of a second. And after the kernel eventually reports a read failure the application may smartly decide to retry a few times as well, again multiplying the time of the delay. Before ripping CDs and DVDs with invalid sectors I therefore set kern.cam.cd.retry_count to 0 and patched vobcopy to skip invalid-sector ranges faster. For some DVDs, this reduces the ripping time from several hours to about 20 minutes. How long did you wait for cdparanoia to die? If you only waited a couple of minutes, it's possible that rebooting may not have been necessary. Fabian signature.asc Description: PGP signature
Re: DTrace script asserts and kills the other process
Yuri y...@rawbw.com wrote:
I am trying to run this dtrace script:
#!/usr/sbin/dtrace -s
pid123:libc::entry
{
self-timestmp[probefunc] = timestmp;
}
pid123:libc::return
/self-timestmp[probefunc] != 0/
{
@function_duration[probefunc] = sum(timestmp -
self-timestmp[probefunc]); timestmp[probefunc] = 0;
}
which I got from here:
http://www.princeton.edu/~unix/Solaris/troubleshoot/dtrace.html
replacing 123 with the pid of some running process.
Result: dtrace utility asserts:
Assertion failed: (dpr != NULL), file
/usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c,
line 751.
Abort trap: 6
Also the target process is killed too:
Killed: 9
8.2-STABLE amd64
This is a known issue. You may be able to work around it by
letting dtrace start the traced process.
There's already a PR about it:
http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/158431
but the limitation isn't mentioned in the wiki:
http://wiki.freebsd.org/DTrace/userland
It's not clear to me if this has worked in the past or if it
works for other architectures (the reporter and I are both using
amd64, too).
Fabian
signature.asc
Description: PGP signature
Re: zfs + uma
Robert Watson rwat...@freebsd.org wrote:
On Fri, 17 Sep 2010, Andre Oppermann wrote:
Although keeping free items around improves performance, it does consume
memory too. And the fact that that memory is not freed on lowmem
condition
makes the situation worse.
Interesting. We may run into related issues with excessive mbuf (cluster)
caching in the per-cpu buckets as well.
Having a general solutions for that is appreciated. Maybe the size of the
free per-cpu buckets should be specified when setting up the UMA zone. Of
certain frequently re-used elements we may want to cache more, other less.
I've been keeping a vague eye out for this over the last few years, and
haven't spotted many problems in production machines I've inspected. You can
use the umastat tool in the tools tree to look at the distribution of memory
over buckets (etc) in UMA manually.
Doesn't build for me on amd64:
f...@r500 /usr/src/tools/tools/umastat $make
Warning: Object directory not changed from original /usr/src/tools/tools/umastat
cc -O2 -pipe -fno-omit-frame-pointer -std=gnu99 -fstack-protector
-Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter
-Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized
-Wno-pointer-sign -c umastat.c
cc1: warnings being treated as errors
umastat.c: In function 'uma_print_bucketlist':
umastat.c:234: warning: format '%llu' expects type 'long long unsigned int',
but argument 3 has type 'uint64_t'
umastat.c:234: warning: format '%llu' expects type 'long long unsigned int',
but argument 4 has type 'uint64_t'
umastat.c: In function 'uma_print_cache':
umastat.c:245: warning: format '%llu' expects type 'long long unsigned int',
but argument 3 has type 'u_int64_t'
umastat.c:246: warning: format '%llu' expects type 'long long unsigned int',
but argument 3 has type 'u_int64_t'
umastat.c: In function 'main':
umastat.c:416: warning: format '%llu' expects type 'long long unsigned int',
but argument 2 has type 'u_int64_t'
umastat.c:418: warning: format '%llu' expects type 'long long unsigned int',
but argument 2 has type 'u_int64_t'
umastat.c:420: warning: format '%llu' expects type 'long long unsigned int',
but argument 2 has type 'u_int64_t'
umastat.c:426: warning: dereferencing type-punned pointer will break
strict-aliasing rules
umastat.c:429: warning: dereferencing type-punned pointer will break
strict-aliasing rules
*** Error code 1
Stop in /usr/src/tools/tools/umastat.
The attached patch seems to work around the problem, I'm not sure if
the casts to void* are better than decreasing the WARN level, though ...
Fabian
From b84b5cf4f24b6886b5db9885f5bea707dcfb11e8 Mon Sep 17 00:00:00 2001
From: Fabian Keil f...@fabiankeil.de
Date: Sat, 18 Sep 2010 13:55:54 +0200
Subject: [PATCH] Work around umastat build failures on amd64.
---
tools/tools/umastat/umastat.c | 16
1 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/tools/tools/umastat/umastat.c b/tools/tools/umastat/umastat.c
index 3c9fe0e..639bf80 100644
--- a/tools/tools/umastat/umastat.c
+++ b/tools/tools/umastat/umastat.c
@@ -230,7 +230,7 @@ uma_print_bucketlist(kvm_t *kvm, struct bucketlist *bucketlist,
}
printf(\n);
- printf(%s}; // total cnt %llu, total entries %llu\n, spaces,
+ printf(%s}; // total cnt %ju, total entries %ju\n, spaces,
total_cnt, total_entries);
}
@@ -242,8 +242,8 @@ uma_print_cache(kvm_t *kvm, struct uma_cache *cache, const char *name,
int ret;
printf(%s%s[%d] = {\n, spaces, name, cpu);
- printf(%s uc_frees = %llu;\n, spaces, cache-uc_frees);
- printf(%s uc_allocs = %llu;\n, spaces, cache-uc_allocs);
+ printf(%s uc_frees = %ju;\n, spaces, cache-uc_frees);
+ printf(%s uc_allocs = %ju;\n, spaces, cache-uc_allocs);
if (cache-uc_freebucket != NULL) {
ret = kread(kvm, cache-uc_freebucket, ub, sizeof(ub), 0);
@@ -412,20 +412,20 @@ main(int argc, char *argv[])
}
printf( Zone {\n);
printf(uz_name = \%s\;\n, name);
- printf(uz_allocs = %llu;\n,
+ printf(uz_allocs = %ju;\n,
uzp_userspace-uz_allocs);
- printf(uz_frees = %llu;\n,
+ printf(uz_frees = %ju;\n,
uzp_userspace-uz_frees);
- printf(uz_fails = %llu;\n,
+ printf(uz_fails = %ju;\n,
uzp_userspace-uz_fails);
printf(uz_fills = %u;\n,
uzp_userspace-uz_fills);
printf(uz_count = %u;\n,
uzp_userspace-uz_count);
- uma_print_bucketlist(kvm, (struct bucketlist *)
+ uma_print_bucketlist(kvm, (void *)
uzp_userspace-uz_full_bucket, uz_full_bucket,
);
- uma_print_bucketlist(kvm, (struct bucketlist *)
+ uma_print_bucketlist(kvm, (void *)
uzp_userspace-uz_free_bucket, uz_free_bucket,
);
--
1.7.2.3
signature.asc
Description: PGP signature
Re: limits for run away Firefox ?
Rick C. Petty rick-freebsd2...@kiwi-computer.com wrote: On Mon, Jan 18, 2010 at 12:25:43PM -0800, Doug Barton wrote: On 01/18/10 11:29, Oliver Fromme wrote: Doug Barton wrote: On 01/17/10 17:07, Julian H. Stacey wrote: Hi hackers I'm tired of my X server occasionaly freezing, swap thrasing, firefox dumps: 4,346,937,344 ~/firefox-bin.core so as a temporary cludge I ran touch ~/firefox-bin.core ; chmod 000 ~/firefox-bin.core Sorry I don't have a solution to your actual problem, but a better way to deal with this is to do: ln -s /dev/null ~/firefox-bin.core I think not generating a core dump at all is better than writing 4 GB to /dev/null. A) The method I proposed is useful for other things too, and as you pointed out it can sometimes be difficult to track down all the ways a given thing is started. What about just adding the limit command to the /usr/local/bin/firefox script? That would guarantee any instantiation of firefox wouldn't dump core. Until the next update ... B) If we're going to be snarky, it would be far better if it didn't need to dump core in the first place. :) I don't think that Oliver was at all snarky. He was merely suggesting a solution which would prevent the core file from being generated at all; the OP was tired of the extra time spent and Oliver's suggestion would certainly reduce this time. The symlink seems to hackish to me, although I've had to use it often in other situations. And in some cases the culprit would unlink(2) it first, so I've had to chflags noschg it, which works better than chmod 000 (if the FS supports it). But I agree that it would be nice to prevent ffox from segfaulting; unfortunately this is one of those apps which segfaults a lot (for me at least). =) I get a lot less segfaults since I disabled ogg support (which never worked for me anyway): about:config - media.ogg.enabled = false Fabian signature.asc Description: PGP signature
Re: Zero-length allocation with posix_memalign()
Jason Evans jas...@freebsd.org wrote: Fabian Keil wrote: Rémi Denis-Courmont, one of the vlc developers, pointed out that passing a zero size to posix_memalign() should actually work, though: | In principle, while useless, there is no reason why allocating an empty | picture should not be possible. posix_memalign() does support zero-length | allocation anyway: | http://www.opengroup.org/onlinepubs/9699919799/functions/posix_memalign.html | | If the size of the space requested is 0, the behavior is | | implementation-defined; the value returned in memptr shall be either a | | null pointer or a unique pointer. Standards: So many to choose from. This behavior for posix_memalign was only defined as of the 2008 standard (see the Issue 7 notes for posix_memalign): https://www.opengroup.org/austin/interps/uploads/40/14543/AI-152.txt Such requirements are unfortunate, because they induce a performance penalty for every call, just so that programs can avoid proper handling of edge cases in the rare situations for which such edge cases are a real possibility. I will add the pessimization to posix_memalign once the 8.0 freeze is over. It will be quite some time before this behavior becomes ubiquitous, so in the meanwhile it's probably a good idea to modify vlc to avoid such allocation requests. Great, thanks. I agree and will forward the vlc patch to the maintainer of the FreeBSD port if getting it committed upstream fails. Fabian signature.asc Description: PGP signature
Zero-length allocation with posix_memalign()
I recently submitted a patch to the vlc developers that prevents
a crash on FreeBSD 8.0 by not calling posix_memalign() with a
size argument of zero.
A simplified test case would be:
#include stdlib.h
int main(int argc, char **argv) {
void *ptr;
posix_memalign(ptr, 16, 0);
return (0);
}
which triggers:
Assertion failed: (size != 0), function arena_malloc, file
/usr/src/lib/libc/stdlib/malloc.c, line 3349.
Rémi Denis-Courmont, one of the vlc developers, pointed out
that passing a zero size to posix_memalign() should actually
work, though:
| In principle, while useless, there is no reason why allocating an empty
| picture should not be possible. posix_memalign() does support zero-length
| allocation anyway:
| http://www.opengroup.org/onlinepubs/9699919799/functions/posix_memalign.html
| | If the size of the space requested is 0, the behavior is
| | implementation-defined; the value returned in memptr shall be either a
| | null pointer or a unique pointer.
http://mailman.videolan.org/pipermail/vlc-devel/2009-July/062299.html
I get the impression that this deviation from the standard could be
easily fixed with something similar to the following, which is mostly
copy and pasted from malloc():
index 5404798..a078d07 100644
--- a/malloc.c
+++ b/malloc.c
@@ -5303,6 +5303,15 @@ posix_memalign(void **memptr, size_t alignment, size_t
size)
int ret;
void *result;
+ if (size == 0) {
+ if (opt_sysv == false)
+ size = 1;
+ else {
+ ret = 0;
+ *memptr = result = NULL;
+ goto RETURN;
+ }
+ }
if (malloc_init())
result = NULL;
else {
I assume the goto RETURN isn't entirely compliant either as
it skips the alignment check, but so does the malloc_init()
failure branch.
Fabian
signature.asc
Description: PGP signature
Re: problems obtaining kernel dump
Roman Bogorodskiy [EMAIL PROTECTED] wrote: I'm having a problem obtaining kernel dump. The box has 512Mb of RAM. In rc.conf I have: dumpdev=/dev/ad4s1b dumpdir=/var/crash swapinfo -h gives the following: Device 1K-blocks UsedAvail Capacity /dev/ad4s1b 1048576 0B 1.0G 0% /var/crash directory exists, and the root partition where it is placed has enough space as well: /dev/ad4s1a989M350M560M38%/ The box runs fresh FreeBSD/i386 -CURRENT. So, I do swapoff and then perform actions to reproduce the crash and it breaks me into ddb prompt. I do: call doadump continue it reboots, and when I run savecore -v /var/crash /dev/ad4s1b it prints: unable to open bounds file, using 0 checking for kernel dump on device /dev/ad4s1b mediasize = 1073741824 sectorsize = 512 magic mismatch on last dump header on /dev/ad4s1b savecore: no dumps found The same happens when I do 'panic' instead of 'call doadump' like handbook suggests. What am I doing wrong? I googled for similar problems, found some mail threads but didn't find meaningful advises though. I'm seeing a similar problem with an amd64 system on a SATA disk running CURRENT from 20080727 (can't update as I'm using Pawel's ZFS patch and upgraded most of the zpools). Calling panic leads to an instant reboot, nothing seems to be dumped. Fabian signature.asc Description: PGP signature
Re: a question regarding sys/shm.h
M. Warner Losh [EMAIL PROTECTED] wrote: In message: [EMAIL PROTECTED] Fabian Keil [EMAIL PROTECTED] writes: : Robert Watson [EMAIL PROTECTED] wrote: : If we do decide to go ahead with the ABI change, there are a number of other : things that should be done simultaneously, such as changing the uid and gid : fields to uid_t and gid_t. : : struct tm's members could be changed as well. struct tm doesn't have a member that's tv_sec. Maybe you mean struct timeval and struct timespec? I did indeed. Sorry for the noise. Fabian signature.asc Description: PGP signature
Re: a question regarding sys/shm.h
Robert Watson [EMAIL PROTECTED] wrote: On Wed, 31 Jan 2007, Dag-Erling Smørgrav wrote: Pascal Hofstee [EMAIL PROTECTED] writes: Any additional sugestions/objections are always greatly appreciated. On 32-bit platforms (i386, powerpc), int is a 32-bit signed integer while size_t is a 32-bit unsigned integer. On 64-bit platforms (amd64, sparc64 etc), int is a 32-bit signed integer while size_t is a 64-bit unsigned integer. In both cases, changing this structure member from int to size_t will break the ABI. This doesn't mean you shouldn't do it, just that it should be done with care. If we do decide to go ahead with the ABI change, there are a number of other things that should be done simultaneously, such as changing the uid and gid fields to uid_t and gid_t. struct tm's members could be changed as well. Quoting a response I got from Juliusz Chroboczek (the author of Polipo) after reporting a compiler warning: |By the way, IEEE 1003.1-2003 says that tv_sec should be a time_t, |hence by making it a long, FreeBSD and NetBSD are violating the POSIX |standard. Could you please file a bug report with them? I didn't find any claims about FreeBSD being IEEE 1003.1-2003 compliant and therefore didn't consider it a bug, but given that the topic is *_t changes and time_t hasn't come up yet, I'd like to mention it anyway. Fabian signature.asc Description: PGP signature
Re: jails, cron and sendmail
Mike Meyer [EMAIL PROTECTED] wrote: In [EMAIL PROTECTED], Dirk Engling [EMAIL PROTECTED] typed: The default configuration doesn't expose sendmail to the publicly visible IP addres. The daemon it runs only listens for connections to the localhost address. Which is rewritten to the jails (externally visible) address on a connect() Yup. I wasn't aware of that strange behavior of jails. That should be fixed. Fixed how? Disallow jailed applications to connect to 127.0.0.1, and thus break most of them, or have them reach 127.0.0.1 on the host system and weaken the security? I think the strange behaviour makes sense and it certainly makes jailing servers easier. Because of the security aspect it's a good idea to have the jail run on a private IP address that's only reachable through packet filter and port forwarding anyway. Don't forward the ports you don't need and the problem is solved. I think the better fix would be to make jails not expose their localhost IP address to the outside world. Exactly. Fabian -- http://www.fabiankeil.de/ signature.asc Description: PGP signature
Re: jails, cron and sendmail
Mike Meyer [EMAIL PROTECTED] wrote: In [EMAIL PROTECTED], Fabian Keil [EMAIL PROTECTED] typed: Mike Meyer [EMAIL PROTECTED] wrote: In [EMAIL PROTECTED], Dirk Engling [EMAIL PROTECTED] typed: The default configuration doesn't expose sendmail to the publicly visible IP addres. The daemon it runs only listens for connections to the localhost address. Which is rewritten to the jails (externally visible) address on a connect() Yup. I wasn't aware of that strange behavior of jails. That should be fixed. Fixed how? Disallow jailed applications to connect to 127.0.0.1, and thus break most of them, or have them reach 127.0.0.1 on the host system and weaken the security? I think the better fix would be to make jails not expose their localhost IP address to the outside world. Exactly. I think I misunderstood what you where saying here, sorry. I assumed you meant the user should run the jail on one of the addresses in the 127.0.0.0/8 range, while you probably were suggesting jails should have their own localhost IP address that is different from their outside IP address? Ok, I'm confused. Exactly how is fixing jails to not expose their localhost IP address to the outside world not fixing this strange behavior of jails? AFAICS jails currently have no localhost IP address they could expose. They have one IP address that is always visible from the host system, and conveniently jailed applications that try to bind to 127.0.0.1 get connected to the one jail IP address, instead of receiving an error or getting through to the host system's localhost. Fabian -- http://www.fabiankeil.de/ signature.asc Description: PGP signature
Re: truncate tool - must be root?
Eric Anderson [EMAIL PROTECTED] wrote: Is it expected that truncate(8) must be used by a superuser? If so, then the man page should probably mention it. If not, then it's broken :) What exactly is truncate(8)? On FreeBSD 6.1-STABLE I only have truncate(1) and it doesn't show any problems. Fabian -- http://www.fabiankeil.de/ signature.asc Description: PGP signature
Re: [ANN] unionfs patchset-11 release
Daichi GOTO [EMAIL PROTECTED] wrote: Fabian Keil wrote: Looks like the attachment was filtered. I got a different panic on FreeBSD TP51.local 6.1-RC FreeBSD 6.1-RC #22: Wed Apr 26 13:25:57 CEST 2006 after mounting an empty directory above /usr/src, applying a patch and using find's -type f option shortly afterwards to show the files in the directory on top: M... We tried as follow, but we could not get the same error :( It looks very fine. # cd /usr/ # mkdir hoge # mount_unionfs -c transparent -o noatime /usr/hoge /usr/src # cd src # find . -type f ; find . -type f ; find . -type f # cd /usr/ # umount /usr/src # rm -r hoge # mkdir hoge # mount_unionfs -c transparent /usr/hoge /usr/src # cd src # find . -type f ; find . -type f ; find . -type f What do you make of it? I didn't give you enough information, sorry. What I'm doing is: [EMAIL PROTECTED] ~ $mkdir /tmp/unionfs-src/ [EMAIL PROTECTED] ~ $mount_unionfs /tmp/unionfs-src /usr/src [EMAIL PROTECTED] ~ $cd /usr/src [EMAIL PROTECTED] /usr/src $patch -p1 ~/test/combi.patch [EMAIL PROTECTED] /usr/src $find /tmp/unionfs-src/ -type f [Panic] ~/test/combi.patch changes about twenty files. I'm not sure if it's important, but /usr was mounted readonly and /tmp is a different file system. My kernel has options WITNESS enabled. The last step doesn't seem to be the only way to get a problem, I tried the first four steps again and umount /usr/src resulted in a reboot. I was running Xorg and didn't get a panic, but I'll try again without Xorg, to see if it's the same problem. Fabian -- http://www.fabiankeil.de/ signature.asc Description: PGP signature
Re: [ANN] unionfs patchset-11 release
Fabian Keil [EMAIL PROTECTED] wrote:
Daichi GOTO [EMAIL PROTECTED] wrote:
Fabian Keil wrote:
I got a different panic on
FreeBSD TP51.local 6.1-RC FreeBSD 6.1-RC #22: Wed Apr 26 13:25:57 CEST
2006
after mounting an empty directory above /usr/src,
applying a patch and using find's -type f option shortly afterwards
to show the files in the directory on top:
We tried as follow, but we could not get the same error :(
It looks very fine.
I didn't give you enough information, sorry.
What I'm doing is:
[EMAIL PROTECTED] ~ $mkdir /tmp/unionfs-src/
[EMAIL PROTECTED] ~ $mount_unionfs /tmp/unionfs-src /usr/src
[EMAIL PROTECTED] ~ $cd /usr/src
[EMAIL PROTECTED] /usr/src $patch -p1 ~/test/combi.patch
[EMAIL PROTECTED] /usr/src $find /tmp/unionfs-src/ -type f
[Panic]
~/test/combi.patch changes about twenty files.
I'm not sure if it's important, but /usr was mounted
readonly and /tmp is a different file system.
My kernel has options WITNESS enabled.
The last step doesn't seem to be the only way to
get a problem, I tried the first four steps again
and umount /usr/src resulted in a reboot.
I was running Xorg and didn't get a panic, but I'll
try again without Xorg, to see if it's the same problem.
OK, I guess it's the same problem:
panic: initiate_write_filepage: dir inum 0 != new 8482
KDB: enter: panic
Locked vnodes
0xc359ddd0: tag syncer, type VNON
usecount 1, writecount 0, refcount 2 mountedhere 0
flags ()
lock type syncer: EXCL (count 1) by thread 0xc334ec00 (pid 41)
0xc393a880: tag ufs, type VDIR
usecount 1, writecount 0, refcount 3 mountedhere 0
flags ()
v_object 0xc3746e70 ref 0 pages 0
lock type ufs: EXCL (count 1) by thread 0xc334ec00 (pid 41)
ino 8469, on dev ad0s3e
exclusive sleep mutex Softdep Lock r = 0 (0xc07839c0) locked @
/usr/src/sys/ufs/ffs/ffs_softdep.c:3730
exclusive sleep mutex Giant r = 0 (0xc072f640) locked @
/usr/src/sys/kern/vfs_subr.c:1608
Locked vnodes
0xc359ddd0: tag syncer, type VNON
usecount 1, writecount 0, refcount 2 mountedhere 0
flags ()
lock type syncer: EXCL (count 1) by thread 0xc334ec00 (pid 41)
0xc393a880: tag ufs, type VDIR
usecount 1, writecount 0, refcount 3 mountedhere 0
flags ()
v_object 0xc3746e70 ref 0 pages 0
lock type ufs: EXCL (count 1) by thread 0xc334ec00 (pid 41)
ino 8469, on dev ad0s3e
panic: from debugger
Uptime: 2m14s
Dumping 511 MB (2 chunks)
chunk 0: 1MB (159 pages) ... ok
chunk 1: 511MB (130656 pages) 495 479 463 447 431 415 399 383 367 351 335 319
303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15
#0 doadump () at pcpu.h:165
165 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) where
#0 doadump () at pcpu.h:165
#1 0xc054a865 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:402
#2 0xc054ab27 in panic (fmt=0xc06c9423 from debugger) at
/usr/src/sys/kern/kern_shutdown.c:558
#3 0xc047f523 in db_panic (addr=-1068086451, have_addr=0, count=-1,
modif=0xd564b8d8 ) at /usr/src/sys/ddb/db_command.c:438
#4 0xc047f49c in db_command (last_cmdp=0xc072b3e4, cmd_table=0x0,
aux_cmd_tablep=0xc06f28c4, aux_cmd_tablep_end=0xc06f28c8)
at /usr/src/sys/ddb/db_command.c:350
#5 0xc047f58d in db_command_loop () at /usr/src/sys/ddb/db_command.c:458
#6 0xc048143d in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_main.c:221
#7 0xc0564dd7 in kdb_trap (type=0, code=0, tf=0xd564ba24) at
/usr/src/sys/kern/subr_kdb.c:473
#8 0xc069e4d2 in trap (frame=
{tf_fs = -1066532856, tf_es = 40, tf_ds = -714866648, tf_edi = 1, tf_esi
= -1066511478, tf_ebp = -714818964, tf_isp = -714818992, tf_ebx = -714818908,
tf_edx = 0, tf_ecx = -1056878592, tf_eax = 18, tf_trapno = 3, tf_err = 0,
tf_eip = -1068086451, tf_cs = 32, tf_eflags = 642, tf_esp = -1066558175, tf_ss
= -1066564882}) at /usr/src/sys/i386/i386/trap.c:593
#9 0xc068feda in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#10 0xc0564b4d in kdb_enter (msg=0x12 Address 0x12 out of bounds) at
cpufunc.h:60
#11 0xc054aabf in panic (fmt=0xc06e538a %s: dir inum %d != new %d) at
/usr/src/sys/kern/kern_shutdown.c:542
#12 0xc0634021 in initiate_write_filepage (pagedep=0xc354bc00, bp=0xcd838268)
at /usr/src/sys/ufs/ffs/ffs_softdep.c:3834
#13 0xc0633d1c in softdep_disk_io_initiation (bp=0xcd838268) at
/usr/src/sys/ufs/ffs/ffs_softdep.c:3740
#14 0xc063c8c4 in ffs_geom_strategy (bo=0xc358aa50, bp=0xcd838268) at buf.h:422
#15 0xc0648db7 in ufs_strategy (ap=0x12) at
/usr/src/sys/ufs/ufs/ufs_vnops.c:1942
#16 0xc06b47c8 in VOP_STRATEGY_APV (vop=0xc0719380, a=0xd564bb68) at
vnode_if.c:1796
#17 0xc059965c in bufstrategy (bo=0xc393a940, bp=0x12) at vnode_if.h:928
#18 0xc05946e6 in bufwrite (bp=0xcd838268) at buf.h:415
#19 0xc0594c31 in bawrite (bp=0x12) at buf.h:399
#20 0xc063cb82 in ffs_syncvnode (vp=0xc393a880, waitfor=3) at
/usr/src/sys/ufs/ffs/ffs_vnops.c:256
#21 0xc063b753 in ffs_sync (mp=0xc352b400, waitfor=3, td
Re: [ANN] unionfs patchset-11 release
Fabian Keil [EMAIL PROTECTED] wrote: Fabian Keil [EMAIL PROTECTED] wrote: Daichi GOTO [EMAIL PROTECTED] wrote: Fabian Keil wrote: What I'm doing is: [EMAIL PROTECTED] ~ $mkdir /tmp/unionfs-src/ [EMAIL PROTECTED] ~ $mount_unionfs /tmp/unionfs-src /usr/src [EMAIL PROTECTED] ~ $cd /usr/src [EMAIL PROTECTED] /usr/src $patch -p1 ~/test/combi.patch [EMAIL PROTECTED] /usr/src $find /tmp/unionfs-src/ -type f [Panic] ~/test/combi.patch changes about twenty files. Not exactly: [EMAIL PROTECTED] ~ $grep +++ ~/test/combi.patch | wc -l 38 I'm not sure if it's important, but /usr was mounted readonly and /tmp is a different file system. I got the same panic with /usr mounted rewritable and both directories on the same file system this time. Running patch -p1 ~/test/combi.patch seems to be enough to trigger the panic. I tried three runs with a smaller patch (unionfs6-p11.diff) without panic, I then took the bigger patch again and a few seconds later the system panicked. Another thing which could be significant or not: After my last mail I closed Xorg and tried to reproduce the panic two times, but couldn't. After a reboot the panic occurred right after the first attempt. I'm not sure anymore if I used the same patch the first times. Fabian -- http://www.fabiankeil.de/ signature.asc Description: PGP signature
