Re: How much memory do I need for buildworld?
Nov 22 16:55:13 mercury kernel: swap_pager: out of swap space Nov 22 16:55:13 mercury kernel: swap_pager_getswapspace(16): failed Nov 22 16:55:13 mercury kernel: pid 22841 (tblgen), uid 0, was killed: out of swap space This machine has 256MB of RAM and one 64MB swap partition. This is most likely the problem: you need more RAM for this particular instance of tblgen. On my -CURRENT i386 box, it takes ~369MiB of RSS to build the X86 disassembler tables. I'm surprised you didn't run into OOM problems earlier, with so little memory. For such router like machines, it is obviously easier to do the build on a fast desktop machine, then install over NFS, or rsync /usr/src and /usr/obj to the target machine. I suggest you have a look at NanoBSD in /usr/src/tools/tools/nanobsd.sh. It will build a disk image for you in roughly 1 hour on a fast machine with 4+ processors by doing a complete world. It allows you to regenerate the image and update a running image easily. If you keep your changes separated in /cfg you will find that you can actually work really quickly in this setup (even though not being able to install packages on the image directly is sometimes a bit of a nuisance). We've wrapped that script with a lot of our own stuff and go from initial config to running in VM in less than 5 minutes, 2.5 minutes for an update after that (using an SSD on the host and FreeBSD in a single processor VM generating the image). Nick signature.asc Description: Message signed with OpenPGP using GPGMail
Re: WITNESS observes 2 LORs on Boot of Release 10.1
On 11/22/2014 03:51 PM, Benjamin Kaduk wrote: On Fri, 21 Nov 2014, Ellis H. Wilson III wrote: Before I start, and this is mainly geared to my responder Benjamin Kaduk, based on your response, are you suggesting that the cnputc WITNESS panic you expected to happen is now completely unavoidable in FreeBSD 10? I.E., is this a spinlock that WITNESS falls over each time but that is provably deadlock free that the developers have decided cannot be BLESSED for some reason? https://lists.freebsd.org/pipermail/freebsd-current/2012-January/031316.html looks to be a better explanation than the previous link I sent ... in short, console output is hard. I guess I just can't wrap my head around why we would ever move to a regime where SKIPSPIN is the default for testing... That just seems like an open invitation for introducing spinlock regressions. I don't think anyone made the conscious decision to do that, it just happened by default as no one spent the time to fix the aforementioned issue. Moving onto the LORs I'm seeing, a question I have as a newbie to WITNESS debugging is how exactly to interpret the output if I see a stacktrace and then a LOR output like the following: lock order reversal: 1st 0x81633d88 entropy harvest mutex (entropy harvest mutex) @ /usr/src/sys/dev/random/random_harvestq.c:198 2nd 0x813b6208 scrlock (scrlock) @ /usr/src/sys/dev/syscons/syscons.c:2682 Does this mean WITNESS has already stored an ordering of #1 harvest_mtx then #2 scp-scr_lock, and somewhere somebody tried to lock scp-scr_lock without first getting harvest_mtx? Or the reverse (WITNESS previously recorded scrlock and then harvest and the lines it spit out were the offenders?) I believe it is the latter (the ordering being printed is the bad one which caused WITNESS to complain). Thanks so much for the additional info Ben. This fleshes out the history of this issue for me significantly. I have filed a bug on this at: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195262 Xin Li was able to identify the ordering that caused the problem and proposed a possible patch to fix it. I can confirm that now I'm booting with solely WITNESS (i.e., not WITNESS_SKIPSPIN) without panic. Thanks! ellis ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
zfs locking vs vnode locking [Was: zfs/vfs lockups, via poudriere]
On 23/11/2014 18:57, Sean Bruno wrote: 31071 100995 rm -mi_switch+0xe1 sleepq_wait+0x3a sleeplk+0x18d __lockmgr_args+0x9ab vop_stdlock+0x3c VOP_LOCK1_APV+0xab _vn_lock+0x43 zfs_lookup+0x45d zfs_freebsd_lookup+0x6d VOP_CACHEDLOOKUP_APV+0xa1 vfs_cache_lookup+0xd6 VOP_LOOKUP_APV+0xa1 lookup+0x5a1 namei+0x534 kern_rmdirat+0x8d amd64_syscall+0x3fb Xfast_syscall+0xfb 31075 100693 mv -mi_switch+0xe1 sleepq_wait+0x3a sleeplk+0x18d __lockmgr_args+0xd5d vop_stdlock+0x3c VOP_LOCK1_APV+0xab _vn_lock+0x43 vputx+0x28a zfs_rename_unlock+0x3e zfs_freebsd_rename+0xe39 VOP_RENAME_APV+0xab kern_renameat+0x4a6 amd64_syscall+0x3fb Xfast_syscall+0xfb Just the stack traces are not sufficient to analyze the problem without examining the relevant vnodes and vnode locks. But I believe that I have seen reports about this kind of problem before. And I think that I understand what's going on. And, as a part of my job, I tried to develop a fix [*] for this problem and had some positive feedback for it. But the fix is not just a few lines of changes. It's a lot of modifications to a lot of files. Besides, my changes alter quite a bit more code than a bare minimum required to fix the problem, which still would be quite a bit of changes. So, right now I would like to describe the problem as I understand it. Some general information about the FreeBSD VFS and its difference from Solaris VFS [**] can be useful, but is not really required. I'll try to explain by example. If we look at any mature and native FreeBSD filesystem with read-write support - ffs, maybe tmpfs - then we can make the following observations. In most of the vnode operation implementations there are no calls to vnode locking functions. E.g. for an operation like vop_remove two vnodes in question are already locked at the VFS layer. In some cases VOPs do locking, but it is very trivial e.g. in vop_create a newly created vnode must be returned locked. Naturally, if we look at the VFS code we see a lot of vnode locking for various purposes. Like locking the vnodes for vop_remove call. Or locking vnodes during their life cycle management, so that, for example, a vnode is not destroyed while there is an ongoing operation on it. Also, we can see locking in VFS namei / lookup implementation where we need to hold onto a directory vnode while looking up a child vnode by name. But there are two vnode operation implementations where we can see a non-trivial vnode locking dance. Those are vop_rename and vop_lookup. Anyone is welcome to take a cursory look at the first hundred or so lines of ufs_rename(). The point of the above observations is that both VFS and a filesystem driver do vnode locking. And, thus, both VFS and the driver must cooperate by using the same locking protocol. Now, if we look at the ZFS ZPL code and most prominently at zfs_rename() we see that there is quite a bit of locking going on there, e.g. see zfs_rename_lock, but the locks in question are all ZFS internal locks. We do not see the vnode locks. From this comes a suspicion, or even a conclusion, that ZFS currently does not use the same vnode locking protocol that is expected from any filesystem driver. There is a weird form of redundancy between the fine grained ZFS locks that got ported over and the FreeBSD vnode locks. In some cases the ZFS locks are always uncontested because the vnode locks held at the VFS level ensure a serialized access. In other cases there is no protection at all, because one thread is in VFS code which uses the vnode locks and another thread is in ZFS code which uses the ZFS locks and thus there is no real synchronization between those threads. My solution to this problem was to completely eliminate (at least) the following ZFS locks kmutex_tz_lock; /* znode modification lock */ krwlock_t z_parent_lock; /* parent lock for directories */ krwlock_t z_name_lock;/* master lock for dirent locks */ zfs_dirlock_t *z_dirlocks;/* directory entry lock list */ and to ensure that the proper vnode locking protocol is followed. That required substantial changes to the rename and lookup code. Finally, this is not really a suggestion to test or discuss my changes, but rather a call to discuss the problem and other possible ways to fix it. I do not preclude any options including making changes to our VFS (and thus ti all the filesystems) :-) [*] https://github.com/avg-I/freebsd/compare/wip/hc/zfs-fbsd-vfs [**] https://clusterhq.com/blog/complexity-freebsd-vfs-using-zfs-example-part-1-2/ https://clusterhq.com/blog/complexity-freebsd-vfs-using-zfs-example-part-2/ -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to
Build failed in Jenkins: FreeBSD_HEAD #1909
See https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/1909/changes
Changes:
[bapt] Do not overwrite any LIBNETBSD or LIBNETBSDDIR if they are already
defined
[delphij] Fix build.
[bapt] Define special case for atf which libraries names are different from
their
declaration
[bapt] s/cft/ctf
[bapt] Forgot to convert demandoc along with mandoc
[bapt] Fix typo
[bapt] Fix dependencies in case of building statically
[bapt] Avoid code duplication by directly adding _INSTALLLIBS and _PRIVATELIBS
into
_LIBRARIES
Reported by:emaste
[bapt] Each time something links to sqlite3 it should link to pthread
[bapt] Convert makefs to LIBADD
Add definition of libnetbsd in src.libnames.mk
[bapt] Convert mandoc to LIBADD
While here remove the ugly LDMANDOC
[ae] Skip L2 addresses lookups for p2p interfaces.
Discussed with: melifaro
Sponsored by: Yandex LLC
[rpaulo] Import libgpio.
This is a thin wrapper around the kernel interface which should make
it easier to write GPIO applications. gpioctl(8) will be converted to
use this library in a separate commit.
Differential Revision: https://reviews.freebsd.org/D1183
Reviewed by:adrian, loos
Discussed on: arm@, embedded@
Relnotes: yes
[brd] Convert to LIBADD.
Reviewed by:bapt
[bapt] Correctly specify -L for internal and private libs
[bapt] Implement LIBADD
LIBADD will automatically set DPADD and LDADD when needed including their
dependencies, LIBADD automatically handles private and internal libs so that
the end user Makefile does not have to care about it.
This allows to reduce overlinking on the base system leaving the framework get
the dependencies properly.
It also allows to built components binaries statically.
To use it:
Replace:
DPADD= ${LIBARCHIVE} ${LIBSSL}
LDADD= -larchive -lssl
by:
LIBADD= archive ssl
Differential Revision: https://reviews.freebsd.org/D1209
Reviewed by:brooks imp
--
[...truncated 118494 lines...]
--- depend_subdir_nfsuserd ---
echo nfsuserd:
/usr/objhttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/tmp/usr/lib/libc.a
.depend
--- depend_subdir_nghook ---
=== usr.sbin/nghook (depend)
--- usr.bin.depend__D ---
echo nl:
/usr/objhttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/tmp/usr/lib/libc.a
.depend
--- depend_subdir_nohup ---
=== usr.bin/nohup (depend)
--- usr.sbin.depend__D ---
--- .depend ---
rm -f .depend
CC='cc ' mkdep -f .depend -a -std=gnu99
https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/usr.sbin/nghook/main.c
--- usr.bin.depend__D ---
--- .depend ---
rm -f .depend
CC='cc ' mkdep -f .depend -a -std=gnu99
https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/usr.bin/nohup/nohup.c
--- depend_subdir_netstat ---
echo netstat:
/usr/objhttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/tmp/usr/lib/libc.a
/usr/objhttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/tmp/usr/lib/libkvm.a
/usr/objhttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/tmp/usr/lib/libmemstat.a
/usr/objhttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/tmp/usr/lib/libutil.a
/usr/objhttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/tmp/usr/lib/libnetgraph.a
.depend
--- depend_subdir_opieinfo ---
=== usr.bin/opieinfo (depend)
--- usr.sbin.depend__D ---
echo nghook:
/usr/objhttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/tmp/usr/lib/libc.a
/usr/objhttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/tmp/usr/lib/libnetgraph.a
.depend
--- depend_subdir_nmtree ---
=== usr.sbin/nmtree (depend)
--- usr.bin.depend__D ---
--- depend_subdir_nohup ---
echo nohup:
/usr/objhttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/tmp/usr/lib/libc.a
.depend
--- depend_subdir_opiekey ---
--- depend_subdir_opieinfo ---
--- .depend ---
rm -f .depend
--- depend_subdir_opiekey ---
=== usr.bin/opiekey (depend)
--- depend_subdir_opieinfo ---
CC='cc ' mkdep -f .depend -a
-Ihttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/usr.bin/opieinfo/../../lib/libopie
-Ihttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/usr.bin/opieinfo/../../contrib/opie
-DINSECURE_OVERRIDE -std=gnu99
https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/usr.bin/opieinfo/../../contrib/opie/opieinfo.c
--- usr.sbin.depend__D ---
--- _sub.depend ---
=== usr.sbin/nmtree/tests (depend)
--- usr.bin.depend__D ---
--- depend_subdir_opiekey ---
--- .depend ---
rm -f .depend
CC='cc ' mkdep -f .depend -a
-Ihttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/usr.bin/opiekey/../../lib/libopie
-Ihttps://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/usr.bin/opiekey/../../contrib/opie
-DINSECURE_OVERRIDE -std=gnu99
https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/ws/usr.bin/opiekey/../../contrib/opie/opiekey.c
--- usr.sbin.depend__D ---
--- scripts.depend ---
--- usr.bin.depend__D ---
--- depend_subdir_opieinfo ---
echo opieinfo:
Jenkins build is back to normal : FreeBSD_HEAD #1910
See https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/1910/changes ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Jenkins build became unstable: FreeBSD_HEAD-tests2 #311
See https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/311/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Jenkins build became unstable: FreeBSD_HEAD-tests2 #311
On Nov 24, 2014, at 21:25, jenkins-ad...@freebsd.org wrote: See https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD-tests2/311/ Hmmm… this build ran out of space on the test image. signature.asc Description: Message signed with OpenPGP using GPGMail
