Re: dev_lock() contention for fdesc syscalls -- possible fix
On Mon, Nov 10, 2014 at 02:49:39AM +0100, Luigi Rizzo wrote:
It was noticed that there is huge dev_lock() contention when multiple
processes do a poll() even on independent file descriptors.
Turns out that not just poll but most syscalls on file descriptors
(as opposed to sockets) in sys/fs/devfs/devfs_vnops.c including
devfs_poll_f(), devfs_ioctl_f() and read/write share the problem
as they use the following pattern
devfs_poll_f() {
...
devfs_fp_check(fp, ...) --
kern/kern_conf.c :: devvn_refthread(fp-f_vnode, ...) --
dev_lock();
dev = vp-v_rdev; // lock on vp ?
... check that dev != NULL
atomic_add_long(dev-si_threadcount, 1);
dev_unlock();
dsw-d_poll()
dev_relthread() --
atomic_subtract_rel_long(dev-si_threadcount, 1);
}
I believe that dev_lock() in devvn_refthread() is protecting
dev = vp-v_rdev
(the cdev entry 'dev' cannot go away for the reasons stated below).
However looking at places where vp-v_rdev is modified, turns out
that it only happens when holding VI_LOCK(vp) -- the places are
devfs_allocv() and devfs_reclaim().
There is one place in zfs where the vnode is created and v_rdev
is set without holding a lock, so nobody can dereference it.
As a consequence i believe that if in devfs_fp_check() we replace
dev_lock() / dev_unlock() with VI_LOCK(vp) / VI_UNLOCK(vp),
we make sure that we can safely dereference vp-v_rdev, and the
cdev cannot go away because the vnode has a reference to it.
The counter uses an atomic instruction (so the release is lockless)
Vnode reference, as well as cdev reference, which is obtained by
dev_ref(), do not provide any protection there. v_rdev is only
coincidentally protected by the vnode interlock.
If you look at larger part of the code, you would note that dev mutex
is held even after v_rdev is dereferenced. The real protection it
provides is against race with destroy_dev(l)(), which could invalidate
dev-so_devsw at any moment when either device thread reference is
not held, or dev mutex is not held. So your patch breaks the
device destruction.
This should be enough to remove the contention.
If you never calls destroy_dev() for the devfs node, you could use
MAKEDEV_ETERNAL flag for make_dev_credf(), which indicates that there
is no risk of race with destroy_dev() for the created node.
Usually, code which could be compiled in kernel statically but also
loaded as module, use MAKEDEV_ETERNAL_KLD flag, which takes care of
module needed to call destroy_dev().
Anyone care to review/test the patch below ?
Yes, there are people who care.
(dev_refthread() still has the single dev_lock() contention,
i don't know how to address that yet)
cheers
luigi
Index: /home/luigi/FreeBSD/head/sys/kern/kern_conf.c
===
--- /home/luigi/FreeBSD/head/sys/kern/kern_conf.c (revision 273452)
+++ /home/luigi/FreeBSD/head/sys/kern/kern_conf.c (working copy)
@@ -224,10 +224,11 @@
}
csw = NULL;
- dev_lock();
+ ASSERT_VI_UNLOCKED(vp, __func__);
+ VI_LOCK(vp); // dev_lock();
dev = vp-v_rdev;
if (dev == NULL) {
- dev_unlock();
+ VI_UNLOCK(vp); // dev_unlock();
return (NULL);
}
cdp = cdev2priv(dev);
@@ -236,7 +237,7 @@
if (csw != NULL)
atomic_add_long(dev-si_threadcount, 1);
}
- dev_unlock();
+ VI_UNLOCK(vp); // dev_unlock();
if (csw != NULL) {
*devp = dev;
*ref = 1;
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: dev_lock() contention for fdesc syscalls -- possible fix
On Mon, Nov 10, 2014 at 10:34:57AM +0200, Konstantin Belousov wrote:
On Mon, Nov 10, 2014 at 02:49:39AM +0100, Luigi Rizzo wrote:
It was noticed that there is huge dev_lock() contention when multiple
processes do a poll() even on independent file descriptors.
Turns out that not just poll but most syscalls on file descriptors
(as opposed to sockets) in sys/fs/devfs/devfs_vnops.c including
devfs_poll_f(), devfs_ioctl_f() and read/write share the problem
as they use the following pattern
devfs_poll_f() {
...
devfs_fp_check(fp, ...) --
kern/kern_conf.c :: devvn_refthread(fp-f_vnode, ...) --
dev_lock();
dev = vp-v_rdev; // lock on vp ?
... check that dev != NULL
atomic_add_long(dev-si_threadcount, 1);
dev_unlock();
dsw-d_poll()
dev_relthread() --
atomic_subtract_rel_long(dev-si_threadcount, 1);
}
I believe that dev_lock() in devvn_refthread() is protecting
dev = vp-v_rdev
(the cdev entry 'dev' cannot go away for the reasons stated below).
However looking at places where vp-v_rdev is modified, turns out
that it only happens when holding VI_LOCK(vp) -- the places are
devfs_allocv() and devfs_reclaim().
There is one place in zfs where the vnode is created and v_rdev
is set without holding a lock, so nobody can dereference it.
As a consequence i believe that if in devfs_fp_check() we replace
dev_lock() / dev_unlock() with VI_LOCK(vp) / VI_UNLOCK(vp),
we make sure that we can safely dereference vp-v_rdev, and the
cdev cannot go away because the vnode has a reference to it.
The counter uses an atomic instruction (so the release is lockless)
Vnode reference, as well as cdev reference, which is obtained by
dev_ref(), do not provide any protection there. v_rdev is only
coincidentally protected by the vnode interlock.
If you look at larger part of the code, you would note that dev mutex
is held even after v_rdev is dereferenced. The real protection it
provides is against race with destroy_dev(l)(), which could invalidate
dev-so_devsw at any moment when either device thread reference is
not held, or dev mutex is not held. So your patch breaks the
device destruction.
I see. Thanks for the clarification.
Would it help to rewrite the part of devvn_refthread as follows:
devvn_refthread() {
// protect vp-v_rdev dereference and dev disappearing
VI_LOCK(vp);
dev = vi-v_rdev;
.. check that dev != NULL
// protect the race on dev-si_devsw
atomic_add_long(dev-si_threadcount, 1);
VI_UNLOCK(vp);
... appropriate memory barrier
csw = dev-si_devsw;
if (csw != NULL) {
// we won the race, even if destroy_devl() runs
// it will keep the cdevsw alive until si_threadcount goes to 0
*devp = dev;
*ref = 1;
} else {
// we lost
*ref = 0;
}
return csw;
}
i.e. tentatively increment si_threadcount before dereferencing si_devsw
and then restoring it if we lose the race ?
It might be necessary to add a barrier in destroy_devl() between clearing
si_devsw and reading si_threadcount.
This should be enough to remove the contention.
If you never calls destroy_dev() for the devfs node, you could use
MAKEDEV_ETERNAL flag for make_dev_credf(), which indicates that there
is no risk of race with destroy_dev() for the created node.
Usually, code which could be compiled in kernel statically but also
loaded as module, use MAKEDEV_ETERNAL_KLD flag, which takes care of
module needed to call destroy_dev().
that I suppose would mean that the module cannot be safely unloaded ?
cheers
luigi
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: dev_lock() contention for fdesc syscalls -- possible fix
On Mon, Nov 10, 2014 at 10:44:12AM +0100, Luigi Rizzo wrote:
On Mon, Nov 10, 2014 at 10:34:57AM +0200, Konstantin Belousov wrote:
On Mon, Nov 10, 2014 at 02:49:39AM +0100, Luigi Rizzo wrote:
It was noticed that there is huge dev_lock() contention when multiple
processes do a poll() even on independent file descriptors.
Turns out that not just poll but most syscalls on file descriptors
(as opposed to sockets) in sys/fs/devfs/devfs_vnops.c including
devfs_poll_f(), devfs_ioctl_f() and read/write share the problem
as they use the following pattern
devfs_poll_f() {
...
devfs_fp_check(fp, ...) --
kern/kern_conf.c :: devvn_refthread(fp-f_vnode, ...) --
dev_lock();
dev = vp-v_rdev; // lock on vp ?
... check that dev != NULL
atomic_add_long(dev-si_threadcount, 1);
dev_unlock();
dsw-d_poll()
dev_relthread() --
atomic_subtract_rel_long(dev-si_threadcount, 1);
}
I believe that dev_lock() in devvn_refthread() is protecting
dev = vp-v_rdev
(the cdev entry 'dev' cannot go away for the reasons stated below).
However looking at places where vp-v_rdev is modified, turns out
that it only happens when holding VI_LOCK(vp) -- the places are
devfs_allocv() and devfs_reclaim().
There is one place in zfs where the vnode is created and v_rdev
is set without holding a lock, so nobody can dereference it.
As a consequence i believe that if in devfs_fp_check() we replace
dev_lock() / dev_unlock() with VI_LOCK(vp) / VI_UNLOCK(vp),
we make sure that we can safely dereference vp-v_rdev, and the
cdev cannot go away because the vnode has a reference to it.
The counter uses an atomic instruction (so the release is lockless)
Vnode reference, as well as cdev reference, which is obtained by
dev_ref(), do not provide any protection there. v_rdev is only
coincidentally protected by the vnode interlock.
If you look at larger part of the code, you would note that dev mutex
is held even after v_rdev is dereferenced. The real protection it
provides is against race with destroy_dev(l)(), which could invalidate
dev-so_devsw at any moment when either device thread reference is
not held, or dev mutex is not held. So your patch breaks the
device destruction.
I see. Thanks for the clarification.
Would it help to rewrite the part of devvn_refthread as follows:
devvn_refthread() {
// protect vp-v_rdev dereference and dev disappearing
VI_LOCK(vp);
dev = vi-v_rdev;
.. check that dev != NULL
// protect the race on dev-si_devsw
atomic_add_long(dev-si_threadcount, 1);
VI_UNLOCK(vp);
... appropriate memory barrier
csw = dev-si_devsw;
if (csw != NULL) {
// we won the race, even if destroy_devl() runs
// it will keep the cdevsw alive until si_threadcount goes to 0
*devp = dev;
*ref = 1;
} else {
// we lost
*ref = 0;
}
return csw;
}
i.e. tentatively increment si_threadcount before dereferencing si_devsw
and then restoring it if we lose the race ?
It might be necessary to add a barrier in destroy_devl() between clearing
si_devsw and reading si_threadcount.
Sure it is neccessary to add a barrier in destroy_devl() then.
From the first look, this might work, but how would you handle the
possibility that cdev memory is already freed when you do the increment ?
The vnode interlock does not protect against it; if you mean that
vnode reclamation already happen and vp-v_rdev must be cleared, this
might need some additional argumentation.
Note that the real patch is more involved, since both dev_refthread()
and devvn_refthread() should be handled. Also, there is some ugly hack
in sound/clone.c.
This should be enough to remove the contention.
If you never calls destroy_dev() for the devfs node, you could use
MAKEDEV_ETERNAL flag for make_dev_credf(), which indicates that there
is no risk of race with destroy_dev() for the created node.
Usually, code which could be compiled in kernel statically but also
loaded as module, use MAKEDEV_ETERNAL_KLD flag, which takes care of
module needed to call destroy_dev().
that I suppose would mean that the module cannot be safely unloaded ?
Well, you are not supposed to use MAKEDEV_ETERNAL for loadable modules
at all. This is what MAKEDEV_ETERNAL_KLD ensures.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD + Google Code-In 2014 = we need ideas.
Hello, List, 2 ideas: 1. port DFBSD's CAM DA driver It's doing much better than FB in heavy load with random IO, eg. squid or postgres databases for zabbix 1.8. It's released within DFBSD V2.10 CAM DA driver enhanced to separate read and write streams, allowing concurrent write completion in the face of many stalled read requests. http://leaf.dragonflybsd.org/mailarchive/commits/2011-04/msg00062.html http://leaf.dragonflybsd.org/mailarchive/commits/2011-04/msg00061.html 2. port DFBSD's swapcache ZFS is so memory hungry, I'd like use the rock UFS for most other than nfs/samba server similar application, but curreently FB has no solutions can utilize SSD to speed up UFS's performance. Maybe it's not quite new technology, but it's useful. Regards Simon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD + Google Code-In 2014 = we need ideas.
Hi! 2 ideas: 1. port DFBSD's CAM DA driver [...] Maybe I'm showing my age here, but Code-In is for school kids aged 13-17, and in my time, there were very few kids at that task level 8-} -- p...@opsec.eu+49 171 3101372 6 years to go ! ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
samba/NSSWITCH interaction in fbsd 10 vs fbsd 8
When I try use the libnss_winbind.so that is generated by samba 3.6 I get the following message: NSSWITCH(nss_load_module): winbind, Undefined symbol nss_module_register. First I have to change its' name to nss_winbins.so.1 however because that is what nsswitch is looking for (BTW where is that documented??) then it finds it but fails as mentioned above. This same samba source generates good nss files in 8.0 but under 10 it fails.. Literally it's the same sources just checked out into a different build system. (10 vs 8). Has there been a change in the API for the nss modules? where it he API for nsswitch files documented? Julian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Changing timezone without reboot/restarting each service?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 After changing timezones in Russia (with replacing /etc/localtime with new file), I found that cron works in old timezone till restart. And all other services do the same, but cron is most obvious here :) Looks like libc reads timezone only once and it could not be chamged for process without restart (which leads to, effectivly, restart of whole server). Is it known problem? I think, it should be fixed somehow. I understand, that re-check timezone file on each time-related call could be expensive, though :( - -- // Lev Serebryakov -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQJ8BAEBCgBmBQJUYLEoXxSAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGOTZEMUNBMEI1RjQzMThCNjc0QjMzMEFF QUIwM0M1OEJGREM0NzhGAAoJEOqwPFi/3EePW08P/1hlYiwRN7AnEjGcnMMIDmW/ jkydPH8NPHr4PrEBexIJE9+bj1UzRSqK9v1t6r8ktdX7Myph1Kb8CaM4nS8+tesS 7Wqblk/o+zHkeBOtF8J9Ar7+7ZdZWd+oG/cTAmZdlxXV1i70s1Qe2TwozO0RKCgm DHtD65jTKGUxWIliRz3GX46NnhRdOeBj+m/2dxe5s0nkP7eUknpJKBBd3IZPAUAQ 5WQvX3YbVwlE9r+pVTWFGvotKMwbmHGtJSyDrsYxyL6T2FY8+8/jl31FSLQNvlXX 9ymWjKnVIECPdRpVfQsEgp6WvRJ+xEKhyDbXCLCiOtTWX8ieJd0qZPFS4J7V87uu SwSs1OVGwfy6zSVI33QFFsp4fJzpFACxUV7YfAO/SDM+CMrGsfSCFfSyutHx1Qip 3EiVAbHKFsCdiTHRiEZfE3cg91Gp25MDWwhmePo521UTS5bI9/vg9KVFF/5QyLgl jHDMz0T8LPUernxqZilJ02vBUu1eHaznbIRgD7zQjuG2YvO5S4vh/Rdt9dbj/cRp DpIfogfEpsvTHVv7N5xWfq0zu/DTCtNtv0eIFZy8WheFb5GvHlI8IwwgVE5CuF9D 4FcIJnOU20LqQjk7LujborqwMj8ifbz48wa4xYc8/G4woKJ/Snsu3Fxdbrtmc5b4 flIOddz8QQmSn28/wOF5 =HDMD -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Changing timezone without reboot/restarting each service?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 After changing timezones in Russia (with replacing /etc/localtime with new file), I found that cron works in old timezone till restart. And all other services do the same, but cron is most obvious here :) Looks like libc reads timezone only once and it could not be chamged for process without restart (which leads to, effectivly, restart of whole server). Is it known problem? I think, it should be fixed somehow. I understand, that re-check timezone file on each time-related call could be expensive, though :( - -- // Lev Serebryakov -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQJ8BAEBCgBmBQJUYLE/XxSAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGOTZEMUNBMEI1RjQzMThCNjc0QjMzMEFF QUIwM0M1OEJGREM0NzhGAAoJEOqwPFi/3EePaAwP/AxFm4IT8GiScZ3TQkpO/89z +LKfHU5qF45M3fRQLU3wPPtp/RYFIb5SGPRLRg9L6SmY1tppx8GlBRU/kORUPizt e9865/4p1RwOlEOJ5vxulJGxMYEjWFGB9kLnkNLgpzMK/88GmT5ntZqVIoC9wuSo fYXME7O2ZOm7Fczaiu5oH8CSlOMBNXXuaXJsuvH7WVWKOLO5XbHZd4UJbkltJQGW LEfHzLcTIU0LEzuZSVIPIzde4cGey0phO7a01XXx2+SIydU077LdV6cB2pKvTYLR t7cifOjoCWkUX/5qknoIQMrY398b6NMpQQURQoG4Uh2WvLDm5OAwYZoHfjroU2+5 7XVSrkKJd67EEw0ve84DVHWAOdtPvciNNMTVZu5z9jkFZwkVKQP7i+ct9E5g5qRq N/hiz6mUrwOfHeXKdnXBg2zc2YLlHxvhJyUc45Gi0W0MmLVMRRFPKkcVblOB5rnJ JKz5fVAXiO8VMUmTCx2TJ/YUzvsxXXkSJ0+abMviV+zACPBdaxKWb4UhEIaNdSMg hrhkd1jOWVdP83ucvXRmIoBxW9Pvfs0r249lFDTRW1PuY+P4JZLvUplvIKo5wR/X mDfeOjSy7qWq2KMVlfHh+3r2wCqRsNWy3JzaRe1npSpD+34vI+7jhPPWS7yUILQi +JB5MQZr0BRwU59Hhi6o =mE5w -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
MK_ vs. WITH_/WITHOUT_ in release/Makefile
Hi all, release/Makefile in CURRENT utilizes MK_* knobs vs. the WITH_/WITHOUT_* knobs seen in release/Makefile in the STABLE/RELEASE branches. Merging a CURRENT version of the Makefile into a RELEASE branch and executing a release build results in an error citing MK_KERNEL_SYMBOLS can't be set by a user. Comparisons of the Makefile between CURRENT and STABLE/RELEASE exposed the difference and changing the knobs from MK_ to WITHOUT_ resolved the error. I have little familiarity with these knobs, but was under the impression the Makefile would not differ like this. Is there documentation describing the use of these knobs between the varying branches and how they are changed from CURRENT to STABLE/RELEASE? -- Take care Rick Miller ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: MK_ vs. WITH_/WITHOUT_ in release/Makefile
On Mon, Nov 10, 2014 at 09:38:08AM -0500, Rick Miller wrote: Hi all, release/Makefile in CURRENT utilizes MK_* knobs vs. the WITH_/WITHOUT_* knobs seen in release/Makefile in the STABLE/RELEASE branches. Merging a CURRENT version of the Makefile into a RELEASE branch and executing a release build results in an error citing MK_KERNEL_SYMBOLS can't be set by a user. Comparisons of the Makefile between CURRENT and STABLE/RELEASE exposed the difference and changing the knobs from MK_ to WITHOUT_ resolved the error. I have little familiarity with these knobs, but was under the impression the Makefile would not differ like this. Is there documentation describing the use of these knobs between the varying branches and how they are changed from CURRENT to STABLE/RELEASE? These changes are result of src.opts.mk changes (only available in head/) that allows specifying MK_FOO=no instead of WITHOUT_FOO=yes on the command line. The changes were applied to the release/Makefile because it allows more granular tuning for different stages of the release build. For example, one might want to build userland/kernel with WITH_DEBUG_FILES=yes, but does not want that to apply to the resulting ISOs, so a global WITH_DEBUG_FILES=yes and target-specific MK_DEBUG_FILES=no do not collide. Glen pgpzIncb_HlTv.pgp Description: PGP signature
What's the least required in base to be functional?
Apologies. That may not have been the best choice of titles. What I'm trying to determine, is what is the very least I will require in base, to actually build a userland build environment. NOTE; this all concerns -CURRENT (recent 11). Point being, while I recognize that clang/llvm is the default on 10+. I have been building/installing world/kernel with make.conf(5) WITHOUT_CLANG=true FAVORITE_COMPILER=gcc src.conf(5) WITHOUT_CLANG=true on RELENG_8, and RELENG_9, and 11 (as of 1 mos ago) Everything worked as anticipated. But a recent (5 days ago) build/install on -CURRENT. Followed by a make delete-old _seemed_ to have an adverse affect. More specifically; having used the above declarations always resulted in the make delete-old removing clang from base. Which was fine. As I had intended to experiment with the different versions of lang/clang, and devel/llvm, via installing from ports. But my recent attempt using the above method, resulted in my being unable to build many ports. x11/* mostly. I ran into problems with xmmintrin.h not being found. Or other problems, where declarations were not supported in gcc(4.8,4.9, or 5). So what exactly *must* be installed in base to allow for a more *granular* approach to testing/building? Used to be IIRC, fmake, or bmake. But that's likely a pretty dated recollection. Thank you for all your time, and consideration. --Chris ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: What's the least required in base to be functional?
On 10 Nov 2014, at 19:54, Chris H bsd-li...@bsdforge.com wrote: Apologies. That may not have been the best choice of titles. What I'm trying to determine, is what is the very least I will require in base, to actually build a userland build environment. NOTE; this all concerns -CURRENT (recent 11). Point being, while I recognize that clang/llvm is the default on 10+. I have been building/installing world/kernel with make.conf(5) WITHOUT_CLANG=true FAVORITE_COMPILER=gcc src.conf(5) WITHOUT_CLANG=true on RELENG_8, and RELENG_9, and 11 (as of 1 mos ago) Everything worked as anticipated. But a recent (5 days ago) build/install on -CURRENT. Followed by a make delete-old _seemed_ to have an adverse affect. More specifically; having used the above declarations always resulted in the make delete-old removing clang from base. Which was fine. As I had intended to experiment with the different versions of lang/clang, and devel/llvm, via installing from ports. But my recent attempt using the above method, resulted in my being unable to build many ports. x11/* mostly. I ran into problems with xmmintrin.h not being found. Or other problems, where declarations were not supported in gcc(4.8,4.9, or 5). So what exactly *must* be installed in base to allow for a more *granular* approach to testing/building? Used to be IIRC, fmake, or bmake. But that's likely a pretty dated recollection. On recent -CURRENT, to build world using the version of gcc in base, and to not build or use the version of clang in base at all, you need at least the following settings in your src.conf: WITH_GCC=x # Enables building gcc for the final world WITH_GCC_BOOTSTRAP=x # Enables building gcc during cross-tools WITH_GNUCXX=x # Enables building libstdc++ and libsupc++ WITHOUT_CLANG_BOOTSTRAP=x # Disables building clang during cross-tools WITHOUT_CLANG=x# Disables building clang for the final world WITHOUT_CLANG_IS_CC=x # Links gcc to /usr/bin/cc, /usr/bin/c++, etc. Note that you can delete WITHOUT_CLANG from your make.conf, just like other WITH_ and WITHOUT_ settings. These only belong in src.conf. -Dimitry signature.asc Description: Message signed with OpenPGP using GPGMail
Re: MK_ vs. WITH_/WITHOUT_ in release/Makefile
On Mon, Nov 10, 2014 at 8:38 AM, Rick Miller vmil...@hostileadmin.com wrote:
Hi all,
release/Makefile in CURRENT utilizes MK_* knobs vs. the WITH_/WITHOUT_*
knobs seen in release/Makefile in the STABLE/RELEASE branches. Merging a
CURRENT version of the Makefile into a RELEASE branch and executing a
release build results in an error citing MK_KERNEL_SYMBOLS can't be set by
a user. Comparisons of the Makefile between CURRENT and STABLE/RELEASE
exposed the difference and changing the knobs from MK_ to WITHOUT_ resolved
the error.
I have little familiarity with these knobs, but was under the impression
the Makefile would not differ like this. Is there documentation describing
the use of these knobs between the varying branches and how they are
changed from CURRENT to STABLE/RELEASE?
According to head/share/Mk/src.opts.mk:
# Users define WITH_FOO and WITHOUT_FOO on the command line or in /etc/src.conf
# and /etc/make.conf files. These translate in the build system to
MK_FOO={yes,no}
# with sensible (usually) defaults.
--
DISCLAIMER:
No electrons were maimed while sending this message. Only slightly bruised.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: What's the least required in base to be functional?
On Mon, 10 Nov 2014 20:28:00 +0100 Dimitry Andric d...@freebsd.org wrote On 10 Nov 2014, at 19:54, Chris H bsd-li...@bsdforge.com wrote: Apologies. That may not have been the best choice of titles. What I'm trying to determine, is what is the very least I will require in base, to actually build a userland build environment. NOTE; this all concerns -CURRENT (recent 11). Point being, while I recognize that clang/llvm is the default on 10+. I have been building/installing world/kernel with make.conf(5) WITHOUT_CLANG=true FAVORITE_COMPILER=gcc src.conf(5) WITHOUT_CLANG=true on RELENG_8, and RELENG_9, and 11 (as of 1 mos ago) Everything worked as anticipated. But a recent (5 days ago) build/install on -CURRENT. Followed by a make delete-old _seemed_ to have an adverse affect. More specifically; having used the above declarations always resulted in the make delete-old removing clang from base. Which was fine. As I had intended to experiment with the different versions of lang/clang, and devel/llvm, via installing from ports. But my recent attempt using the above method, resulted in my being unable to build many ports. x11/* mostly. I ran into problems with xmmintrin.h not being found. Or other problems, where declarations were not supported in gcc(4.8,4.9, or 5). So what exactly *must* be installed in base to allow for a more *granular* approach to testing/building? Used to be IIRC, fmake, or bmake. But that's likely a pretty dated recollection. On recent -CURRENT, to build world using the version of gcc in base, and to not build or use the version of clang in base at all, you need at least the following settings in your src.conf: WITH_GCC=x # Enables building gcc for the final world WITH_GCC_BOOTSTRAP=x # Enables building gcc during cross-tools WITH_GNUCXX=x # Enables building libstdc++ and libsupc++ WITHOUT_CLANG_BOOTSTRAP=x # Disables building clang during cross-tools WITHOUT_CLANG=x# Disables building clang for the final world WITHOUT_CLANG_IS_CC=x # Links gcc to /usr/bin/cc, /usr/bin/c++, etc. Note that you can delete WITHOUT_CLANG from your make.conf, just like other WITH_ and WITHOUT_ settings. These only belong in src.conf. -Dimitry Thank you, Dimitry. Perfect! So that I can become better acquainted. Where can I find (read) more about my options in base? KNOBS, and such. I don't recall reading about these in the developers handbook, and even then, especially where -CURRENT is concerned, they move/change pretty quickly. :) Thanks again. --Chris ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: What's the least required in base to be functional?
On 10 Nov 2014, at 21:04, Chris H bsd-li...@bsdforge.com wrote: On Mon, 10 Nov 2014 20:28:00 +0100 Dimitry Andric d...@freebsd.org wrote ... Note that you can delete WITHOUT_CLANG from your make.conf, just like other WITH_ and WITHOUT_ settings. These only belong in src.conf. -Dimitry Thank you, Dimitry. Perfect! So that I can become better acquainted. Where can I find (read) more about my options in base? KNOBS, and such. I don't recall reading about these in the developers handbook, and even then, especially where -CURRENT is concerned, they move/change pretty quickly. :) You can read the build(7), src.conf(5) and make.conf(5) man pages. -Dimitry signature.asc Description: Message signed with OpenPGP using GPGMail
Re: What's the least required in base to be functional?
On Mon, 10 Nov 2014 21:16:46 +0100 Dimitry Andric d...@freebsd.org wrote On 10 Nov 2014, at 21:04, Chris H bsd-li...@bsdforge.com wrote: On Mon, 10 Nov 2014 20:28:00 +0100 Dimitry Andric d...@freebsd.org wrote ... Note that you can delete WITHOUT_CLANG from your make.conf, just like other WITH_ and WITHOUT_ settings. These only belong in src.conf. -Dimitry Thank you, Dimitry. Perfect! So that I can become better acquainted. Where can I find (read) more about my options in base? KNOBS, and such. I don't recall reading about these in the developers handbook, and even then, especially where -CURRENT is concerned, they move/change pretty quickly. :) You can read the build(7), src.conf(5) and make.conf(5) man pages. -Dimitry D'OH! Sorry for the noise, and thanks. :) --Chris ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD + Google Code-In 2014 = we need ideas.
On Sun, Nov 9, 2014 at 8:59 PM, Wojciech A. Koszek wkos...@freebsd.org wrote: Hello, This year we'd like to participate in the Google Code-In 2014. This is Google Summer of Code, but for younger people: age range is 13--17. If you're one of them, we highly encourage you to apply! ***This year coding tasks are possible, so feel free to add those*** To submit idea which you'd like to see done in GCI, visit: http://bit.ly/FreeBSD_GCIN2014 Regardless of who you are, please use the form to submit ideas. Don't add stuff via Wiki, since this year we'll do direct import of all ideas from Google Forms. To see tasks from previous year that are yet to be copied to Google Forms: https://wiki.freebsd.org/GoogleCodeIn/2014Tasks Thanks to GCI in the previous years, we gained one more FreeBSD developer. We'd like to partcipate this year too. We need: 1) ideas. How about converting various utility functions to use libxo? I think that's within the grasp of a high-schooler. 2) mentors 3) participants. Just like in previous years we'll decide whether we're ready. Deadlines: --- November 12, 2014 - The 10-12 Mentoring organizations are announced for Google Code-in 2014 and the orgs can start entering their tasks into the Google system (the tasks will not be viewable to students until the contest opens on Dec 1). December 1, 2014 17:00 UTC - Google Code-in 2014 contest opens for students January 19, 2015 17:00 UTC - Google Code-in 2014 student work ends --- MORE INFO: [0] http://www.google-melange.com/gci/homepage/google/gci2014 [1] gci-ment...@googlegroups.com [2] https://developers.google.com/open-source/gci/resources/mentor-and-orgadmin-info -- Wojciech A. Koszek wkos...@freebsd.czest.pl http://FreeBSD.czest.pl/~wkoszek/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD + Google Code-In 2014 = we need ideas.
On Mon, Nov 10, 2014 at 01:56:27PM -0700, Alan Somers wrote: On Sun, Nov 9, 2014 at 8:59 PM, Wojciech A. Koszek wkos...@freebsd.org wrote: Hello, This year we'd like to participate in the Google Code-In 2014. This is Google Summer of Code, but for younger people: age range is 13--17. If you're one of them, we highly encourage you to apply! ***This year coding tasks are possible, so feel free to add those*** To submit idea which you'd like to see done in GCI, visit: http://bit.ly/FreeBSD_GCIN2014 Regardless of who you are, please use the form to submit ideas. Don't add stuff via Wiki, since this year we'll do direct import of all ideas from Google Forms. To see tasks from previous year that are yet to be copied to Google Forms: https://wiki.freebsd.org/GoogleCodeIn/2014Tasks Thanks to GCI in the previous years, we gained one more FreeBSD developer. We'd like to partcipate this year too. We need: 1) ideas. How about converting various utility functions to use libxo? I think that's within the grasp of a high-schooler. This sounds good. Feel free to add 1 task for each such utility. Wojciech 2) mentors 3) participants. Just like in previous years we'll decide whether we're ready. Deadlines: --- November 12, 2014 - The 10-12 Mentoring organizations are announced for Google Code-in 2014 and the orgs can start entering their tasks into the Google system (the tasks will not be viewable to students until the contest opens on Dec 1). December 1, 2014 17:00 UTC - Google Code-in 2014 contest opens for students January 19, 2015 17:00 UTC - Google Code-in 2014 student work ends --- MORE INFO: [0] http://www.google-melange.com/gci/homepage/google/gci2014 [1] gci-ment...@googlegroups.com [2] https://developers.google.com/open-source/gci/resources/mentor-and-orgadmin-info -- Wojciech A. Koszek wkos...@freebsd.czest.pl http://FreeBSD.czest.pl/~wkoszek/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org -- Wojciech A. Koszek wkos...@freebsd.czest.pl http://FreeBSD.czest.pl/~wkoszek/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Unable to build world w/o clang on 11
Greetings, I'm attempting to build/install world/kernel on a fresh install of 11 on bare metal, from the bootonly iso from 10-26. I understand that clang is the default for 10+. But had hoped to install it from ports *after* kernel/world. I used what I *thought* was the correct direction to do this: src.conf(5) WITH_GCC=true WITH_GCC_BOOTSTRAP=true WITH_GNUCXX=true WITHOUT_CLANG_BOOTSTRAP=true WITHOUT_CLANG=true WITHOUT_CLANG_IS_CC=true But after attempting to 'make -j6 buildworld'. It failed. So while still in /usr/src, I make clean, cd /usr/obj, and chflags -R noschg *, then rm -rf *, followed by cd /usr/src, and make buildworld. This too fails. I inadvertently clobbered the output I had intended to post along with this message. But I remember it being during the CC of rescue. If that helps any. Thank you for all your time, and consideration. --Chris ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
HEADS-UP: amd64 UEFI boot is broken
UEFI booting is broken for amd64 as of r273582 (Oct 24). The symptom is an apparent hang after the loader transfers control to the kernel (no output from the kernel is observed). Until this is addressed you can checkout an earlier revision or revert r273582 locally. -Ed ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Unable to build world w/o clang on 11
On Mon, 10 Nov 2014 16:54:52 -0800 Chris H bsd-li...@bsdforge.com wrote Greetings, I'm attempting to build/install world/kernel on a fresh install of 11 on bare metal, from the bootonly iso from 10-26. I understand that clang is the default for 10+. But had hoped to install it from ports *after* kernel/world. I used what I *thought* was the correct direction to do this: src.conf(5) WITH_GCC=true WITH_GCC_BOOTSTRAP=true WITH_GNUCXX=true WITHOUT_CLANG_BOOTSTRAP=true WITHOUT_CLANG=true WITHOUT_CLANG_IS_CC=true But after attempting to 'make -j6 buildworld'. It failed. So while still in /usr/src, I make clean, cd /usr/obj, and chflags -R noschg *, then rm -rf *, followed by cd /usr/src, and make buildworld. This too fails. I inadvertently clobbered the output I had intended to post along with this message. But I remember it being during the CC of rescue. If that helps any. Thank you for all your time, and consideration. --Chris Additional attempts, and information, in an attempt to buildworld without clang. FreeBSD dev FreeBSD 11.0-CURRENT #0 r273635: Sat Oct 25 14:23:40 UTC 2014 r...@grind.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 Path: /usr/src Working Copy Root Path: /usr/src URL: svn://svn.freebsd.org/base/head Relative URL: ^/head Repository Root: svn://svn.freebsd.org/base Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 274349 Node Kind: directory Schedule: normal Last Changed Author: emaste Last Changed Rev: 274349 Last Changed Date: 2014-11-10 10:20:46 -0800 (Mon, 10 Nov 2014) Path: /usr/ports Working Copy Root Path: /usr/ports URL: svn://svn.freebsd.org/ports/head Relative URL: ^/head Repository Root: svn://svn.freebsd.org/ports Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5 Revision: 372406 Node Kind: directory Schedule: normal Last Changed Author: pawel Last Changed Rev: 372406 Last Changed Date: 2014-11-10 09:25:04 -0800 (Mon, 10 Nov 2014) src.conf ONLY contains: WITHOUT_CLANG_BOOTSTRAP=true WITHOUT_CLANG=true WITHOUT_CLANG_IS_CC=true make -j6 buildworld fails with the following: --- cddl/lib__L --- --- ddt.So --- cc -fpic -DPIC -O2 -pipe -I/usr/src/cddl/lib/libzpool/../../../sys/cddl/comp at/opensolaris -I/usr/src/cddl/lib/libzpool/../../compat/opensolaris/include -I/ usr/src/cddl/lib/libzpool/../../compat/opensolaris/lib/libumem -I/usr/src/cddl/l ib/libzpool/../../contrib/opensolaris/lib/libzpool/common -I/usr/src/cddl/lib/li bzpool/../../../sys/cddl/contrib/opensolaris/uts/common/sys -I/usr/src/cddl/lib/ libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs -I/usr/src/cddl /lib/libzpool/../../../sys/cddl/contrib/opensolaris/common/zfs -I/usr/src/cddl/l ib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common -I/usr/src/cddl/lib /libzpool/../../contrib/opensolaris/head -I/usr/src/cddl/lib/libzpool/../../lib/ libumem -I/usr/src/cddl/lib/libzpool/../../contrib/opensolaris/lib/libnvpair -DW ANTS_MUTEX_OWNED -I/usr/src/cddl/lib/libzpool/../../../lib/libpthread/thread -I/ usr/src/cddl/lib/libzpool/../../../lib/libpthread/sys -I/usr/src/cddl/lib/libzpo ol/../../../lib/libthr/arch/amd64/include -g -DDEBUG=1 -DNEED_SOLARIS_BOOLEAN -s td=iso9899:1999 -fstack-protector -Wno-pointer-sign -Wno-unknown-pragmas -Wno-em pty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compa re -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-co nversion -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Wno-parenthes es -Qunused-arguments -c /usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/op ensolaris/uts/common/fs/zfs/ddt.c -o ddt.So --- lib__L --- In file included from /usr/src/lib/libdpv/dialog_util.c:43: In file included from /usr/src/lib/libdpv/dialog_util.h:34: /usr/src/lib/libdpv/dialogrc.h:34:10: fatal error: 'figpar.h' file not found #include figpar.h ^ 1 error generated. /usr/src/lib/libdpv/dialogrc.c:34:10: fatal error: 'figpar.h' file not found #include figpar.h ^ --- depend_subdir_libedit --- === lib/libedit (depend) --- depend_subdir_libdpv --- 1 error generated. /usr/src/lib/libdpv/dprompt.c:40:10: fatal error: 'string_m.h' file not found #include string_m.h ^ 1 error generated. --- depend_subdir_libedit --- --- common.h --- sh /usr/src/lib/libedit/makelist -h /usr/src/lib/libedit/common.c common.h --- depend_subdir_libdpv --- /usr/src/lib/libdpv/dpv.c:42:10: fatal error: 'string_m.h' file not found #include string_m.h ^ 1 error generated. --- depend_subdir_libedit --- --- emacs.h --- sh /usr/src/lib/libedit/makelist -h /usr/src/lib/libedit/emacs.c emacs.h --- kerberos5/lib__L --- --- gss_oid.So --- --- lib__L --- --- depend_subdir_libdpv --- In file included from /usr/src/lib/libdpv/status.c:36: In file included from /usr/src/lib/libdpv/dialog_util.h:34: /usr/src/lib/libdpv/dialogrc.h:34:10: fatal error: 'figpar.h' file not found #include figpar.h ^ 1 error generated. --- kerberos5/lib__L --- cc -fpic -DPIC -O2 -pipe
Re: Unable to build world w/o clang on 11
On Mon, Nov 10, 2014 at 06:37:35PM -0800, Chris H wrote: === lib/libdpv (depend) rm -f .depend CC='cc ' mkdep -f .depend -a-I/usr/src/lib/libdpv -std=gnu99 /usr/src/lib /libdpv/dialog_util.c /usr/src/lib/libdpv/dialogrc.c /usr/src/lib/libdpv/dprompt c /usr/src/lib/libdpv/dpv.c /usr/src/lib/libdpv/status.c /usr/src/lib/libdpv/ut il.c In file included from /usr/src/lib/libdpv/dialog_util.c:43: In file included from /usr/src/lib/libdpv/dialog_util.h:34: /usr/src/lib/libdpv/dialogrc.h:34:10: fatal error: 'figpar.h' file not found #include figpar.h ^ 1 error generated. /usr/src/lib/libdpv/dialogrc.c:34:10: fatal error: 'figpar.h' file not found Until such time that you can get the tree to build without -j, don't use it. Now, the obvious question: do you have figpar.h in /usr/src? -- Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Changing timezone without reboot/restarting each service?
On Mon, Nov 10, 2014, at 06:36, Lev Serebryakov wrote: After changing timezones in Russia (with replacing /etc/localtime with new file), I found that cron works in old timezone till restart. And all other services do the same, but cron is most obvious here :) Looks like libc reads timezone only once and it could not be chamged for process without restart (which leads to, effectivly, restart of whole server). Is it known problem? I think, it should be fixed somehow. I understand, that re-check timezone file on each time-related call could be expensive, though :( I think this was one of the crowning achievements of systemd, but I'm sure someone can come up with something much more sane than that to address this problem. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Changing timezone without reboot/restarting each service?
On 2014-11-10 22:28, Mark Felder wrote: On Mon, Nov 10, 2014, at 06:36, Lev Serebryakov wrote: After changing timezones in Russia (with replacing /etc/localtime with new file), I found that cron works in old timezone till restart. And all other services do the same, but cron is most obvious here :) Looks like libc reads timezone only once and it could not be chamged for process without restart (which leads to, effectivly, restart of whole server). Is it known problem? I think, it should be fixed somehow. I understand, that re-check timezone file on each time-related call could be expensive, though :( I think this was one of the crowning achievements of systemd, but I'm sure someone can come up with something much more sane than that to address this problem. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org jkh@ mentioned this specifically when he gave his talk at EuroBSDCon and MeetBSD, about how Apple solved this in LaunchD, because apparently originally libc DID check /etc/localtime constantly. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: Unable to build world w/o clang on 11
On Mon, 10 Nov 2014 19:22:31 -0800 Steve Kargl s...@troutmask.apl.washington.edu wrote On Mon, Nov 10, 2014 at 06:37:35PM -0800, Chris H wrote: === lib/libdpv (depend) rm -f .depend CC='cc ' mkdep -f .depend -a-I/usr/src/lib/libdpv -std=gnu99 /usr/src/lib /libdpv/dialog_util.c /usr/src/lib/libdpv/dialogrc.c /usr/src/lib/libdpv/dprompt c /usr/src/lib/libdpv/dpv.c /usr/src/lib/libdpv/status.c /usr/src/lib/libdpv/ut il.c In file included from /usr/src/lib/libdpv/dialog_util.c:43: In file included from /usr/src/lib/libdpv/dialog_util.h:34: /usr/src/lib/libdpv/dialogrc.h:34:10: fatal error: 'figpar.h' file not found #include figpar.h ^ 1 error generated. /usr/src/lib/libdpv/dialogrc.c:34:10: fatal error: 'figpar.h' file not found Until such time that you can get the tree to build without -j, don't use it. Now, the obvious question: do you have figpar.h in /usr/src? Ugh... it's been a hectic day. Apparently it wasn't included in Revision: 274349 Last Changed Date: 2014-11-10 10:20:46 -0800 (Mon, 10 Nov 2014) :P I'll blow away src, and ports, and checkout what's ever available now. Hoping it's *now* included. ;) Thank you *very* much rubbing my face in the obvious, Steve. :) --Chris -- Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
