libmap.conf vs packages and whatnot
Hello,
turns out that we got some additional cost when executing binaries.
I'm not declaring I'm going to do any work on this, but I'd like to
state why I don't like current approach. It's just my $0,03.
So we got libmap.conf + /usr/local/etc/libmap.d directory.
This results in:
lstat(/etc,{ mode=drwxr-xr-x ,inode=22,size=117,blksize=7680 }) = 0 (0x0)
lstat(/etc/libmap.conf,{ mode=-rw-r--r-- ,inode=384351,size=47,blksize=4096
}) = 0 (0x0)
open(/etc/libmap.conf,O_CLOEXEC,0146) = 3 (0x3)
fstat(3,{ mode=-rw-r--r-- ,inode=384351,size=47,blksize=4096 }) = 0 (0x0)
mmap(0x0,47,PROT_READ,MAP_PRIVATE,3,0x0) = 34366185472 (0x800626000)
close(3) = 0 (0x0)
lstat(/usr,{ mode=drwxr-xr-x ,inode=9,size=17,blksize=4096 }) = 0 (0x0)
lstat(/usr/local,{ mode=drwxr-xr-x ,inode=563,size=17,blksize=4096 }) = 0
(0x0)
lstat(/usr/local/etc,{ mode=drwxr-xr-x ,inode=16407,size=44,blksize=4096 }) =
0 (0x0)
lstat(/usr/local/etc/libmap.d,{ mode=drwxr-xr-x
,inode=12994,size=2,blksize=4096 }) = 0 (0x0)
open(/usr/local/etc/libmap.d,O_NONBLOCK|O_DIRECTORY|O_CLOEXEC,0165) = 3 (0x3)
fstatfs(0x3,0x7fffcaa0,0x0,0x400,0x1000,0x8080808080808080) = 0 (0x0)
getdirentries(0x3,0x800624000,0x1000,0x800623028,0x1000,0x8080808080808080) =
24 (0x18)
getdirentries(0x3,0x800624000,0x1000,0x800623028,0x1000,0x8080808080808080) = 0
(0x0)
close(3) = 0 (0x0)
For each exec, even though in vast majority of cases there is no need
for this. Also note the directory is empty, otherwise this is another
round of lstats + open for each file.
I'm told this will be needed for ports so that they can shuffle libraries
for interested programs. I don't know problem area very well, so please
correct me if this stuff cannot work.
First off, if there is no way around needing to parse data, I would
suggest generating a binary blob which can be mmapped and quickly
traversed.
1st case: we want new lib for each program
just replace a symlink (or better yet, a hardlink) to the old one.
2nd case: only selected programs should get new lib
For each prog in need, one can create /usr/local/etc/libmap.d/${prog}.
And there you go, one open and if it failed there is nothing to do.
Path in question can be a symlink to a bigger file with a set of
remmapped libraries, or possibly can 'include ${some_set}'.
It is unclear to me whether ports tree guarantees unique names of
binaries. If not, maybe there will be an easy way to guarantee lookups
or unique stuff anyway (putting some crap in during compilation/linkage?).
This way the cost for uninterested progs is greatly reduced. Interested
progs also get this faster, as long as there are more than 3 files (as
with current approach one would have to parse them all).
--
Mateusz Guzik mjguzik gmail.com
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: OOM killer and kernel cache reclamation rate limit in vm_pageout_scan()
On Wed, Oct 15, 2014 at 11:56:33PM -0600, Justin T. Gibbs wrote: avg pointed out the rate limiting code in vm_pageout_scan() during discussion about PR 187594. While it certainly can contribute to the problems discussed in that PR, a bigger problem is that it can allow the OOM killer to be triggered even though there is plenty of reclaimable memory available in the system. Any load that can consume enough pages within the polling interval to hit the v_free_min threshold (e.g. multiple 'dd if=/dev/zero of=/file/on/zfs') can make this happen. The product I?m working on does not have swap configured and treats any OOM trigger as fatal, so it is very obvious when this happens. :-) I?ve tried several things to mitigate the problem. The first was to ignore rate limiting for pass 2. However, even though ZFS is guaranteed to receive some feedback prior to OOM being declared, my testing showed that a trivial load (a couple dd operations) could still consume enough of the reclaimed space to leave the system below its target at the end of pass 2. After removing the rate limiting entirely, I?ve so far been unable to kill the system via a ZFS induced load. I understand the motivation behind the rate limiting, but the current implementation seems too simplistic to be safe. The documentation for the Solaris slab allocator provides good motivation for their approach of using a ?sliding average? to reign in temporary bursts of usage without unduly harming efficient service for the recorded steady-state memory demand. Regardless of the approach taken, I believe that the OOM killer must be a last resort and shouldn?t be called when there are caches that can be culled. One other thing I?ve noticed in my testing with ZFS is that it needs feedback and a little time to react to memory pressure. Calling it?s lowmem handler just once isn?t enough for it to limit in-flight writes so it can avoid reuse of pages that it just freed up. But, it doesn?t take too long to react ( 1sec in the profiling I?ve done). Is there a way in vm_pageout_scan() that we can better record that progress is being made (pages were freed in the pass, even if some/all of them were consumed again) and allow more passes before the OOM killer is invoked in this case? ? Justin https://docs.freebsd.org/cgi/getmsg.cgi?fetch=103436+0+/usr/local/www/db/text/2014/freebsd-hackers/20141012.freebsd-hackers might have some relevance. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
kernel page fault with nfs
Hi For some days now I've had problems with my current (last test with r273178M). Sometimes when accessing a nfs-share there is a pagefault: ### Fatal trap 12: page fault while in kernel mode cpuid = 7; apic id = 07 fault virtual address = 0xfe07cfe60400 fault code = supervisor read data, page not present instruction pointer = 0x20:0x80d4d4b6 stack pointer = 0x28:0xfe086088b380 frame pointer = 0x28:0xfe086088b3f0 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 43868 (mplayer) #0 0x80926d29 in shutdown_nice (howto=1) at /usr/src/sys/kern/kern_shutdown.c:207 #1 0x80926a2d in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:444 #2 0x80926f80 in panic (fmt=0x104 Address 0x104 out of bounds) at /usr/src/sys/kern/kern_shutdown.c:698 #3 0x8035f147 in panic_cmd_del (arg=0x0) at /usr/src/sys/ddb/db_command.c:244 #4 0x8035ed5d in db_command (cmd_table=0x0) at /usr/src/sys/ddb/db_command.c:439 #5 0x8035ead4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:488 #6 0x803615e0 in db_trap (type=value optimized out, code=0) at /usr/src/sys/ddb/db_main.c:247 #7 0x80966db1 in kdb_trap (type=12, code=0, tf=0xfe086088b2d0) at /usr/src/sys/kern/subr_kdb.c:626 #8 0x80d4f92c in trap_fatal (frame=0xfe086088b2d0, eva=value optimized out) at /usr/src/sys/amd64/amd64/trap.c:835 #9 0x80d4fcbc in trap_pfault (frame=0xfe086088b2d0, usermode=0) at atomic.h:161 #10 0x80d4f2de in trap (frame=0xfe086088b2d0) at /usr/src/sys/amd64/amd64/trap.c:594 #11 0x80d33822 in Xtss () at /usr/src/sys/amd64/amd64/exception.S:154 #12 0x80d4d4b6 in stack_save_td (st=value optimized out, td=value optimized out) at /usr/src/sys/amd64/amd64/stack_machdep.c:74 #13 0x809f30b2 in foffset_unlock (fp=value optimized out, val=value optimized out, flags=value optimized out) at /usr/src/sys/kern/vfs_vnops.c:700 #14 0x8082faad in ncl_bioread (vp=0xf80201dd7490, uio=0xfe086088b7d8, ioflag=value optimized out, cred=0xf8015816a600) at /usr/src/sys/fs/nfsclient/nfs_clbio.c:511 #15 0x80e64381 in VOP_MARKATIME_APV (vop=value optimized out, a=0xfe086088b650) at vnode_if.c:856 #16 0x809f4dd5 in vn_read (fp=0xf80272490cd0, uio=0xfe086088b7d8, active_cred=0xf8015816a600, flags=128, td=0xf800) at vnode_if.h:859 #17 0x809f5502 in get_advice (fp=0xfe086088b730, uio=0x400) at /usr/src/sys/kern/vfs_vnops.c:729 #18 0x809f2b80 in vn_io_fault1 () at /usr/src/sys/kern/vfs_vnops.c:1058 #19 0x809f0d3b in vn_io_fault (fp=0xf80272490cd0, uio=0xfe086088b970, active_cred=0x400, flags=128, td=0xf800) at /usr/src/sys/kern/vfs_vnops.c:128 #20 0x80981d95 in freebsd6_pread (td=0xf802d93204a8, uap=0xf9fb094c00a8) at /usr/src/sys/kern/sys_generic.c:217 #21 0x80981ab8 in sys_cap_fcntls_get (td=value optimized out, uap=0x800) at /usr/src/sys/kern/sys_capability.c:576 #22 0x80981a43 in sys_cap_fcntls_get (td=value optimized out, uap=0x0) at sx.h:183 #23 0x80d503cb in amd64_syscall (td=0x45e400, traced=0) at subr_syscall.c:87 #24 0x80d33b0b in Xprot () at /usr/src/sys/amd64/amd64/exception.S:324 #25 0x000806a7fe6a in ?? () ## Any hints would be appreciated. Please let me know if further information is required. Thanks in advance, Tobias ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: HOWTO articles for migrating from Linux to FreeBSD, especially for pkg?
On Fri, Aug 15, 2014 at 12:01 PM, Craig Rodrigues rodr...@freebsd.org wrote: On Wed, Aug 13, 2014 at 6:34 PM, Russell L. Carter rcar...@pinyon.org wrote: One complaint I have about the FreeBSD project, is that the core project contributors and developers rely too much on e-mail for communication. This certainly works, and I use it too, but new and casual users getting into FreeBSD may get lost in the maze of FreeBSD mailing lists. It would be nice if more of the core project contributors used the web forums ( http://forums.freebsd.org ), since stuff like that shows up nicely in web searches, and it is easier for newcomers to find stuff, and jump in and contribute to threads, versus mailing lists. I just noticed this: https://forums.freebsd.org/threads/new-forums-software-online.48495/ The FreeBSD Forums Administration team just upgraded the forum software from phpBB to Xenforo. They migrated 260,000 messages..wow, this is a huge achievement! Xenforo is nice stuff, so hopefully more FreeBSD core developers can take advantage of the forums, in addition to the traditional mailing lists. -- Craig ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
ssh None cipher
Whilst trying to replicate data from my FreeNAS to my FreeBSD home theater PC on my local LAN, I came across this bug preventing use of the None cipher: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=163127 I think I could enable the None cipher by recompiling base with a flag in /etc/src.conf. Is there any harm in enabling this by default, but having the None cipher remain disabled in /etc/ssh/sshd_config? That way people wouldn't have it on my default, but wouldn't have to recompile to enable it. Thoughts? -- -- From: Benjamin Woods woods...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: ssh None cipher
On Fri, 17 Oct 2014, Ben Woods wrote: Whilst trying to replicate data from my FreeNAS to my FreeBSD home theater PC on my local LAN, I came across this bug preventing use of the None cipher: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=163127 I think I could enable the None cipher by recompiling base with a flag in /etc/src.conf. I agree. Is there any harm in enabling this by default, but having the None cipher remain disabled in /etc/ssh/sshd_config? That way people wouldn't have it on my default, but wouldn't have to recompile to enable it. I do not see any immediate and concrete harm that doing so would cause, yet that is insufficient for me to think that doing so would be a good idea. -Ben ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: ssh None cipher
On 2014-10-17 22:43, Benjamin Kaduk wrote: On Fri, 17 Oct 2014, Ben Woods wrote: Whilst trying to replicate data from my FreeNAS to my FreeBSD home theater PC on my local LAN, I came across this bug preventing use of the None cipher: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=163127 I think I could enable the None cipher by recompiling base with a flag in /etc/src.conf. I agree. Is there any harm in enabling this by default, but having the None cipher remain disabled in /etc/ssh/sshd_config? That way people wouldn't have it on my default, but wouldn't have to recompile to enable it. I do not see any immediate and concrete harm that doing so would cause, yet that is insufficient for me to think that doing so would be a good idea. -Ben ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org I've been using openssh-portable from ports with the none cipher patch to get around this. IIRC, upstream openssh refuses to merge the none cipher patches because you shouldn't do that. But I'd vote for having it compiled in and just disabled by default. It will refuse to let you have a shell without encryption, and prints a big fat hairy warning when encryption is disabled. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: HOWTO articles for migrating from Linux to FreeBSD, especially for pkg?
Hi, On Fri, 15 Aug 2014 12:01:30 -0700 Craig Rodrigues rodr...@freebsd.org wrote: On Wed, Aug 13, 2014 at 6:34 PM, Russell L. Carter rcar...@pinyon.org wrote: One complaint I have about the FreeBSD project, is that the core project contributors and developers rely too much on e-mail for communication. This certainly works, and I use it too, but new and casual users getting into FreeBSD may get lost in the maze of FreeBSD mailing lists. It would be nice if more of the core project contributors mailing lists have two advantages I would not like to miss. A) people can create their own archive and use it when they are offline B) Google Co. offer good search tools to search the online archives used the web forums ( http://forums.freebsd.org ), since stuff like I also noticed while travelling that some locations block the forum. Ok, it is rare. that shows up nicely in web searches, and it is easier This is also true for the e-mails. Erich ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
