Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 08:13:52 PM Ngie Cooper wrote:
> > On Apr 14, 2017, at 20:12, Peter Wemm  wrote:
> > 
> > On Friday, April 14, 2017 07:36:55 PM Peter Wemm wrote:
> >> On Friday, April 14, 2017 02:14:16 PM Ngie Cooper wrote:
>  On Apr 14, 2017, at 14:10, Maxim Sobolev  wrote:
>  
>  Peter, Ngie,
>  
>  Looks like out of that refactoring came a logical bug that is present
>  in
>  the head, which causes syslod to first to shutdown the socket for
>  reading
>  and then try to select/recv on it (which is somewhat stupid). And that
>  issue has been masked by shutdown() on datagram socket becoming
>  effectively a NOP in 11 & head 20 months ago. It only affects head
>  though, 11-stable still has the old code which does not include that
>  half-closed socket into the select list. Attached patch is expected to
>  fix head, Peter, it would be nice if you can give it a try (restoring
>  latest changes into uipc_sockets.c) and let me know if it helps.
>  
>  Thanks!
> >>> 
> >>> CCing hrs@ for input as he did the refactoring.
> >>> Thanks!
> >>> -Ngie
> >>> 
> >>> PS LGTM with the change. Will wait for feedback from wemm@.
> >> 
> >> This is definitely not working.  I get ENOSPC  and listen queue overflows
> >> on /var/run/logpriv now.
> >> 
> >> Grabbing an old 10.3 /usr/sbin/syslogd and placing it on the top of the
> >> 12.x one worked fine, aside from the include statements.
> > 
> > This can't be right:
> > if (SecureMode || res->ai_family == AF_LOCAL) {
> > 
> >/* Forbid communication in secure mode. */
> >if (shutdown(s, SHUT_RD) < 0 &&
> >
> >errno != ENOTCONN) {
> >
> >logerror("shutdown");
> >if (!Debug)
> >
> >die(0);
> >
> >}
> >dprintf("listening on socket\n");
> >sl_recv = NULL;
> >   
> >   }
> > 
> > This appears to disable unix domain sockets like /var/run/log and
> > /var/run/logpriv.
> 
> ACK. This looks like a fun bug.

> -Ngie

I suspect it's meant to be "if (SecureMode && res->ai_family != AF_LOCAL) {" 
as a simple logic inversion error of another line earlier.  However there's an 
awful lot of strange things in this code.

1) listen(s, 5) - on datagram sockets.
2) dprintf("shutdown") in code regardless of whether the shutdown is going to 
happen.
3) dprintf("listening on socket") in code that only happens when we're NOT 
going to listen.
4) dprintf("sending on socket") in the code path when we're going to listen.
5) shutdown on all unix domain sockets, regardless of securemode..

This code block makes my head spin.

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316953 - head/sbin/savecore

[Rodney W. Grimes]

> Author: ngie
> Date: Sat Apr 15 06:53:07 2017
> New Revision: 316953
> URL: https://svnweb.freebsd.org/changeset/base/316953
> 
> Log:
>   Switch back to non-IEC units for 1024 bytes
>   
>   I was swayed a little too quickly when I saw the wiki page discussing
>   kB vs KiB. Switch back as none of the code in base openly uses
>   IEC units via humanize_number(3) (which was my next step), and there's
>   a large degree of dislike with IEC vs more SI-like units.
>   
>   MFC after:  7 weeks
>   Submitted by:   jhb, rgrimes
>   Sponsored by:   Dell EMC Isilon
> 
> Modified:
>   head/sbin/savecore/savecore.c
> 
> Modified: head/sbin/savecore/savecore.c
> ==
> --- head/sbin/savecore/savecore.c Sat Apr 15 03:28:13 2017
> (r316952)
> +++ head/sbin/savecore/savecore.c Sat Apr 15 06:53:07 2017
> (r316953)
> @@ -322,7 +322,7 @@ check_space(const char *savedir, off_t d
>   if (available < needed) {
>   syslog(LOG_WARNING,
>   "no dump: not enough free space on device (need at least "
> - "%jdKiB for dump; %jdKiB available; %jdKiB reserved)",
> + "%jdkB for dump; %jdkB available; %jdkB reserved)",
>   (intmax_t)needed,
>   (intmax_t)available + minfree,
>   (intmax_t)minfree);
Thank you!

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316953 - head/sbin/savecore

[Ngie Cooper]

Author: ngie
Date: Sat Apr 15 06:53:07 2017
New Revision: 316953
URL: https://svnweb.freebsd.org/changeset/base/316953

Log:
  Switch back to non-IEC units for 1024 bytes
  
  I was swayed a little too quickly when I saw the wiki page discussing
  kB vs KiB. Switch back as none of the code in base openly uses
  IEC units via humanize_number(3) (which was my next step), and there's
  a large degree of dislike with IEC vs more SI-like units.
  
  MFC after:7 weeks
  Submitted by: jhb, rgrimes
  Sponsored by: Dell EMC Isilon

Modified:
  head/sbin/savecore/savecore.c

Modified: head/sbin/savecore/savecore.c
==
--- head/sbin/savecore/savecore.c   Sat Apr 15 03:28:13 2017
(r316952)
+++ head/sbin/savecore/savecore.c   Sat Apr 15 06:53:07 2017
(r316953)
@@ -322,7 +322,7 @@ check_space(const char *savedir, off_t d
if (available < needed) {
syslog(LOG_WARNING,
"no dump: not enough free space on device (need at least "
-   "%jdKiB for dump; %jdKiB available; %jdKiB reserved)",
+   "%jdkB for dump; %jdkB available; %jdkB reserved)",
(intmax_t)needed,
(intmax_t)available + minfree,
(intmax_t)minfree);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> On Fri, Apr 14, 2017 at 06:30:17PM -0700, Rodney W. Grimes wrote:
> > > The patch to add compression support is here and should largely still
> > > work:
> > > https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff
> > > 
> > > I've been hesitant about pushing it forward:
> > > - The dump_write* APIs need some simplification after the addition of
> > >   encrypted dump support and support for dumping to 4Kn drives.
> > > - I'm not sure how encryption should compose with compression. It seems
> > >   intuitively obvious that we should compress before encrypting if the
> > >   compression is to be of any use, but I don't know enough to know
> > >   whether the compression might somehow compromise the effectiveness of
> > >   the encryption.
> > > 
> > > If anyone has some insight on the second of these two points, I'd
> > > appreciate hearing it.
> > 
> > I have a large amount of reworking and modulization of the dump code
> > incuding intergration of your (markj) compressed dumps.  Layer isnt
> > implemented but is in the plan.  I should not of held off on the net
> > dump code as it got smashed by encrypted dumps, then again by
> > the libif'ing for all the Intel drives that had been netdump modified.
> > 
> > Basically now starting over :-(
> 
> Could you post your patches somewhere? I've been sitting on this (and
> the netdump patches, for which I have quite a few modifications) for far
> too long, and would like to finish them and get them in soon. I'll note
> that the netdump code posted a few years ago had some problems that are
> fixed in Isilon's version, which I'm working on rebasing on HEAD. In

Isnt the code I rebaed in December to -12 your code from Isiolon
Or tell me you handed me your patches to upstream, and then continued
to evolved the code without letting me know?   Are YOUR patckes some
place public?  I dont think mine are, but I do have a link in 
http://people.freebsd.org/~rgrimes pointing to the other Netdump version
that I think is an old version of your code.

> particular, I simplified the driver integration a bit, changed the code
> to avoid allocating mbufs from UMA after a panic, plumbed a
> configuration interface through dumpon, and fixed some problems with
> netdumpd. I'm working on integrating netdump with Isilon's internal
> infrastructure at the moment. The conversion of em and igb to iflib does
> complicate things a bit; I haven't yet looked at how hard it would be
> to support netdump in iflib'ed drivers.
> 
> > Minidump is an lkm for me, and main dump is almost an lkm for me too.
> 
> Does "lkm" mean "loadable kernel module"? If so, why?

Yes, so you dont have to reboot to write and debug  new versions, so
you can have a kernel without minidump if you want, and I am sure
there are others.  More importantly, why not?  Modules are good,
staticially linked rarely used code is bad.

I also have a version of minidump that can be asked, if I panicked
right now how big would hte dump be?

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Mark Johnston]

On Fri, Apr 14, 2017 at 07:00:36PM -0700, Conrad Meyer wrote:
> Larry,
> 
> You just need to run netdumpd on the nearby server.  It could be a
> port (although I'm not aware that it is ported yet).

I was working on a port makefile for netdumpd before this thread
started. :)

It would be a bit more usable as a port; I prefer it that way since
Isilon's infrastructure folks would like to run netdumpd on 11.0.
For what it's worth, I've got my revision of the code here:
https://github.com/markjdb/netdumpd

This version fixes some performance problems in the original, and runs
most of the code in capability mode. The capsicum integration hasn't
yet been reviewed, however.

> 
> Best,
> Conrad
> 
> On Fri, Apr 14, 2017 at 6:37 PM, Larry Rosenman  wrote:
> > On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
> > wrote:
> >
> >> Yeah, I have the following:
> > > borg.lerctr.org /home/ler $ swapctl -l
> > > Device:   1024-blocks Used:
> > > /dev/mfid0p38388608 0
> > > /dev/mfid1p38388608 0
> > > /dev/mfid2p38388608 0
> > > /dev/mfid3p38388608 0
> > > /dev/mfid4p38388608 0
> > > /dev/mfid5p38388608 0
> > > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > > hw.physmem: 137368682496
> > > borg.lerctr.org /home/ler $
> > >
> > > SO 6 8G partitions (48G), but the dump is larger than 8G.
> >
> > Larry,
> >   This is a very good concern and point given todays more
> > common huge memory foot prints and lots of spindles.  I'll
> > keep this in they back of my mind as I tromp around in the
> > dump code.  I have another solution that may work for you
> > and that is to use Netdump rather than swapdump.  This
> > basically eliminates the trip to swap space and you end
> > up going to savecore style output on the netdump server.
> >
> > --
> > Rod Grimes 
> > rgri...@freebsd.org
> >
> > What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
> > since that’s what is “next to” this server?
> >
> >
> >
> 
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Mark Johnston]

On Fri, Apr 14, 2017 at 06:30:17PM -0700, Rodney W. Grimes wrote:
> > The patch to add compression support is here and should largely still
> > work:
> > https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff
> > 
> > I've been hesitant about pushing it forward:
> > - The dump_write* APIs need some simplification after the addition of
> >   encrypted dump support and support for dumping to 4Kn drives.
> > - I'm not sure how encryption should compose with compression. It seems
> >   intuitively obvious that we should compress before encrypting if the
> >   compression is to be of any use, but I don't know enough to know
> >   whether the compression might somehow compromise the effectiveness of
> >   the encryption.
> > 
> > If anyone has some insight on the second of these two points, I'd
> > appreciate hearing it.
> 
> I have a large amount of reworking and modulization of the dump code
> incuding intergration of your (markj) compressed dumps.  Layer isnt
> implemented but is in the plan.  I should not of held off on the net
> dump code as it got smashed by encrypted dumps, then again by
> the libif'ing for all the Intel drives that had been netdump modified.
> 
> Basically now starting over :-(

Could you post your patches somewhere? I've been sitting on this (and
the netdump patches, for which I have quite a few modifications) for far
too long, and would like to finish them and get them in soon. I'll note
that the netdump code posted a few years ago had some problems that are
fixed in Isilon's version, which I'm working on rebasing on HEAD. In
particular, I simplified the driver integration a bit, changed the code
to avoid allocating mbufs from UMA after a panic, plumbed a
configuration interface through dumpon, and fixed some problems with
netdumpd. I'm working on integrating netdump with Isilon's internal
infrastructure at the moment. The conversion of em and igb to iflib does
complicate things a bit; I haven't yet looked at how hard it would be
to support netdump in iflib'ed drivers.

> Minidump is an lkm for me, and main dump is almost an lkm for me too.

Does "lkm" mean "loadable kernel module"? If so, why?
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

[ Charset UTF-8 unsupported, converting... ]
> 
> > On Apr 14, 2017, at 20:05, Rodney W. Grimes 
> >  wrote:
> > 
> >>> 
> >>> On Apr 14, 2017, at 18:49, Rodney W. Grimes 
> >>>  wrote:
> >>> 
>  On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> > 
> > Log:
> > savecore: fix space calculation with respect to `minfree` in 
> > check_space(..)
> > 
> > - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> >   representable data to INT_MAX. Check the values received from
> >   strtoll(3), trimming trailing whitespace off the end to maintain
> >   POLA.
> > - Use `KiB` instead of `kB` when describing free space, total space,
> >   etc. I am now fully aware of `KiB` being the IEC standard for 1024
> >   bytes and `kB` being the IEC standard for 1000 bytes.
>  
>  I will just rant lightly that no one actually uses this in the real 
>  world.
>  
>  Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
>  kilobyte is a power of 2.  The End.
>  
>  (Next up we'll have to rename 4k displays to
>  4k)
> >>> 
> >>> Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
> >>> a place we do this, so please, lets not start doing this here?
> >> 
> >> humanize_number(3) from libutil uses IEC units.
> > 
> > And how many things bother to use this library function?  Do the
> > ones that do call it produce the traditional output that has been
> > around for 40 years?
> > 
> >>> Yes, these are newer standards, perhaps some day we should make a global
> >>> switch to them, but lets not start mixing and matching things.
> >> 
> >> I understand and agree. I?m not 100% sold on that one way or another, but
> >> since I was going to redo the number representation in save core with
> >> humanize_number(3), because reading `KiB` is not ideal
> >^^^
> > I hope we are not already reading KiB anyplace?.
> 
> I meant it?s a lot harder for humans to read `KiB`
> instead of ``.

And I am agreeing on the scaled, and disagreeing on the KiB, another words
please revert to printing KB as the former code did as far as a Unit
goes.  Lets not use Ki Mi Gi, etc.

> >> usability wise, and I don?t want to reinvent the wheel normalizing numbers
> >> and printing out the unit.
> >> 
> >> Perhaps there should be a flag baked into humanize_number, etc for parsing 
> >> IEC vs non-IEC unit values?
> > 
> > I dont think it parses anything, but as far as producing strings from values
> > it already has an IEC flag: HN_IEC_PREFIXES, please lets not use this flag,
> > and if we are using it anyplace lets see if we can remove that use.
> 
> I don?t see it used anywhere in the tree, based on a quick grep.

YEA!!!  Thanks for doing the search.

> > Also be careful, this function only accepts signed int 64, which means
> > we are not gona be able to use this in all places that probably need
> > this, so perhaps a larger can of paint for a bigger bike shead is needed?
> 
> I don?t necessarily follow the above statement 100%. Are you warning
> against mass-conversion to libutil (if so, I agree? this was just a case
> where it really helps readability in savecore(8))?

Yes, I am warning against mass conversion, or for that that matter much
conversion to humanize_number because it shall need to be revamped if
it is ever to become a real solutions due to it being a very narrow
minded function (no unsigned implementation for very large values
that are probably the ones that could benifit the most from this
type of function.) 

You also may want to still report full sizes in output (you maybe
I have not read the final code:)
>From kernel boot:
real memory  = 134217728 (128 MB)
avail memory = 120832000 (115 MB)

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316952 - head/share/mk

[Simon J. Gerraty]

Author: sjg
Date: Sat Apr 15 03:28:13 2017
New Revision: 316952
URL: https://svnweb.freebsd.org/changeset/base/316952

Log:
  Latest version, we need to realpath anyway to check if an obj
  dependency is also a src dependency.

Modified:
  head/share/mk/meta2deps.py

Modified: head/share/mk/meta2deps.py
==
--- head/share/mk/meta2deps.py  Sat Apr 15 02:24:22 2017(r316951)
+++ head/share/mk/meta2deps.py  Sat Apr 15 03:28:13 2017(r316952)
@@ -492,6 +492,21 @@ class MetaFile:
 if not file:
 f.close()
 
+def is_src(self, base, dir, rdir):
+"""is base in srctop"""
+for dir in [dir,rdir]:
+if not dir:
+continue
+path = '/'.join([dir,base])
+srctop = self.find_top(path, self.srctops)
+if srctop:
+if self.dpdeps:
+self.add(self.file_deps, path.replace(srctop,''), 'file')
+self.add(self.src_deps, dir.replace(srctop,''), 'src')
+self.seenit(dir)
+return True
+return False
+
 def parse_path(self, path, cwd, op=None, w=[]):
 """look at a path for the op specified"""
 
@@ -520,10 +535,9 @@ class MetaFile:
 # to the src dir, we may need to add dependencies for each
 rdir = dir
 dir = abspath(dir, cwd, self.last_dir, self.debug, self.debug_out)
-if rdir == dir or rdir.find('./') > 0:
-rdir = None
-if os.path.islink(dir):
 rdir = os.path.realpath(dir)
+if rdir == dir:
+rdir = None
 # now put path back together
 path = '/'.join([dir,base])
 if self.debug > 1:
@@ -544,17 +558,9 @@ class MetaFile:
 # finally, we get down to it
 if dir == self.cwd or dir == self.curdir:
 return
-srctop = self.find_top(path, self.srctops)
-if srctop:
-if self.dpdeps:
-self.add(self.file_deps, path.replace(srctop,''), 'file')
-self.add(self.src_deps, dir.replace(srctop,''), 'src')
+if self.is_src(base, dir, rdir):
 self.seenit(w[2])
-self.seenit(dir)
-if rdir and not rdir.startswith(srctop):
-dir = rdir  # for below
-rdir = None
-else:
+if not rdir:
 return
 
 objroot = None
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> 
> > On Apr 14, 2017, at 19:40, Ngie Cooper (yaneurabeya) 
> >  wrote:
> > 
> >> 
> >> On Apr 14, 2017, at 18:49, Rodney W. Grimes 
> >>  wrote:
> >> 
> >>> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
>  Author: ngie
>  Date: Fri Apr 14 19:41:48 2017
>  New Revision: 316938
>  URL: https://svnweb.freebsd.org/changeset/base/316938
>  
>  Log:
>  savecore: fix space calculation with respect to `minfree` in 
>  check_space(..)
>  
>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>    representable data to INT_MAX. Check the values received from
>    strtoll(3), trimming trailing whitespace off the end to maintain
>    POLA.
>  - Use `KiB` instead of `kB` when describing free space, total space,
>    etc. I am now fully aware of `KiB` being the IEC standard for 1024
>    bytes and `kB` being the IEC standard for 1000 bytes.
> >>> 
> >>> I will just rant lightly that no one actually uses this in the real world.
> >>> 
> >>> Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
> >>> kilobyte is a power of 2.  The End.
> >>> 
> >>> (Next up we'll have to rename 4k displays to
> >>> 4k)
> >> 
> >> Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
> >> a place we do this, so please, lets not start doing this here?
> > 
> > humanize_number(3) from libutil uses IEC units.
> > 
> >> Yes, these are newer standards, perhaps some day we should make a global
> >> switch to them, but lets not start mixing and matching things.
> > 
> > I understand and agree. I?m not 100% sold on that one way or another, but 
> > since I was going to redo the number representation in save core with 
> > humanize_number(3), because reading `KiB` is not ideal 
> > usability wise, and I don?t want to reinvent the wheel normalizing numbers 
> > and printing out the unit.
> 
> *unit. ?> *unit, KiB seemed like a logical next step after discussing it at 
> long length in the CR.

I do not see any long length discussion in the CR. I see an assertion that
KiB is the new IEC standard, I see a miss understanding of what kB and KB
means, and I see someone wanting to push the tree towards IEC.

I see here on the mailling list a push to NOT start IEC'ing things, at
least by myself and jhb.

I would not consider 2 or 3 peole have a 15 line converstaion in a CR
as reasonable grounds for a change like this.

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 20:12, Peter Wemm  wrote:
> 
> On Friday, April 14, 2017 07:36:55 PM Peter Wemm wrote:
>> On Friday, April 14, 2017 02:14:16 PM Ngie Cooper wrote:
 On Apr 14, 2017, at 14:10, Maxim Sobolev  wrote:
 
 Peter, Ngie,
 
 Looks like out of that refactoring came a logical bug that is present in
 the head, which causes syslod to first to shutdown the socket for
 reading
 and then try to select/recv on it (which is somewhat stupid). And that
 issue has been masked by shutdown() on datagram socket becoming
 effectively a NOP in 11 & head 20 months ago. It only affects head
 though, 11-stable still has the old code which does not include that
 half-closed socket into the select list. Attached patch is expected to
 fix head, Peter, it would be nice if you can give it a try (restoring
 latest changes into uipc_sockets.c) and let me know if it helps.
 
 Thanks!
>>> 
>>> CCing hrs@ for input as he did the refactoring.
>>> Thanks!
>>> -Ngie
>>> 
>>> PS LGTM with the change. Will wait for feedback from wemm@.
>> 
>> This is definitely not working.  I get ENOSPC  and listen queue overflows on
>> /var/run/logpriv now.
>> 
>> Grabbing an old 10.3 /usr/sbin/syslogd and placing it on the top of the 12.x
>> one worked fine, aside from the include statements.
> 
> This can't be right:
> if (SecureMode || res->ai_family == AF_LOCAL) {
>/* Forbid communication in secure mode. */
>if (shutdown(s, SHUT_RD) < 0 &&
>errno != ENOTCONN) {
>logerror("shutdown");
>if (!Debug)
>die(0);
>}
>dprintf("listening on socket\n");
>sl_recv = NULL;
>   }
> 
> This appears to disable unix domain sockets like /var/run/log and
> /var/run/logpriv.

ACK. This looks like a fun bug.

-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 07:36:55 PM Peter Wemm wrote:
> On Friday, April 14, 2017 02:14:16 PM Ngie Cooper wrote:
> > > On Apr 14, 2017, at 14:10, Maxim Sobolev  wrote:
> > > 
> > > Peter, Ngie,
> > > 
> > > Looks like out of that refactoring came a logical bug that is present in
> > > the head, which causes syslod to first to shutdown the socket for
> > > reading
> > > and then try to select/recv on it (which is somewhat stupid). And that
> > > issue has been masked by shutdown() on datagram socket becoming
> > > effectively a NOP in 11 & head 20 months ago. It only affects head
> > > though, 11-stable still has the old code which does not include that
> > > half-closed socket into the select list. Attached patch is expected to
> > > fix head, Peter, it would be nice if you can give it a try (restoring
> > > latest changes into uipc_sockets.c) and let me know if it helps.
> > > 
> > > Thanks!
> > 
> > CCing hrs@ for input as he did the refactoring.
> > Thanks!
> > -Ngie
> > 
> > PS LGTM with the change. Will wait for feedback from wemm@.
> 
> This is definitely not working.  I get ENOSPC  and listen queue overflows on
> /var/run/logpriv now.
> 
> Grabbing an old 10.3 /usr/sbin/syslogd and placing it on the top of the 12.x
> one worked fine, aside from the include statements.

This can't be right:
 if (SecureMode || res->ai_family == AF_LOCAL) {
/* Forbid communication in secure mode. */
if (shutdown(s, SHUT_RD) < 0 &&
errno != ENOTCONN) {
logerror("shutdown");
if (!Debug)
die(0);
}
dprintf("listening on socket\n");
sl_recv = NULL;
   }

This appears to disable unix domain sockets like /var/run/log and 
/var/run/logpriv.

--
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 20:05, Rodney W. Grimes  
> wrote:
> 
>>> 
>>> On Apr 14, 2017, at 18:49, Rodney W. Grimes 
>>>  wrote:
>>> 
 On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
> 
> Log:
> savecore: fix space calculation with respect to `minfree` in 
> check_space(..)
> 
> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>   representable data to INT_MAX. Check the values received from
>   strtoll(3), trimming trailing whitespace off the end to maintain
>   POLA.
> - Use `KiB` instead of `kB` when describing free space, total space,
>   etc. I am now fully aware of `KiB` being the IEC standard for 1024
>   bytes and `kB` being the IEC standard for 1000 bytes.
 
 I will just rant lightly that no one actually uses this in the real world.
 
 Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
 kilobyte is a power of 2.  The End.
 
 (Next up we'll have to rename 4k displays to
 4k)
>>> 
>>> Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
>>> a place we do this, so please, lets not start doing this here?
>> 
>> humanize_number(3) from libutil uses IEC units.
> 
> And how many things bother to use this library function?  Do the
> ones that do call it produce the traditional output that has been
> around for 40 years?
> 
>>> Yes, these are newer standards, perhaps some day we should make a global
>>> switch to them, but lets not start mixing and matching things.
>> 
>> I understand and agree. I?m not 100% sold on that one way or another, but
>> since I was going to redo the number representation in save core with
>> humanize_number(3), because reading `KiB` is not ideal
>^^^
> I hope we are not already reading KiB anyplace….

I meant it’s a lot harder for humans to read `KiB` instead of 
``.

>> usability wise, and I don?t want to reinvent the wheel normalizing numbers
>> and printing out the unit.
>> 
>> Perhaps there should be a flag baked into humanize_number, etc for parsing 
>> IEC vs non-IEC unit values?
> 
> I dont think it parses anything, but as far as producing strings from values
> it already has an IEC flag: HN_IEC_PREFIXES, please lets not use this flag,
> and if we are using it anyplace lets see if we can remove that use.

I don’t see it used anywhere in the tree, based on a quick grep.

> Also be careful, this function only accepts signed int 64, which means
> we are not gona be able to use this in all places that probably need
> this, so perhaps a larger can of paint for a bigger bike shead is needed?

I don’t necessarily follow the above statement 100%. Are you warning against 
mass-conversion to libutil (if so, I agree… this was just a case where it 
really helps readability in savecore(8))?

Thanks!
-Ngie



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> 
> > On Apr 14, 2017, at 18:49, Rodney W. Grimes 
> >  wrote:
> > 
> >> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
> >>> Author: ngie
> >>> Date: Fri Apr 14 19:41:48 2017
> >>> New Revision: 316938
> >>> URL: https://svnweb.freebsd.org/changeset/base/316938
> >>> 
> >>> Log:
> >>>  savecore: fix space calculation with respect to `minfree` in 
> >>> check_space(..)
> >>> 
> >>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> >>>representable data to INT_MAX. Check the values received from
> >>>strtoll(3), trimming trailing whitespace off the end to maintain
> >>>POLA.
> >>>  - Use `KiB` instead of `kB` when describing free space, total space,
> >>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
> >>>bytes and `kB` being the IEC standard for 1000 bytes.
> >> 
> >> I will just rant lightly that no one actually uses this in the real world.
> >> 
> >> Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
> >> kilobyte is a power of 2.  The End.
> >> 
> >> (Next up we'll have to rename 4k displays to
> >> 4k)
> > 
> > Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
> > a place we do this, so please, lets not start doing this here?
> 
> humanize_number(3) from libutil uses IEC units.

And how many things bother to use this library function?  Do the
ones that do call it produce the traditional output that has been
around for 40 years?

> > Yes, these are newer standards, perhaps some day we should make a global
> > switch to them, but lets not start mixing and matching things.
> 
> I understand and agree. I?m not 100% sold on that one way or another, but
> since I was going to redo the number representation in save core with
> humanize_number(3), because reading `KiB` is not ideal
^^^
I hope we are not already reading KiB anyplace

> usability wise, and I don?t want to reinvent the wheel normalizing numbers
> and printing out the unit.
> 
> Perhaps there should be a flag baked into humanize_number, etc for parsing 
> IEC vs non-IEC unit values?

I dont think it parses anything, but as far as producing strings from values
it already has an IEC flag: HN_IEC_PREFIXES, please lets not use this flag,
and if we are using it anyplace lets see if we can remove that use.

Also be careful, this function only accepts signed int 64, which means
we are not gona be able to use this in all places that probably need
this, so perhaps a larger can of paint for a bigger bike shead is needed?

> Thanks for the input :)!
input++;  

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 19:40, Ngie Cooper (yaneurabeya)  
> wrote:
> 
>> 
>> On Apr 14, 2017, at 18:49, Rodney W. Grimes  
>> wrote:
>> 
>>> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
 Author: ngie
 Date: Fri Apr 14 19:41:48 2017
 New Revision: 316938
 URL: https://svnweb.freebsd.org/changeset/base/316938
 
 Log:
 savecore: fix space calculation with respect to `minfree` in 
 check_space(..)
 
 - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
   representable data to INT_MAX. Check the values received from
   strtoll(3), trimming trailing whitespace off the end to maintain
   POLA.
 - Use `KiB` instead of `kB` when describing free space, total space,
   etc. I am now fully aware of `KiB` being the IEC standard for 1024
   bytes and `kB` being the IEC standard for 1000 bytes.
>>> 
>>> I will just rant lightly that no one actually uses this in the real world.
>>> 
>>> Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
>>> kilobyte is a power of 2.  The End.
>>> 
>>> (Next up we'll have to rename 4k displays to
>>> 4k)
>> 
>> Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
>> a place we do this, so please, lets not start doing this here?
> 
> humanize_number(3) from libutil uses IEC units.
> 
>> Yes, these are newer standards, perhaps some day we should make a global
>> switch to them, but lets not start mixing and matching things.
> 
> I understand and agree. I’m not 100% sold on that one way or another, but 
> since I was going to redo the number representation in save core with 
> humanize_number(3), because reading `KiB` is not ideal 
> usability wise, and I don’t want to reinvent the wheel normalizing numbers 
> and printing out the unit.

*unit. —> *unit, KiB seemed like a logical next step after discussing it at 
long length in the CR.

-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 18:49, Rodney W. Grimes  
> wrote:
> 
>> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
>>> Author: ngie
>>> Date: Fri Apr 14 19:41:48 2017
>>> New Revision: 316938
>>> URL: https://svnweb.freebsd.org/changeset/base/316938
>>> 
>>> Log:
>>>  savecore: fix space calculation with respect to `minfree` in 
>>> check_space(..)
>>> 
>>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>>representable data to INT_MAX. Check the values received from
>>>strtoll(3), trimming trailing whitespace off the end to maintain
>>>POLA.
>>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>>bytes and `kB` being the IEC standard for 1000 bytes.
>> 
>> I will just rant lightly that no one actually uses this in the real world.
>> 
>> Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
>> kilobyte is a power of 2.  The End.
>> 
>> (Next up we'll have to rename 4k displays to
>> 4k)
> 
> Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
> a place we do this, so please, lets not start doing this here?

humanize_number(3) from libutil uses IEC units.

> Yes, these are newer standards, perhaps some day we should make a global
> switch to them, but lets not start mixing and matching things.

I understand and agree. I’m not 100% sold on that one way or another, but since 
I was going to redo the number representation in save core with 
humanize_number(3), because reading `KiB` is not ideal 
usability wise, and I don’t want to reinvent the wheel normalizing numbers and 
printing out the unit.

Perhaps there should be a flag baked into humanize_number, etc for parsing IEC 
vs non-IEC unit values?

Thanks for the input :)!
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 02:14:16 PM Ngie Cooper wrote:
> > On Apr 14, 2017, at 14:10, Maxim Sobolev  wrote:
> > 
> > Peter, Ngie,
> > 
> > Looks like out of that refactoring came a logical bug that is present in
> > the head, which causes syslod to first to shutdown the socket for reading
> > and then try to select/recv on it (which is somewhat stupid). And that
> > issue has been masked by shutdown() on datagram socket becoming
> > effectively a NOP in 11 & head 20 months ago. It only affects head
> > though, 11-stable still has the old code which does not include that
> > half-closed socket into the select list. Attached patch is expected to
> > fix head, Peter, it would be nice if you can give it a try (restoring
> > latest changes into uipc_sockets.c) and let me know if it helps.
> > 
> > Thanks!
> 
> CCing hrs@ for input as he did the refactoring.
> Thanks!
> -Ngie
> 
> PS LGTM with the change. Will wait for feedback from wemm@.

This is definitely not working.  I get ENOSPC  and listen queue overflows on 
/var/run/logpriv now.

Grabbing an old 10.3 /usr/sbin/syslogd and placing it on the top of the 12.x 
one worked fine, aside from the include statements.

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 02:21:05 PM Maxim Sobolev wrote:
> Peter, It is actually the other way around. If you take syslogd code out of
> 11-stable and earlier that would work just fine with or without r316874.
> But since r285910 syslogd in head had been refactored a lot and I think
> that particular bug has introduced that has been masked by the shutdown()
> on datagram sockets becoming a NOP after r285910. Then r316874 restored our
> historical behavior for the shutdown(2) and bingo, bug in the new syslogd
> code is now causing it to spin when shutdown() != NOP.
> 
> -Max

Hmm, there's a new problem:

 45104 auditd   CALL  socket(PF_LOCAL,0x1002,0)
 45104 auditd   RET   socket 3
 45104 auditd   CALL  connect(0x3,0x7fffdbd8,0x6a)
 45104 auditd   STRU  struct sockaddr { AF_LOCAL, /var/run/logpriv }
 45104 auditd   NAMI  "/var/run/logpriv"
 45104 auditd   RET   connect 0
 45104 auditd   CALL  sendto(0x3,0x7fffe130,0x2f,0,0,0)
 45104 auditd   RET   sendto -1 errno 55 No buffer space available
 45104 auditd   CALL  open(0x800da5c67,0x15)
 45104 auditd   NAMI  "/dev/console"
 45104 auditd   RET   open 4
.. and it all goes to /dev/console instead.

On restarting syslogd:
Apr 15 02:17:43 repoman2 syslogd: exiting on signal 15
sonewconn: pcb 0xf80051e72680: Listen queue overflow: 16 already in queue 
already

Umm.. did the patch forget to listen to incoming connections or something?

I haven't seen this before anywhere except when syslogd is wedged.


> 
> On Fri, Apr 14, 2017 at 12:46 PM, Peter Wemm  wrote:
> > On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> > > Thanks, Peter. I will try to look into this asap.
> > 
> > I don't understand what is going on yet. Presumably there must be other
> > changes in play that affect udp/select sometime between the original 2015
> > change and this. The syslogd -s code is Old(TM).  I'm also wondering
> > whether
> > the -s code even works at all since the 2015 / r285910 change...
> > 
> > > -Max
> > > 
> > > On Apr 14, 2017 12:32 PM, "Peter Wemm"  wrote:
> > > > On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> > > > > On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > > > > > Author: sobomax
> > > > > > Date: Fri Apr 14 17:23:28 2017
> > > > > > New Revision: 316874
> > > > > > URL: https://svnweb.freebsd.org/changeset/base/316874
> > > > > > 
> > > > > > Log:
> > > > > >   Restore ability to shutdown DGRAM sockets, still forcing
> > 
> > ENOTCONN to
> > 
> > > > be
> > > > 
> > > > > > returned by the shutdown(2) system call. This ability has been
> > 
> > lost as
> > 
> > > > > > part
> > > > > > of the svn revision 285910.
> > > > > > 
> > > > > >   Reviewed by:  ed, rwatson, glebius, hiren
> > > > > >   MFC after:2 weeks
> > > > > >   Differential Revision:https://reviews.freebsd.org/D10351
> > > > > 
> > > > > This appears to have broken syslogd and had a major change in
> > 
> > behavior
> > 
> > > > with
> > > > 
> > > > > regards to select(2).
> > > > > 
> > > > > If you run syslogd with the -s flag (which is default), it now spins
> > 
> > at
> > 
> > > > 100%
> > > > 
> > > > > cpu as all the shutdown sockets now return readable from select.
> > > > > 
> > > > > Old releases / jails also manifest this behavor.  If it wasn't for
> > > > > losing
> > > > > the ability to run old branch binaries I'd suggest changing syslogd
> > > > > instead, but we depend on this in the cluster and I expect others do
> > > > > too.
> > > > > 
> > > > > I'm not 100% certain that this change is the culprit but a heads-up
> > > > > can't
> > > > > hurt. I'll try reverting it on the freebsd cluster next, after
> > > > > fixing
> > > > > the
> > > > > broken auditing changes.
> > > > > 
> > > > > -Peter
> > > > 
> > > > I can confirm that reverting r316874 fixes syslogd and backwards
> > > > compatability
> > > > with old branches.
> > > > 
> > > > --
> > > > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > > > KI6FJV
> > > > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
> > 
> > --
> > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > KI6FJV
> > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316945 - in head: etc/defaults etc/periodic/daily share/man/man5

[Alan Somers]

On Fri, Apr 14, 2017 at 7:55 PM, Rodney W. Grimes
 wrote:
> [ Charset UTF-8 unsupported, converting... ]
>> Author: asomers
>> Date: Fri Apr 14 22:59:14 2017
>> New Revision: 316945
>> URL: https://svnweb.freebsd.org/changeset/base/316945
>>
>> Log:
>>   Add 410.status-mfi, a periodic script for mfi(4) arrays
>>
>>   PR: 176049
>>   Submitted by:   docon...@gsoft.com.au
>>   Reviewed by:scottl, Larry Rosenman 
>>   MFC after:  3 weeks
>>   Relnotes:   yes
>>
>> Added:
>>   head/etc/periodic/daily/410.status-mfi   (contents, props changed)
>> Modified:
>>   head/etc/defaults/periodic.conf
>>   head/etc/periodic/daily/Makefile
>>   head/share/man/man5/periodic.conf.5
>>

>> Modified: head/etc/periodic/daily/Makefile
>> ==
>> --- head/etc/periodic/daily/Makefile  Fri Apr 14 22:02:08 2017
>> (r316944)
>> +++ head/etc/periodic/daily/Makefile  Fri Apr 14 22:59:14 2017
>> (r316945)
>> @@ -13,6 +13,7 @@ FILES=  100.clean-disks \
>>   400.status-disks \
>>   401.status-graid \
>>   406.status-gmirror \
>> + 410.status-mfi \
>>   407.status-graid3 \
>>   408.status-gstripe \
>>   409.status-gconcat \
>
> Insered into the middle of a number sorted list?

Mea culpa.  Already fixed in r316946.



> Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316951 - head/usr.sbin/syslogd

[Maxim Sobolev]

Author: sobomax
Date: Sat Apr 15 02:24:22 2017
New Revision: 316951
URL: https://svnweb.freebsd.org/changeset/base/316951

Log:
  Since shutdown(2) on datagram socket is no longer a NOP after rev 316874
  don't bother to select/recv on that socket. This prevents syslogd(8)
  from spinning endlessly when started with the -s option (default).
  
  Reported by:  peter

Modified:
  head/usr.sbin/syslogd/syslogd.c

Modified: head/usr.sbin/syslogd/syslogd.c
==
--- head/usr.sbin/syslogd/syslogd.c Sat Apr 15 01:52:49 2017
(r316950)
+++ head/usr.sbin/syslogd/syslogd.c Sat Apr 15 02:24:22 2017
(r316951)
@@ -702,7 +702,7 @@ main(int argc, char *argv[])
sizeof(fd_mask));
 
STAILQ_FOREACH(sl, &shead, next) {
-   if (sl->sl_socket != -1)
+   if (sl->sl_socket != -1 && sl->sl_recv != NULL)
FD_SET(sl->sl_socket, fdsr);
}
i = select(fdsrmax + 1, fdsr, NULL, NULL,
@@ -2877,6 +2877,7 @@ socksetup(struct peer *pe)
struct addrinfo hints, *res, *res0;
int error;
char *cp;
+   int (*sl_recv)(struct socklist *);
/*
 * We have to handle this case for backwards compatibility:
 * If there are two (or more) colons but no '[' and ']',
@@ -3003,6 +3004,7 @@ socksetup(struct peer *pe)
}
dprintf("new socket fd is %d\n", s);
listen(s, 5);
+   sl_recv = socklist_recv_sock;
dprintf("shutdown\n");
if (SecureMode || res->ai_family == AF_LOCAL) {
/* Forbid communication in secure mode. */
@@ -3013,13 +3015,14 @@ socksetup(struct peer *pe)
die(0);
}
dprintf("listening on socket\n");
+   sl_recv = NULL;
} else
dprintf("sending on socket\n");
addsock(res->ai_addr, res->ai_addrlen,
&(struct socklist){
.sl_socket = s,
.sl_peer = pe,
-   .sl_recv = socklist_recv_sock
+   .sl_recv = sl_recv
});
}
freeaddrinfo(res0);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Conrad Meyer]

That has not actually been committed yet.  Mark Johnston has a patch for it.

On Fri, Apr 14, 2017 at 7:17 PM, Larry Rosenman  wrote:
>
> On 4/14/17, 9:16 PM, "Conrad Meyer"  wrote:
>
> On Fri, Apr 14, 2017 at 7:02 PM, Larry Rosenman  wrote:
> > It looked like the netdump code might need some stuff for current 
> –CURRENT.
>
> Not for the server (netdumpd) — it's just a simple UDP daemon that
> writes incoming UDP streams to disk.
>
> Best,
> Conrad
>
>
> I was referring to the code to actually dump to netdumpd.
>
>
>
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 9:16 PM, "Conrad Meyer"  wrote:

On Fri, Apr 14, 2017 at 7:02 PM, Larry Rosenman  wrote:
> It looked like the netdump code might need some stuff for current 
–CURRENT.

Not for the server (netdumpd) — it's just a simple UDP daemon that
writes incoming UDP streams to disk.

Best,
Conrad


I was referring to the code to actually dump to netdumpd.



___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Conrad Meyer]

On Fri, Apr 14, 2017 at 7:02 PM, Larry Rosenman  wrote:
> It looked like the netdump code might need some stuff for current –CURRENT.

Not for the server (netdumpd) — it's just a simple UDP daemon that
writes incoming UDP streams to disk.

Best,
Conrad
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> > On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
> > wrote:
> > 
> > > Yeah, I have the following:
> > > borg.lerctr.org /home/ler $ swapctl -l
> > > Device:   1024-blocks Used:
> > > /dev/mfid0p38388608 0
> > > /dev/mfid1p38388608 0
> > > /dev/mfid2p38388608 0
> > > /dev/mfid3p38388608 0
> > > /dev/mfid4p38388608 0
> > > /dev/mfid5p38388608 0
> > > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > > hw.physmem: 137368682496
> > > borg.lerctr.org /home/ler $
> > > 
> > > SO 6 8G partitions (48G), but the dump is larger than 8G. 
> > 
> > Larry,
> >   This is a very good concern and point given todays more
> > common huge memory foot prints and lots of spindles.  I'll
> > keep this in they back of my mind as I tromp around in the
> > dump code.  I have another solution that may work for you
> > and that is to use Netdump rather than swapdump.  This
> > basically eliminates the trip to swap space and you end
> > up going to savecore style output on the netdump server.
> > 
> > -- 
> > Rod Grimes 
> > rgri...@freebsd.org
> > 
> > What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
> > since that?s what is ?next to? this server?  
> 
> As a netdump server FreeNAS would be fairly trivial to compile the netdump
> client code for.I should have running 11.0 and 10.2 code around
> some place.

You know, maybe we should get the client side code pushed in, that
has been stable and not stomped on by other changes and is usable
for those of us who have kernels that can do netdump. 

It is a fairly simple daemon that listens on a socket and stores
the dumps.

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

It looked like the netdump code might need some stuff for current –CURRENT. 


-- 
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: l...@lerctr.org
US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
 
 

On 4/14/17, 9:00 PM, "Conrad Meyer"  wrote:

Larry,

You just need to run netdumpd on the nearby server.  It could be a
port (although I'm not aware that it is ported yet).

Best,
Conrad

On Fri, Apr 14, 2017 at 6:37 PM, Larry Rosenman  wrote:
> On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
wrote:
>
>> Yeah, I have the following:
> > borg.lerctr.org /home/ler $ swapctl -l
> > Device:   1024-blocks Used:
> > /dev/mfid0p38388608 0
> > /dev/mfid1p38388608 0
> > /dev/mfid2p38388608 0
> > /dev/mfid3p38388608 0
> > /dev/mfid4p38388608 0
> > /dev/mfid5p38388608 0
> > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > hw.physmem: 137368682496
> > borg.lerctr.org /home/ler $
> >
> > SO 6 8G partitions (48G), but the dump is larger than 8G.
>
> Larry,
>   This is a very good concern and point given todays more
> common huge memory foot prints and lots of spindles.  I'll
> keep this in they back of my mind as I tromp around in the
> dump code.  I have another solution that may work for you
> and that is to use Netdump rather than swapdump.  This
> basically eliminates the trip to swap space and you end
> up going to savecore style output on the netdump server.
>
> --
> Rod Grimes 
rgri...@freebsd.org
>
> What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
since that’s what is “next to” this server?
>
>
>



___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 8:59 PM, "Rodney W. Grimes"  wrote:

> On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
wrote:
> 
> > Yeah, I have the following:
> > borg.lerctr.org /home/ler $ swapctl -l
> > Device:   1024-blocks Used:
> > /dev/mfid0p38388608 0
> > /dev/mfid1p38388608 0
> > /dev/mfid2p38388608 0
> > /dev/mfid3p38388608 0
> > /dev/mfid4p38388608 0
> > /dev/mfid5p38388608 0
> > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > hw.physmem: 137368682496
> > borg.lerctr.org /home/ler $
> > 
> > SO 6 8G partitions (48G), but the dump is larger than 8G. 
> 
> Larry,
>   This is a very good concern and point given todays more
> common huge memory foot prints and lots of spindles.  I'll
> keep this in they back of my mind as I tromp around in the
> dump code.  I have another solution that may work for you
> and that is to use Netdump rather than swapdump.  This
> basically eliminates the trip to swap space and you end
> up going to savecore style output on the netdump server.
> 
> -- 
> Rod Grimes 
rgri...@freebsd.org
> 
> What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
since that?s what is ?next to? this server?  

As a netdump server FreeNAS would be fairly trivial to compile the netdump
client code for.I should have running 11.0 and 10.2 code around
some place.

-- 
Rod Grimes 
rgri...@freebsd.org

I’m also running:
borg.lerctr.org /home/ler $ uname -a
FreeBSD borg.lerctr.org 12.0-CURRENT FreeBSD 12.0-CURRENT #23 r316945: Fri Apr 
14 18:37:13 CDT 2017 r...@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER  amd64
borg.lerctr.org /home/ler $




___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Conrad Meyer]

Larry,

You just need to run netdumpd on the nearby server.  It could be a
port (although I'm not aware that it is ported yet).

Best,
Conrad

On Fri, Apr 14, 2017 at 6:37 PM, Larry Rosenman  wrote:
> On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
> wrote:
>
>> Yeah, I have the following:
> > borg.lerctr.org /home/ler $ swapctl -l
> > Device:   1024-blocks Used:
> > /dev/mfid0p38388608 0
> > /dev/mfid1p38388608 0
> > /dev/mfid2p38388608 0
> > /dev/mfid3p38388608 0
> > /dev/mfid4p38388608 0
> > /dev/mfid5p38388608 0
> > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > hw.physmem: 137368682496
> > borg.lerctr.org /home/ler $
> >
> > SO 6 8G partitions (48G), but the dump is larger than 8G.
>
> Larry,
>   This is a very good concern and point given todays more
> common huge memory foot prints and lots of spindles.  I'll
> keep this in they back of my mind as I tromp around in the
> dump code.  I have another solution that may work for you
> and that is to use Netdump rather than swapdump.  This
> basically eliminates the trip to swap space and you end
> up going to savecore style output on the netdump server.
>
> --
> Rod Grimes 
> rgri...@freebsd.org
>
> What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
> since that’s what is “next to” this server?
>
>
>
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
> wrote:
> 
> > Yeah, I have the following:
> > borg.lerctr.org /home/ler $ swapctl -l
> > Device:   1024-blocks Used:
> > /dev/mfid0p38388608 0
> > /dev/mfid1p38388608 0
> > /dev/mfid2p38388608 0
> > /dev/mfid3p38388608 0
> > /dev/mfid4p38388608 0
> > /dev/mfid5p38388608 0
> > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > hw.physmem: 137368682496
> > borg.lerctr.org /home/ler $
> > 
> > SO 6 8G partitions (48G), but the dump is larger than 8G. 
> 
> Larry,
>   This is a very good concern and point given todays more
> common huge memory foot prints and lots of spindles.  I'll
> keep this in they back of my mind as I tromp around in the
> dump code.  I have another solution that may work for you
> and that is to use Netdump rather than swapdump.  This
> basically eliminates the trip to swap space and you end
> up going to savecore style output on the netdump server.
> 
> -- 
> Rod Grimes 
> rgri...@freebsd.org
> 
> What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
> since that?s what is ?next to? this server?  

As a netdump server FreeNAS would be fairly trivial to compile the netdump
client code for.I should have running 11.0 and 10.2 code around
some place.

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316945 - in head: etc/defaults etc/periodic/daily share/man/man5

[Rodney W. Grimes]

[ Charset UTF-8 unsupported, converting... ]
> Author: asomers
> Date: Fri Apr 14 22:59:14 2017
> New Revision: 316945
> URL: https://svnweb.freebsd.org/changeset/base/316945
> 
> Log:
>   Add 410.status-mfi, a periodic script for mfi(4) arrays
>   
>   PR: 176049
>   Submitted by:   docon...@gsoft.com.au
>   Reviewed by:scottl, Larry Rosenman 
>   MFC after:  3 weeks
>   Relnotes:   yes
> 
> Added:
>   head/etc/periodic/daily/410.status-mfi   (contents, props changed)
> Modified:
>   head/etc/defaults/periodic.conf
>   head/etc/periodic/daily/Makefile
>   head/share/man/man5/periodic.conf.5
> 
> Modified: head/etc/defaults/periodic.conf
> ==
> --- head/etc/defaults/periodic.conf   Fri Apr 14 22:02:08 2017
> (r316944)
> +++ head/etc/defaults/periodic.conf   Fri Apr 14 22:59:14 2017
> (r316945)
> @@ -112,6 +112,9 @@ daily_status_gstripe_enable="NO"  # Che
>  # 409.status-gconcat
>  daily_status_gconcat_enable="NO" # Check gconcat(8)
>  
> +# 410.status-mfi
> +daily_status_mfi_enable="NO" # Check mfiutil(8)
> +
>  # 420.status-network
>  daily_status_network_enable="YES"# Check network status
>  daily_status_network_usedns="YES"# DNS lookups are ok
> 
> Added: head/etc/periodic/daily/410.status-mfi
> ==
> --- /dev/null 00:00:00 1970   (empty, because file is newly added)
> +++ head/etc/periodic/daily/410.status-mfiFri Apr 14 22:59:14 2017
> (r316945)
> @@ -0,0 +1,33 @@
> +#!/bin/sh
> +#
> +# $FreeBSD$
> +#
> +
> +# If there is a global system configuration file, suck it in.
> +#
> +if [ -r /etc/defaults/periodic.conf ]
> +then
> +. /etc/defaults/periodic.conf
> +source_periodic_confs
> +fi
> +
> +case "$daily_status_mfi_enable" in
> +[Yy][Ee][Ss])
> +   echo
> +   echo 'Checking status of mfi(4) devices:'
> +
> +   if mfiutil show volumes; then
> +   if mfiutil show volumes | grep -q DEGRADED; then
> +   rc=3
> +   else
> +   rc=0
> +   fi
> +   else
> +   rc=2
> +   fi
> +   ;;
> +
> +*)  rc=0;;
> +esac
> +
> +exit $rc
> 
> Modified: head/etc/periodic/daily/Makefile
> ==
> --- head/etc/periodic/daily/Makefile  Fri Apr 14 22:02:08 2017
> (r316944)
> +++ head/etc/periodic/daily/Makefile  Fri Apr 14 22:59:14 2017
> (r316945)
> @@ -13,6 +13,7 @@ FILES=  100.clean-disks \
>   400.status-disks \
>   401.status-graid \
>   406.status-gmirror \
> + 410.status-mfi \
>   407.status-graid3 \
>   408.status-gstripe \
>   409.status-gconcat \

Insered into the middle of a number sorted list?

> 
> Modified: head/share/man/man5/periodic.conf.5
> ==
> --- head/share/man/man5/periodic.conf.5   Fri Apr 14 22:02:08 2017
> (r316944)
> +++ head/share/man/man5/periodic.conf.5   Fri Apr 14 22:59:14 2017
> (r316945)
> @@ -399,6 +399,15 @@ if you want to run
>  on your
>  .Xr gconcat 8
>  devices.
> +.It Va daily_status_mfi_enable
> +.Pq Vt bool
> +Set to
> +.Dq Li YES
> +if you want to run
> +.Nm mfiutil Cm status
> +on your
> +.Xr mfi 4
> +devices.
>  .It Va daily_status_network_enable
>  .Pq Vt bool
>  Set to
> 
> 

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316950 - head/share/mk

[Simon J. Gerraty]

Author: sjg
Date: Sat Apr 15 01:52:49 2017
New Revision: 316950
URL: https://svnweb.freebsd.org/changeset/base/316950

Log:
  Update to latest, handle case of __objdir=obj

Modified:
  head/share/mk/auto.obj.mk

Modified: head/share/mk/auto.obj.mk
==
--- head/share/mk/auto.obj.mk   Sat Apr 15 01:44:45 2017(r316949)
+++ head/share/mk/auto.obj.mk   Sat Apr 15 01:52:49 2017(r316950)
@@ -58,7 +58,10 @@ __objdir_made != echo ${__objdir}/; umas
 # This causes make to use the specified directory as .OBJDIR
 .OBJDIR: ${__objdir}
 .if ${.OBJDIR:tA} != ${__objdir:tA} && ${__objdir_made:Uno:M${__objdir}/*} != 
""
+# watch out for __objdir being relative path
+.if !(${__objdir:M/*} == "" && ${.OBJDIR:tA} == ${${.CURDIR}/${__objdir}:L:tA})
 .error could not use ${__objdir}: .OBJDIR=${.OBJDIR}
 .endif
 .endif
 .endif
+.endif
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> > 
> > Log:
> >   savecore: fix space calculation with respect to `minfree` in 
> > check_space(..)
> >   
> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> > representable data to INT_MAX. Check the values received from
> > strtoll(3), trimming trailing whitespace off the end to maintain
> > POLA.
> >   - Use `KiB` instead of `kB` when describing free space, total space,
> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
> > bytes and `kB` being the IEC standard for 1000 bytes.
> 
> I will just rant lightly that no one actually uses this in the real world.
> 
> Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
> kilobyte is a power of 2.  The End.
> 
> (Next up we'll have to rename 4k displays to
> 4k)

Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
a place we do this, so please, lets not start doing this here?

Yes, these are newer standards, perhaps some day we should make a global
switch to them, but lets not start mixing and matching things.

> -- 
> John Baldwin
-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316949 - in head: contrib/bmake contrib/bmake/mk contrib/bmake/unit-tests usr.bin/bmake

[Simon J. Gerraty]

Author: sjg
Date: Sat Apr 15 01:44:45 2017
New Revision: 316949
URL: https://svnweb.freebsd.org/changeset/base/316949

Log:
  Update to bmake 20170413

Modified:
  head/contrib/bmake/ChangeLog
  head/contrib/bmake/Makefile
  head/contrib/bmake/job.c
  head/contrib/bmake/main.c
  head/contrib/bmake/mk/ChangeLog
  head/contrib/bmake/mk/auto.obj.mk
  head/contrib/bmake/mk/dirdeps.mk
  head/contrib/bmake/mk/install-mk
  head/contrib/bmake/mk/meta.stage.mk
  head/contrib/bmake/mk/meta2deps.py
  head/contrib/bmake/mk/mkopt.sh
  head/contrib/bmake/str.c
  head/contrib/bmake/unit-tests/modmatch.exp
  head/contrib/bmake/unit-tests/modmatch.mk
  head/usr.bin/bmake/Makefile
Directory Properties:
  head/contrib/bmake/   (props changed)

Modified: head/contrib/bmake/ChangeLog
==
--- head/contrib/bmake/ChangeLogSat Apr 15 00:51:35 2017
(r316948)
+++ head/contrib/bmake/ChangeLogSat Apr 15 01:44:45 2017
(r316949)
@@ -1,8 +1,35 @@
+2017-04-13  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170413
+ Merge with NetBSD make, pick up
+ o main.c: when setting .OBJDIR ignore '$' in paths.
+
+   * job.c: use MALLOC_OPTIONS to set malloc_options.
+
+2017-04-11  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170411
+ Merge with NetBSD make, pick up
+ o str.c: Str_Match: allow [^a-z] to behave as expected.
+
+2017-03-26  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170326
+ Merge with NetBSD make, pick up
+ o main.c: purge relative paths from realpath cache when .OBJDIR
+   is changed.
+
+2017-03-11  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170311
+ Merge with NetBSD make, pick up
+  o main.c: only use -C arg "as is" if it starts with '/'.
+
 2017-03-01  Simon J. Gerraty  
 
* Makefile (_MAKE_VERSION): 20170301
  Merge with NetBSD make, pick up
- o main.c: use -C arg as is rather than getcwd()
+ o main.c: use -C arg "as is" rather than getcwd()
if they identify the same directory.
  o parse.c: ensure loadfile buffer is \n terminated in non-mmap case
 

Modified: head/contrib/bmake/Makefile
==
--- head/contrib/bmake/Makefile Sat Apr 15 00:51:35 2017(r316948)
+++ head/contrib/bmake/Makefile Sat Apr 15 01:44:45 2017(r316949)
@@ -1,7 +1,7 @@
-#  $Id: Makefile,v 1.81 2017/03/01 17:01:23 sjg Exp $
+#  $Id: Makefile,v 1.85 2017/04/13 16:29:40 sjg Exp $
 
 # Base version on src date
-_MAKE_VERSION= 20170301
+_MAKE_VERSION= 20170413
 
 PROG=  bmake
 

Modified: head/contrib/bmake/job.c
==
--- head/contrib/bmake/job.cSat Apr 15 00:51:35 2017(r316948)
+++ head/contrib/bmake/job.cSat Apr 15 01:44:45 2017(r316949)
@@ -389,7 +389,10 @@ static void JobSigLock(sigset_t *);
 static void JobSigUnlock(sigset_t *);
 static void JobSigReset(void);
 
-const char *malloc_options="A";
+#if !defined(MALLOC_OPTIONS)
+# define MALLOC_OPTIONS "A"
+#endif
+const char *malloc_options= MALLOC_OPTIONS;
 
 static void
 job_table_dump(const char *where)

Modified: head/contrib/bmake/main.c
==
--- head/contrib/bmake/main.c   Sat Apr 15 00:51:35 2017(r316948)
+++ head/contrib/bmake/main.c   Sat Apr 15 01:44:45 2017(r316949)
@@ -1,4 +1,4 @@
-/* $NetBSD: main.c,v 1.257 2017/02/08 17:47:36 christos Exp $  */
+/* $NetBSD: main.c,v 1.260 2017/04/13 13:55:23 christos Exp $  */
 
 /*
  * Copyright (c) 1988, 1989, 1990, 1993
@@ -69,7 +69,7 @@
  */
 
 #ifndef MAKE_NATIVE
-static char rcsid[] = "$NetBSD: main.c,v 1.257 2017/02/08 17:47:36 christos 
Exp $";
+static char rcsid[] = "$NetBSD: main.c,v 1.260 2017/04/13 13:55:23 christos 
Exp $";
 #else
 #include 
 #ifndef lint
@@ -81,7 +81,7 @@ __COPYRIGHT("@(#) Copyright (c) 1988, 19
 #if 0
 static char sccsid[] = "@(#)main.c 8.3 (Berkeley) 3/19/94";
 #else
-__RCSID("$NetBSD: main.c,v 1.257 2017/02/08 17:47:36 christos Exp $");
+__RCSID("$NetBSD: main.c,v 1.260 2017/04/13 13:55:23 christos Exp $");
 #endif
 #endif /* not lint */
 #endif
@@ -722,22 +722,15 @@ Boolean
 Main_SetObjdir(const char *fmt, ...)
 {
struct stat sb;
-   char *p, *path;
-   char buf[MAXPATHLEN + 1], pbuf[MAXPATHLEN + 1];
+   char *path;
+   char buf[MAXPATHLEN + 1];
Boolean rc = FALSE;
va_list ap;
 
va_start(ap, fmt);
-   vsnprintf(path = pbuf, MAXPATHLEN, fmt, ap);
+   vsnprintf(path = buf, MAXPATHLEN, fmt, ap);
va_end(ap);
 
-   /* expand variable substitutions */
-   if (strchr(path, '$') != 0) {
-   snprintf(buf, MAXPATHLEN, "%s", path);
-   path = p = Var_Subst(NULL, b

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

[ Charset UTF-8 unsupported, converting... ]
> On Fri, Apr 14, 2017 at 01:45:22PM -0700, Ngie Cooper (yaneurabeya) wrote:
> 
> > 
> > > On Apr 14, 2017, at 13:42, Slawa Olhovchenkov  wrote:
> > 
> > ?
> > 
> > > No, something like:
> > > % swapinfo
> > > Device  1K-blocks UsedAvail Capacity
> > > /dev/ada0p2  335544320 33554432 0%
> > > /dev/ada1p2  335544320 33554432 0%
> > > Total671088640 67108864 0%
> > 
> > Striping dumps across multiple swap devices isn?t supported. Sorry.
> 
> What about support in the future?

Added to my project list at people.freebsd.org/~rgrimes.  No promises,
but I'll look at it while I am doing the other work.

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 8:35 PM, "Rodney W. Grimes"  wrote:

> Yeah, I have the following:
> borg.lerctr.org /home/ler $ swapctl -l
> Device:   1024-blocks Used:
> /dev/mfid0p38388608 0
> /dev/mfid1p38388608 0
> /dev/mfid2p38388608 0
> /dev/mfid3p38388608 0
> /dev/mfid4p38388608 0
> /dev/mfid5p38388608 0
> borg.lerctr.org /home/ler $ sysctl hw.physmem
> hw.physmem: 137368682496
> borg.lerctr.org /home/ler $
> 
> SO 6 8G partitions (48G), but the dump is larger than 8G. 

Larry,
  This is a very good concern and point given todays more
common huge memory foot prints and lots of spindles.  I'll
keep this in they back of my mind as I tromp around in the
dump code.  I have another solution that may work for you
and that is to use Netdump rather than swapdump.  This
basically eliminates the trip to swap space and you end
up going to savecore style output on the netdump server.

-- 
Rod Grimes 
rgri...@freebsd.org

What does it take for NetDump to work to a FreeNAS (9.10 nightly) server since 
that’s what is “next to” this server?  


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> On 4/14/17, 3:39 PM, "Ngie Cooper (yaneurabeya)" 
>  wrote:
> 
> 
> > On Apr 14, 2017, at 13:37, Larry Rosenman  wrote:
> > 
> > On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)" 
>  wrote:
> > 
> >> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
> >> 
> >> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
>  wrote:
> >> 
> >>> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
> >>> 
> >>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> >>> 
>  On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  
> wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> > 
> > Log:
> > savecore: fix space calculation with respect to `minfree` in 
> check_space(..)
> > 
> > - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> >  representable data to INT_MAX. Check the values received from
> >  strtoll(3), trimming trailing whitespace off the end to maintain
> >  POLA.
> > - Use `KiB` instead of `kB` when describing free space, total space,
> >  etc. I am now fully aware of `KiB` being the IEC standard for 1024
> >  bytes and `kB` being the IEC standard for 1000 bytes.
> > - Store available number of KiB in `available` so it can be more
> >  easily queried and compared to ensure that there are enough KiB to
> >  store the dump image on disk.
> > - Print out the reserved space on disk, per `minfree`, so end-users
> >  can troubleshoot why check_space(..) is reporting that there isn't
> >  enough free space.
> > 
> > MFC after:7 weeks
> > Reviewed by:  Anton Rang  (earlier diff), cem 
> (earlier diff)
> > Tested with:  positive/negative cases (see review); make tinderbox
> > Sponsored by: Dell EMC Isilon
> > Differential Revision:D10379
>  
>  The free space calculation is still uselessly conservative, because 
> it
>  doesn't account for the fact that core dumps will always be either
>  spare or compressed.  The result is that savecore will frequently
>  refuse to save corefiles even when there's plenty of space.  I
>  proposed removing the space check altogether in
>  https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>  after the compressed core dump feature was merged, because then 
> mostly
>  accurate space checks will be possible.  AFAIK the compressed core
>  dump feature still hasn't been finished.
> >>> 
> >>> Is posible (in the future) to use multiple swaps (on multiple disks)
> >>> for save core dumps?
> >> 
> >>   Multiple swap devices is already handled by savecore(8), if one uses 
> fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
> devices.
> >> 
> >>   As far as saving to multiple disks is concerned, I would hope that 
> one is using a redundancy capable filesystem (zfs) or RAID-like technology 
> (gmirror, graid, LSI Fusion?s RAID product line) to stripe and/or mirror the 
> data across multiple disks.
> > 
> >?
> > 
> >> How do I use multiple devices to have the system dump on all of my 
> swap?  I got a message about not enough space, but there (I think) was enough 
> between multiple drives?.
> > 
> >Something like:
> > 
> >- Create a zpool
> >- Mount zpool to /crashdumps
> >- Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
> ?dumpdir=/crashdumps?
> > 
> >?
> >HTH,
> >-Ngie
> > 
> >PS The issue with lack of space might be the issue that Alan brought 
> up earlier with compressed dumps and overly conservative free space checks, 
> or it might be the fact that dumpdir (default: /var/crash) is full.
> > 
> > 
> > I was talking about the actual crashdump to swap by the system.  
> /var/crash has 10T of space (my root pool).
> 
> If your memory is bigger than your swap, you?re unfortunately not able to 
> save the mini dump if the size of the saved pages exceed the space on swap. I 
> think this is where markj?s compressed dumps feature will come in handy.
> Thanks!
> -Ngie
> 
> 
> Yeah, I have the following:
> borg.lerctr.org /home/ler $ swapctl -l
> Device:   1024-blocks Used:
> /dev/mfid0p38388608 0
> /dev/mfid1p38388608 0
> /dev/mfid2p38388608 0
> /dev/mfid3p38388608 0
> /dev/mfid4p38388608 0
> /dev/mfid5p38388608 0
> borg.lerctr.org /home/ler $ sysctl hw.physmem
> hw.physmem: 137368682496
> borg.lerctr.org /home/ler $
> 
> SO 6 8G partitions (48G), but the dump is larger than 8G. 

Larry,
  This is a very good concern and point given today

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> > On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> > > Author: ngie
> > > Date: Fri Apr 14 19:41:48 2017
> > > New Revision: 316938
> > > URL: https://svnweb.freebsd.org/changeset/base/316938
> > >
> > > Log:
> > >   savecore: fix space calculation with respect to `minfree` in 
> > > check_space(..)
> > >
> > >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> > > representable data to INT_MAX. Check the values received from
> > > strtoll(3), trimming trailing whitespace off the end to maintain
> > > POLA.
> > >   - Use `KiB` instead of `kB` when describing free space, total space,
> > > etc. I am now fully aware of `KiB` being the IEC standard for 1024
> > > bytes and `kB` being the IEC standard for 1000 bytes.
> > >   - Store available number of KiB in `available` so it can be more
> > > easily queried and compared to ensure that there are enough KiB to
> > > store the dump image on disk.
> > >   - Print out the reserved space on disk, per `minfree`, so end-users
> > > can troubleshoot why check_space(..) is reporting that there isn't
> > > enough free space.
> > >
> > >   MFC after:7 weeks
> > >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
> > > diff)
> > >   Tested with:  positive/negative cases (see review); make tinderbox
> > >   Sponsored by: Dell EMC Isilon
> > >   Differential Revision:D10379
> > 
> > The free space calculation is still uselessly conservative, because it
> > doesn't account for the fact that core dumps will always be either
> > spare or compressed.  The result is that savecore will frequently
> > refuse to save corefiles even when there's plenty of space.  I
> > proposed removing the space check altogether in
> > https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> > after the compressed core dump feature was merged, because then mostly
> > accurate space checks will be possible.  AFAIK the compressed core
> > dump feature still hasn't been finished.
> 
> I had held off on it for a while because it was going to conflict with
> the work to add encrypted dump support, which of course has finished.
> 
> The patch to add compression support is here and should largely still
> work:
> https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff
> 
> I've been hesitant about pushing it forward:
> - The dump_write* APIs need some simplification after the addition of
>   encrypted dump support and support for dumping to 4Kn drives.
> - I'm not sure how encryption should compose with compression. It seems
>   intuitively obvious that we should compress before encrypting if the
>   compression is to be of any use, but I don't know enough to know
>   whether the compression might somehow compromise the effectiveness of
>   the encryption.
> 
> If anyone has some insight on the second of these two points, I'd
> appreciate hearing it.

I have a large amount of reworking and modulization of the dump code
incuding intergration of your (markj) compressed dumps.  Layer isnt
implemented but is in the plan.  I should not of held off on the net
dump code as it got smashed by encrypted dumps, then again by
the libif'ing for all the Intel drives that had been netdump modified.

Basically now starting over :-(

Minidump is an lkm for me, and main dump is almost an lkm for me too.
-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316936 - head/sys/dev/cxgbe/iw_cxgbe

[Rodney W. Grimes]

[ Charset UTF-8 unsupported, converting... ]
> Author: np
> Date: Fri Apr 14 19:15:31 2017
> New Revision: 316936
> URL: https://svnweb.freebsd.org/changeset/base/316936
> 
> Log:
>   cxgbe/iw_cxgbe: hw supports 64K (not 32K) Protection Domains.
>   
>   MFC after:  3 days
>   Sponsored by:   Chelsio Communications
> 
> Modified:
>   head/sys/dev/cxgbe/iw_cxgbe/t4.h
> 
> Modified: head/sys/dev/cxgbe/iw_cxgbe/t4.h
> ==
> --- head/sys/dev/cxgbe/iw_cxgbe/t4.h  Fri Apr 14 18:56:00 2017
> (r316935)
> +++ head/sys/dev/cxgbe/iw_cxgbe/t4.h  Fri Apr 14 19:15:31 2017
> (r316936)
> @@ -61,7 +61,7 @@
>  
>  #define T4_MAX_NUM_QP (1<<16)
>  #define T4_MAX_NUM_CQ (1<<15)
> -#define T4_MAX_NUM_PD (1<<15)
> +#define T4_MAX_NUM_PD 65536

Why the change in methods of expressing powers of 2 here?
This, imho, would better match the near by code as
#define T4_MAX_NUM_PD (1<<16)

>  #define T4_EQ_STATUS_ENTRIES (L1_CACHE_BYTES > 64 ? 2 : 1)
>  #define T4_MAX_EQ_SIZE (65520 - T4_EQ_STATUS_ENTRIES)
>  #define T4_MAX_IQ_SIZE (65520 - 1)
> 
> 

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316948 - vendor/NetBSD/bmake/20170413

[Simon J. Gerraty]

Author: sjg
Date: Sat Apr 15 00:51:35 2017
New Revision: 316948
URL: https://svnweb.freebsd.org/changeset/base/316948

Log:
  tag bmake-20170413

Added:
  vendor/NetBSD/bmake/20170413/
 - copied from r316947, vendor/NetBSD/bmake/dist/
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316947 - in vendor/NetBSD/bmake/dist: . mk unit-tests

[Simon J. Gerraty]

Author: sjg
Date: Sat Apr 15 00:51:18 2017
New Revision: 316947
URL: https://svnweb.freebsd.org/changeset/base/316947

Log:
  Import bmake-20170413

Modified:
  vendor/NetBSD/bmake/dist/ChangeLog
  vendor/NetBSD/bmake/dist/Makefile
  vendor/NetBSD/bmake/dist/job.c
  vendor/NetBSD/bmake/dist/main.c
  vendor/NetBSD/bmake/dist/mk/ChangeLog
  vendor/NetBSD/bmake/dist/mk/auto.obj.mk
  vendor/NetBSD/bmake/dist/mk/dirdeps.mk
  vendor/NetBSD/bmake/dist/mk/install-mk
  vendor/NetBSD/bmake/dist/mk/meta.stage.mk
  vendor/NetBSD/bmake/dist/mk/meta2deps.py
  vendor/NetBSD/bmake/dist/mk/mkopt.sh
  vendor/NetBSD/bmake/dist/str.c
  vendor/NetBSD/bmake/dist/unit-tests/modmatch.exp
  vendor/NetBSD/bmake/dist/unit-tests/modmatch.mk

Modified: vendor/NetBSD/bmake/dist/ChangeLog
==
--- vendor/NetBSD/bmake/dist/ChangeLog  Sat Apr 15 00:39:45 2017
(r316946)
+++ vendor/NetBSD/bmake/dist/ChangeLog  Sat Apr 15 00:51:18 2017
(r316947)
@@ -1,8 +1,35 @@
+2017-04-13  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170413
+ Merge with NetBSD make, pick up
+ o main.c: when setting .OBJDIR ignore '$' in paths.
+
+   * job.c: use MALLOC_OPTIONS to set malloc_options.
+
+2017-04-11  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170411
+ Merge with NetBSD make, pick up
+ o str.c: Str_Match: allow [^a-z] to behave as expected.
+
+2017-03-26  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170326
+ Merge with NetBSD make, pick up
+ o main.c: purge relative paths from realpath cache when .OBJDIR
+   is changed.
+
+2017-03-11  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170311
+ Merge with NetBSD make, pick up
+  o main.c: only use -C arg "as is" if it starts with '/'.
+
 2017-03-01  Simon J. Gerraty  
 
* Makefile (_MAKE_VERSION): 20170301
  Merge with NetBSD make, pick up
- o main.c: use -C arg as is rather than getcwd()
+ o main.c: use -C arg "as is" rather than getcwd()
if they identify the same directory.
  o parse.c: ensure loadfile buffer is \n terminated in non-mmap case
 

Modified: vendor/NetBSD/bmake/dist/Makefile
==
--- vendor/NetBSD/bmake/dist/Makefile   Sat Apr 15 00:39:45 2017
(r316946)
+++ vendor/NetBSD/bmake/dist/Makefile   Sat Apr 15 00:51:18 2017
(r316947)
@@ -1,7 +1,7 @@
-#  $Id: Makefile,v 1.81 2017/03/01 17:01:23 sjg Exp $
+#  $Id: Makefile,v 1.85 2017/04/13 16:29:40 sjg Exp $
 
 # Base version on src date
-_MAKE_VERSION= 20170301
+_MAKE_VERSION= 20170413
 
 PROG=  bmake
 

Modified: vendor/NetBSD/bmake/dist/job.c
==
--- vendor/NetBSD/bmake/dist/job.c  Sat Apr 15 00:39:45 2017
(r316946)
+++ vendor/NetBSD/bmake/dist/job.c  Sat Apr 15 00:51:18 2017
(r316947)
@@ -373,7 +373,10 @@ static void JobSigLock(sigset_t *);
 static void JobSigUnlock(sigset_t *);
 static void JobSigReset(void);
 
-const char *malloc_options="A";
+#if !defined(MALLOC_OPTIONS)
+# define MALLOC_OPTIONS "A"
+#endif
+const char *malloc_options= MALLOC_OPTIONS;
 
 static void
 job_table_dump(const char *where)

Modified: vendor/NetBSD/bmake/dist/main.c
==
--- vendor/NetBSD/bmake/dist/main.c Sat Apr 15 00:39:45 2017
(r316946)
+++ vendor/NetBSD/bmake/dist/main.c Sat Apr 15 00:51:18 2017
(r316947)
@@ -1,4 +1,4 @@
-/* $NetBSD: main.c,v 1.257 2017/02/08 17:47:36 christos Exp $  */
+/* $NetBSD: main.c,v 1.260 2017/04/13 13:55:23 christos Exp $  */
 
 /*
  * Copyright (c) 1988, 1989, 1990, 1993
@@ -69,7 +69,7 @@
  */
 
 #ifndef MAKE_NATIVE
-static char rcsid[] = "$NetBSD: main.c,v 1.257 2017/02/08 17:47:36 christos 
Exp $";
+static char rcsid[] = "$NetBSD: main.c,v 1.260 2017/04/13 13:55:23 christos 
Exp $";
 #else
 #include 
 #ifndef lint
@@ -81,7 +81,7 @@ __COPYRIGHT("@(#) Copyright (c) 1988, 19
 #if 0
 static char sccsid[] = "@(#)main.c 8.3 (Berkeley) 3/19/94";
 #else
-__RCSID("$NetBSD: main.c,v 1.257 2017/02/08 17:47:36 christos Exp $");
+__RCSID("$NetBSD: main.c,v 1.260 2017/04/13 13:55:23 christos Exp $");
 #endif
 #endif /* not lint */
 #endif
@@ -458,7 +458,8 @@ rearg:  
(void)fprintf(stderr, "%s: %s.\n", progname, 
strerror(errno));
exit(2);
}
-   if (stat(argvalue, &sa) != -1 &&
+   if (argvalue[0] == '/' &&
+   stat(argvalue, &sa) != -1 &&
stat(curdir, &sb) != -1 &&
sa.st_ino == sb.st_ino &&
sa.st_dev == sb.st_dev)
@@ -721,22 +722,15 @@ Bool

svn commit: r316946 - head/etc/periodic/daily

[Alan Somers]

Author: asomers
Date: Sat Apr 15 00:39:45 2017
New Revision: 316946
URL: https://svnweb.freebsd.org/changeset/base/316946

Log:
  Reorder Makefile entries from r316945
  
  PR:   176049
  Reported by:  Oliver Pinter
  MFC after:3 weeks
  X-MFC-With:   316945

Modified:
  head/etc/periodic/daily/Makefile

Modified: head/etc/periodic/daily/Makefile
==
--- head/etc/periodic/daily/MakefileFri Apr 14 22:59:14 2017
(r316945)
+++ head/etc/periodic/daily/MakefileSat Apr 15 00:39:45 2017
(r316946)
@@ -13,10 +13,10 @@ FILES=  100.clean-disks \
400.status-disks \
401.status-graid \
406.status-gmirror \
-   410.status-mfi \
407.status-graid3 \
408.status-gstripe \
409.status-gconcat \
+   410.status-mfi \
420.status-network \
430.status-uptime \
450.status-security \
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 02:10:56 PM Maxim Sobolev wrote:
> Peter, Ngie,
> 
> Looks like out of that refactoring came a logical bug that is present in
> the head, which causes syslod to first to shutdown the socket for reading
> and then try to select/recv on it (which is somewhat stupid). And that
> issue has been masked by shutdown() on datagram socket becoming effectively
> a NOP in 11 & head 20 months ago. It only affects head though, 11-stable
> still has the old code which does not include that half-closed socket into
> the select list. Attached patch is expected to fix head, Peter, it would be
> nice if you can give it a try (restoring latest changes into
> uipc_sockets.c) and let me know if it helps.

Confirmed - resting uipc_socket.c to HEAD and applying the patch to syslogd 
does solve the problem we encountered.  Thanks!

> Thanks!
> 
> On Fri, Apr 14, 2017 at 12:48 PM, Ngie Cooper (yaneurabeya) <
> 
> yaneurab...@gmail.com> wrote:
> > > On Apr 14, 2017, at 12:46, Peter Wemm  wrote:
> > > 
> > > On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> > >> Thanks, Peter. I will try to look into this asap.
> > > 
> > > I don't understand what is going on yet. Presumably there must be other
> > > changes in play that affect udp/select sometime between the original
> > > 2015
> > > change and this. The syslogd -s code is Old(TM).  I'm also wondering
> > 
> > whether
> > 
> > > the -s code even works at all since the 2015 / r285910 change...
> > 
> > syslogd has been refactored a lot on ^/head. I don’t think it’s safe to
> > say that the ^/head and ^/stable/11 and earlier copies will function the
> > same.
> > Thanks,
> > -Ngie

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316945 - in head: etc/defaults etc/periodic/daily share/man/man5

[Oliver Pinter]

On Saturday, April 15, 2017, Alan Somers  wrote:

> Author: asomers
> Date: Fri Apr 14 22:59:14 2017
> New Revision: 316945
> URL: https://svnweb.freebsd.org/changeset/base/316945
>
> Log:
>   Add 410.status-mfi, a periodic script for mfi(4) arrays
>
>   PR:   176049
>   Submitted by: docon...@gsoft.com.au 
>   Reviewed by:  scottl, Larry Rosenman >
>   MFC after:3 weeks
>   Relnotes: yes
>
> Added:
>   head/etc/periodic/daily/410.status-mfi   (contents, props changed)
> Modified:
>   head/etc/defaults/periodic.conf
>   head/etc/periodic/daily/Makefile
>   head/share/man/man5/periodic.conf.5
>
> Modified: head/etc/defaults/periodic.conf
> 
> ==
> --- head/etc/defaults/periodic.conf Fri Apr 14 22:02:08 2017
> (r316944)
> +++ head/etc/defaults/periodic.conf Fri Apr 14 22:59:14 2017
> (r316945)
> @@ -112,6 +112,9 @@ daily_status_gstripe_enable="NO"#
> Che
>  # 409.status-gconcat
>  daily_status_gconcat_enable="NO"   # Check gconcat(8)
>
> +# 410.status-mfi
> +daily_status_mfi_enable="NO"   # Check mfiutil(8)
> +
>  # 420.status-network
>  daily_status_network_enable="YES"  # Check network
> status
>  daily_status_network_usedns="YES"  # DNS lookups are
> ok
>
> Added: head/etc/periodic/daily/410.status-mfi
> 
> ==
> --- /dev/null   00:00:00 1970   (empty, because file is newly added)
> +++ head/etc/periodic/daily/410.status-mfi  Fri Apr 14 22:59:14 2017
>   (r316945)
> @@ -0,0 +1,33 @@
> +#!/bin/sh
> +#
> +# $FreeBSD$
> +#
> +
> +# If there is a global system configuration file, suck it in.
> +#
> +if [ -r /etc/defaults/periodic.conf ]
> +then
> +. /etc/defaults/periodic.conf
> +source_periodic_confs
> +fi
> +
> +case "$daily_status_mfi_enable" in
> +[Yy][Ee][Ss])
> +   echo
> +   echo 'Checking status of mfi(4) devices:'
> +
> +   if mfiutil show volumes; then
> +   if mfiutil show volumes | grep -q DEGRADED; then
> +   rc=3
> +   else
> +   rc=0
> +   fi
> +   else
> +   rc=2
> +   fi
> +   ;;
> +
> +*)  rc=0;;
> +esac
> +
> +exit $rc
>
> Modified: head/etc/periodic/daily/Makefile
> 
> ==
> --- head/etc/periodic/daily/MakefileFri Apr 14 22:02:08 2017
> (r316944)
> +++ head/etc/periodic/daily/MakefileFri Apr 14 22:59:14 2017
> (r316945)
> @@ -13,6 +13,7 @@ FILES=100.clean-disks \
> 400.status-disks \
> 401.status-graid \
> 406.status-gmirror \
> +   410.status-mfi \


This should be in different place, this list is ordered, as I see from this
context.


> 407.status-graid3 \
> 408.status-gstripe \
> 409.status-gconcat \
>
> Modified: head/share/man/man5/periodic.conf.5
> 
> ==
> --- head/share/man/man5/periodic.conf.5 Fri Apr 14 22:02:08 2017
> (r316944)
> +++ head/share/man/man5/periodic.conf.5 Fri Apr 14 22:59:14 2017
> (r316945)
> @@ -399,6 +399,15 @@ if you want to run
>  on your
>  .Xr gconcat 8
>  devices.
> +.It Va daily_status_mfi_enable
> +.Pq Vt bool
> +Set to
> +.Dq Li YES
> +if you want to run
> +.Nm mfiutil Cm status
> +on your
> +.Xr mfi 4
> +devices.
>  .It Va daily_status_network_enable
>  .Pq Vt bool
>  Set to
> ___
> svn-src-h...@freebsd.org  mailing list
> https://lists.freebsd.org/mailman/listinfo/svn-src-head
> To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org
> "
>
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r313343 - head/sys/arm/arm

[Ronald Klop]

With this change merged to 11-CURRENT my SHEEVAPLUG/arm displays different  
information in dmesg:


messages:Apr 15 00:53:07 sheeva2 kernel: Little-endian DC enabled IC  
enabled WA disabled DC streaming enabled
messages:Apr 15 00:53:07 sheeva2 kernel: WB enabled LABT branch prediction  
enabled
messages.0.bz2:Apr 11 20:02:06 sheeva2 kernel: Little-endian DC enabled IC  
disabled WA disabled DC streaming enabled
messages.0.bz2:Apr 11 20:02:06 sheeva2 kernel: WB enabled LABT branch  
prediction disabled


IC changed from disabled to enabled and LABT branch prediction is  
displayed as enabled also.


Is this only cosmetically? Or does it really change behaviour?

Anybody willing to MFC this 11?

Regards,
Ronald.


On Mon, 06 Feb 2017 15:58:24 +0100, Stanislav Galabov  
 wrote:



Author: sgalabov
Date: Mon Feb  6 14:58:24 2017
New Revision: 313343
URL: https://svnweb.freebsd.org/changeset/base/313343

Log:
  sys/arm/arm/identcpu-v4.c: fix identify_arm_cpu()
 identify_arm_cpu() in sys/arm/arm/identcpu-v4.c incorrectly uses a
  u_int8_t variable to store the result of cpu_get_control().
  It should really use a u_int variable, the same way as done for  
cpu_ident()

  in the same function, as both cpuid and control registers are 32-bit..
  This issue causes users of identcpu-v4 to incorrectly report things  
such as

  icache status (bit 12 in cpu control register) and basically anything
  defined in bits above bit 7 :-)
 Reviewed by:   manu
  Sponsored by: Smartcom - Bulgaria AD
  Differential Revision:https://reviews.freebsd.org/D9460

Modified:
  head/sys/arm/arm/identcpu-v4.c

Modified: head/sys/arm/arm/identcpu-v4.c
==
--- head/sys/arm/arm/identcpu-v4.c  Mon Feb  6 14:41:34 2017
(r313342)
+++ head/sys/arm/arm/identcpu-v4.c  Mon Feb  6 14:58:24 2017
(r313343)
@@ -294,8 +294,7 @@ u_int cpu_pfr(int num)
 void
 identify_arm_cpu(void)
 {
-   u_int cpuid;
-   u_int8_t ctrl;
+   u_int cpuid, ctrl;
int i;
ctrl = cpu_get_control();
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316945 - in head: etc/defaults etc/periodic/daily share/man/man5

[Alan Somers]

Author: asomers
Date: Fri Apr 14 22:59:14 2017
New Revision: 316945
URL: https://svnweb.freebsd.org/changeset/base/316945

Log:
  Add 410.status-mfi, a periodic script for mfi(4) arrays
  
  PR:   176049
  Submitted by: docon...@gsoft.com.au
  Reviewed by:  scottl, Larry Rosenman 
  MFC after:3 weeks
  Relnotes: yes

Added:
  head/etc/periodic/daily/410.status-mfi   (contents, props changed)
Modified:
  head/etc/defaults/periodic.conf
  head/etc/periodic/daily/Makefile
  head/share/man/man5/periodic.conf.5

Modified: head/etc/defaults/periodic.conf
==
--- head/etc/defaults/periodic.conf Fri Apr 14 22:02:08 2017
(r316944)
+++ head/etc/defaults/periodic.conf Fri Apr 14 22:59:14 2017
(r316945)
@@ -112,6 +112,9 @@ daily_status_gstripe_enable="NO"# Che
 # 409.status-gconcat
 daily_status_gconcat_enable="NO"   # Check gconcat(8)
 
+# 410.status-mfi
+daily_status_mfi_enable="NO"   # Check mfiutil(8)
+
 # 420.status-network
 daily_status_network_enable="YES"  # Check network status
 daily_status_network_usedns="YES"  # DNS lookups are ok

Added: head/etc/periodic/daily/410.status-mfi
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/etc/periodic/daily/410.status-mfi  Fri Apr 14 22:59:14 2017
(r316945)
@@ -0,0 +1,33 @@
+#!/bin/sh
+#
+# $FreeBSD$
+#
+
+# If there is a global system configuration file, suck it in.
+#
+if [ -r /etc/defaults/periodic.conf ]
+then
+. /etc/defaults/periodic.conf
+source_periodic_confs
+fi
+
+case "$daily_status_mfi_enable" in
+[Yy][Ee][Ss])
+   echo
+   echo 'Checking status of mfi(4) devices:'
+
+   if mfiutil show volumes; then
+   if mfiutil show volumes | grep -q DEGRADED; then
+   rc=3
+   else
+   rc=0
+   fi
+   else
+   rc=2
+   fi
+   ;;
+
+*)  rc=0;;
+esac
+
+exit $rc

Modified: head/etc/periodic/daily/Makefile
==
--- head/etc/periodic/daily/MakefileFri Apr 14 22:02:08 2017
(r316944)
+++ head/etc/periodic/daily/MakefileFri Apr 14 22:59:14 2017
(r316945)
@@ -13,6 +13,7 @@ FILES=100.clean-disks \
400.status-disks \
401.status-graid \
406.status-gmirror \
+   410.status-mfi \
407.status-graid3 \
408.status-gstripe \
409.status-gconcat \

Modified: head/share/man/man5/periodic.conf.5
==
--- head/share/man/man5/periodic.conf.5 Fri Apr 14 22:02:08 2017
(r316944)
+++ head/share/man/man5/periodic.conf.5 Fri Apr 14 22:59:14 2017
(r316945)
@@ -399,6 +399,15 @@ if you want to run
 on your
 .Xr gconcat 8
 devices.
+.It Va daily_status_mfi_enable
+.Pq Vt bool
+Set to
+.Dq Li YES
+if you want to run
+.Nm mfiutil Cm status
+on your
+.Xr mfi 4
+devices.
 .It Va daily_status_network_enable
 .Pq Vt bool
 Set to
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Mark Johnston]

On Fri, Apr 14, 2017 at 02:28:57PM -0700, Xin LI wrote:
> On Fri, Apr 14, 2017 at 1:29 PM, Mark Johnston  wrote:
> > - I'm not sure how encryption should compose with compression. It seems
> >   intuitively obvious that we should compress before encrypting if the
> >   compression is to be of any use, but I don't know enough to know
> >   whether the compression might somehow compromise the effectiveness of
> >   the encryption.
> 
> I think the biggest concern is the added code involved in the dump
> process (which happen when the kernel is already unhealthy), which can
> jeopardize it and defeat the usefulness of having a crash dump being
> set up in the first place.  

I agree in principle but this doesn't appear to cause problems in
practice. zlib allocates memory only at initialization time, so its
requirements in panic context are quite minimal. I wrote a little bit of
zlib glue, used currently for compressed userland core dumps, which
works properly in panic context as well.

> And with textdumps available, the benefit
> of having compression is limited because we can request for minidump
> or full dumps only when the textdumps are not good enough for
> diagnosing the kernel bug.

Sure, but in this case the compression may be vital.

> 
> I don't think security (e.g. leaking information because of the use of
> compression) is a very big concern in this context because in order
> for the potential attacker to read the raw material needs a
> compromised system (unlike an attack from the network, where someone
> who controls the network would have access to the raw material); the
> dump is usually quite large, and measuring downtime would be hard at
> that scale.

Ok.

> 
> By the way (not meant to bikeshed) if I was to do this I'd prefer
> using lz4 or something that compresses faster than zlib.

I agree, but I think the existing lz4 implementation in the kernel is
not so well suited to running after a panic. It seems fixable, but
compression speed also isn't hugely important here IMO.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316944 - in stable/11: . sys/netinet sys/netinet6

[Steven Hartland]

Author: smh
Date: Fri Apr 14 22:02:08 2017
New Revision: 316944
URL: https://svnweb.freebsd.org/changeset/base/316944

Log:
  MFC r316313, r316328:
  
  Allow explicitly assigned IPv4 & IPv6 loopback addresses to be used in
  jails.
  
  Relnotes: Yes
  Sponsored by: Multiplay

Modified:
  stable/11/UPDATING
  stable/11/sys/netinet/in_jail.c
  stable/11/sys/netinet6/in6_jail.c
Directory Properties:
  stable/11/   (props changed)

Modified: stable/11/UPDATING
==
--- stable/11/UPDATING  Fri Apr 14 21:49:20 2017(r316943)
+++ stable/11/UPDATING  Fri Apr 14 22:02:08 2017(r316944)
@@ -16,6 +16,11 @@ from older versions of FreeBSD, try WITH
 the tip of head, and then rebuild without this option. The bootstrap process
 from older version of current across the gcc/clang cutover is a bit fragile.
 
+20170414:
+   Binds and sends to the loopback addresses, IPv6 and IPv4, will now
+   use any explicitly assigned loopback address available in the jail
+   instead of using the first assigned address of the jail.
+
 20170402:
Clang, llvm, lldb, compiler-rt and libc++ have been upgraded to 4.0.0.
Please see the 20141231 entry below for information about prerequisites

Modified: stable/11/sys/netinet/in_jail.c
==
--- stable/11/sys/netinet/in_jail.c Fri Apr 14 21:49:20 2017
(r316943)
+++ stable/11/sys/netinet/in_jail.c Fri Apr 14 22:02:08 2017
(r316944)
@@ -306,11 +306,6 @@ prison_local_ip4(struct ucred *cred, str
}
 
ia0.s_addr = ntohl(ia->s_addr);
-   if (ia0.s_addr == INADDR_LOOPBACK) {
-   ia->s_addr = pr->pr_ip4[0].s_addr;
-   mtx_unlock(&pr->pr_mtx);
-   return (0);
-   }
 
if (ia0.s_addr == INADDR_ANY) {
/*
@@ -323,6 +318,11 @@ prison_local_ip4(struct ucred *cred, str
}
 
error = prison_check_ip4_locked(pr, ia);
+   if (error == EADDRNOTAVAIL && ia0.s_addr == INADDR_LOOPBACK) {
+   ia->s_addr = pr->pr_ip4[0].s_addr;
+   error = 0;
+   }
+
mtx_unlock(&pr->pr_mtx);
return (error);
 }
@@ -354,7 +354,8 @@ prison_remote_ip4(struct ucred *cred, st
return (EAFNOSUPPORT);
}
 
-   if (ntohl(ia->s_addr) == INADDR_LOOPBACK) {
+   if (ntohl(ia->s_addr) == INADDR_LOOPBACK &&
+   prison_check_ip4_locked(pr, ia) == EADDRNOTAVAIL) {
ia->s_addr = pr->pr_ip4[0].s_addr;
mtx_unlock(&pr->pr_mtx);
return (0);
@@ -370,9 +371,8 @@ prison_remote_ip4(struct ucred *cred, st
 /*
  * Check if given address belongs to the jail referenced by cred/prison.
  *
- * Returns 0 if jail doesn't restrict IPv4 or if address belongs to jail,
- * EADDRNOTAVAIL if the address doesn't belong, or EAFNOSUPPORT if the jail
- * doesn't allow IPv4.  Address passed in in NBO.
+ * Returns 0 if address belongs to jail,
+ * EADDRNOTAVAIL if the address doesn't belong to the jail.
  */
 int
 prison_check_ip4_locked(const struct prison *pr, const struct in_addr *ia)

Modified: stable/11/sys/netinet6/in6_jail.c
==
--- stable/11/sys/netinet6/in6_jail.c   Fri Apr 14 21:49:20 2017
(r316943)
+++ stable/11/sys/netinet6/in6_jail.c   Fri Apr 14 22:02:08 2017
(r316944)
@@ -293,12 +293,6 @@ prison_local_ip6(struct ucred *cred, str
return (EAFNOSUPPORT);
}
 
-   if (IN6_IS_ADDR_LOOPBACK(ia6)) {
-   bcopy(&pr->pr_ip6[0], ia6, sizeof(struct in6_addr));
-   mtx_unlock(&pr->pr_mtx);
-   return (0);
-   }
-
if (IN6_IS_ADDR_UNSPECIFIED(ia6)) {
/*
 * In case there is only 1 IPv6 address, and v6only is true,
@@ -311,6 +305,11 @@ prison_local_ip6(struct ucred *cred, str
}
 
error = prison_check_ip6_locked(pr, ia6);
+   if (error == EADDRNOTAVAIL && IN6_IS_ADDR_LOOPBACK(ia6)) {
+   bcopy(&pr->pr_ip6[0], ia6, sizeof(struct in6_addr));
+   error = 0;
+   }
+
mtx_unlock(&pr->pr_mtx);
return (error);
 }
@@ -341,7 +340,8 @@ prison_remote_ip6(struct ucred *cred, st
return (EAFNOSUPPORT);
}
 
-   if (IN6_IS_ADDR_LOOPBACK(ia6)) {
+   if (IN6_IS_ADDR_LOOPBACK(ia6) &&
+prison_check_ip6_locked(pr, ia6) == EADDRNOTAVAIL) {
bcopy(&pr->pr_ip6[0], ia6, sizeof(struct in6_addr));
mtx_unlock(&pr->pr_mtx);
return (0);
@@ -357,9 +357,8 @@ prison_remote_ip6(struct ucred *cred, st
 /*
  * Check if given address belongs to the jail referenced by cred/prison.
  *
- * R

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 02:21:05 PM Maxim Sobolev wrote:
> Peter, It is actually the other way around. If you take syslogd code out of
> 11-stable and earlier that would work just fine with or without r316874.
> But since r285910 syslogd in head had been refactored a lot and I think
> that particular bug has introduced that has been masked by the shutdown()
> on datagram sockets becoming a NOP after r285910. Then r316874 restored our
> historical behavior for the shutdown(2) and bingo, bug in the new syslogd
> code is now causing it to spin when shutdown() != NOP.

Ok, this makes sense.

Just to be sure I'm on the same page, I should apply the syslogd.diff from a 
few messages ago and restore the shutdown(2) kernel code and give it a spin.  
Correct?

-Peter

> -Max
> 
> On Fri, Apr 14, 2017 at 12:46 PM, Peter Wemm  wrote:
> > On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> > > Thanks, Peter. I will try to look into this asap.
> > 
> > I don't understand what is going on yet. Presumably there must be other
> > changes in play that affect udp/select sometime between the original 2015
> > change and this. The syslogd -s code is Old(TM).  I'm also wondering
> > whether
> > the -s code even works at all since the 2015 / r285910 change...
> > 
> > > -Max
> > > 
> > > On Apr 14, 2017 12:32 PM, "Peter Wemm"  wrote:
> > > > On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> > > > > On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > > > > > Author: sobomax
> > > > > > Date: Fri Apr 14 17:23:28 2017
> > > > > > New Revision: 316874
> > > > > > URL: https://svnweb.freebsd.org/changeset/base/316874
> > > > > > 
> > > > > > Log:
> > > > > >   Restore ability to shutdown DGRAM sockets, still forcing
> > 
> > ENOTCONN to
> > 
> > > > be
> > > > 
> > > > > > returned by the shutdown(2) system call. This ability has been
> > 
> > lost as
> > 
> > > > > > part
> > > > > > of the svn revision 285910.
> > > > > > 
> > > > > >   Reviewed by:  ed, rwatson, glebius, hiren
> > > > > >   MFC after:2 weeks
> > > > > >   Differential Revision:https://reviews.freebsd.org/D10351
> > > > > 
> > > > > This appears to have broken syslogd and had a major change in
> > 
> > behavior
> > 
> > > > with
> > > > 
> > > > > regards to select(2).
> > > > > 
> > > > > If you run syslogd with the -s flag (which is default), it now spins
> > 
> > at
> > 
> > > > 100%
> > > > 
> > > > > cpu as all the shutdown sockets now return readable from select.
> > > > > 
> > > > > Old releases / jails also manifest this behavor.  If it wasn't for
> > > > > losing
> > > > > the ability to run old branch binaries I'd suggest changing syslogd
> > > > > instead, but we depend on this in the cluster and I expect others do
> > > > > too.
> > > > > 
> > > > > I'm not 100% certain that this change is the culprit but a heads-up
> > > > > can't
> > > > > hurt. I'll try reverting it on the freebsd cluster next, after
> > > > > fixing
> > > > > the
> > > > > broken auditing changes.
> > > > > 
> > > > > -Peter
> > > > 
> > > > I can confirm that reverting r316874 fixes syslogd and backwards
> > > > compatability
> > > > with old branches.
> > > > 
> > > > --
> > > > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > > > KI6FJV
> > > > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
> > 
> > --
> > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > KI6FJV
> > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 2:40 PM, John Baldwin  wrote:
> 
> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
>> Author: ngie
>> Date: Fri Apr 14 19:41:48 2017
>> New Revision: 316938
>> URL: https://svnweb.freebsd.org/changeset/base/316938
>> 
>> Log:
>>  savecore: fix space calculation with respect to `minfree` in check_space(..)
>> 
>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>representable data to INT_MAX. Check the values received from
>>strtoll(3), trimming trailing whitespace off the end to maintain
>>POLA.
>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>bytes and `kB` being the IEC standard for 1000 bytes.
> 
> I will just rant lightly that no one actually uses this in the real world.
> 
> Good lucking finding a "16 GiB" DIMM on crucial.com  or 
> a 4Kin drive.  A
> kilobyte is a power of 2.  The End.
> 
> (Next up we'll have to rename 4k displays to
> 4k)


https://media0.giphy.com/media/7rj2ZgttvgomY/giphy.gif 
 :P.
-Ngie


signature.asc
Description: Message signed with OpenPGP

svn commit: r316943 - in stable/11/sys: conf kern netinet netinet6 sys

[Steven Hartland]

Author: smh
Date: Fri Apr 14 21:49:20 2017
New Revision: 316943
URL: https://svnweb.freebsd.org/changeset/base/316943

Log:
  MFC r303863:
  
  Move IPv4 & IPv6 specific jail functions to netinet and netinet6 files.
  
  Sponsored by: Multiplay

Added:
  stable/11/sys/netinet/in_jail.c
 - copied unchanged from r303863, head/sys/netinet/in_jail.c
  stable/11/sys/netinet6/in6_jail.c
 - copied unchanged from r303863, head/sys/netinet6/in6_jail.c
Modified:
  stable/11/sys/conf/files
  stable/11/sys/kern/kern_jail.c
  stable/11/sys/sys/jail.h
Directory Properties:
  stable/11/   (props changed)

Modified: stable/11/sys/conf/files
==
--- stable/11/sys/conf/filesFri Apr 14 21:42:27 2017(r316942)
+++ stable/11/sys/conf/filesFri Apr 14 21:49:20 2017(r316943)
@@ -3805,6 +3805,7 @@ netinet/in_fib.c  optional inet
 netinet/in_gif.c   optional gif inet | netgraph_gif inet
 netinet/ip_gre.c   optional gre inet
 netinet/ip_id.coptional inet
+netinet/in_jail.c  optional inet
 netinet/in_mcast.c optional inet
 netinet/in_pcb.c   optional inet | inet6
 netinet/in_pcbgroup.c  optional inet pcbgroup | inet6 pcbgroup
@@ -3871,6 +3872,7 @@ netinet6/in6_cksum.c  optional inet6
 netinet6/in6_fib.c optional inet6
 netinet6/in6_gif.c optional gif inet6 | netgraph_gif inet6
 netinet6/in6_ifattach.coptional inet6
+netinet6/in6_jail.coptional inet6
 netinet6/in6_mcast.c   optional inet6
 netinet6/in6_pcb.c optional inet6
 netinet6/in6_pcbgroup.coptional inet6 pcbgroup

Modified: stable/11/sys/kern/kern_jail.c
==
--- stable/11/sys/kern/kern_jail.c  Fri Apr 14 21:42:27 2017
(r316942)
+++ stable/11/sys/kern/kern_jail.c  Fri Apr 14 21:49:20 2017
(r316943)
@@ -130,14 +130,6 @@ static void prison_racct_attach(struct p
 static void prison_racct_modify(struct prison *pr);
 static void prison_racct_detach(struct prison *pr);
 #endif
-#ifdef INET
-static int _prison_check_ip4(const struct prison *, const struct in_addr *);
-static int prison_restrict_ip4(struct prison *pr, struct in_addr *newip4);
-#endif
-#ifdef INET6
-static int _prison_check_ip6(struct prison *pr, struct in6_addr *ia6);
-static int prison_restrict_ip6(struct prison *pr, struct in6_addr *newip6);
-#endif
 
 /* Flags for prison_deref */
 #definePD_DEREF0x01
@@ -252,54 +244,6 @@ prison0_init(void)
strlcpy(prison0.pr_osrelease, osrelease, sizeof(prison0.pr_osrelease));
 }
 
-#ifdef INET
-static int
-qcmp_v4(const void *ip1, const void *ip2)
-{
-   in_addr_t iaa, iab;
-
-   /*
-* We need to compare in HBO here to get the list sorted as expected
-* by the result of the code.  Sorting NBO addresses gives you
-* interesting results.  If you do not understand, do not try.
-*/
-   iaa = ntohl(((const struct in_addr *)ip1)->s_addr);
-   iab = ntohl(((const struct in_addr *)ip2)->s_addr);
-
-   /*
-* Do not simply return the difference of the two numbers, the int is
-* not wide enough.
-*/
-   if (iaa > iab)
-   return (1);
-   else if (iaa < iab)
-   return (-1);
-   else
-   return (0);
-}
-#endif
-
-#ifdef INET6
-static int
-qcmp_v6(const void *ip1, const void *ip2)
-{
-   const struct in6_addr *ia6a, *ia6b;
-   int i, rc;
-
-   ia6a = (const struct in6_addr *)ip1;
-   ia6b = (const struct in6_addr *)ip2;
-
-   rc = 0;
-   for (i = 0; rc == 0 && i < sizeof(struct in6_addr); i++) {
-   if (ia6a->s6_addr[i] > ia6b->s6_addr[i])
-   rc = 1;
-   else if (ia6a->s6_addr[i] < ia6b->s6_addr[i])
-   rc = -1;
-   }
-   return (rc);
-}
-#endif
-
 /*
  * struct jail_args {
  * struct jail *jail;
@@ -845,7 +789,8 @@ kern_jail_set(struct thread *td, struct 
 * address to connect from.
 */
if (ip4s > 1)
-   qsort(ip4 + 1, ip4s - 1, sizeof(*ip4), qcmp_v4);
+   qsort(ip4 + 1, ip4s - 1, sizeof(*ip4),
+   prison_qcmp_v4);
/*
 * Check for duplicate addresses and do some simple
 * zero and broadcast checks. If users give other bogus
@@ -893,7 +838,8 @@ kern_jail_set(struct thread *td, struct 
ip6 = malloc(ip6s * sizeof(*ip6), M_PRISON, M_WAITOK);
bcopy(op, ip6, ip6s * sizeof(*ip6));
if (ip6s > 1)
-   qsort(ip6 + 1, ip6s - 1, sizeof(*ip6

svn commit: r316942 - in stable/10/bin/sh: . tests/expansion

[Jilles Tjoelker]

Author: jilles
Date: Fri Apr 14 21:42:27 2017
New Revision: 316942
URL: https://svnweb.freebsd.org/changeset/base/316942

Log:
  MFC r314686: sh: Fix crash if a -T trap is taken during command substitution.
  
  Code like  t=$(stat -f %m "$file")  segfaulted if -T was active and a trap
  was taken while the shell was waiting for the child process to finish.
  
  What happened was that the dotrap() call in waitforjob() was hit. This
  re-entered command execution (including expand.c) at a point not expected by
  expbackq(), and global state (unallocated stack string and argbackq) was
  corrupted.
  
  To fix this, change expbackq() to prepare for command execution to be
  re-entered.
  
  In stable/10, there is more global state that needs to be restored than in
  stable/11 and head.
  
  Reported by:  bdrewery

Added:
  stable/10/bin/sh/tests/expansion/cmdsubst21.0
 - copied unchanged from r314686, head/bin/sh/tests/expansion/cmdsubst21.0
  stable/10/bin/sh/tests/expansion/cmdsubst22.0
 - copied unchanged from r314686, head/bin/sh/tests/expansion/cmdsubst22.0
Modified:
  stable/10/bin/sh/expand.c
  stable/10/bin/sh/tests/expansion/Makefile
Directory Properties:
  stable/10/   (props changed)

Modified: stable/10/bin/sh/expand.c
==
--- stable/10/bin/sh/expand.c   Fri Apr 14 20:15:34 2017(r316941)
+++ stable/10/bin/sh/expand.c   Fri Apr 14 21:42:27 2017(r316942)
@@ -439,9 +439,6 @@ expbackq(union node *cmd, int quoted, in
p = grabstackstr(dest);
evalbackcmd(cmd, &in);
ungrabstackstr(p, dest);
-   ifsfirst = saveifs;
-   ifslastp = savelastp;
-   argbackq = saveargbackq;
 
p = in.buf;
lastc = '\0';
@@ -479,14 +476,20 @@ expbackq(union node *cmd, int quoted, in
close(in.fd);
if (in.buf)
ckfree(in.buf);
-   if (in.jp)
+   if (in.jp) {
+   p = grabstackstr(dest);
exitstatus = waitforjob(in.jp, (int *)NULL);
-   if (quoted == 0)
-   recordregion(startloc, dest - stackblock(), 0);
+   ungrabstackstr(p, dest);
+   }
TRACE(("expbackq: size=%td: \"%.*s\"\n",
((dest - stackblock()) - startloc),
(int)((dest - stackblock()) - startloc),
stackblock() + startloc));
+   ifsfirst = saveifs;
+   ifslastp = savelastp;
+   if (quoted == 0)
+   recordregion(startloc, dest - stackblock(), 0);
+   argbackq = saveargbackq;
expdest = dest;
INTON;
 }

Modified: stable/10/bin/sh/tests/expansion/Makefile
==
--- stable/10/bin/sh/tests/expansion/Makefile   Fri Apr 14 20:15:34 2017
(r316941)
+++ stable/10/bin/sh/tests/expansion/Makefile   Fri Apr 14 21:42:27 2017
(r316942)
@@ -41,6 +41,8 @@ FILES+=   cmdsubst17.0
 FILES+=cmdsubst18.0
 FILES+=cmdsubst19.0
 FILES+=cmdsubst20.0
+FILES+=cmdsubst21.0
+FILES+=cmdsubst22.0
 FILES+=export1.0
 FILES+=export2.0
 FILES+=export3.0

Copied: stable/10/bin/sh/tests/expansion/cmdsubst21.0 (from r314686, 
head/bin/sh/tests/expansion/cmdsubst21.0)
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ stable/10/bin/sh/tests/expansion/cmdsubst21.0   Fri Apr 14 21:42:27 
2017(r316942, copy of r314686, head/bin/sh/tests/expansion/cmdsubst21.0)
@@ -0,0 +1,6 @@
+# $FreeBSD$
+
+set -T
+trapped=''
+trap "trapped=x$trapped" TERM
+[ "x$($SH -c "kill $$")y" = xy ] && [ "$trapped" = x ]

Copied: stable/10/bin/sh/tests/expansion/cmdsubst22.0 (from r314686, 
head/bin/sh/tests/expansion/cmdsubst22.0)
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ stable/10/bin/sh/tests/expansion/cmdsubst22.0   Fri Apr 14 21:42:27 
2017(r316942, copy of r314686, head/bin/sh/tests/expansion/cmdsubst22.0)
@@ -0,0 +1,6 @@
+# $FreeBSD$
+
+set -T
+trapped=''
+trap "trapped=x$trapped" TERM
+[ "x$(:; kill $$)y" = xy ] && [ "$trapped" = x ]
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[John Baldwin]

On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
> 
> Log:
>   savecore: fix space calculation with respect to `minfree` in check_space(..)
>   
>   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> representable data to INT_MAX. Check the values received from
> strtoll(3), trimming trailing whitespace off the end to maintain
> POLA.
>   - Use `KiB` instead of `kB` when describing free space, total space,
> etc. I am now fully aware of `KiB` being the IEC standard for 1024
> bytes and `kB` being the IEC standard for 1000 bytes.

I will just rant lightly that no one actually uses this in the real world.

Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
kilobyte is a power of 2.  The End.

(Next up we'll have to rename 4k displays to
4k)

-- 
John Baldwin
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316941 - head/sys/kern

[Bruce Evans]

On Fri, 14 Apr 2017, Edward Tomasz Napierala wrote:


Log:
 Don't try to write out bufs that have already failed with ENXIO.
 This fixes some panics after disconnecting mounted disks.

 Submitted by:  imp (slightly different version, which I've then lost)
 Reviewed by:   kib, imp, mckusick
 MFC after: 2 weeks
 Differential Revision: https://reviews.freebsd.org/D9674

Modified:
 head/sys/kern/vfs_bio.c

Modified: head/sys/kern/vfs_bio.c
==
--- head/sys/kern/vfs_bio.c Fri Apr 14 20:15:17 2017(r316940)
+++ head/sys/kern/vfs_bio.c Fri Apr 14 20:15:34 2017(r316941)
@@ -2290,18 +2290,28 @@ brelse(struct buf *bp)
bdirty(bp);
}
if (bp->b_iocmd == BIO_WRITE && (bp->b_ioflags & BIO_ERROR) &&
+   (bp->b_error != ENXIO || !LIST_EMPTY(&bp->b_dep)) &&
!(bp->b_flags & B_INVAL)) {
/*
-* Failed write, redirty.  Must clear BIO_ERROR to prevent
-* pages from being scrapped.
+* Failed write, redirty.  All errors except ENXIO (which
+* means the device is gone) are expected to be potentially
+* transient - underlying media might work if tried again
+* after EIO, and memory might be available after an ENOMEM.
+*
+* Do this also for buffers that failed with ENXIO, but have
+* non-empty dependencies - the soft updates code might need
+* to access the buffer to untangle them.
+*
+* Must clear BIO_ERROR to prevent pages from being scrapped.
 */


This is hard to fix, but I have used a version that only retries after
EIO for 15-20 years.  I didn't think of ENOMEM.

The media is unlikely to come back after EIO too.  For removable media,
you might be able to get the write done to new media, but a panic reading
from the new media is just as likely.  Geom "tasting" might prevent the
new media being used.

ENXIO is actually the one error that can often be recovered from.  I
wrote a form of "tasting" in a toy OS 30-35 years ago.  It handled
removal of "mounted" disks with pending writes too well, in a way that
made recovery from non-transient I/O errors almost impossible without
turning off the system.  ENXIO was treated as a transient I/O error.
It was recovered from perfectly if the user could find the original
media and unremove it.  The "tasting" usually worked to detect different
media and disallow writing cached data to a different disk.  Media
errors were common, and when one occurred for writing the method of
replacing the disk by a garbage one did't work since it was a different
disk.  The most common one was writing to a write protected disk, and
that was recoverable by removing the write protection.  But often you
really didn't want to write to that disk, but wanted to write somewhere.
The only way to continue was to reboot to discard the write.

Bruce
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316822 - in head/sys: kern netinet netinet6

[Gleb Smirnoff]

  Andrey,

On Fri, Apr 14, 2017 at 09:00:48AM +, Andrey V. Elsukov wrote:
A> Log:
A>   Rework r316770 to make it protocol independent and general, like we
A>   do for streaming sockets.
A>   
A>   And do more cleanup in the sbappendaddr_locked_internal() to prevent
A>   leak information from existing mbuf to the one, that will be possible
A>   created later by netgraph.

Thanks!

Your comment actually made me think that this can be improved...

A> @@ -794,8 +794,20 @@ sbappendaddr_locked_internal(struct sock
A>  return (0);
A>  m->m_len = asa->sa_len;
A>  bcopy(asa, mtod(m, caddr_t), asa->sa_len);
A> -if (m0)
A> +if (m0) {
A>  m_clrprotoflags(m0);
A> +m_tag_delete_chain(m0);
A> +/*
A> + * Clear some persistent info from pkthdr.
A> + * We don't use m_demote(), because some netgraph consumers
A> + * expect M_PKTHDR presence.
A> + */

Looks like you are right and only netgraph wants to see M_PKTHDR there.
Rest of datagram socket consumers just use m_nextpkt to differentiate
records from parts of records.

I'll see if we can fix ng_ksocket and use same code on datagram sockbufs
as we do on stream ones.

-- 
Totus tuus, Glebius.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Xin LI]

On Fri, Apr 14, 2017 at 1:29 PM, Mark Johnston  wrote:
> - I'm not sure how encryption should compose with compression. It seems
>   intuitively obvious that we should compress before encrypting if the
>   compression is to be of any use, but I don't know enough to know
>   whether the compression might somehow compromise the effectiveness of
>   the encryption.

I think the biggest concern is the added code involved in the dump
process (which happen when the kernel is already unhealthy), which can
jeopardize it and defeat the usefulness of having a crash dump being
set up in the first place.  And with textdumps available, the benefit
of having compression is limited because we can request for minidump
or full dumps only when the textdumps are not good enough for
diagnosing the kernel bug.

I don't think security (e.g. leaking information because of the use of
compression) is a very big concern in this context because in order
for the potential attacker to read the raw material needs a
compromised system (unlike an attack from the network, where someone
who controls the network would have access to the raw material); the
dump is usually quite large, and measuring downtime would be hard at
that scale.

By the way (not meant to bikeshed) if I was to do this I'd prefer
using lz4 or something that compresses faster than zlib.

Cheers,
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Maxim Sobolev]

Peter, It is actually the other way around. If you take syslogd code out of
11-stable and earlier that would work just fine with or without r316874.
But since r285910 syslogd in head had been refactored a lot and I think
that particular bug has introduced that has been masked by the shutdown()
on datagram sockets becoming a NOP after r285910. Then r316874 restored our
historical behavior for the shutdown(2) and bingo, bug in the new syslogd
code is now causing it to spin when shutdown() != NOP.

-Max

On Fri, Apr 14, 2017 at 12:46 PM, Peter Wemm  wrote:

> On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> > Thanks, Peter. I will try to look into this asap.
>
> I don't understand what is going on yet. Presumably there must be other
> changes in play that affect udp/select sometime between the original 2015
> change and this. The syslogd -s code is Old(TM).  I'm also wondering
> whether
> the -s code even works at all since the 2015 / r285910 change...
>
> > -Max
> >
> > On Apr 14, 2017 12:32 PM, "Peter Wemm"  wrote:
> > > On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> > > > On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > > > > Author: sobomax
> > > > > Date: Fri Apr 14 17:23:28 2017
> > > > > New Revision: 316874
> > > > > URL: https://svnweb.freebsd.org/changeset/base/316874
> > > > >
> > > > > Log:
> > > > >   Restore ability to shutdown DGRAM sockets, still forcing
> ENOTCONN to
> > >
> > > be
> > >
> > > > > returned by the shutdown(2) system call. This ability has been
> lost as
> > > > > part
> > > > > of the svn revision 285910.
> > > > >
> > > > >   Reviewed by:  ed, rwatson, glebius, hiren
> > > > >   MFC after:2 weeks
> > > > >   Differential Revision:https://reviews.freebsd.org/D10351
> > > >
> > > > This appears to have broken syslogd and had a major change in
> behavior
> > >
> > > with
> > >
> > > > regards to select(2).
> > > >
> > > > If you run syslogd with the -s flag (which is default), it now spins
> at
> > >
> > > 100%
> > >
> > > > cpu as all the shutdown sockets now return readable from select.
> > > >
> > > > Old releases / jails also manifest this behavor.  If it wasn't for
> > > > losing
> > > > the ability to run old branch binaries I'd suggest changing syslogd
> > > > instead, but we depend on this in the cluster and I expect others do
> > > > too.
> > > >
> > > > I'm not 100% certain that this change is the culprit but a heads-up
> > > > can't
> > > > hurt. I'll try reverting it on the freebsd cluster next, after fixing
> > > > the
> > > > broken auditing changes.
> > > >
> > > > -Peter
> > >
> > > I can confirm that reverting r316874 fixes syslogd and backwards
> > > compatability
> > > with old branches.
> > >
> > > --
> > > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > > KI6FJV
> > > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
>
> --
> Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> KI6FJV
> UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
>
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 14:10, Maxim Sobolev  wrote:
> 
> Peter, Ngie,
> 
> Looks like out of that refactoring came a logical bug that is present in the 
> head, which causes syslod to first to shutdown the socket for reading and 
> then try to select/recv on it (which is somewhat stupid). And that issue has 
> been masked by shutdown() on datagram socket becoming effectively a NOP in 11 
> & head 20 months ago. It only affects head though, 11-stable still has the 
> old code which does not include that half-closed socket into the select list. 
> Attached patch is expected to fix head, Peter, it would be nice if you can 
> give it a try (restoring latest changes into uipc_sockets.c) and let me know 
> if it helps.
> 
> Thanks!

CCing hrs@ for input as he did the refactoring.
Thanks!
-Ngie

PS LGTM with the change. Will wait for feedback from wemm@.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316874 - head/sys/kern

[Maxim Sobolev]

Peter, Ngie,

Looks like out of that refactoring came a logical bug that is present in
the head, which causes syslod to first to shutdown the socket for reading
and then try to select/recv on it (which is somewhat stupid). And that
issue has been masked by shutdown() on datagram socket becoming effectively
a NOP in 11 & head 20 months ago. It only affects head though, 11-stable
still has the old code which does not include that half-closed socket into
the select list. Attached patch is expected to fix head, Peter, it would be
nice if you can give it a try (restoring latest changes into
uipc_sockets.c) and let me know if it helps.

Thanks!

On Fri, Apr 14, 2017 at 12:48 PM, Ngie Cooper (yaneurabeya) <
yaneurab...@gmail.com> wrote:

>
> > On Apr 14, 2017, at 12:46, Peter Wemm  wrote:
> >
> > On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> >> Thanks, Peter. I will try to look into this asap.
> >
> > I don't understand what is going on yet. Presumably there must be other
> > changes in play that affect udp/select sometime between the original 2015
> > change and this. The syslogd -s code is Old(TM).  I'm also wondering
> whether
> > the -s code even works at all since the 2015 / r285910 change...
>
> syslogd has been refactored a lot on ^/head. I don’t think it’s safe to
> say that the ^/head and ^/stable/11 and earlier copies will function the
> same.
> Thanks,
> -Ngie
>



-- 
Maksym Sobolyev
Sippy Software, Inc.
Internet Telephony (VoIP) Experts
Tel (Canada): +1-778-783-0474
Tel (Toll-Free): +1-855-747-7779
Fax: +1-866-857-6942
Web: http://www.sippysoft.com
MSN: sa...@sippysoft.com
Skype: SippySoft
Index: syslogd.c
===
--- syslogd.c   (revision 316854)
+++ syslogd.c   (working copy)
@@ -702,7 +702,7 @@
sizeof(fd_mask));
 
STAILQ_FOREACH(sl, &shead, next) {
-   if (sl->sl_socket != -1)
+   if (sl->sl_socket != -1 && sl->sl_recv != NULL)
FD_SET(sl->sl_socket, fdsr);
}
i = select(fdsrmax + 1, fdsr, NULL, NULL,
@@ -2877,6 +2877,7 @@
struct addrinfo hints, *res, *res0;
int error;
char *cp;
+   int (*sl_recv)(struct socklist *);
/*
 * We have to handle this case for backwards compatibility:
 * If there are two (or more) colons but no '[' and ']',
@@ -3003,6 +3004,7 @@
}
dprintf("new socket fd is %d\n", s);
listen(s, 5);
+   sl_recv = socklist_recv_sock;
dprintf("shutdown\n");
if (SecureMode || res->ai_family == AF_LOCAL) {
/* Forbid communication in secure mode. */
@@ -3013,6 +3015,7 @@
die(0);
}
dprintf("listening on socket\n");
+   sl_recv = NULL;
} else
dprintf("sending on socket\n");
addsock(res->ai_addr, res->ai_addrlen,
@@ -3019,7 +3022,7 @@
&(struct socklist){
.sl_socket = s,
.sl_peer = pe,
-   .sl_recv = socklist_recv_sock
+   .sl_recv = sl_recv
});
}
freeaddrinfo(res0);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Xin LI]

No, this is not the threat model for swaps.

On Fri, Apr 14, 2017 at 1:47 PM, Ronald Klop  wrote:
> On Fri, 14 Apr 2017 22:29:18 +0200, Mark Johnston  wrote:
>
>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>>>
>>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>>> > Author: ngie
>>> > Date: Fri Apr 14 19:41:48 2017
>>> > New Revision: 316938
>>> > URL: https://svnweb.freebsd.org/changeset/base/316938
>>> >
>>> > Log:
>>> >   savecore: fix space calculation with respect to `minfree` in
>>> > check_space(..)
>>> >
>>> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>> > representable data to INT_MAX. Check the values received from
>>> > strtoll(3), trimming trailing whitespace off the end to maintain
>>> > POLA.
>>> >   - Use `KiB` instead of `kB` when describing free space, total space,
>>> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>> > bytes and `kB` being the IEC standard for 1000 bytes.
>>> >   - Store available number of KiB in `available` so it can be more
>>> > easily queried and compared to ensure that there are enough KiB to
>>> > store the dump image on disk.
>>> >   - Print out the reserved space on disk, per `minfree`, so end-users
>>> > can troubleshoot why check_space(..) is reporting that there isn't
>>> > enough free space.
>>> >
>>> >   MFC after:7 weeks
>>> >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier
>>> > diff)
>>> >   Tested with:  positive/negative cases (see review); make tinderbox
>>> >   Sponsored by: Dell EMC Isilon
>>> >   Differential Revision:D10379
>>>
>>> The free space calculation is still uselessly conservative, because it
>>> doesn't account for the fact that core dumps will always be either
>>> spare or compressed.  The result is that savecore will frequently
>>> refuse to save corefiles even when there's plenty of space.  I
>>> proposed removing the space check altogether in
>>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>>> after the compressed core dump feature was merged, because then mostly
>>> accurate space checks will be possible.  AFAIK the compressed core
>>> dump feature still hasn't been finished.
>>
>>
>> I had held off on it for a while because it was going to conflict with
>> the work to add encrypted dump support, which of course has finished.
>>
>> The patch to add compression support is here and should largely still
>> work:
>>
>> https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff
>>
>> I've been hesitant about pushing it forward:
>> - The dump_write* APIs need some simplification after the addition of
>>   encrypted dump support and support for dumping to 4Kn drives.
>> - I'm not sure how encryption should compose with compression. It seems
>>   intuitively obvious that we should compress before encrypting if the
>>   compression is to be of any use, but I don't know enough to know
>>   whether the compression might somehow compromise the effectiveness of
>>   the encryption.
>>
>> If anyone has some insight on the second of these two points, I'd
>> appreciate hearing it.
>
>
> About compression before encryption there is some information. For example:
> https://en.wikipedia.org/wiki/CRIME
> Openssh removed compression in pre-authentication in
> http://www.openssh.com/txt/release-7.4
>
> Regards,
> Ronald.
>
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Conrad Meyer]

On Fri, Apr 14, 2017 at 1:41 PM, Alan Somers  wrote:
> On Fri, Apr 14, 2017 at 2:29 PM, Mark Johnston  wrote:
>> I've been hesitant about pushing it forward:
>> - The dump_write* APIs need some simplification after the addition of
>>   encrypted dump support and support for dumping to 4Kn drives.
>> - I'm not sure how encryption should compose with compression. It seems
>>   intuitively obvious that we should compress before encrypting if the
>>   compression is to be of any use, but I don't know enough to know
>>   whether the compression might somehow compromise the effectiveness of
>>   the encryption.
>>
>> If anyone has some insight on the second of these two points, I'd
>> appreciate hearing it.
>
> I think compress then encrypt should be ok.  AFAIK all attacks against
> compress-then-encrypt systems have involved either incredibly short
> payloads that are easy to guess, or a stream of separately compressed
> blocks that can be fingerprinted.  But core dumps are very long, and
> they can't be fingerprinted in whole because they're unique.  If you
> were to encrypt each page individually then pages could be
> fingerprinted, so don't do that.  Instead, compress the entire core
> dump as a single stream and you should be ok.

+1.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:48, Larry Rosenman  wrote:

…

> No can do, the rest of the space is pool.

>_>… that sucks. This is part of the reason why I still make my swap partitions 
>greater than or equal to the amount of memory I have in systems :/..
The best recommendation I have is either: wait for compressed dump support, 
repartition/shuffle around data on your disks so the configuration fits, or try 
and get ccd to work *shrugs*.
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 3:46 PM, "Ngie Cooper (yaneurabeya)"  wrote:


> On Apr 14, 2017, at 13:43, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:39 PM, "Ngie Cooper (yaneurabeya)" 
 wrote:
> 
>> On Apr 14, 2017, at 13:37, Larry Rosenman  wrote:
>> 
>> On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)"  
wrote:
>> 
>>> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
>>> 
>>> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
 wrote:
>>> 
 On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
 
 On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
 
> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>> Author: ngie
>> Date: Fri Apr 14 19:41:48 2017
>> New Revision: 316938
>> URL: https://svnweb.freebsd.org/changeset/base/316938
>> 
>> Log:
>> savecore: fix space calculation with respect to `minfree` in 
check_space(..)
>> 
>> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>> representable data to INT_MAX. Check the values received from
>> strtoll(3), trimming trailing whitespace off the end to maintain
>> POLA.
>> - Use `KiB` instead of `kB` when describing free space, total space,
>> etc. I am now fully aware of `KiB` being the IEC standard for 1024
>> bytes and `kB` being the IEC standard for 1000 bytes.
>> - Store available number of KiB in `available` so it can be more
>> easily queried and compared to ensure that there are enough KiB to
>> store the dump image on disk.
>> - Print out the reserved space on disk, per `minfree`, so end-users
>> can troubleshoot why check_space(..) is reporting that there isn't
>> enough free space.
>> 
>> MFC after:7 weeks
>> Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
diff)
>> Tested with:  positive/negative cases (see review); make tinderbox
>> Sponsored by: Dell EMC Isilon
>> Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.
 
 Is posible (in the future) to use multiple swaps (on multiple disks)
 for save core dumps?
>>> 
>>>  Multiple swap devices is already handled by savecore(8), if one uses 
fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
devices.
>>> 
>>>  As far as saving to multiple disks is concerned, I would hope that one 
is using a redundancy capable filesystem (zfs) or RAID-like technology 
(gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror the 
data across multiple disks.
>> 
>>   …
>> 
>>> How do I use multiple devices to have the system dump on all of my 
swap?  I got a message about not enough space, but there (I think) was enough 
between multiple drives….
>> 
>>   Something like:
>> 
>>   - Create a zpool
>>   - Mount zpool to /crashdumps
>>   - Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
‘dumpdir=/crashdumps’
>> 
>>   ?
>>   HTH,
>>   -Ngie
>> 
>>   PS The issue with lack of space might be the issue that Alan brought 
up earlier with compressed dumps and overly conservative free space checks, or 
it might be the fact that dumpdir (default: /var/crash) is full.
>> 
>> 
>> I was talking about the actual crashdump to swap by the system.  
/var/crash has 10T of space (my root pool).
> 
>If your memory is bigger than your swap, you’re unfortunately not able 
to save the mini dump if the size of the saved pages exceed the space on swap. 
I think this is where markj’s compressed dumps feature will come in handy.
>Thanks!
>-Ngie
> 
> 
> Yeah, I have the following:
> borg.lerctr.org /home/ler $ swapctl -l
> Device:   1024-blocks Used:
> /dev/mfid0p38388608 0
> /dev/mfid1p38388608 0
> /dev/mfid2p38388608 0
> /dev/mfid3p38388608 0
> /dev/mfid4p38388608 0
> /dev/mfid5p38388608 0
> borg.lerctr.org /home/ler $ sysctl hw.physmem
> hw.physmem: 137368682496
> borg.lerctr.org /home/ler $
> 
> SO 6 8G partitions (48G), but the dump is larger than 8G.
 

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:46, Slawa Olhovchenkov  wrote:
> 
> On Fri, Apr 14, 2017 at 01:45:22PM -0700, Ngie Cooper (yaneurabeya) wrote:
> 
>> 
>>> On Apr 14, 2017, at 13:42, Slawa Olhovchenkov  wrote:
>> 
>> …
>> 
>>> No, something like:
>>> % swapinfo
>>> Device  1K-blocks UsedAvail Capacity
>>> /dev/ada0p2  335544320 33554432 0%
>>> /dev/ada1p2  335544320 33554432 0%
>>> Total671088640 67108864 0%
>> 
>> Striping dumps across multiple swap devices isn’t supported. Sorry.
> 
> What about support in the future?

*shrugs* I don’t have a crystal ball to say yay or nay, but saving full dumps 
out to disk seems foolhardy to say the least. Shrinking down the dataset seems 
like a more sensible problem to solve than striping data across disks.
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Ronald Klop]

On Fri, 14 Apr 2017 22:29:18 +0200, Mark Johnston   
wrote:



On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:

On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
>
> Log:
>   savecore: fix space calculation with respect to `minfree` in  
check_space(..)

>
>   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> representable data to INT_MAX. Check the values received from
> strtoll(3), trimming trailing whitespace off the end to maintain
> POLA.
>   - Use `KiB` instead of `kB` when describing free space, total space,
> etc. I am now fully aware of `KiB` being the IEC standard for 1024
> bytes and `kB` being the IEC standard for 1000 bytes.
>   - Store available number of KiB in `available` so it can be more
> easily queried and compared to ensure that there are enough KiB to
> store the dump image on disk.
>   - Print out the reserved space on disk, per `minfree`, so end-users
> can troubleshoot why check_space(..) is reporting that there isn't
> enough free space.
>
>   MFC after:7 weeks
>   Reviewed by:  Anton Rang  (earlier diff), cem  
(earlier diff)

>   Tested with:  positive/negative cases (see review); make tinderbox
>   Sponsored by: Dell EMC Isilon
>   Differential Revision:D10379

The free space calculation is still uselessly conservative, because it
doesn't account for the fact that core dumps will always be either
spare or compressed.  The result is that savecore will frequently
refuse to save corefiles even when there's plenty of space.  I
proposed removing the space check altogether in
https://reviews.freebsd.org/D2587.  However, I agreed to wait until
after the compressed core dump feature was merged, because then mostly
accurate space checks will be possible.  AFAIK the compressed core
dump feature still hasn't been finished.


I had held off on it for a while because it was going to conflict with
the work to add encrypted dump support, which of course has finished.

The patch to add compression support is here and should largely still
work:
https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff

I've been hesitant about pushing it forward:
- The dump_write* APIs need some simplification after the addition of
  encrypted dump support and support for dumping to 4Kn drives.
- I'm not sure how encryption should compose with compression. It seems
  intuitively obvious that we should compress before encrypting if the
  compression is to be of any use, but I don't know enough to know
  whether the compression might somehow compromise the effectiveness of
  the encryption.

If anyone has some insight on the second of these two points, I'd
appreciate hearing it.


About compression before encryption there is some information. For  
example: https://en.wikipedia.org/wiki/CRIME
Openssh removed compression in pre-authentication in  
http://www.openssh.com/txt/release-7.4


Regards,
Ronald.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:43, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:39 PM, "Ngie Cooper (yaneurabeya)" 
>  wrote:
> 
>> On Apr 14, 2017, at 13:37, Larry Rosenman  wrote:
>> 
>> On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)"  
>> wrote:
>> 
>>> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
>>> 
>>> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
>>>  wrote:
>>> 
 On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
 
 On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
 
> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>> Author: ngie
>> Date: Fri Apr 14 19:41:48 2017
>> New Revision: 316938
>> URL: https://svnweb.freebsd.org/changeset/base/316938
>> 
>> Log:
>> savecore: fix space calculation with respect to `minfree` in 
>> check_space(..)
>> 
>> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>> representable data to INT_MAX. Check the values received from
>> strtoll(3), trimming trailing whitespace off the end to maintain
>> POLA.
>> - Use `KiB` instead of `kB` when describing free space, total space,
>> etc. I am now fully aware of `KiB` being the IEC standard for 1024
>> bytes and `kB` being the IEC standard for 1000 bytes.
>> - Store available number of KiB in `available` so it can be more
>> easily queried and compared to ensure that there are enough KiB to
>> store the dump image on disk.
>> - Print out the reserved space on disk, per `minfree`, so end-users
>> can troubleshoot why check_space(..) is reporting that there isn't
>> enough free space.
>> 
>> MFC after:7 weeks
>> Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
>> diff)
>> Tested with:  positive/negative cases (see review); make tinderbox
>> Sponsored by: Dell EMC Isilon
>> Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.
 
 Is posible (in the future) to use multiple swaps (on multiple disks)
 for save core dumps?
>>> 
>>>  Multiple swap devices is already handled by savecore(8), if one uses 
>>> fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
>>> devices.
>>> 
>>>  As far as saving to multiple disks is concerned, I would hope that one is 
>>> using a redundancy capable filesystem (zfs) or RAID-like technology 
>>> (gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror 
>>> the data across multiple disks.
>> 
>>   …
>> 
>>> How do I use multiple devices to have the system dump on all of my swap?  I 
>>> got a message about not enough space, but there (I think) was enough 
>>> between multiple drives….
>> 
>>   Something like:
>> 
>>   - Create a zpool
>>   - Mount zpool to /crashdumps
>>   - Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
>> ‘dumpdir=/crashdumps’
>> 
>>   ?
>>   HTH,
>>   -Ngie
>> 
>>   PS The issue with lack of space might be the issue that Alan brought up 
>> earlier with compressed dumps and overly conservative free space checks, or 
>> it might be the fact that dumpdir (default: /var/crash) is full.
>> 
>> 
>> I was talking about the actual crashdump to swap by the system.  /var/crash 
>> has 10T of space (my root pool).
> 
>If your memory is bigger than your swap, you’re unfortunately not able to 
> save the mini dump if the size of the saved pages exceed the space on swap. I 
> think this is where markj’s compressed dumps feature will come in handy.
>Thanks!
>-Ngie
> 
> 
> Yeah, I have the following:
> borg.lerctr.org /home/ler $ swapctl -l
> Device:   1024-blocks Used:
> /dev/mfid0p38388608 0
> /dev/mfid1p38388608 0
> /dev/mfid2p38388608 0
> /dev/mfid3p38388608 0
> /dev/mfid4p38388608 0
> /dev/mfid5p38388608 0
> borg.lerctr.org /home/ler $ sysctl hw.physmem
> hw.physmem: 137368682496
> borg.lerctr.org /home/ler $
> 
> SO 6 8G partitions (48G), but the dump is larger than 8G.

Yup, that’s not going to work. Since you’re using mfi(4), I’d try creating 
another 48GB large RAID-0 volume to help deal with that issue.
Cheers,
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Fri, Apr 14, 2017 at 01:45:22PM -0700, Ngie Cooper (yaneurabeya) wrote:

> 
> > On Apr 14, 2017, at 13:42, Slawa Olhovchenkov  wrote:
> 
> …
> 
> > No, something like:
> > % swapinfo
> > Device  1K-blocks UsedAvail Capacity
> > /dev/ada0p2  335544320 33554432 0%
> > /dev/ada1p2  335544320 33554432 0%
> > Total671088640 67108864 0%
> 
> Striping dumps across multiple swap devices isn’t supported. Sorry.

What about support in the future?

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:42, Slawa Olhovchenkov  wrote:

…

> No, something like:
> % swapinfo
> Device  1K-blocks UsedAvail Capacity
> /dev/ada0p2  335544320 33554432 0%
> /dev/ada1p2  335544320 33554432 0%
> Total671088640 67108864 0%

Striping dumps across multiple swap devices isn’t supported. Sorry.
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 3:39 PM, "Ngie Cooper (yaneurabeya)"  wrote:


> On Apr 14, 2017, at 13:37, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)"  
wrote:
> 
>> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
>> 
>> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
 wrote:
>> 
>>> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
>>> 
>>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>>> 
 On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
> 
> Log:
> savecore: fix space calculation with respect to `minfree` in 
check_space(..)
> 
> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>  representable data to INT_MAX. Check the values received from
>  strtoll(3), trimming trailing whitespace off the end to maintain
>  POLA.
> - Use `KiB` instead of `kB` when describing free space, total space,
>  etc. I am now fully aware of `KiB` being the IEC standard for 1024
>  bytes and `kB` being the IEC standard for 1000 bytes.
> - Store available number of KiB in `available` so it can be more
>  easily queried and compared to ensure that there are enough KiB to
>  store the dump image on disk.
> - Print out the reserved space on disk, per `minfree`, so end-users
>  can troubleshoot why check_space(..) is reporting that there isn't
>  enough free space.
> 
> MFC after:7 weeks
> Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
diff)
> Tested with:  positive/negative cases (see review); make tinderbox
> Sponsored by: Dell EMC Isilon
> Differential Revision:D10379
 
 The free space calculation is still uselessly conservative, because it
 doesn't account for the fact that core dumps will always be either
 spare or compressed.  The result is that savecore will frequently
 refuse to save corefiles even when there's plenty of space.  I
 proposed removing the space check altogether in
 https://reviews.freebsd.org/D2587.  However, I agreed to wait until
 after the compressed core dump feature was merged, because then mostly
 accurate space checks will be possible.  AFAIK the compressed core
 dump feature still hasn't been finished.
>>> 
>>> Is posible (in the future) to use multiple swaps (on multiple disks)
>>> for save core dumps?
>> 
>>   Multiple swap devices is already handled by savecore(8), if one uses 
fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
devices.
>> 
>>   As far as saving to multiple disks is concerned, I would hope that one 
is using a redundancy capable filesystem (zfs) or RAID-like technology 
(gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror the 
data across multiple disks.
> 
>…
> 
>> How do I use multiple devices to have the system dump on all of my swap? 
 I got a message about not enough space, but there (I think) was enough between 
multiple drives….
> 
>Something like:
> 
>- Create a zpool
>- Mount zpool to /crashdumps
>- Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
‘dumpdir=/crashdumps’
> 
>?
>HTH,
>-Ngie
> 
>PS The issue with lack of space might be the issue that Alan brought 
up earlier with compressed dumps and overly conservative free space checks, or 
it might be the fact that dumpdir (default: /var/crash) is full.
> 
> 
> I was talking about the actual crashdump to swap by the system.  
/var/crash has 10T of space (my root pool).

If your memory is bigger than your swap, you’re unfortunately not able to 
save the mini dump if the size of the saved pages exceed the space on swap. I 
think this is where markj’s compressed dumps feature will come in handy.
Thanks!
-Ngie


Yeah, I have the following:
borg.lerctr.org /home/ler $ swapctl -l
Device:   1024-blocks Used:
/dev/mfid0p38388608 0
/dev/mfid1p38388608 0
/dev/mfid2p38388608 0
/dev/mfid3p38388608 0
/dev/mfid4p38388608 0
/dev/mfid5p38388608 0
borg.lerctr.org /home/ler $ sysctl hw.physmem
hw.physmem: 137368682496
borg.lerctr.org /home/ler $

SO 6 8G partitions (48G), but the dump is larger than 8G. 





___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Fri, Apr 14, 2017 at 01:33:36PM -0700, Ngie Cooper (yaneurabeya) wrote:

> 
> > On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
> > 
> > On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
> >  wrote:
> > 
> >> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
> >> 
> >> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> >> 
> >>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>  Author: ngie
>  Date: Fri Apr 14 19:41:48 2017
>  New Revision: 316938
>  URL: https://svnweb.freebsd.org/changeset/base/316938
>  
>  Log:
>  savecore: fix space calculation with respect to `minfree` in 
>  check_space(..)
>  
>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>    representable data to INT_MAX. Check the values received from
>    strtoll(3), trimming trailing whitespace off the end to maintain
>    POLA.
>  - Use `KiB` instead of `kB` when describing free space, total space,
>    etc. I am now fully aware of `KiB` being the IEC standard for 1024
>    bytes and `kB` being the IEC standard for 1000 bytes.
>  - Store available number of KiB in `available` so it can be more
>    easily queried and compared to ensure that there are enough KiB to
>    store the dump image on disk.
>  - Print out the reserved space on disk, per `minfree`, so end-users
>    can troubleshoot why check_space(..) is reporting that there isn't
>    enough free space.
>  
>  MFC after:7 weeks
>  Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
>  diff)
>  Tested with:  positive/negative cases (see review); make tinderbox
>  Sponsored by: Dell EMC Isilon
>  Differential Revision:D10379
> >>> 
> >>> The free space calculation is still uselessly conservative, because it
> >>> doesn't account for the fact that core dumps will always be either
> >>> spare or compressed.  The result is that savecore will frequently
> >>> refuse to save corefiles even when there's plenty of space.  I
> >>> proposed removing the space check altogether in
> >>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> >>> after the compressed core dump feature was merged, because then mostly
> >>> accurate space checks will be possible.  AFAIK the compressed core
> >>> dump feature still hasn't been finished.
> >> 
> >> Is posible (in the future) to use multiple swaps (on multiple disks)
> >> for save core dumps?
> > 
> >Multiple swap devices is already handled by savecore(8), if one uses 
> > fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
> > devices.
> > 
> >As far as saving to multiple disks is concerned, I would hope that one 
> > is using a redundancy capable filesystem (zfs) or RAID-like technology 
> > (gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror 
> > the data across multiple disks.
> 
> …
> 
> > How do I use multiple devices to have the system dump on all of my swap?  I 
> > got a message about not enough space, but there (I think) was enough 
> > between multiple drives….
> 
> Something like:
> 
> - Create a zpool
> - Mount zpool to /crashdumps
> - Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
> ‘dumpdir=/crashdumps’
> 
> ?

No, something like:
% swapinfo
Device  1K-blocks UsedAvail Capacity
/dev/ada0p2  335544320 33554432 0%
/dev/ada1p2  335544320 33554432 0%
Total671088640 67108864 0%

> 
> PS The issue with lack of space might be the issue that Alan brought up 
> earlier with compressed dumps and overly conservative free space checks, or 
> it might be the fact that dumpdir (default: /var/crash) is full.


___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Alan Somers]

On Fri, Apr 14, 2017 at 2:29 PM, Mark Johnston  wrote:
> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>> > Author: ngie
>> > Date: Fri Apr 14 19:41:48 2017
>> > New Revision: 316938
>> > URL: https://svnweb.freebsd.org/changeset/base/316938
>> >
>> > Log:
>> >   savecore: fix space calculation with respect to `minfree` in 
>> > check_space(..)
>> >
>> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>> > representable data to INT_MAX. Check the values received from
>> > strtoll(3), trimming trailing whitespace off the end to maintain
>> > POLA.
>> >   - Use `KiB` instead of `kB` when describing free space, total space,
>> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
>> > bytes and `kB` being the IEC standard for 1000 bytes.
>> >   - Store available number of KiB in `available` so it can be more
>> > easily queried and compared to ensure that there are enough KiB to
>> > store the dump image on disk.
>> >   - Print out the reserved space on disk, per `minfree`, so end-users
>> > can troubleshoot why check_space(..) is reporting that there isn't
>> > enough free space.
>> >
>> >   MFC after:7 weeks
>> >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
>> > diff)
>> >   Tested with:  positive/negative cases (see review); make tinderbox
>> >   Sponsored by: Dell EMC Isilon
>> >   Differential Revision:D10379
>>
>> The free space calculation is still uselessly conservative, because it
>> doesn't account for the fact that core dumps will always be either
>> spare or compressed.  The result is that savecore will frequently
>> refuse to save corefiles even when there's plenty of space.  I
>> proposed removing the space check altogether in
>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>> after the compressed core dump feature was merged, because then mostly
>> accurate space checks will be possible.  AFAIK the compressed core
>> dump feature still hasn't been finished.
>
> I had held off on it for a while because it was going to conflict with
> the work to add encrypted dump support, which of course has finished.
>
> The patch to add compression support is here and should largely still
> work:
> https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff
>
> I've been hesitant about pushing it forward:
> - The dump_write* APIs need some simplification after the addition of
>   encrypted dump support and support for dumping to 4Kn drives.
> - I'm not sure how encryption should compose with compression. It seems
>   intuitively obvious that we should compress before encrypting if the
>   compression is to be of any use, but I don't know enough to know
>   whether the compression might somehow compromise the effectiveness of
>   the encryption.
>
> If anyone has some insight on the second of these two points, I'd
> appreciate hearing it.

I think compress then encrypt should be ok.  AFAIK all attacks against
compress-then-encrypt systems have involved either incredibly short
payloads that are easy to guess, or a stream of separately compressed
blocks that can be fingerprinted.  But core dumps are very long, and
they can't be fingerprinted in whole because they're unique.  If you
were to encrypt each page individually then pages could be
fingerprinted, so don't do that.  Instead, compress the entire core
dump as a single stream and you should be ok.

-Alan
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:37, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)"  
> wrote:
> 
>> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
>> 
>> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
>>  wrote:
>> 
>>> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
>>> 
>>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>>> 
 On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
> 
> Log:
> savecore: fix space calculation with respect to `minfree` in 
> check_space(..)
> 
> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>  representable data to INT_MAX. Check the values received from
>  strtoll(3), trimming trailing whitespace off the end to maintain
>  POLA.
> - Use `KiB` instead of `kB` when describing free space, total space,
>  etc. I am now fully aware of `KiB` being the IEC standard for 1024
>  bytes and `kB` being the IEC standard for 1000 bytes.
> - Store available number of KiB in `available` so it can be more
>  easily queried and compared to ensure that there are enough KiB to
>  store the dump image on disk.
> - Print out the reserved space on disk, per `minfree`, so end-users
>  can troubleshoot why check_space(..) is reporting that there isn't
>  enough free space.
> 
> MFC after:7 weeks
> Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
> Tested with:  positive/negative cases (see review); make tinderbox
> Sponsored by: Dell EMC Isilon
> Differential Revision:D10379
 
 The free space calculation is still uselessly conservative, because it
 doesn't account for the fact that core dumps will always be either
 spare or compressed.  The result is that savecore will frequently
 refuse to save corefiles even when there's plenty of space.  I
 proposed removing the space check altogether in
 https://reviews.freebsd.org/D2587.  However, I agreed to wait until
 after the compressed core dump feature was merged, because then mostly
 accurate space checks will be possible.  AFAIK the compressed core
 dump feature still hasn't been finished.
>>> 
>>> Is posible (in the future) to use multiple swaps (on multiple disks)
>>> for save core dumps?
>> 
>>   Multiple swap devices is already handled by savecore(8), if one uses 
>> fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
>> devices.
>> 
>>   As far as saving to multiple disks is concerned, I would hope that one is 
>> using a redundancy capable filesystem (zfs) or RAID-like technology 
>> (gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror the 
>> data across multiple disks.
> 
>…
> 
>> How do I use multiple devices to have the system dump on all of my swap?  I 
>> got a message about not enough space, but there (I think) was enough between 
>> multiple drives….
> 
>Something like:
> 
>- Create a zpool
>- Mount zpool to /crashdumps
>- Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
> ‘dumpdir=/crashdumps’
> 
>?
>HTH,
>-Ngie
> 
>PS The issue with lack of space might be the issue that Alan brought up 
> earlier with compressed dumps and overly conservative free space checks, or 
> it might be the fact that dumpdir (default: /var/crash) is full.
> 
> 
> I was talking about the actual crashdump to swap by the system.  /var/crash 
> has 10T of space (my root pool).

If your memory is bigger than your swap, you’re unfortunately not able to save 
the mini dump if the size of the saved pages exceed the space on swap. I think 
this is where markj’s compressed dumps feature will come in handy.
Thanks!
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)"  wrote:


> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
 wrote:
> 
>> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
>> 
>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>> 
>>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
 Author: ngie
 Date: Fri Apr 14 19:41:48 2017
 New Revision: 316938
 URL: https://svnweb.freebsd.org/changeset/base/316938
 
 Log:
 savecore: fix space calculation with respect to `minfree` in 
check_space(..)
 
 - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
   representable data to INT_MAX. Check the values received from
   strtoll(3), trimming trailing whitespace off the end to maintain
   POLA.
 - Use `KiB` instead of `kB` when describing free space, total space,
   etc. I am now fully aware of `KiB` being the IEC standard for 1024
   bytes and `kB` being the IEC standard for 1000 bytes.
 - Store available number of KiB in `available` so it can be more
   easily queried and compared to ensure that there are enough KiB to
   store the dump image on disk.
 - Print out the reserved space on disk, per `minfree`, so end-users
   can troubleshoot why check_space(..) is reporting that there isn't
   enough free space.
 
 MFC after:7 weeks
 Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
diff)
 Tested with:  positive/negative cases (see review); make tinderbox
 Sponsored by: Dell EMC Isilon
 Differential Revision:D10379
>>> 
>>> The free space calculation is still uselessly conservative, because it
>>> doesn't account for the fact that core dumps will always be either
>>> spare or compressed.  The result is that savecore will frequently
>>> refuse to save corefiles even when there's plenty of space.  I
>>> proposed removing the space check altogether in
>>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>>> after the compressed core dump feature was merged, because then mostly
>>> accurate space checks will be possible.  AFAIK the compressed core
>>> dump feature still hasn't been finished.
>> 
>> Is posible (in the future) to use multiple swaps (on multiple disks)
>> for save core dumps?
> 
>Multiple swap devices is already handled by savecore(8), if one uses 
fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
devices.
> 
>As far as saving to multiple disks is concerned, I would hope that one 
is using a redundancy capable filesystem (zfs) or RAID-like technology 
(gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror the 
data across multiple disks.

…

> How do I use multiple devices to have the system dump on all of my swap?  
I got a message about not enough space, but there (I think) was enough between 
multiple drives….

Something like:

- Create a zpool
- Mount zpool to /crashdumps
- Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
‘dumpdir=/crashdumps’

?
HTH,
-Ngie

PS The issue with lack of space might be the issue that Alan brought up 
earlier with compressed dumps and overly conservative free space checks, or it 
might be the fact that dumpdir (default: /var/crash) is full.


I was talking about the actual crashdump to swap by the system.  /var/crash has 
10T of space (my root pool).




___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
>  wrote:
> 
>> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
>> 
>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>> 
>>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
 Author: ngie
 Date: Fri Apr 14 19:41:48 2017
 New Revision: 316938
 URL: https://svnweb.freebsd.org/changeset/base/316938
 
 Log:
 savecore: fix space calculation with respect to `minfree` in 
 check_space(..)
 
 - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
   representable data to INT_MAX. Check the values received from
   strtoll(3), trimming trailing whitespace off the end to maintain
   POLA.
 - Use `KiB` instead of `kB` when describing free space, total space,
   etc. I am now fully aware of `KiB` being the IEC standard for 1024
   bytes and `kB` being the IEC standard for 1000 bytes.
 - Store available number of KiB in `available` so it can be more
   easily queried and compared to ensure that there are enough KiB to
   store the dump image on disk.
 - Print out the reserved space on disk, per `minfree`, so end-users
   can troubleshoot why check_space(..) is reporting that there isn't
   enough free space.
 
 MFC after:7 weeks
 Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
 Tested with:  positive/negative cases (see review); make tinderbox
 Sponsored by: Dell EMC Isilon
 Differential Revision:D10379
>>> 
>>> The free space calculation is still uselessly conservative, because it
>>> doesn't account for the fact that core dumps will always be either
>>> spare or compressed.  The result is that savecore will frequently
>>> refuse to save corefiles even when there's plenty of space.  I
>>> proposed removing the space check altogether in
>>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>>> after the compressed core dump feature was merged, because then mostly
>>> accurate space checks will be possible.  AFAIK the compressed core
>>> dump feature still hasn't been finished.
>> 
>> Is posible (in the future) to use multiple swaps (on multiple disks)
>> for save core dumps?
> 
>Multiple swap devices is already handled by savecore(8), if one uses 
> fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
> devices.
> 
>As far as saving to multiple disks is concerned, I would hope that one is 
> using a redundancy capable filesystem (zfs) or RAID-like technology (gmirror, 
> graid, LSI Fusion’s RAID product line) to stripe and/or mirror the data 
> across multiple disks.

…

> How do I use multiple devices to have the system dump on all of my swap?  I 
> got a message about not enough space, but there (I think) was enough between 
> multiple drives….

Something like:

- Create a zpool
- Mount zpool to /crashdumps
- Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
‘dumpdir=/crashdumps’

?
HTH,
-Ngie

PS The issue with lack of space might be the issue that Alan brought up earlier 
with compressed dumps and overly conservative free space checks, or it might be 
the fact that dumpdir (default: /var/crash) is full.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Mark Johnston]

On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> >
> > Log:
> >   savecore: fix space calculation with respect to `minfree` in 
> > check_space(..)
> >
> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> > representable data to INT_MAX. Check the values received from
> > strtoll(3), trimming trailing whitespace off the end to maintain
> > POLA.
> >   - Use `KiB` instead of `kB` when describing free space, total space,
> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
> > bytes and `kB` being the IEC standard for 1000 bytes.
> >   - Store available number of KiB in `available` so it can be more
> > easily queried and compared to ensure that there are enough KiB to
> > store the dump image on disk.
> >   - Print out the reserved space on disk, per `minfree`, so end-users
> > can troubleshoot why check_space(..) is reporting that there isn't
> > enough free space.
> >
> >   MFC after:7 weeks
> >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
> >   Tested with:  positive/negative cases (see review); make tinderbox
> >   Sponsored by: Dell EMC Isilon
> >   Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.

I had held off on it for a while because it was going to conflict with
the work to add encrypted dump support, which of course has finished.

The patch to add compression support is here and should largely still
work:
https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff

I've been hesitant about pushing it forward:
- The dump_write* APIs need some simplification after the addition of
  encrypted dump support and support for dumping to 4Kn drives.
- I'm not sure how encryption should compose with compression. It seems
  intuitively obvious that we should compress before encrypting if the
  compression is to be of any use, but I don't know enough to know
  whether the compression might somehow compromise the effectiveness of
  the encryption.

If anyone has some insight on the second of these two points, I'd
appreciate hearing it.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)"  wrote:


> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
> 
> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> 
>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>>> Author: ngie
>>> Date: Fri Apr 14 19:41:48 2017
>>> New Revision: 316938
>>> URL: https://svnweb.freebsd.org/changeset/base/316938
>>> 
>>> Log:
>>>  savecore: fix space calculation with respect to `minfree` in 
check_space(..)
>>> 
>>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>>representable data to INT_MAX. Check the values received from
>>>strtoll(3), trimming trailing whitespace off the end to maintain
>>>POLA.
>>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>>bytes and `kB` being the IEC standard for 1000 bytes.
>>>  - Store available number of KiB in `available` so it can be more
>>>easily queried and compared to ensure that there are enough KiB to
>>>store the dump image on disk.
>>>  - Print out the reserved space on disk, per `minfree`, so end-users
>>>can troubleshoot why check_space(..) is reporting that there isn't
>>>enough free space.
>>> 
>>>  MFC after:7 weeks
>>>  Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
diff)
>>>  Tested with:  positive/negative cases (see review); make tinderbox
>>>  Sponsored by: Dell EMC Isilon
>>>  Differential Revision:D10379
>> 
>> The free space calculation is still uselessly conservative, because it
>> doesn't account for the fact that core dumps will always be either
>> spare or compressed.  The result is that savecore will frequently
>> refuse to save corefiles even when there's plenty of space.  I
>> proposed removing the space check altogether in
>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>> after the compressed core dump feature was merged, because then mostly
>> accurate space checks will be possible.  AFAIK the compressed core
>> dump feature still hasn't been finished.
> 
> Is posible (in the future) to use multiple swaps (on multiple disks)
> for save core dumps?

Multiple swap devices is already handled by savecore(8), if one uses 
fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
devices.

As far as saving to multiple disks is concerned, I would hope that one is 
using a redundancy capable filesystem (zfs) or RAID-like technology (gmirror, 
graid, LSI Fusion’s RAID product line) to stripe and/or mirror the data across 
multiple disks.



Thanks!
-Ngie

How do I use multiple devices to have the system dump on all of my swap?  I got 
a message about not enough space, but there (I think) was enough between 
multiple drives….




___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
> 
> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> 
>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>>> Author: ngie
>>> Date: Fri Apr 14 19:41:48 2017
>>> New Revision: 316938
>>> URL: https://svnweb.freebsd.org/changeset/base/316938
>>> 
>>> Log:
>>>  savecore: fix space calculation with respect to `minfree` in 
>>> check_space(..)
>>> 
>>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>>representable data to INT_MAX. Check the values received from
>>>strtoll(3), trimming trailing whitespace off the end to maintain
>>>POLA.
>>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>>bytes and `kB` being the IEC standard for 1000 bytes.
>>>  - Store available number of KiB in `available` so it can be more
>>>easily queried and compared to ensure that there are enough KiB to
>>>store the dump image on disk.
>>>  - Print out the reserved space on disk, per `minfree`, so end-users
>>>can troubleshoot why check_space(..) is reporting that there isn't
>>>enough free space.
>>> 
>>>  MFC after:7 weeks
>>>  Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
>>>  Tested with:  positive/negative cases (see review); make tinderbox
>>>  Sponsored by: Dell EMC Isilon
>>>  Differential Revision:D10379
>> 
>> The free space calculation is still uselessly conservative, because it
>> doesn't account for the fact that core dumps will always be either
>> spare or compressed.  The result is that savecore will frequently
>> refuse to save corefiles even when there's plenty of space.  I
>> proposed removing the space check altogether in
>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>> after the compressed core dump feature was merged, because then mostly
>> accurate space checks will be possible.  AFAIK the compressed core
>> dump feature still hasn't been finished.
> 
> Is posible (in the future) to use multiple swaps (on multiple disks)
> for save core dumps?

Multiple swap devices is already handled by savecore(8), if one uses fstab(5) 
or dumpon(8). Otherwise, you must invoke savecore(8) on individual devices.

As far as saving to multiple disks is concerned, I would hope that one is using 
a redundancy capable filesystem (zfs) or RAID-like technology (gmirror, graid, 
LSI Fusion’s RAID product line) to stripe and/or mirror the data across 
multiple disks.

Thanks!
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

svn commit: r316941 - head/sys/kern

[Edward Tomasz Napierala]

Author: trasz
Date: Fri Apr 14 20:15:34 2017
New Revision: 316941
URL: https://svnweb.freebsd.org/changeset/base/316941

Log:
  Don't try to write out bufs that have already failed with ENXIO.
  This fixes some panics after disconnecting mounted disks.
  
  Submitted by: imp (slightly different version, which I've then lost)
  Reviewed by:  kib, imp, mckusick
  MFC after:2 weeks
  Differential Revision:https://reviews.freebsd.org/D9674

Modified:
  head/sys/kern/vfs_bio.c

Modified: head/sys/kern/vfs_bio.c
==
--- head/sys/kern/vfs_bio.c Fri Apr 14 20:15:17 2017(r316940)
+++ head/sys/kern/vfs_bio.c Fri Apr 14 20:15:34 2017(r316941)
@@ -2290,18 +2290,28 @@ brelse(struct buf *bp)
bdirty(bp);
}
if (bp->b_iocmd == BIO_WRITE && (bp->b_ioflags & BIO_ERROR) &&
+   (bp->b_error != ENXIO || !LIST_EMPTY(&bp->b_dep)) &&
!(bp->b_flags & B_INVAL)) {
/*
-* Failed write, redirty.  Must clear BIO_ERROR to prevent
-* pages from being scrapped.
+* Failed write, redirty.  All errors except ENXIO (which
+* means the device is gone) are expected to be potentially
+* transient - underlying media might work if tried again
+* after EIO, and memory might be available after an ENOMEM.
+*
+* Do this also for buffers that failed with ENXIO, but have
+* non-empty dependencies - the soft updates code might need
+* to access the buffer to untangle them.
+*
+* Must clear BIO_ERROR to prevent pages from being scrapped.
 */
bp->b_ioflags &= ~BIO_ERROR;
bdirty(bp);
} else if ((bp->b_flags & (B_NOCACHE | B_INVAL)) ||
(bp->b_ioflags & BIO_ERROR) || (bp->b_bufsize <= 0)) {
/*
-* Either a failed read I/O or we were asked to free or not
-* cache the buffer.
+* Either a failed read I/O, or we were asked to free or not
+* cache the buffer, or we failed to write to a device that's
+* no longer present.
 */
bp->b_flags |= B_INVAL;
if (!LIST_EMPTY(&bp->b_dep))
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316940 - head/sys/dev/cxgbe/iw_cxgbe

[Navdeep Parhar]

Author: np
Date: Fri Apr 14 20:15:17 2017
New Revision: 316940
URL: https://svnweb.freebsd.org/changeset/base/316940

Log:
  cxgbe/iw_cxgbe: Report the actual values of various parameters as
  configured by the firmware.
  
  MFC after:3 days
  Sponsored by: Chelsio Communications

Modified:
  head/sys/dev/cxgbe/iw_cxgbe/provider.c
  head/sys/dev/cxgbe/iw_cxgbe/qp.c
  head/sys/dev/cxgbe/iw_cxgbe/t4.h

Modified: head/sys/dev/cxgbe/iw_cxgbe/provider.c
==
--- head/sys/dev/cxgbe/iw_cxgbe/provider.c  Fri Apr 14 20:10:18 2017
(r316939)
+++ head/sys/dev/cxgbe/iw_cxgbe/provider.c  Fri Apr 14 20:15:17 2017
(r316940)
@@ -307,6 +307,7 @@ c4iw_query_device(struct ib_device *ibde
 {
struct c4iw_dev *dev = to_c4iw_dev(ibdev);
struct adapter *sc = dev->rdev.adap;
+   const int spg_ndesc = sc->params.sge.spg_len / EQ_ESIZE;
 
CTR3(KTR_IW_CXGBE, "%s ibdev %p, props %p", __func__, ibdev, props);
 
@@ -320,13 +321,15 @@ c4iw_query_device(struct ib_device *ibde
props->vendor_id = pci_get_vendor(sc->dev);
props->vendor_part_id = pci_get_device(sc->dev);
props->max_mr_size = T4_MAX_MR_SIZE;
-   props->max_qp = T4_MAX_NUM_QP;
-   props->max_qp_wr = T4_MAX_QP_DEPTH;
+   props->max_qp = sc->vres.qp.size / 2;
+   props->max_qp_wr = T4_MAX_QP_DEPTH(spg_ndesc);
props->max_sge = T4_MAX_RECV_SGE;
props->max_sge_rd = 1;
-   props->max_qp_rd_atom = c4iw_max_read_depth;
-   props->max_qp_init_rd_atom = c4iw_max_read_depth;
-   props->max_cq = T4_MAX_NUM_CQ;
+   props->max_res_rd_atom = sc->params.max_ird_adapter;
+   props->max_qp_rd_atom = min(sc->params.max_ordird_qp,
+   c4iw_max_read_depth);
+   props->max_qp_init_rd_atom = props->max_qp_rd_atom;
+   props->max_cq = sc->vres.qp.size;
props->max_cqe = T4_MAX_CQ_DEPTH;
props->max_mr = c4iw_num_stags(&dev->rdev);
props->max_pd = T4_MAX_NUM_PD;

Modified: head/sys/dev/cxgbe/iw_cxgbe/qp.c
==
--- head/sys/dev/cxgbe/iw_cxgbe/qp.cFri Apr 14 20:10:18 2017
(r316939)
+++ head/sys/dev/cxgbe/iw_cxgbe/qp.cFri Apr 14 20:15:17 2017
(r316940)
@@ -133,6 +133,7 @@ static int create_qp(struct c4iw_rdev *r
int ret;
int eqsize;
struct wrqe *wr;
+   const int spg_ndesc = sc->params.sge.spg_len / EQ_ESIZE;
 
wq->sq.qid = c4iw_get_qpid(rdev, uctx);
if (!wq->sq.qid)
@@ -214,8 +215,7 @@ static int create_qp(struct c4iw_rdev *r
res->u.sqrq.op = FW_RI_RES_OP_WRITE;
 
/* eqsize is the number of 64B entries plus the status page size. */
-   eqsize = wq->sq.size * T4_SQ_NUM_SLOTS +
-   (sc->params.sge.spg_len / EQ_ESIZE);
+   eqsize = wq->sq.size * T4_SQ_NUM_SLOTS + spg_ndesc;
 
res->u.sqrq.fetchszm_to_iqid = cpu_to_be32(
V_FW_RI_RES_WR_HOSTFCMODE(0) |  /* no host cidx updates */
@@ -237,8 +237,7 @@ static int create_qp(struct c4iw_rdev *r
res->u.sqrq.op = FW_RI_RES_OP_WRITE;
 
/* eqsize is the number of 64B entries plus the status page size. */
-   eqsize = wq->rq.size * T4_RQ_NUM_SLOTS +
-   (sc->params.sge.spg_len / EQ_ESIZE);
+   eqsize = wq->rq.size * T4_RQ_NUM_SLOTS + spg_ndesc;
res->u.sqrq.fetchszm_to_iqid = cpu_to_be32(
V_FW_RI_RES_WR_HOSTFCMODE(0) |  /* no host cidx updates */
V_FW_RI_RES_WR_CPRIO(0) |   /* don't keep in chip cache */
@@ -1523,7 +1522,7 @@ c4iw_create_qp(struct ib_pd *pd, struct 
struct c4iw_create_qp_resp uresp;
int sqsize, rqsize;
struct c4iw_ucontext *ucontext;
-   int ret;
+   int ret, spg_ndesc;
struct c4iw_mm_entry *mm1, *mm2, *mm3, *mm4;
 
CTR2(KTR_IW_CXGBE, "%s ib_pd %p", __func__, pd);
@@ -1541,12 +1540,13 @@ c4iw_create_qp(struct ib_pd *pd, struct 
if (attrs->cap.max_inline_data > T4_MAX_SEND_INLINE)
return ERR_PTR(-EINVAL);
 
+   spg_ndesc = rhp->rdev.adap->params.sge.spg_len / EQ_ESIZE;
rqsize = roundup(attrs->cap.max_recv_wr + 1, 16);
-   if (rqsize > T4_MAX_RQ_SIZE)
+   if (rqsize > T4_MAX_RQ_SIZE(spg_ndesc))
return ERR_PTR(-E2BIG);
 
sqsize = roundup(attrs->cap.max_send_wr + 1, 16);
-   if (sqsize > T4_MAX_SQ_SIZE)
+   if (sqsize > T4_MAX_SQ_SIZE(spg_ndesc))
return ERR_PTR(-E2BIG);
 
ucontext = pd->uobject ? to_c4iw_ucontext(pd->uobject->context) : NULL;
@@ -1556,9 +1556,10 @@ c4iw_create_qp(struct ib_pd *pd, struct 
if (!qhp)
return ERR_PTR(-ENOMEM);
qhp->wq.sq.size = sqsize;
-   qhp->wq.sq.memsize = (sqsize + 1) * sizeof *qhp->wq.sq.queue;
+   qhp->wq.sq.memsize = (sqsize + spg_ndesc) * sizeof *qhp->wq.sq.queue +
+   16 * sizeof(__be64);
qhp->wq.rq.size = rqsi

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:

> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> >
> > Log:
> >   savecore: fix space calculation with respect to `minfree` in 
> > check_space(..)
> >
> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> > representable data to INT_MAX. Check the values received from
> > strtoll(3), trimming trailing whitespace off the end to maintain
> > POLA.
> >   - Use `KiB` instead of `kB` when describing free space, total space,
> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
> > bytes and `kB` being the IEC standard for 1000 bytes.
> >   - Store available number of KiB in `available` so it can be more
> > easily queried and compared to ensure that there are enough KiB to
> > store the dump image on disk.
> >   - Print out the reserved space on disk, per `minfree`, so end-users
> > can troubleshoot why check_space(..) is reporting that there isn't
> > enough free space.
> >
> >   MFC after:7 weeks
> >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
> >   Tested with:  positive/negative cases (see review); make tinderbox
> >   Sponsored by: Dell EMC Isilon
> >   Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.

Is posible (in the future) to use multiple swaps (on multiple disks)
for save core dumps?
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316939 - head/lib/lib80211

[Adrian Chadd]

Author: adrian
Date: Fri Apr 14 20:10:18 2017
New Revision: 316939
URL: https://svnweb.freebsd.org/changeset/base/316939

Log:
  [lib80211] fix a missing cleanup path.
  
  PR:   lib/218655
  Reported by:  

Modified:
  head/lib/lib80211/lib80211_regdomain.c

Modified: head/lib/lib80211/lib80211_regdomain.c
==
--- head/lib/lib80211/lib80211_regdomain.c  Fri Apr 14 19:41:48 2017
(r316938)
+++ head/lib/lib80211/lib80211_regdomain.c  Fri Apr 14 20:10:18 2017
(r316939)
@@ -569,6 +569,7 @@ cleanup_bands(netband_head *head)
nb = LIST_FIRST(head);
if (nb == NULL)
break;
+   LIST_REMOVE(nb, next);
free(nb);
}
 }
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:04, Ngie Cooper (yaneurabeya)  
> wrote:
> 
> 
>> On Apr 14, 2017, at 13:02, Alan Somers  wrote:
> 
> ...
> 
>> Right: check_size() needs to know the on-disk size of the core dump.
>> But it doesn't have any way to know the on-disk size until after it's
>> written everything to disk.  That's why I simply disabled the check at
>> $WORK.  Another possibility would be for savecore to ignore the
>> up-front check, but handle ENOSPC by deleting the fragmentary vmcore
>> file.  I never tried to implement that.
> 
> Let me dig through the gzip format a bit. There should be a reasonably good 
> upper bound approximation that we can use for compressed mini dumps.

Also: deleting truncated dumps sounds great to me: their value when truncated 
is pretty low from what I remember.
-Ngie



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:02, Alan Somers  wrote:

...

> Right: check_size() needs to know the on-disk size of the core dump.
> But it doesn't have any way to know the on-disk size until after it's
> written everything to disk.  That's why I simply disabled the check at
> $WORK.  Another possibility would be for savecore to ignore the
> up-front check, but handle ENOSPC by deleting the fragmentary vmcore
> file.  I never tried to implement that.

Let me dig through the gzip format a bit. There should be a reasonably good 
upper bound approximation that we can use for compressed minidumps.
Thanks!
-Ngie



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Alan Somers]

On Fri, Apr 14, 2017 at 1:58 PM, Ngie Cooper (yaneurabeya)
 wrote:
>
>> On Apr 14, 2017, at 12:49, Alan Somers  wrote:
>>
>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>>> Author: ngie
>>> Date: Fri Apr 14 19:41:48 2017
>>> New Revision: 316938
>>> URL: https://svnweb.freebsd.org/changeset/base/316938
>>>
>>> Log:
>>>  savecore: fix space calculation with respect to `minfree` in 
>>> check_space(..)
>>>
>>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>>representable data to INT_MAX. Check the values received from
>>>strtoll(3), trimming trailing whitespace off the end to maintain
>>>POLA.
>>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>>bytes and `kB` being the IEC standard for 1000 bytes.
>>>  - Store available number of KiB in `available` so it can be more
>>>easily queried and compared to ensure that there are enough KiB to
>>>store the dump image on disk.
>>>  - Print out the reserved space on disk, per `minfree`, so end-users
>>>can troubleshoot why check_space(..) is reporting that there isn't
>>>enough free space.
>>>
>>>  MFC after:7 weeks
>>>  Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
>>>  Tested with:  positive/negative cases (see review); make tinderbox
>>>  Sponsored by: Dell EMC Isilon
>>>  Differential Revision:D10379
>>
>> The free space calculation is still uselessly conservative, because it
>> doesn't account for the fact that core dumps will always be either
>> spare or compressed.  The result is that savecore will frequently
>> refuse to save corefiles even when there's plenty of space.  I
>> proposed removing the space check altogether in
>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>> after the compressed core dump feature was merged, because then mostly
>> accurate space checks will be possible.  AFAIK the compressed core
>> dump feature still hasn't been finished.
>
> That CR concerns me.
>
> The whole being able to check how much space we have on disk before dumping 
> is a wonderful tool because it prevents users from having /var/crash be full 
> of truncated full dumps, text dumps, and/or mini dumps.
>
> The real problem that you’re bringing up is that the size being passed in to 
> check_size(..) is wrong. It needs to be the on-disk size (which can be a 
> pessimization from gzip/zlib), not the in-memory size of all of the dumped 
> pages.
>
> Thanks for the note — this helps me with another related issue at $work that 
> will likely prevent me from being able to write proper tests to test out 
> panic dumps on OneFS :).
>
> Cheers!
> -Ngie

Right: check_size() needs to know the on-disk size of the core dump.
But it doesn't have any way to know the on-disk size until after it's
written everything to disk.  That's why I simply disabled the check at
$WORK.  Another possibility would be for savecore to ignore the
up-front check, but handle ENOSPC by deleting the fragmentary vmcore
file.  I never tried to implement that.
-Alan
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 12:49, Alan Somers  wrote:
> 
> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>> Author: ngie
>> Date: Fri Apr 14 19:41:48 2017
>> New Revision: 316938
>> URL: https://svnweb.freebsd.org/changeset/base/316938
>> 
>> Log:
>>  savecore: fix space calculation with respect to `minfree` in check_space(..)
>> 
>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>representable data to INT_MAX. Check the values received from
>>strtoll(3), trimming trailing whitespace off the end to maintain
>>POLA.
>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>bytes and `kB` being the IEC standard for 1000 bytes.
>>  - Store available number of KiB in `available` so it can be more
>>easily queried and compared to ensure that there are enough KiB to
>>store the dump image on disk.
>>  - Print out the reserved space on disk, per `minfree`, so end-users
>>can troubleshoot why check_space(..) is reporting that there isn't
>>enough free space.
>> 
>>  MFC after:7 weeks
>>  Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
>>  Tested with:  positive/negative cases (see review); make tinderbox
>>  Sponsored by: Dell EMC Isilon
>>  Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.

That CR concerns me.

The whole being able to check how much space we have on disk before dumping is 
a wonderful tool because it prevents users from having /var/crash be full of 
truncated full dumps, text dumps, and/or mini dumps.

The real problem that you’re bringing up is that the size being passed in to 
check_size(..) is wrong. It needs to be the on-disk size (which can be a 
pessimization from gzip/zlib), not the in-memory size of all of the dumped 
pages.

Thanks for the note — this helps me with another related issue at $work that 
will likely prevent me from being able to write proper tests to test out panic 
dumps on OneFS :).

Cheers!
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Alan Somers]

On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
>
> Log:
>   savecore: fix space calculation with respect to `minfree` in check_space(..)
>
>   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> representable data to INT_MAX. Check the values received from
> strtoll(3), trimming trailing whitespace off the end to maintain
> POLA.
>   - Use `KiB` instead of `kB` when describing free space, total space,
> etc. I am now fully aware of `KiB` being the IEC standard for 1024
> bytes and `kB` being the IEC standard for 1000 bytes.
>   - Store available number of KiB in `available` so it can be more
> easily queried and compared to ensure that there are enough KiB to
> store the dump image on disk.
>   - Print out the reserved space on disk, per `minfree`, so end-users
> can troubleshoot why check_space(..) is reporting that there isn't
> enough free space.
>
>   MFC after:7 weeks
>   Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
>   Tested with:  positive/negative cases (see review); make tinderbox
>   Sponsored by: Dell EMC Isilon
>   Differential Revision:D10379

The free space calculation is still uselessly conservative, because it
doesn't account for the fact that core dumps will always be either
spare or compressed.  The result is that savecore will frequently
refuse to save corefiles even when there's plenty of space.  I
proposed removing the space check altogether in
https://reviews.freebsd.org/D2587.  However, I agreed to wait until
after the compressed core dump feature was merged, because then mostly
accurate space checks will be possible.  AFAIK the compressed core
dump feature still hasn't been finished.

-Alan
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 12:46, Peter Wemm  wrote:
> 
> On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
>> Thanks, Peter. I will try to look into this asap.
> 
> I don't understand what is going on yet. Presumably there must be other
> changes in play that affect udp/select sometime between the original 2015
> change and this. The syslogd -s code is Old(TM).  I'm also wondering whether
> the -s code even works at all since the 2015 / r285910 change...

syslogd has been refactored a lot on ^/head. I don’t think it’s safe to say 
that the ^/head and ^/stable/11 and earlier copies will function the same.
Thanks,
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> Thanks, Peter. I will try to look into this asap.

I don't understand what is going on yet. Presumably there must be other 
changes in play that affect udp/select sometime between the original 2015 
change and this. The syslogd -s code is Old(TM).  I'm also wondering whether 
the -s code even works at all since the 2015 / r285910 change...

> -Max
> 
> On Apr 14, 2017 12:32 PM, "Peter Wemm"  wrote:
> > On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> > > On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > > > Author: sobomax
> > > > Date: Fri Apr 14 17:23:28 2017
> > > > New Revision: 316874
> > > > URL: https://svnweb.freebsd.org/changeset/base/316874
> > > > 
> > > > Log:
> > > >   Restore ability to shutdown DGRAM sockets, still forcing ENOTCONN to
> > 
> > be
> > 
> > > > returned by the shutdown(2) system call. This ability has been lost as
> > > > part
> > > > of the svn revision 285910.
> > > > 
> > > >   Reviewed by:  ed, rwatson, glebius, hiren
> > > >   MFC after:2 weeks
> > > >   Differential Revision:https://reviews.freebsd.org/D10351
> > > 
> > > This appears to have broken syslogd and had a major change in behavior
> > 
> > with
> > 
> > > regards to select(2).
> > > 
> > > If you run syslogd with the -s flag (which is default), it now spins at
> > 
> > 100%
> > 
> > > cpu as all the shutdown sockets now return readable from select.
> > > 
> > > Old releases / jails also manifest this behavor.  If it wasn't for
> > > losing
> > > the ability to run old branch binaries I'd suggest changing syslogd
> > > instead, but we depend on this in the cluster and I expect others do
> > > too.
> > > 
> > > I'm not 100% certain that this change is the culprit but a heads-up
> > > can't
> > > hurt. I'll try reverting it on the freebsd cluster next, after fixing
> > > the
> > > broken auditing changes.
> > > 
> > > -Peter
> > 
> > I can confirm that reverting r316874 fixes syslogd and backwards
> > compatability
> > with old branches.
> > 
> > --
> > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > KI6FJV
> > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316874 - head/sys/kern

[Maxim Sobolev]

Thanks, Peter. I will try to look into this asap.

-Max

On Apr 14, 2017 12:32 PM, "Peter Wemm"  wrote:

> On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> > On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > > Author: sobomax
> > > Date: Fri Apr 14 17:23:28 2017
> > > New Revision: 316874
> > > URL: https://svnweb.freebsd.org/changeset/base/316874
> > >
> > > Log:
> > >   Restore ability to shutdown DGRAM sockets, still forcing ENOTCONN to
> be
> > >
> > > returned by the shutdown(2) system call. This ability has been lost as
> > > part
> > > of the svn revision 285910.
> > >
> > >   Reviewed by:  ed, rwatson, glebius, hiren
> > >   MFC after:2 weeks
> > >   Differential Revision:https://reviews.freebsd.org/D10351
> >
> > This appears to have broken syslogd and had a major change in behavior
> with
> > regards to select(2).
> >
> > If you run syslogd with the -s flag (which is default), it now spins at
> 100%
> > cpu as all the shutdown sockets now return readable from select.
> >
> > Old releases / jails also manifest this behavor.  If it wasn't for losing
> > the ability to run old branch binaries I'd suggest changing syslogd
> > instead, but we depend on this in the cluster and I expect others do too.
> >
> > I'm not 100% certain that this change is the culprit but a heads-up can't
> > hurt. I'll try reverting it on the freebsd cluster next, after fixing the
> > broken auditing changes.
> >
> > -Peter
>
> I can confirm that reverting r316874 fixes syslogd and backwards
> compatability
> with old branches.
>
> --
> Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> KI6FJV
> UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316938 - head/sbin/savecore

[Ngie Cooper]

Author: ngie
Date: Fri Apr 14 19:41:48 2017
New Revision: 316938
URL: https://svnweb.freebsd.org/changeset/base/316938

Log:
  savecore: fix space calculation with respect to `minfree` in check_space(..)
  
  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
representable data to INT_MAX. Check the values received from
strtoll(3), trimming trailing whitespace off the end to maintain
POLA.
  - Use `KiB` instead of `kB` when describing free space, total space,
etc. I am now fully aware of `KiB` being the IEC standard for 1024
bytes and `kB` being the IEC standard for 1000 bytes.
  - Store available number of KiB in `available` so it can be more
easily queried and compared to ensure that there are enough KiB to
store the dump image on disk.
  - Print out the reserved space on disk, per `minfree`, so end-users
can troubleshoot why check_space(..) is reporting that there isn't
enough free space.
  
  MFC after:7 weeks
  Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
  Tested with:  positive/negative cases (see review); make tinderbox
  Sponsored by: Dell EMC Isilon
  Differential Revision:D10379

Modified:
  head/sbin/savecore/savecore.c

Modified: head/sbin/savecore/savecore.c
==
--- head/sbin/savecore/savecore.c   Fri Apr 14 19:18:50 2017
(r316937)
+++ head/sbin/savecore/savecore.c   Fri Apr 14 19:41:48 2017
(r316938)
@@ -68,6 +68,7 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -278,7 +279,7 @@ static int
 check_space(const char *savedir, off_t dumpsize, int bounds)
 {
FILE *fp;
-   off_t minfree, spacefree, totfree, needed;
+   off_t available, minfree, spacefree, totfree, needed;
struct statfs fsbuf;
char buf[100];
 
@@ -294,19 +295,37 @@ check_space(const char *savedir, off_t d
else {
if (fgets(buf, sizeof(buf), fp) == NULL)
minfree = 0;
-   else
-   minfree = atoi(buf);
+   else {
+   char *endp;
+
+   errno = 0;
+   minfree = strtoll(buf, &endp, 10);
+   if (minfree == 0 && errno != 0)
+   minfree = -1;
+   else {
+   while (*endp != '\0' && isspace(*endp))
+   endp++;
+   if (*endp != '\0' || minfree < 0)
+   minfree = -1;
+   }
+   if (minfree < 0)
+   syslog(LOG_WARNING,
+   "`minfree` didn't contain a valid size "
+   "(`%s`). Defaulting to 0", buf);
+   }
(void)fclose(fp);
}
 
+   available = minfree > 0 ? spacefree - minfree : totfree;
needed = dumpsize / 1024 + 2;   /* 2 for info file */
needed -= saved_dump_size(bounds);
-   if ((minfree > 0 ? spacefree : totfree) - needed < minfree) {
+   if (available < needed) {
syslog(LOG_WARNING,
-   "no dump: not enough free space on device (%lldkB "
-   "available; need at least %lldkB)",
-   (long long)(minfree > 0 ? spacefree : totfree),
-   (long long)needed);
+   "no dump: not enough free space on device (need at least "
+   "%jdKiB for dump; %jdKiB available; %jdKiB reserved)",
+   (intmax_t)needed,
+   (intmax_t)available + minfree,
+   (intmax_t)minfree);
return (0);
}
if (spacefree - needed < 0)
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > Author: sobomax
> > Date: Fri Apr 14 17:23:28 2017
> > New Revision: 316874
> > URL: https://svnweb.freebsd.org/changeset/base/316874
> > 
> > Log:
> >   Restore ability to shutdown DGRAM sockets, still forcing ENOTCONN to be
> > 
> > returned by the shutdown(2) system call. This ability has been lost as
> > part
> > of the svn revision 285910.
> > 
> >   Reviewed by:  ed, rwatson, glebius, hiren
> >   MFC after:2 weeks
> >   Differential Revision:https://reviews.freebsd.org/D10351
> 
> This appears to have broken syslogd and had a major change in behavior with
> regards to select(2).
> 
> If you run syslogd with the -s flag (which is default), it now spins at 100%
> cpu as all the shutdown sockets now return readable from select.
> 
> Old releases / jails also manifest this behavor.  If it wasn't for losing
> the ability to run old branch binaries I'd suggest changing syslogd
> instead, but we depend on this in the cluster and I expect others do too.
> 
> I'm not 100% certain that this change is the culprit but a heads-up can't
> hurt. I'll try reverting it on the freebsd cluster next, after fixing the
> broken auditing changes.
> 
> -Peter

I can confirm that reverting r316874 fixes syslogd and backwards compatability 
with old branches.

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

svn commit: r316937 - in head: contrib/ofed/libcxgb4/src sys/dev/cxgbe/iw_cxgbe

[Navdeep Parhar]

Author: np
Date: Fri Apr 14 19:18:50 2017
New Revision: 316937
URL: https://svnweb.freebsd.org/changeset/base/316937

Log:
  cxgbe/iw_cxgbe: Report accurate page_size_cap in ib_query_device.
  
  MFC after:3 days
  Sponsored by: Chelsio Communications

Modified:
  head/contrib/ofed/libcxgb4/src/t4.h
  head/sys/dev/cxgbe/iw_cxgbe/t4.h

Modified: head/contrib/ofed/libcxgb4/src/t4.h
==
--- head/contrib/ofed/libcxgb4/src/t4.h Fri Apr 14 19:15:31 2017
(r316936)
+++ head/contrib/ofed/libcxgb4/src/t4.h Fri Apr 14 19:18:50 2017
(r316937)
@@ -102,7 +102,7 @@
 #define T4_MAX_CQ_DEPTH (T4_MAX_IQ_SIZE - 1)
 #define T4_MAX_NUM_STAG (1<<15)
 #define T4_MAX_MR_SIZE (~0ULL - 1)
-#define T4_PAGESIZE_MASK 0x000  /* 4KB-128MB */
+#define T4_PAGESIZE_MASK 0x000  /* 4KB-8TB */
 #define T4_STAG_UNSET 0x
 #define T4_FW_MAJ 0
 

Modified: head/sys/dev/cxgbe/iw_cxgbe/t4.h
==
--- head/sys/dev/cxgbe/iw_cxgbe/t4.hFri Apr 14 19:15:31 2017
(r316936)
+++ head/sys/dev/cxgbe/iw_cxgbe/t4.hFri Apr 14 19:18:50 2017
(r316937)
@@ -70,7 +70,7 @@
 #define T4_MAX_QP_DEPTH (T4_MAX_RQ_SIZE - 1)
 #define T4_MAX_CQ_DEPTH (T4_MAX_IQ_SIZE - 1)
 #define T4_MAX_MR_SIZE (~0ULL - 1)
-#define T4_PAGESIZE_MASK 0x000  /* 4KB-128MB */
+#define T4_PAGESIZE_MASK 0x000  /* 4KB-8TB */
 #define T4_STAG_UNSET 0x
 #define T4_FW_MAJ 0
 #define T4_EQ_STATUS_ENTRIES (L1_CACHE_BYTES > 64 ? 2 : 1)
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316936 - head/sys/dev/cxgbe/iw_cxgbe

[Navdeep Parhar]

Author: np
Date: Fri Apr 14 19:15:31 2017
New Revision: 316936
URL: https://svnweb.freebsd.org/changeset/base/316936

Log:
  cxgbe/iw_cxgbe: hw supports 64K (not 32K) Protection Domains.
  
  MFC after:3 days
  Sponsored by: Chelsio Communications

Modified:
  head/sys/dev/cxgbe/iw_cxgbe/t4.h

Modified: head/sys/dev/cxgbe/iw_cxgbe/t4.h
==
--- head/sys/dev/cxgbe/iw_cxgbe/t4.hFri Apr 14 18:56:00 2017
(r316935)
+++ head/sys/dev/cxgbe/iw_cxgbe/t4.hFri Apr 14 19:15:31 2017
(r316936)
@@ -61,7 +61,7 @@
 
 #define T4_MAX_NUM_QP (1<<16)
 #define T4_MAX_NUM_CQ (1<<15)
-#define T4_MAX_NUM_PD (1<<15)
+#define T4_MAX_NUM_PD 65536
 #define T4_EQ_STATUS_ENTRIES (L1_CACHE_BYTES > 64 ? 2 : 1)
 #define T4_MAX_EQ_SIZE (65520 - T4_EQ_STATUS_ENTRIES)
 #define T4_MAX_IQ_SIZE (65520 - 1)
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316935 - vendor/illumos/dist/lib/libzfs/common

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 18:56:00 2017
New Revision: 316935
URL: https://svnweb.freebsd.org/changeset/base/316935

Log:
  5704 libzfs can only handle 255 file descriptors
  
  illumos/illumos-gate@bde3d612a7c090234c60e6e4578821237a5db135
  
https://github.com/illumos/illumos-gate/commit/bde3d612a7c090234c60e6e4578821237a5db135
  
  https://www.illumos.org/issues/5704
libzfs uses fopen(), at least in libzfs_init(). If there are more than 255
filedescriptors open, fopen() will fail unless you give 'F' as the last mode
character. The fix would be to give 'rF' instead of 'r' as mode to fopen().
  
  Reviewed by: Josef 'Jeff' Sipek 
  Reviewed by: John Kennedy 
  Approved by: Richard Lowe 
  Author: Simon Klinkert 

Modified:
  vendor/illumos/dist/lib/libzfs/common/libzfs_util.c

Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_util.c
==
--- vendor/illumos/dist/lib/libzfs/common/libzfs_util.c Fri Apr 14 18:54:11 
2017(r316934)
+++ vendor/illumos/dist/lib/libzfs/common/libzfs_util.c Fri Apr 14 18:56:00 
2017(r316935)
@@ -626,13 +626,13 @@ libzfs_init(void)
return (NULL);
}
 
-   if ((hdl->libzfs_mnttab = fopen(MNTTAB, "r")) == NULL) {
+   if ((hdl->libzfs_mnttab = fopen(MNTTAB, "rF")) == NULL) {
(void) close(hdl->libzfs_fd);
free(hdl);
return (NULL);
}
 
-   hdl->libzfs_sharetab = fopen("/etc/dfs/sharetab", "r");
+   hdl->libzfs_sharetab = fopen("/etc/dfs/sharetab", "rF");
 
if (libzfs_core_init() != 0) {
(void) close(hdl->libzfs_fd);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316934 - vendor/illumos/dist/lib/libzfs/common

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 18:54:11 2017
New Revision: 316934
URL: https://svnweb.freebsd.org/changeset/base/316934

Log:
  7340 receive manual origin should override automatic origin
  
  illumos/illumos-gate@ed4e7a6a5cbc5e8986dc649ad54435210487b102
  
https://github.com/illumos/illumos-gate/commit/ed4e7a6a5cbc5e8986dc649ad54435210487b102
  
  https://www.illumos.org/issues/7340
When -o origin= is specified as part of a ZFS receive, that origin
should override the automatic detection in libzfs.
  
  Reviewed by: George Wilson 
  Reviewed by: Matthew Ahrens 
  Approved by: Robert Mustacchi 
  Author: Paul Dagnelie 

Modified:
  vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c

Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c
==
--- vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c Fri Apr 14 
18:52:48 2017(r316933)
+++ vendor/illumos/dist/lib/libzfs/common/libzfs_sendrecv.c Fri Apr 14 
18:54:11 2017(r316934)
@@ -3173,7 +3173,12 @@ zfs_receive_one(libzfs_handle_t *hdl, in
/*
 * Determine the name of the origin snapshot, store in zc_string.
 */
-   if (drrb->drr_flags & DRR_FLAG_CLONE) {
+   if (originsnap) {
+   (void) strncpy(zc.zc_string, originsnap, sizeof (zc.zc_string));
+   if (flags->verbose)
+   (void) printf("using provided clone origin %s\n",
+   zc.zc_string);
+   } else if (drrb->drr_flags & DRR_FLAG_CLONE) {
if (guid_to_name(hdl, zc.zc_value,
drrb->drr_fromguid, B_FALSE, zc.zc_string) != 0) {
zcmd_free_nvlists(&zc);
@@ -3184,11 +3189,6 @@ zfs_receive_one(libzfs_handle_t *hdl, in
}
if (flags->verbose)
(void) printf("found clone origin %s\n", zc.zc_string);
-   } else if (originsnap) {
-   (void) strncpy(zc.zc_string, originsnap, sizeof (zc.zc_string));
-   if (flags->verbose)
-   (void) printf("using provided clone origin %s\n",
-   zc.zc_string);
}
 
boolean_t resuming = DMU_GET_FEATUREFLAGS(drrb->drr_versioninfo) &
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316933 - vendor/illumos/dist/lib/libzfs/common

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 18:52:48 2017
New Revision: 316933
URL: https://svnweb.freebsd.org/changeset/base/316933

Log:
  5142 libzfs support raidz root pool (loader project)
  
  illumos/illumos-gate@d5f26ad8122c3762fb16413a17bfb497db86a782
  
https://github.com/illumos/illumos-gate/commit/d5f26ad8122c3762fb16413a17bfb497db86a782
  
  https://www.illumos.org/issues/5142
the current libzfs only allows simple disk and mirror setup for boot pool, 
as
loader does support booting from raidz, this feature will remove raidz
restriction from boot pool setup.
  
  Reviewed by: George Wilson 
  Reviewed by: Yuri Pankov 
  Reviewed by: Andrew Stormont 
  Reviewed by: Albert Lee 
  Approved by: Robert Mustacchi 
  Author: Toomas Soome 

Modified:
  vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c

Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c
==
--- vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Fri Apr 14 18:51:16 
2017(r316932)
+++ vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Fri Apr 14 18:52:48 
2017(r316933)
@@ -2267,6 +2267,7 @@ vdev_get_physpaths(nvlist_t *nv, char *p
return (ret);
}
} else if (strcmp(type, VDEV_TYPE_MIRROR) == 0 ||
+   strcmp(type, VDEV_TYPE_RAIDZ) == 0 ||
strcmp(type, VDEV_TYPE_REPLACING) == 0 ||
(is_spare = (strcmp(type, VDEV_TYPE_SPARE) == 0))) {
nvlist_t **child;
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316932 - vendor/illumos/dist/lib/libzfs/common

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 18:51:16 2017
New Revision: 316932
URL: https://svnweb.freebsd.org/changeset/base/316932

Log:
  6280 libzfs: unshare_one() could fail with EZFS_SHARENFSFAILED
  
  illumos/illumos-gate@d1672efb6feac57c42788e27f739dfa3c4f3baf7
  
https://github.com/illumos/illumos-gate/commit/d1672efb6feac57c42788e27f739dfa3c4f3baf7
  
  https://www.illumos.org/issues/6280
The unshare_one() in libzfs could fail with EZFS_SHARENFSFAILED at line 834
here:
831/* make sure libshare initialized */
832if ((err = zfs_init_libshare(hdl, SA_INIT_SHARE_API)) != SA_OK) {
833free(mntpt);/* don't need the copy anymore */
834return (zfs_error_fmt(hdl, EZFS_SHARENFSFAILED,
835dgettext(TEXT_DOMAIN, "cannot unshare '%s': %s"),
836name, _sa_errorstr(err)));
837}
The correct error should be EZFS_UNSHARENFSFAILED instead.
  
  Reviewed by: Toomas Soome 
  Reviewed by: Dan McDonald 
  Reviewed by: Matthew Ahrens 
  Approved by: Gordon Ross 
  Author: Marcel Telka 

Modified:
  vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c

Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c
==
--- vendor/illumos/dist/lib/libzfs/common/libzfs_mount.cFri Apr 14 
18:49:44 2017(r316931)
+++ vendor/illumos/dist/lib/libzfs/common/libzfs_mount.cFri Apr 14 
18:51:16 2017(r316932)
@@ -20,6 +20,7 @@
  */
 
 /*
+ * Copyright 2015 Nexenta Systems, Inc.  All rights reserved.
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
  * Copyright (c) 2014, 2016 by Delphix. All rights reserved.
  * Copyright 2016 Igor Kozhukhov 
@@ -914,7 +915,7 @@ unshare_one(libzfs_handle_t *hdl, const 
if ((err = zfs_init_libshare_arg(hdl, SA_INIT_ONE_SHARE_FROM_NAME,
(void *)name)) != SA_OK) {
free(mntpt);/* don't need the copy anymore */
-   return (zfs_error_fmt(hdl, EZFS_SHARENFSFAILED,
+   return (zfs_error_fmt(hdl, EZFS_UNSHARENFSFAILED,
dgettext(TEXT_DOMAIN, "cannot unshare '%s': %s"),
name, _sa_errorstr(err)));
}
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> Author: sobomax
> Date: Fri Apr 14 17:23:28 2017
> New Revision: 316874
> URL: https://svnweb.freebsd.org/changeset/base/316874
> 
> Log:
>   Restore ability to shutdown DGRAM sockets, still forcing ENOTCONN to be
> returned by the shutdown(2) system call. This ability has been lost as part
> of the svn revision 285910.
> 
>   Reviewed by:ed, rwatson, glebius, hiren
>   MFC after:  2 weeks
>   Differential Revision:  https://reviews.freebsd.org/D10351

This appears to have broken syslogd and had a major change in behavior with 
regards to select(2).

If you run syslogd with the -s flag (which is default), it now spins at 100% 
cpu as all the shutdown sockets now return readable from select.

Old releases / jails also manifest this behavor.  If it wasn't for losing the 
ability to run old branch binaries I'd suggest changing syslogd instead, but 
we depend on this in the cluster and I expect others do too.

I'm not 100% certain that this change is the culprit but a heads-up can't 
hurt. I'll try reverting it on the freebsd cluster next, after fixing the 
broken auditing changes.

-Peter
-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

svn commit: r316931 - vendor/illumos/dist/lib/libzfs/common

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 18:49:44 2017
New Revision: 316931
URL: https://svnweb.freebsd.org/changeset/base/316931

Log:
  6268 zfs diff confused by moving a file to another directory
  
  illumos/illumos-gate@aab04418a72c0a29040a5da7eec08efe19dbef04
  
https://github.com/illumos/illumos-gate/commit/aab04418a72c0a29040a5da7eec08efe19dbef04
  
  https://www.illumos.org/issues/6268
The zfs diff command presents a description of the changes that have 
occurred
to files within a filesystem between two snapshots. If a file is renamed, 
the
tool is capable of reporting this, e.g.:
cd /some/zfs/dataset/subdir
mv file0 file1
Will result in a diff record like:
R/some/zfs/dataset/subdir/file0  ->  /some/zfs/dataset/subdir/file1
Unfortunately, it seems that rename detection only uses the base filename to
determine if a file has been renamed or simply modified. This leads to
misreporting only the original filename, omitting the more relevant 
destination
filename entirely. For example:
cd /some/zfs/dataset/subdir
mv file0 ../otherdir/file0
Will result in a diff entry:
M/some/zfs/dataset/subdir/file0
But it should really emit:
R/some/zfs/dataset/subdir/file0  ->  
/some/zfs/dataset/otherdir/file0
  
  Reviewed by: Matthew Ahrens 
  Reviewed by: Justin Gibbs 
  Approved by: Dan McDonald 
  Author: Joshua M. Clulow 

Modified:
  vendor/illumos/dist/lib/libzfs/common/libzfs_diff.c

Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_diff.c
==
--- vendor/illumos/dist/lib/libzfs/common/libzfs_diff.c Fri Apr 14 18:43:10 
2017(r316930)
+++ vendor/illumos/dist/lib/libzfs/common/libzfs_diff.c Fri Apr 14 18:49:44 
2017(r316931)
@@ -57,15 +57,6 @@
 #defineZDIFF_REMOVED   '-'
 #defineZDIFF_RENAMED   'R'
 
-static boolean_t
-do_name_cmp(const char *fpath, const char *tpath)
-{
-   char *fname, *tname;
-   fname = strrchr(fpath, '/') + 1;
-   tname = strrchr(tpath, '/') + 1;
-   return (strcmp(fname, tname) == 0);
-}
-
 typedef struct differ_info {
zfs_handle_t *zhp;
char *fromsnap;
@@ -260,7 +251,6 @@ static int
 write_inuse_diffs_one(FILE *fp, differ_info_t *di, uint64_t dobj)
 {
struct zfs_stat fsb, tsb;
-   boolean_t same_name;
mode_t fmode, tmode;
char fobjname[MAXPATHLEN], tobjname[MAXPATHLEN];
int fobjerr, tobjerr;
@@ -321,7 +311,6 @@ write_inuse_diffs_one(FILE *fp, differ_i
 
if (fmode != tmode && fsb.zs_gen == tsb.zs_gen)
tsb.zs_gen++;   /* Force a generational difference */
-   same_name = do_name_cmp(fobjname, tobjname);
 
/* Simple modification or no change */
if (fsb.zs_gen == tsb.zs_gen) {
@@ -332,7 +321,7 @@ write_inuse_diffs_one(FILE *fp, differ_i
if (change) {
print_link_change(fp, di, change,
change > 0 ? fobjname : tobjname, &tsb);
-   } else if (same_name) {
+   } else if (strcmp(fobjname, tobjname) == 0) {
print_file(fp, di, ZDIFF_MODIFIED, fobjname, &tsb);
} else {
print_rename(fp, di, fobjname, tobjname, &tsb);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316930 - vendor-sys/illumos/dist/uts/common/fs/zfs

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 18:43:10 2017
New Revision: 316930
URL: https://svnweb.freebsd.org/changeset/base/316930

Log:
  5814 bpobj_iterate_impl(): Close a refcount leak iterating on a sublist.
  
  illumos/illumos-gate@b67dde11a73a9455d641403cbbb65ec2add41b41
  
https://github.com/illumos/illumos-gate/commit/b67dde11a73a9455d641403cbbb65ec2add41b41
  
  https://www.illumos.org/issues/5814
Lets pull in this patch from freebsd:
http://svnweb.freebsd.org/base?view=revision&revision=271781
bpobj_iterate_impl(): Close a refcount leak iterating on a sublist.
If bpobj_space() returned non-zero here, the sublist would have been
left open, along with the bonus buffer hold it requires. This call
does not invoke any calls to bpobj_close() itself.
This bug doesn't have any known vector, but was found on inspection.
MFC after: 1 week
Sponsored by: Spectra Logic
Affects: All ZFS versions starting 21 May 2010 (illumos cde58dbc)
MFSpectraBSD: r1050998 on 2014/03/26
Fix bpobj_iterate_impl() to properly call bpobj_close() if bpobj_space()
returns an error.
  
  Reviewed by: Prakash Surya 
  Reviewed by: Matthew Ahrens 
  Reviewed by: Paul Dagnelie 
  Reviewed by: Simon Klinkert 
  Approved by: Gordon Ross 
  Author: Will Andrews 

Modified:
  vendor-sys/illumos/dist/uts/common/fs/zfs/bpobj.c

Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/bpobj.c
==
--- vendor-sys/illumos/dist/uts/common/fs/zfs/bpobj.c   Fri Apr 14 18:41:37 
2017(r316929)
+++ vendor-sys/illumos/dist/uts/common/fs/zfs/bpobj.c   Fri Apr 14 18:43:10 
2017(r316930)
@@ -301,8 +301,10 @@ bpobj_iterate_impl(bpobj_t *bpo, bpobj_i
if (free) {
err = bpobj_space(&sublist,
&used_before, &comp_before, &uncomp_before);
-   if (err)
+   if (err != 0) {
+   bpobj_close(&sublist);
break;
+   }
}
err = bpobj_iterate_impl(&sublist, func, arg, tx, free);
if (free) {
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316929 - vendor-sys/illumos/dist/uts/common/fs/zfs

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 18:41:37 2017
New Revision: 316929
URL: https://svnweb.freebsd.org/changeset/base/316929

Log:
  6914 kernel virtual memory fragmentation leads to hang
  
  illumos/illumos-gate@af868f46a5b794687741d5424de9e3a2d684a84a
  
https://github.com/illumos/illumos-gate/commit/af868f46a5b794687741d5424de9e3a2d684a84a
  
  https://www.illumos.org/issues/6914
This change allows the kernel to use more virtual address space. This will
allow us to devote 1.5x physmem for the zio arena, and an additional 1.5x
physmem for the kernel heap.
We saw a hang when unable to find any 128K contiguous memory segments. 
Looking
at the core file we see many threads in stacks similar to this:
> ff68c9c87c00::findstack -v
stack pointer for thread ff68c9c87c00: ff02cd63d8b0
[ ff02cd63d8b0 _resume_from_idle+0xf4() ]
  ff02cd63d8e0 swtch+0x141()
  ff02cd63d920 cv_wait+0x70(ff6009b1b01e, ff6009b1b020)
  ff02cd63da50 vmem_xalloc+0x640(ff6009b1b000, 2, 1000, 0, 0, 
0, 0, ff020004)
  ff02cd63dac0 vmem_alloc+0x135(ff6009b1b000, 2, 4)
  ff02cd63db60 segkmem_xalloc+0x171(ff6009b1b000, 0, 2, 4, 0, 
fb885fe0, fbcefa10)
  ff02cd63dbc0 segkmem_alloc_vn+0x4a(ff6009b1b000, 2, 4, 
fbcefa10)
  ff02cd63dbf0 segkmem_zio_alloc+0x20(ff6009b1b000, 2, 4)
  ff02cd63dd20 vmem_xalloc+0x5b1(ff6009b1c000, 2, 1000, 0, 0, 
0, 0, 4)
  ff02cd63dd90 vmem_alloc+0x135(ff6009b1c000, 2, 4)
  ff02cd63de20 kmem_slab_create+0x8d(ff605fd37008, 4)
  ff02cd63de80 kmem_slab_alloc+0x11e(ff605fd37008, 4)
  ff02cd63dee0 kmem_cache_alloc+0x233(ff605fd37008, 4)
  ff02cd63df10 zio_data_buf_alloc+0x5b(2)
  ff02cd63df70 arc_get_data_buf+0x92(ff6265a70588, 2, 
ff901fd796f8)
  ff02cd63dfb0 arc_buf_alloc_impl+0x9c(ff6265a70588, 
ff6d233ab0b8)
  
  Reviewed by: George Wilson 
  Reviewed by: Adam Leventhal 
  Reviewed by: John Kennedy 
  Reviewed by: Igor Kozhukhov 
  Reviewed by: Josef 'Jeff' Sipek 
  Approved by: Garrett D'Amore 
  Author: Matthew Ahrens 

Modified:
  vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c

Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c
==
--- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Fri Apr 14 18:38:53 
2017(r316928)
+++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Fri Apr 14 18:41:37 
2017(r316929)
@@ -5885,18 +5885,6 @@ arc_init(void)
/* Convert seconds to clock ticks */
arc_min_prefetch_lifespan = 1 * hz;
 
-   /* Start out with 1/8 of all memory */
-   arc_c = allmem / 8;
-
-#ifdef _KERNEL
-   /*
-* On architectures where the physical memory can be larger
-* than the addressable space (intel in 32-bit mode), we may
-* need to limit the cache to 1/8 of VM size.
-*/
-   arc_c = MIN(arc_c, vmem_size(heap_arena, VMEM_ALLOC | VMEM_FREE) / 8);
-#endif
-
/* set min cache to 1/32 of all memory, or 64MB, whichever is more */
arc_c_min = MAX(allmem / 32, 64 << 20);
/* set max to 3/4 of all memory, or all but 1GB, whichever is more */
@@ -5934,6 +5922,15 @@ arc_init(void)
/* limit meta-data to 1/4 of the arc capacity */
arc_meta_limit = arc_c_max / 4;
 
+#ifdef _KERNEL
+   /*
+* Metadata is stored in the kernel's heap.  Don't let us
+* use more than half the heap for the ARC.
+*/
+   arc_meta_limit = MIN(arc_meta_limit,
+   vmem_size(heap_arena, VMEM_ALLOC | VMEM_FREE) / 2);
+#endif
+
/* Allow the tunable to override if it is reasonable */
if (zfs_arc_meta_limit > 0 && zfs_arc_meta_limit <= arc_c_max)
arc_meta_limit = zfs_arc_meta_limit;
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316927 - vendor-sys/illumos/dist/uts/common/fs/zfs

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 18:38:21 2017
New Revision: 316927
URL: https://svnweb.freebsd.org/changeset/base/316927

Log:
  5379 modifying a mmap()-ed file does not update its timestamps
  
  illumos/illumos-gate@80e10fd0d22bbf0d18bfdae035e06f44c68ae8e6
  
https://github.com/illumos/illumos-gate/commit/80e10fd0d22bbf0d18bfdae035e06f44c68ae8e6
  
  https://www.illumos.org/issues/5379
The following is based on a review of the illumos code and on a similar 
problem
reported for FreeBSD where the relevant code is different.
Looking at this block of code 
http://src.illumos.org/source/xref/illumos-gate/
usr/src/uts/common/fs/zfs/zfs_vnops.c#4187 I see code to set up an
sa_bulk_attr_t object, I see code to set up mtime and ctime values, but I do
not see code to actually apply the attributes...
I would expect there to be a call to sa_bulk_update(), there is such a call 
in
zfs_write() for instance.
mmap_write.c [Magnifier] - demo (1.42 KB) Andriy Gapon, 2015-11-11 01:53 PM
  
  Reviewed by: Matthew Ahrens 
  Reviewed by: Prashanth Sreenivasa 
  Reviewed by: Dan McDonald 
  Approved by: Gordon Ross 
  Author: Andriy Gapon 

Modified:
  vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c

Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c
==
--- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c   Fri Apr 14 
18:34:03 2017(r316926)
+++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c   Fri Apr 14 
18:38:21 2017(r316927)
@@ -4266,6 +4266,8 @@ zfs_putapage(vnode_t *vp, page_t *pp, u_
&zp->z_pflags, 8);
zfs_tstamp_update_setup(zp, CONTENT_MODIFIED, mtime, ctime,
B_TRUE);
+   err = sa_bulk_update(zp->z_sa_hdl, bulk, count, tx);
+   ASSERT0(err);
zfs_log_write(zfsvfs->z_log, tx, TX_WRITE, zp, off, len, 0);
}
dmu_tx_commit(tx);
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316928 - vendor-sys/illumos/dist/uts/common/fs/zfs

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 18:38:53 2017
New Revision: 316928
URL: https://svnweb.freebsd.org/changeset/base/316928

Log:
  7256 low probability race in zfs_get_data
  
  illumos/illumos-gate@0c94e1af6784c69a1dea25e0e35dd13b2b91e2e5
  
https://github.com/illumos/illumos-gate/commit/0c94e1af6784c69a1dea25e0e35dd13b2b91e2e5
  
  https://www.illumos.org/issues/7256
   error = dmu_sync(zio, lr->lr_common.lrc_txg,
zfs_get_done, zgd);
   ASSERT(error || lr->lr_length <= zp->z_blksz);
It's possible, although extremely rare, that the zfs_get_done() callback is
executed before dmu_sync() returns.
In that case the znode's range lock is dropped and the znode is 
unreferenced.
Thus, the assertion can access some invalid or wrong data via the zp 
pointer.
size variable caches the correct value of z_blksz and can be safely used 
here.
  
  Reviewed by: Matt Ahrens 
  Reviewed by: Pavel Zakharov 
  Approved by: Dan McDonald 
  Author: Andriy Gapon 

Modified:
  vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c

Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c
==
--- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c   Fri Apr 14 
18:38:21 2017(r316927)
+++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vnops.c   Fri Apr 14 
18:38:53 2017(r316928)
@@ -1142,7 +1142,7 @@ zfs_get_data(void *arg, lr_write_t *lr, 
 
error = dmu_sync(zio, lr->lr_common.lrc_txg,
zfs_get_done, zgd);
-   ASSERT(error || lr->lr_length <= zp->z_blksz);
+   ASSERT(error || lr->lr_length <= size);
 
/*
 * On success, we need to wait for the write I/O
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"