Re: Binary update to -STABLE? And if so, what do I get?

2019-02-14 Thread Freddie Cash
On Thu, Feb 14, 2019 at 10:13 AM Kevin Oberman  wrote:

> On Thu, Feb 14, 2019 at 3:10 AM Pete French 
> wrote:
> > On 14/02/2019 01:43, Jason Tubnor wrote:
> > > I also have hit this IPv6 issue (I thought I was going crazy until I
> > worked
> > > it out) and other iflib issues in 12.0, which have been fixed in
> -STABLE
> > > that really should be patched in 12.0 or bring forward an early 12.1
> > > release. For our use case, 12.0 is just too buggy for production at
> this
> > > rate and we won't touch it, which is a shame because there is a lot of
> > good
> > > work in there that we would like to use but it is trumped by the
> > breakages.
> >
> > Any reason behind not running STBLE out of interest ? Yes, 12 has been
> > buggy with regards to networking, but these things get fixed very fast
> > and I now have all my machines on the lattest STABLE in production, as
> > of yesterday.
> >
> > -pete.
> >
>
> Generally, not many.
>
> Far and away the biggest is the requirement to build from sources. It's not
> a big deal for me, but if I still had many systems to deal with, that would
> be a pain.
>

Just as one can setup a poudriere/synth system for building custom binary
package repositories (so one builds packages on one system for easy
installation on multiple systems using binary packages), one can also setup
a custom freebsd-update server (so one builds the OS on one system, for
easy installation on multiple servers using binary updates).  And that can
be done to track -STABLE or -CURRENT, I believe.

Granted, I have never done it, nor looked too deeply into the documentation
around it, but I do know it's possible. :)  At least in theory.  :D

IOW, the days of needing to compile everything on each individual machine
are behind us.

-- 
Freddie Cash
fjwc...@gmail.com
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Binary update to -STABLE? And if so, what do I get?

2019-02-14 Thread Kevin Oberman
On Thu, Feb 14, 2019 at 3:10 AM Pete French 
wrote:

>
>
> On 14/02/2019 01:43, Jason Tubnor wrote:
> > I also have hit this IPv6 issue (I thought I was going crazy until I
> worked
> > it out) and other iflib issues in 12.0, which have been fixed in -STABLE
> > that really should be patched in 12.0 or bring forward an early 12.1
> > release. For our use case, 12.0 is just too buggy for production at this
> > rate and we won't touch it, which is a shame because there is a lot of
> good
> > work in there that we would like to use but it is trumped by the
> breakages.
>
> Any reason behind not running STBLE out of interest ? Yes, 12 has been
> buggy with regards to networking, but these things get fixed very fast
> and I now have all my machines on the lattest STABLE in production, as
> of yesterday.
>
> -pete.
>

Generally, not many.

Far and away the biggest is the requirement to build from sources. It's not
a big deal for me, but if I still had many systems to deal with, that would
be a pain.

I might also mention that just before the pre-release freeze and after a
release, STABLE and be a bit unstable as developers rush to get things in
before the freeze or to add things that they did not want to MFC with
little test time before a release. In recent years this issue has
significantly improved, though.

I run either HEAD or STABLE on my personal system and RELEASE on my only
server, the latter just so I can do binary updates. My server is on hold at
11.2 due to the IPv6 issue and I am amazed that this BIG oops did not
result in an EN and a patched release. Lack of IPv6 is not, for many
people, a minor issue.

The bottom line is that the only real reasons I see for not running stable
is the lack of binary updates, and issues with systems being slightly out
of sync if all are not updated to the same SVN revision at all times. Those
are very big reasons for many.
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkober...@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: 11.2-STABLE kernel wired memory leak

2019-02-14 Thread Stefan Esser
Am 13.02.19 um 10:59 schrieb Andriy Gapon:
> On 12/02/2019 20:17, Eugene Grosbein wrote:
>> 13.02.2019 1:14, Eugene Grosbein wrote:
>>
>>> Use following command to see how much memory is wasted in your case:
>>>
>>> vmstat -z | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, $1}' | sort 
>>> -k1,1 -rn | head
>>
>> Oops, small correction:
>>
>> vmstat -z | sed 's/:/,/' | awk -F, '{printf "%10s %s\n", $2*$5/1024/1024, 
>> $1}' | sort -k1,1 -rn | head
> 
> I have a much uglier but somewhat more informative "one-liner" for
> post-processing vmstat -z output:
> 
> vmstat -z | tail +3 | awk -F '[:,] *' 'BEGIN { total=0; cache=0; used=0 } {u =
> $2 * $4; c = $2 * $5; t = u + c; cache += c; used += u; total += t; name=$1;
> gsub(" ", "_", name); print t, name, u, c} END { print total, "TOTAL", used,
> cache } ' | sort -n | perl -a -p -e 'while (($j, $_) = each(@F)) { 1 while
> s/^(-?\d+)(\d{3})/$1,$2/; print $_, " "} print "\n"' | column -t
> 
> This would be much nicer as a small python script.

Or, since you are already using perl:


#!/usr/local/bin/perl5





open STDIN, "vmstat -z |" or die "Failed to run vmstat program";
open STDOUT, "| sort -n @ARGV -k 2" or die "Failed to pipe through sort";

$fmt="%-30s %8.3f %8.3f %8.3f %6.1f%%\n";
while () {
($n, $s, undef, $u, $c) = split /[:,] */;
next unless $s > 0;
$n =~ s/ /_/g;
$s /= 1024 * 1024;
$u *= $s;
$c *= $s;
$t =  $u + $c;
next unless $t > 0;
printf $fmt, $n, $t, $u, $c, 100 * $c / $t;
$cache += $c;
$used  += $u;
$total += $t;
}
printf $fmt, TOTAL, $total, $used, $cache, 100 * $cached / $total;
close STDOUT;


This script takes additional parameters, e.g. "-r" to reverse the
sort or "-r -k5" to print those entries with the highest percentage
of unused memory at the top.

(I chose to suppress lines with a current "total" of 0 - you may
want to remove the "next" command above the printf in the loop to
see them, but they did not seem useful to me.)

> Or, even, we could add a sort option for vmstat -z / -m.

Yes, and the hardest part might be to select option characters
for the various useful report formats. ;-)

Regards, STefan
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


FreeBSD CI Weekly Report 2019-02-10

2019-02-14 Thread Li-Wen Hsu
(bcc -current and -stable for more audience)

FreeBSD CI Weekly Report 2019-02-10
===

Here is a summary of the FreeBSD Continuous Integration results for
the period from 2019-02-04 to 2019-02-10.

During this period, we have:

* 2564 builds (97.4% passed, 2.6% failed) were executed on aarch64,
amd64, armv6, armv7, i386, mips, mips64, powerpc, powerpc64,
powerpcspe, riscv64, sparc64 architectures for head, stable/12,
stable/11 branches.
* 546 test runs (21.2% passed, 63.4% unstable, 15.4% exception) were
executed on amd64, i386, riscv64 architectures for head, stable/12,
stable/11 branches.
* 12 doc buils (100% passed)

If any of the issues found by CI are in your area of interest or
expertise please investigate the PRs listed below.

Web version of this report is available at
https://hackmd.io/s/ryA8Crr4N and archive is available at
http://hackfoldr.org/freebsd-ci-report/, any help is welcome.

## Fixed Jobs

* https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test
  r343713 enabled COVERAGE and KCOV by default causes the test VM
exits while executing dtrace test cases, it then got disabled by
r343949.

* https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc
  Error message: `x86_64-unknown-freebsd12.0-gcc: error: unrecognized
command line option '-fsanitize-coverage=trace-pc,trace-cmp'; did you
mean '-fsanitize-coverage=trace-pc'?`
  Fixed in : https://svnweb.freebsd.org/changeset/base/343746

## Fixed Tests

* lib.msun.{cbrt_test.cbrtl_powl,trig_test.reduction}
* https://bugs.freebsd.org/234040
* https://svnweb.freebsd.org/changeset/base/343916
* https://svnweb.freebsd.org/changeset/base/343917

## Failing Tests

* https://ci.freebsd.org/job/FreeBSD-head-amd64-test/
* lib.libc.sys.sendfile_test.hdtr_positive_v4
* lib.libc.sys.sendfile_test.hdtr_positive_v6
  See https://bugs.freebsd.org/235200 and
https://bugs.freebsd.org/234809 for deails.
  WIP: https://bugs.freebsd.org/234809
* lib.libc.regex.exhaust_test.regcomp_too_big
* lib.libregex.exhaust_test.regcomp_too_big
  These two began failing since r343964, further examination is needed.

* https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/
* There are 60 (-1 since last report) failing cases, see
https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/lastCompletedBuild/testReport/
for more details

* https://ci.freebsd.org/job/FreeBSD-head-i386-test/
* sys.netmap.ctrl-api-test.main
* sys.opencrypto.runtests.main
* lib.libc.regex.exhaust_test.regcomp_too_big
* lib.libregex.exhaust_test.regcomp_too_big
* sys.kern.coredump_phnum_test.coredump_phnum
  WIP: https://reviews.freebsd.org/D18495
* lib.libc.sys.sendfile_test.hdtr_positive_v4
* lib.libc.sys.sendfile_test.hdtr_positive_v6
  see https://bugs.freebsd.org/235200 and
https://bugs.freebsd.org/234809 for deails.
  WIP: https://bugs.freebsd.org/234809

* https://ci.freebsd.org/job/FreeBSD-stable-12-i386-test/
* sbin.bectl.bectl_test.bectl_mount
* sys.netmap.ctrl-api-test.main
* sys.opencrypto.runtests.main
* lib.libc.regex.exhaust_test.regcomp_too_big
* lib.libregex.exhaust_test.regcomp_too_big
* sys.kern.coredump_phnum_test.coredump_phnum
  WIP: https://reviews.freebsd.org/D18495

* https://ci.freebsd.org/job/FreeBSD-stable-11-amd64-test/
* usr.bin.procstat.procstat_test.kernel_stacks

* https://ci.freebsd.org/job/FreeBSD-stable-11-i386-test/
* sys.netmap.ctrl-api-test.main
* sys.opencrypto.runtests.main
* usr.bin.procstat.procstat_test.kernel_stacks
* local.kyua.* (31 cases)
* local.lutok.* (3 cases)

## Disabled Tests

* lib.libc.sys.mmap_test.mmap_truncate_signal
  https://bugs.freebsd.org/211924
* sys.fs.tmpfs.mount_test.large
  https://bugs.freebsd.org/212862
* sys.fs.tmpfs.link_test.kqueue
  https://bugs.freebsd.org/213662
* sys.kqueue.libkqueue.kqueue_test.main
  https://bugs.freebsd.org/233586
* usr.bin.procstat.procstat_test.command_line_arguments
  https://bugs.freebsd.org/233587
* usr.bin.procstat.procstat_test.environment
  https://bugs.freebsd.org/233588

## Open Issues

### Cause build fails

* [29: genassym.o build race](https://bugs.freebsd.org/29)
* (Updated) patch available:
https://people.freebsd.org/~bdrewery/patches/PR29.diff
* [233735: Possible build race: genoffset.o /usr/src/sys/sys/types.h:
error: machine/endian.h: No such file or
directory](https://bugs.freebsd.org/233735)
* [233769: Possible build race: ld: error: unable to find library
-lgcc_s](https://bugs.freebsd.org/233769)

### Others
[Tickets related to testing@](https://preview.tinyurl.com/y9maauwg)

## Other News

* https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/1131/
  zfs test became very slow (~100 mins -> ~200 mins) after enabling
COVERAGE and KCOV by default (then disabled in r343949)
___
freebsd-stable@freebsd.org mailing list

Re: Binary update to -STABLE? And if so, what do I get?

2019-02-14 Thread Pete French




On 14/02/2019 01:43, Jason Tubnor wrote:

I also have hit this IPv6 issue (I thought I was going crazy until I worked
it out) and other iflib issues in 12.0, which have been fixed in -STABLE
that really should be patched in 12.0 or bring forward an early 12.1
release. For our use case, 12.0 is just too buggy for production at this
rate and we won't touch it, which is a shame because there is a lot of good
work in there that we would like to use but it is trumped by the breakages.


Any reason behind not running STBLE out of interest ? Yes, 12 has been 
buggy with regards to networking, but these things get fixed very fast 
and I now have all my machines on the lattest STABLE in production, as 
of yesterday.


-pete.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"