Re: Mirror the freebsd-update server?

[Rainer Duffner]

> Am 13.09.2017 um 00:09 schrieb Jason Tubnor :
> 
> I found this useful.  I made some adjustments for what I needed but those
> updates are sweet now :-)
> https://wiki.freebsd.org/VladimirKrstulja/Guides/FreeBSDUpdateReverseProxy 
> 

"This is a simple cache. That means it doesn't consider the files as a whole 
repository, which in turn means updates to your cache are not atomic. It'd be 
advised to nuke your cache before your update run, as its point is only to 
retain the files in a local cache for some short period of time required for 
all your machines to be updated.“


The problem is, my updates don’t work that way.

These servers all belong to different customers (well, some have a couple) and 
I can’t patch them in „some short period of time“.
It’s continuous process.
I have other stuff to do. And I have to negotiate downtimes etc.


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Mirror the freebsd-update server?

[Jason Tubnor]

On 13 September 2017 at 07:52, Kurt Jaeger  wrote:

> Hi!
>
> > > is it possible or even allowed to just mirror one of the
> freebsd-update servers locally?
> [...]
>
> > Take a look at https://www.freebsd.org/doc/en/articles/hubs/index.html.
> > This doc discusses setting up a mirror (found via quick Google
> > search).
>
> That's a mirror of some FTP server, but a fbsd-update repo is
> somewhat different. A fbsd-update repo setup is described here:
>
> https://www.freebsd.org/doc/en_US.ISO8859-1/articles/
> freebsd-update-server/article.html
>

I found this useful.  I made some adjustments for what I needed but those
updates are sweet now :-)
https://wiki.freebsd.org/VladimirKrstulja/Guides/FreeBSDUpdateReverseProxy
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Mirror the freebsd-update server?

[Kurt Jaeger]

Hi!

> > is it possible or even allowed to just mirror one of the freebsd-update 
> > servers locally?
[...]

> Take a look at https://www.freebsd.org/doc/en/articles/hubs/index.html.
> This doc discusses setting up a mirror (found via quick Google
> search).

That's a mirror of some FTP server, but a fbsd-update repo is
somewhat different. A fbsd-update repo setup is described here:

https://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/article.html

-- 
p...@opsec.eu+49 171 3101372 3 years to go !
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Mirror the freebsd-update server?

[Rainer Duffner]

> Am 12.09.2017 um 23:45 schrieb Chris Gordon :
> 
> Take a look at https://www.freebsd.org/doc/en/articles/hubs/index.html 
> .  This doc 
> discusses setting up a mirror (found via quick Google search).



Yes, but it doesn’t talk about freebsd-update - which is quite a different 
beast than FTP/WWW mirrors?



___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Mirror the freebsd-update server?

[Chris Gordon]

> On Sep 12, 2017, at 11:03 AM, rai...@ultra-secure.de wrote:
> 
> Hi,
> 
> is it possible or even allowed to just mirror one of the freebsd-update 
> servers locally?
> 
> I don't want to build the patches myself (I have no custom kernels, but best 
> I can do is not completely f' it up - so why bother?).
> 
> I have between 100 und 200 servers that need updates - and on upgrades, it 
> kind of pulls on my nerves.
> 
> 
> Unfortunately, I'm not in a position to cache them via squid or so.

Take a look at https://www.freebsd.org/doc/en/articles/hubs/index.html.  This 
doc discusses setting up a mirror (found via quick Google search).

Chris  
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 11.1 coredumping in sendfile, as used by a uwsgi process

[Christoph Brinkhaus]

On Tue, Sep 12, 2017 at 04:29:49PM +0200, Mark Martinec wrote:

Hello Mark!
> 2017-09-12 15:46, Steven Hartland wrote:
> > Could you post the decoded crash info from /var/crash/...
> 
> Using crashinfo(8) I suppose?
> 
> > I would also create a bug report:
> > https://bugs.freebsd.org/bugzilla/enter_bug.cgi?product=Base%20System
> 
> Done (with additional info):
> 
>Bug 59 - 11.1-R crashing in sendfile syscall, as used by a uwsgi 
> process
>https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=59
>
I have had issues on a quite weak system, it has 2G RAM.
Since FreeBSD-11.1 zfs arc is compressed by default which takes
some resources. With that feature disabled by
vfs.zfs.compressed_arc_enabled="0"
in /boot/loader.conf the system is as stable as it has been before.
I am not sure if there is a relation to your specific situation.

Kind regards,
Christoph
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Mirror the freebsd-update server?

[rainer]

Hi,

is it possible or even allowed to just mirror one of the freebsd-update 
servers locally?


I don't want to build the patches myself (I have no custom kernels, but 
best I can do is not completely f' it up - so why bother?).


I have between 100 und 200 servers that need updates - and on upgrades, 
it kind of pulls on my nerves.



Unfortunately, I'm not in a position to cache them via squid or so.


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 11.1 coredumping in sendfile, as used by a uwsgi process

[Mark Martinec]

2017-09-12 15:46, Steven Hartland wrote:

Could you post the decoded crash info from /var/crash/...


Using crashinfo(8) I suppose?


I would also create a bug report:
https://bugs.freebsd.org/bugzilla/enter_bug.cgi?product=Base%20System


Done (with additional info):

  Bug 59 - 11.1-R crashing in sendfile syscall, as used by a uwsgi 
process

  https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=59


Mark

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 11.1 coredumping in sendfile, as used by a uwsgi process

[Steven Hartland]

Could you post the decoded crash info from /var/crash/...

I would also create a bug report:
https://bugs.freebsd.org/bugzilla/enter_bug.cgi?product=Base%20System

    Regards
    Steve
On 12/09/2017 14:40, Mark Martinec wrote:

A couple of days ago I have upgraded an Intel box from FreeBSD 10.3 to
11.1-RELEASE-p1, and reinstalled all the packages, built on the same 
OS version.

This host is running nginx web server with an uwsgi as a backend.
The file system is ZFS (recent as of 10.3, zpool not yet upgraded
to new 11.1 features).

Ever since the upgrade, this host is crashing/rebooting two or three 
times
per day. The reported crash location is always the same: it is in a 
sendfile

function (same addresses each time), the running process is always uwsgi:


Sep 12 15:03:12 xxx syslogd: kernel boot file is /boot/kernel/kernel
Sep 12 15:03:12 xxx kernel: [22677]
Sep 12 15:03:12 xxx kernel: [22677]
Sep 12 15:03:12 xxx kernel: [22677] Fatal trap 12: page fault while in 
kernel mode

Sep 12 15:03:12 xxx kernel: [22677] cpuid = 7; apic id = 07
Sep 12 15:03:12 xxx kernel: [22677] fault virtual address = 0xe8
Sep 12 15:03:12 xxx kernel: [22677] fault code    = 
supervisor write data, page not present
Sep 12 15:03:12 xxx kernel: [22677] instruction pointer   = 
0x20:0x80afefb2
Sep 12 15:03:12 xxx kernel: [22677] stack pointer = 
0x28:0xfe02397da5a0
Sep 12 15:03:12 xxx kernel: [22677] frame pointer = 
0x28:0xfe02397da5e0
Sep 12 15:03:12 xxx kernel: [22677] code segment  = base 
0x0, limit 0xf, type 0x1b
Sep 12 15:03:12 xxx kernel: [22677]   = DPL 0, pres 1, 
long 1, def32 0, gran 1
Sep 12 15:03:12 xxx kernel: [22677] processor eflags  = interrupt 
enabled, resume, IOPL = 0
Sep 12 15:03:12 xxx kernel: [22677] current process   = 34504 
(uwsgi)

Sep 12 15:03:12 xxx kernel: [22677] trap number   = 12
Sep 12 15:03:12 xxx kernel: [22677] panic: page fault
Sep 12 15:03:12 xxx kernel: [22677] cpuid = 7
Sep 12 15:03:12 xxx kernel: [22677] KDB: stack backtrace:
Sep 12 15:03:12 xxx kernel: [22677] #0 0x80aada97 at 
kdb_backtrace+0x67

Sep 12 15:03:12 xxx kernel: [22677] #1 0x80a6bb76 at vpanic+0x186
Sep 12 15:03:12 xxx kernel: [22677] #2 0x80a6b9e3 at panic+0x43
Sep 12 15:03:12 xxx kernel: [22677] #3 0x80edf832 at 
trap_fatal+0x322
Sep 12 15:03:12 xxx kernel: [22677] #4 0x80edf889 at 
trap_pfault+0x49

Sep 12 15:03:12 xxx kernel: [22677] #5 0x80edf0c6 at trap+0x286
Sep 12 15:03:12 xxx kernel: [22677] #6 0x80ec3641 at calltrap+0x8
Sep 12 15:03:12 xxx kernel: [22677] #7 0x80a6a2af at 
sendfile_iodone+0xbf
Sep 12 15:03:12 xxx kernel: [22677] #8 0x80a69eae at 
vn_sendfile+0x124e
Sep 12 15:03:12 xxx kernel: [22677] #9 0x80a6a4dd at 
sendfile+0x13d
Sep 12 15:03:12 xxx kernel: [22677] #10 0x80ee0394 at 
amd64_syscall+0x6c4
Sep 12 15:03:12 xxx kernel: [22677] #11 0x80ec392b at 
Xfast_syscall+0xfb

Sep 12 15:03:12 xxx kernel: [22677] Uptime: 6h17m57s
Sep 12 15:03:12 xxx kernel: [22677] Dumping 983 out of 8129 
MB:..2%..12%..22%..31%..41%..51%..61%..72%..82%..92%Copyright (c) 
1992-2017 The FreeBSD Project.
Sep 12 15:03:12 xxx kernel: Copyright (c) 1979, 1980, 1983, 1986, 
1988, 1989, 1991, 1992, 1993, 1994
Sep 12 15:03:12 xxx kernel: The Regents of the University of 
California. All rights reserved.
Sep 12 15:03:12 xxx kernel: FreeBSD is a registered trademark of The 
FreeBSD Foundation.
Sep 12 15:03:12 xxx kernel: FreeBSD 11.1-RELEASE-p1 #0: Wed Aug  9 
11:55:48 UTC 2017

[...]
Sep 12 15:03:12 xxx savecore: reboot after panic: page fault
Sep 12 15:03:12 xxx savecore: writing core to /var/crash/vmcore.4


This host with the same services was very stable under 10.3, same ZFS 
pool.


We have several other hosts running 11.1 with no incidents, running 
various

services (but admittedly no other host has a comparably busy web server).
Interestingly the nginx has a sendfile feature enabled too, but this does
not cause a crash (on this or other hosts), only the sendfile as used
by uwsgi seems to be the problem.

For the time being I have disabled the use of sendfile in uwsgi, we'll 
see

is this avoids the trouble.

Suggestions?

  Mark
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

11.1 coredumping in sendfile, as used by a uwsgi process

[Mark Martinec]

A couple of days ago I have upgraded an Intel box from FreeBSD 10.3 to
11.1-RELEASE-p1, and reinstalled all the packages, built on the same OS 
version.

This host is running nginx web server with an uwsgi as a backend.
The file system is ZFS (recent as of 10.3, zpool not yet upgraded
to new 11.1 features).

Ever since the upgrade, this host is crashing/rebooting two or three 
times
per day. The reported crash location is always the same: it is in a 
sendfile
function (same addresses each time), the running process is always 
uwsgi:



Sep 12 15:03:12 xxx syslogd: kernel boot file is /boot/kernel/kernel
Sep 12 15:03:12 xxx kernel: [22677]
Sep 12 15:03:12 xxx kernel: [22677]
Sep 12 15:03:12 xxx kernel: [22677] Fatal trap 12: page fault while in 
kernel mode

Sep 12 15:03:12 xxx kernel: [22677] cpuid = 7; apic id = 07
Sep 12 15:03:12 xxx kernel: [22677] fault virtual address = 0xe8
Sep 12 15:03:12 xxx kernel: [22677] fault code= 
supervisor write data, page not present
Sep 12 15:03:12 xxx kernel: [22677] instruction pointer   = 
0x20:0x80afefb2
Sep 12 15:03:12 xxx kernel: [22677] stack pointer = 
0x28:0xfe02397da5a0
Sep 12 15:03:12 xxx kernel: [22677] frame pointer = 
0x28:0xfe02397da5e0
Sep 12 15:03:12 xxx kernel: [22677] code segment  = base 
0x0, limit 0xf, type 0x1b
Sep 12 15:03:12 xxx kernel: [22677]   = DPL 0, pres 1, 
long 1, def32 0, gran 1
Sep 12 15:03:12 xxx kernel: [22677] processor eflags  = interrupt 
enabled, resume, IOPL = 0
Sep 12 15:03:12 xxx kernel: [22677] current process   = 34504 
(uwsgi)

Sep 12 15:03:12 xxx kernel: [22677] trap number   = 12
Sep 12 15:03:12 xxx kernel: [22677] panic: page fault
Sep 12 15:03:12 xxx kernel: [22677] cpuid = 7
Sep 12 15:03:12 xxx kernel: [22677] KDB: stack backtrace:
Sep 12 15:03:12 xxx kernel: [22677] #0 0x80aada97 at 
kdb_backtrace+0x67
Sep 12 15:03:12 xxx kernel: [22677] #1 0x80a6bb76 at 
vpanic+0x186

Sep 12 15:03:12 xxx kernel: [22677] #2 0x80a6b9e3 at panic+0x43
Sep 12 15:03:12 xxx kernel: [22677] #3 0x80edf832 at 
trap_fatal+0x322
Sep 12 15:03:12 xxx kernel: [22677] #4 0x80edf889 at 
trap_pfault+0x49

Sep 12 15:03:12 xxx kernel: [22677] #5 0x80edf0c6 at trap+0x286
Sep 12 15:03:12 xxx kernel: [22677] #6 0x80ec3641 at 
calltrap+0x8
Sep 12 15:03:12 xxx kernel: [22677] #7 0x80a6a2af at 
sendfile_iodone+0xbf
Sep 12 15:03:12 xxx kernel: [22677] #8 0x80a69eae at 
vn_sendfile+0x124e
Sep 12 15:03:12 xxx kernel: [22677] #9 0x80a6a4dd at 
sendfile+0x13d
Sep 12 15:03:12 xxx kernel: [22677] #10 0x80ee0394 at 
amd64_syscall+0x6c4
Sep 12 15:03:12 xxx kernel: [22677] #11 0x80ec392b at 
Xfast_syscall+0xfb

Sep 12 15:03:12 xxx kernel: [22677] Uptime: 6h17m57s
Sep 12 15:03:12 xxx kernel: [22677] Dumping 983 out of 8129 
MB:..2%..12%..22%..31%..41%..51%..61%..72%..82%..92%Copyright (c) 
1992-2017 The FreeBSD Project.
Sep 12 15:03:12 xxx kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 
1989, 1991, 1992, 1993, 1994
Sep 12 15:03:12 xxx kernel: The Regents of the University of California. 
All rights reserved.
Sep 12 15:03:12 xxx kernel: FreeBSD is a registered trademark of The 
FreeBSD Foundation.
Sep 12 15:03:12 xxx kernel: FreeBSD 11.1-RELEASE-p1 #0: Wed Aug  9 
11:55:48 UTC 2017

[...]
Sep 12 15:03:12 xxx savecore: reboot after panic: page fault
Sep 12 15:03:12 xxx savecore: writing core to /var/crash/vmcore.4


This host with the same services was very stable under 10.3, same ZFS 
pool.


We have several other hosts running 11.1 with no incidents, running 
various
services (but admittedly no other host has a comparably busy web 
server).
Interestingly the nginx has a sendfile feature enabled too, but this 
does

not cause a crash (on this or other hosts), only the sendfile as used
by uwsgi seems to be the problem.

For the time being I have disabled the use of sendfile in uwsgi, we'll 
see

is this avoids the trouble.

Suggestions?

  Mark
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Unusually high "Wired" memory

[Borja Marcos]

> On 11 Sep 2017, at 11:25, Borja Marcos  wrote:
> 
>> Since I’ve updated a machine to 11.1-STABLE I am seeing a rather unusual 
>> growth of Wired memory.
>> 
>> Any hints on what might have changed from 11-RELEASE to 11.1-RELEASE and 
>> 11.1-STABLE?

vmstat -z and vmstat -m follow

% vmstat -z
ITEM   SIZE  LIMIT USED FREE  REQ FAIL SLEEP

UMA Kegs:   384,  0, 126,   4, 126,   0,   0
UMA Zones: 4736,  0, 143,   0, 143,   0,   0
UMA Slabs:   80,  0,   89605,   70495,14041377,   0,   0
UMA Hash:   256,  0,   4,  11,  15,   0,   0
4 Bucket:32,  0,  27,2348, 4568149,   0,   0
6 Bucket:48,  0,   0,   0,  470783,   0,   0
8 Bucket:64,  0,   0,   0, 6493588,  13,   0
12 Bucket:   96,  0,   0,   0, 2905574,   2,   0
16 Bucket:  128,  0,  28, 747, 1375609,   0,   0
32 Bucket:  256,  0,   0,   0, 1342213,14372,   0
64 Bucket:  512,  0,   0,   0, 4462146,120046,   0
128 Bucket:1024,  0,   0,   0,  846934,11702,   0
256 Bucket:2048,  0,   0,   0,  402301,3139,   0
vmem btag:   56,  0,   18945,   10875,  367860, 216,   0
VM OBJECT:  240,  0,   28017,   37279, 5670269,   0,   0
RADIX NODE: 144,  0,   27077,   26410,19244770,   0,   0
MAP:240,  0,   3,  61,   3,   0,   0
KMAP ENTRY: 128,  0,   6,  87,   6,   0,   0
MAP ENTRY:  128,  0,4327,3454, 5294482,   0,   0
VMSPACE:   2504,  0,  28, 128,3950,   0,   0
fakepg: 104,  0,   4, 148,   4,   0,   0
mt_zone:  16400,  0, 307,   0, 307,   0,   0
16:  16,  0,   40214,   36843,86753581,   0,   0
32:  32,  0,   29825,   43925,312018796,   0,   0
64:  64,  0,   44824,  186002,119290303,   0,   0
128:128,  0,   14044,   17669,192809899,   0,   0
256:256,  0,   26953,   42242,139829494,   0,   0
512:512,  0,  169943,  212729,97319297,   0,   0
1024:  1024,  0,2844, 240,113460194,   0,   0
2048:  2048,  0, 204,  40,134422120,   0,   0
4096:  4096,  0,8295, 170,19109860,   0,   0
8192:  8192,  0, 157,  12, 5847918,   0,   0
16384:16384,  0,8091,  78, 5635701,   0,   0
32768:32768,  0, 127,  13, 3738362,   0,   0
65536:65536,  0,  67,  21, 4328503,   0,   0
64 pcpu:  8,  0,1764, 796,2125,   0,   0
SLEEPQUEUE:  80,  0,1691, 975,1691,   0,   0
Files:   80,  0,,4789,13481817,   0,   0
filedesc0: 1104,  0,  50,  34,3971,   0,   0
TURNSTILE:  136,  0,1691,1009,1691,   0,   0
rl_entry:40,  0, 696,3104, 696,   0,   0
umtx pi: 96,  0,   0,   0,   0,   0,   0
umtx_shm:88,  0,   0,   0,   0,   0,   0
MAC labels:  40,  0,   0,   0,   0,   0,   0
PROC:  1416,  0,  49, 311,3970,   0,   0
THREAD:1456,  0,1458, 232,   35807,   0,   0
cpuset:  96,  0, 933,1199,1985,   0,   0
audit_record:  1248,  0,   0,   0,   0,   0,   0
mbuf_packet:256, 1677735,   16377,   0,14185830,   0,   0
mbuf:   256, 1677735,   5,2008,14156009,   0,   0
mbuf_cluster:  2048, 262144,   16377, 451, 2251293,   0,   0
mbuf_jumbo_page:   4096, 131072,   1,  12,  601300,   0,   0
mbuf_jumbo_9k: 9216,  38836,   0,   0,   0,   0,   0
mbuf_jumbo_16k:   16384,  21845,   0,   0,   0,   0,   0
g_bio:  376,  0,15235172, 248,477839025,   0,   0
ttyinq: 160,  0, 135, 190, 735,   0,   0
ttyoutq:256,  0,  72,  93, 392,   0,   0
FPU_save_area:  832,  0,   0,   0,   0,   0,   0
CTL IO: 672,  0,   0,   0,   0,   0,   0
beio:   368,  0,   0,   0,   0,   0,   0
taskq_zone:  48,  0,   0,  83, 4883426,   0,   0
VNODE:  472,  0,  116663,   57169, 7020248,   0,   0
VNODEPOLL:  120,  0,   0,   0,   0,