Re: Relatively deterministic panic with sendfile(2) when running tests in the sxlock code

2018-10-14 Thread Enji Cooper (yaneurabeya) 

> On Oct 14, 2018, at 10:17 PM, Enji Cooper (yaneurabeya) 
>  wrote:

...

> Oh yipes. I guess passing in a server socket (a bound and listening socket) 
> instead of a client socket (connect’ed to a server socket) for `s` will 
> result in a crash?
> 
> From 
> https://github.com/ngie-eign/freebsd/blob/95b96470a3a0270c36c4e7fb5eedc150fe124fac/lib/libc/tests/sys/sendfile_test.c#L479
>  
> :
> ATF_TC_BODY(s_negative_not_connected_socket, tc)
> {
>   int client_sock, error, fd, port;
> 
>   port = XXX_TEST_PORT_BASE + __LINE__;
>   client_sock = setup_tcp_server(XXX_TEST_DOMAIN, port);
> 
>   fd = open(SOURCE_FILE, O_CREAT|O_RDWR);
>   ATF_REQUIRE_MSG(fd != -1, "open failed: %s", strerror(errno));
> 
>   error = sendfile(fd, client_sock, 0, 0, NULL, NULL, SF_FLAGS(0, 0));
>   ATF_REQUIRE_ERRNO(ENOTCONN, error == -1);
> 
>   (void)close(fd);
>   (void)close(client_sock);
> }
> Let me see if I can track this down..

Can’t repro this on 11.2-RELEASE. Trying 11.2-STABLE.
Thanks!
-Enji



signature.asc
Description: Message signed with OpenPGP

Re: Relatively deterministic panic with sendfile(2) when running tests in the sxlock code

2018-10-14 Thread Enji Cooper (yaneurabeya) 

> On Oct 14, 2018, at 10:12 PM, Enji Cooper (yaneurabeya) 
>  wrote:
> 
>> On Oct 14, 2018, at 9:45 PM, Enji Cooper (yaneurabeya) 
>> mailto:yaneurab...@gmail.com>> wrote:
>> 
>> 
>> 
>>> On Oct 14, 2018, at 7:25 PM, Gleb Smirnoff >> > wrote:
>>> 
>>>  Hi Enji,
>>> 
>>> On Sun, Oct 14, 2018 at 06:51:42PM -0700, Enji Cooper (yaneurabeya) wrote:
>>> E> Hi,
>>> E>  I’m seeing a semi-deterministic panic on 12.0-ALPHA9 related to 
>>> sendfile(2) when running sendfile_test on the host: 
>>> https://pastebin.com/raw/6Y7xg0ki ; it 
>>> looks like it’s crashing in the sxlock code when calling sblock on a 
>>> sockbuf. Are there any commands in gdb you would like me to run to display 
>>> lock state?
>>> E>  Repro:
>>> E>
>>> E> mkdir /path/to/git/checkout
>>> E> cd /path/to/git/checkout
>>> E> git clone https://github.com/ngie-eign/freebsd/tree/sendfile_tests 
>>>  .
>>> E> git checkout sendfile_tests
>>> E> (cd lib/libc/tests/sys/; make obj; make; sudo make install)
>>> E> kyua test -k /usr/tests/lib/libc/sys/Kyuafile sendfile_test
>>> 
>>> I'd like to reproduce it myself, but looks like URL is
>>> wrong:
>>> 
>>> glebius@erla:/usr/src:|>git clone 
>>> https://github.com/ngie-eign/freebsd/tree/sendfile_tests 
>>> 
>>> Клонирование в «sendfile_tests»…
>>> fatal: repository 
>>> 'https://github.com/ngie-eign/freebsd/tree/sendfile_tests/ 
>>> ' not found
>> 
>> Mea culpa. It should be:
>> 
>> $ git clone https://github.com/ngie-eign/freebsd.git 
>>  .
>> 
>> Another note is that I’m running GENERIC-NODEBUG, not GENERIC-DEBUG.
>> 
>> I suspect that it’s crashing on :hdtr_negative_bad_pointers or : 
>> s_negative_not_descriptor, because the other items don’t seem terribly 
>> plausible.
>> 
>> The test case (source) can be found here: 
>> https://github.com/ngie-eign/freebsd/blob/95b96470a3a0270c36c4e7fb5eedc150fe124fac/lib/libc/tests/sys/sendfile_test.c
>>  
>> 
> Aha! It was actually :s_negative_not_connected_socket.
> 
> Updated repro: use `kyua test -k /usr/tests/lib/libc/sys/Kyuafile 
> sendfile_test:s_negative_not_connected_socket` instead of the other kyua call 
> I provided.

Oh yipes. I guess passing in a server socket (a bound and listening socket) 
instead of a client socket (connect’ed to a server socket) for `s` will result 
in a crash?

From 
https://github.com/ngie-eign/freebsd/blob/95b96470a3a0270c36c4e7fb5eedc150fe124fac/lib/libc/tests/sys/sendfile_test.c#L479
 
:
ATF_TC_BODY(s_negative_not_connected_socket, tc)
{
int client_sock, error, fd, port;

port = XXX_TEST_PORT_BASE + __LINE__;
client_sock = setup_tcp_server(XXX_TEST_DOMAIN, port);

fd = open(SOURCE_FILE, O_CREAT|O_RDWR);
ATF_REQUIRE_MSG(fd != -1, "open failed: %s", strerror(errno));

error = sendfile(fd, client_sock, 0, 0, NULL, NULL, SF_FLAGS(0, 0));
ATF_REQUIRE_ERRNO(ENOTCONN, error == -1);

(void)close(fd);
(void)close(client_sock);
}
Let me see if I can track this down..

Thanks!
-Enji


signature.asc
Description: Message signed with OpenPGP

Re: Relatively deterministic panic with sendfile(2) when running tests in the sxlock code

2018-10-14 Thread Enji Cooper (yaneurabeya) 


> On Oct 14, 2018, at 9:45 PM, Enji Cooper (yaneurabeya) 
>  wrote:
> 
> 
> 
>> On Oct 14, 2018, at 7:25 PM, Gleb Smirnoff > > wrote:
>> 
>>  Hi Enji,
>> 
>> On Sun, Oct 14, 2018 at 06:51:42PM -0700, Enji Cooper (yaneurabeya) wrote:
>> E> Hi,
>> E>   I’m seeing a semi-deterministic panic on 12.0-ALPHA9 related to 
>> sendfile(2) when running sendfile_test on the host: 
>> https://pastebin.com/raw/6Y7xg0ki ; it 
>> looks like it’s crashing in the sxlock code when calling sblock on a 
>> sockbuf. Are there any commands in gdb you would like me to run to display 
>> lock state?
>> E>   Repro:
>> E>
>> E> mkdir /path/to/git/checkout
>> E> cd /path/to/git/checkout
>> E> git clone https://github.com/ngie-eign/freebsd/tree/sendfile_tests 
>>  .
>> E> git checkout sendfile_tests
>> E> (cd lib/libc/tests/sys/; make obj; make; sudo make install)
>> E> kyua test -k /usr/tests/lib/libc/sys/Kyuafile sendfile_test
>> 
>> I'd like to reproduce it myself, but looks like URL is
>> wrong:
>> 
>> glebius@erla:/usr/src:|>git clone 
>> https://github.com/ngie-eign/freebsd/tree/sendfile_tests 
>> 
>> Клонирование в «sendfile_tests»…
>> fatal: repository 'https://github.com/ngie-eign/freebsd/tree/sendfile_tests/ 
>> ' not found
> 
> Mea culpa. It should be:
> 
> $ git clone https://github.com/ngie-eign/freebsd.git 
>  .
> 
> Another note is that I’m running GENERIC-NODEBUG, not GENERIC-DEBUG.
> 
> I suspect that it’s crashing on :hdtr_negative_bad_pointers or : 
> s_negative_not_descriptor, because the other items don’t seem terribly 
> plausible.
> 
> The test case (source) can be found here: 
> https://github.com/ngie-eign/freebsd/blob/95b96470a3a0270c36c4e7fb5eedc150fe124fac/lib/libc/tests/sys/sendfile_test.c
>  
> 
Aha! It was actually :s_negative_not_connected_socket.

Updated repro: use `kyua test -k /usr/tests/lib/libc/sys/Kyuafile 
sendfile_test:s_negative_not_connected_socket` instead of the other kyua call I 
provided.

Thanks!
-Enji


signature.asc
Description: Message signed with OpenPGP

Re: Relatively deterministic panic with sendfile(2) when running tests in the sxlock code

2018-10-14 Thread Enji Cooper (yaneurabeya) 


> On Oct 14, 2018, at 7:25 PM, Gleb Smirnoff  wrote:
> 
>  Hi Enji,
> 
> On Sun, Oct 14, 2018 at 06:51:42PM -0700, Enji Cooper (yaneurabeya) wrote:
> E> Hi,
> E>I’m seeing a semi-deterministic panic on 12.0-ALPHA9 related to 
> sendfile(2) when running sendfile_test on the host: 
> https://pastebin.com/raw/6Y7xg0ki; it looks like it’s crashing in the sxlock 
> code when calling sblock on a sockbuf. Are there any commands in gdb you 
> would like me to run to display lock state?
> E>Repro:
> E>
> E> mkdir /path/to/git/checkout
> E> cd /path/to/git/checkout
> E> git clone https://github.com/ngie-eign/freebsd/tree/sendfile_tests .
> E> git checkout sendfile_tests
> E> (cd lib/libc/tests/sys/; make obj; make; sudo make install)
> E> kyua test -k /usr/tests/lib/libc/sys/Kyuafile sendfile_test
> 
> I'd like to reproduce it myself, but looks like URL is
> wrong:
> 
> glebius@erla:/usr/src:|>git clone 
> https://github.com/ngie-eign/freebsd/tree/sendfile_tests
> Клонирование в «sendfile_tests»…
> fatal: repository 'https://github.com/ngie-eign/freebsd/tree/sendfile_tests/' 
> not found

Mea culpa. It should be:

$ git clone https://github.com/ngie-eign/freebsd.git 
 .

Another note is that I’m running GENERIC-NODEBUG, not GENERIC-DEBUG.

I suspect that it’s crashing on :hdtr_negative_bad_pointers or : 
s_negative_not_descriptor, because the other items don’t seem terribly 
plausible.

The test case (source) can be found here: 
https://github.com/ngie-eign/freebsd/blob/95b96470a3a0270c36c4e7fb5eedc150fe124fac/lib/libc/tests/sys/sendfile_test.c

Thanks!
-Enji


signature.asc
Description: Message signed with OpenPGP

Re: Relatively deterministic panic with sendfile(2) when running tests in the sxlock code

2018-10-14 Thread Gleb Smirnoff 
  Hi Enji,

On Sun, Oct 14, 2018 at 06:51:42PM -0700, Enji Cooper (yaneurabeya) wrote:
E> Hi,
E>  I’m seeing a semi-deterministic panic on 12.0-ALPHA9 related to 
sendfile(2) when running sendfile_test on the host: 
https://pastebin.com/raw/6Y7xg0ki; it looks like it’s crashing in the sxlock 
code when calling sblock on a sockbuf. Are there any commands in gdb you would 
like me to run to display lock state?
E>  Repro:
E> 
E> mkdir /path/to/git/checkout
E> cd /path/to/git/checkout
E> git clone https://github.com/ngie-eign/freebsd/tree/sendfile_tests .
E> git checkout sendfile_tests
E> (cd lib/libc/tests/sys/; make obj; make; sudo make install)
E> kyua test -k /usr/tests/lib/libc/sys/Kyuafile sendfile_test

I'd like to reproduce it myself, but looks like URL is
wrong:

glebius@erla:/usr/src:|>git clone 
https://github.com/ngie-eign/freebsd/tree/sendfile_tests 
Клонирование в «sendfile_tests»…
fatal: repository 'https://github.com/ngie-eign/freebsd/tree/sendfile_tests/' 
not found


-- 
Gleb Smirnoff
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: 12.0-ALPHA5 - ZFS default ARC max apparently forcing system to run out of memory

2018-10-14 Thread Rebecca Cran 
On 9/27/18 9:00 PM, Allan Jude wrote:

>
> It doesn't appear like ZFS is dominating memory usage there. Using less
> than the 8GB you indicated that setting the max to solved the problem...


You're right, the problem appeared again despite having limited ZFS:


FreeBSD tau.bluestop.org 12.0-ALPHA8 FreeBSD 12.0-ALPHA8
dc3b1a4cf0e(master) MYKERNEL  amd64

vfs.zfs.arc_max: 8589934592


Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(15): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(14): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(15): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(8): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(15): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(8): failed
Oct  9 22:17:50 tau syslogd: last message repeated 1 times
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(4): failed
Oct  9 22:17:50 tau syslogd: last message repeated 2 times
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(5): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(4): failed
Oct  9 22:17:50 tau syslogd: last message repeated 1 times
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(3): failed
Oct  9 22:17:50 tau syslogd: last message repeated 1 times
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(15): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(3): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(15): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(3): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(5): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(3): failed
Oct  9 22:17:50 tau syslogd: last message repeated 1 times
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(15): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(3): failed
Oct  9 22:17:50 tau kernel: swap_pager_getswapspace(15): failed
Oct  9 22:21:51 tau kernel: swap_pager_getswapspace(1): failed
Oct  9 22:36:03 tau kernel: pid 9236 (cc1plus), uid 0, was killed: out
of swap space
Oct  9 22:40:55 tau kernel: pid 29131 (cc1plus), uid 0, was killed: out
of swap space
Oct  9 22:40:56 tau kernel: pid 82673 (firefox), uid 1001, was killed:
out of swap space
Oct  9 22:40:57 tau kernel: pid 30363 (firefox), uid 1001: exited on
signal 11 (core dumped)
Oct  9 22:40:59 tau kernel: pid 46714 (firefox), uid 1001: exited on
signal 11 (core dumped)
Oct  9 22:41:00 tau kernel: pid 62612 (firefox), uid 1001: exited on
signal 11 (core dumped)


-- 
Rebecca Cran

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Relatively deterministic panic with sendfile(2) when running tests in the sxlock code

2018-10-14 Thread Enji Cooper (yaneurabeya) 
Hi,
I’m seeing a semi-deterministic panic on 12.0-ALPHA9 related to 
sendfile(2) when running sendfile_test on the host: 
https://pastebin.com/raw/6Y7xg0ki; it looks like it’s crashing in the sxlock 
code when calling sblock on a sockbuf. Are there any commands in gdb you would 
like me to run to display lock state?
Repro:

mkdir /path/to/git/checkout
cd /path/to/git/checkout
git clone https://github.com/ngie-eign/freebsd/tree/sendfile_tests .
git checkout sendfile_tests
(cd lib/libc/tests/sys/; make obj; make; sudo make install)
kyua test -k /usr/tests/lib/libc/sys/Kyuafile sendfile_test

Thank you!
-Enji


signature.asc
Description: Message signed with OpenPGP

RE: Problem compiling rust: observations on swap

2018-10-14 Thread Cy Schubert 
Set TAR in make.conf to gnu tar from ports. Some tarballs will cause bsdtar to 
exhaust memory and swap. There was discussion a while ago suggesting this is a 
bug in vmm.

---
Sent using a tiny phone keyboard.
Apologies for any typos and autocorrect.
Also, this old phone only supports top post. Apologies.

Cy Schubert
 or 
The need of the many outweighs the greed of the few.
---

-Original Message-
From: Graham Perrin
Sent: 14/10/2018 15:19
To: freebsd-current@freebsd.org
Subject: Problem compiling rust: observations on swap

On 06/10/2018 23:41, Rebecca Cran wrote:

> On 10/6/18 6:40 AM, Greg V wrote:
>
>> BTW, this error message doesn't say much, but if cargo fails, you
>> might be out of memory.
>
> I was going to suggest being out of memory too. I've seen the rust build
> cause my system to run out of all 32GB RAM and 2GB swap.

Yeah, by coincidence I mentioned this yesterday in IRC:



– whilst poudriere built both llvm and rust. Around 6 of 8 GB swap used on a 
system with 16 GB real memory.

(I allowed a larger than usual partition for swap when I first installed the 
system.)
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSL 1.1.1 Update report (ongoing)

2018-10-14 Thread Eric McCorkle 
* ptlib; Fails to build, due to C compiler errors arising from
source-level incompatibilities.  This gets dragged in by opal, which
ends up being a dependency of ekiga, which is a dependency of gnome3.
Resolved by adding it to the exclude list, which actually does not seem
to be breaking the build of opal.

* ffmpeg: autoconf fails to detect openssl.  Probably easily fixable,
but the trivial workaround is to tick the GNUTLS option (emacs ends
up dragging in GNUTLS anyway, so it doesn't add more packages)

On 10/14/18 1:31 PM, Eric McCorkle wrote:
> More:
> 
> * ImageMagick (unrelated to OpenSSL 1.1.1): This fails with the OpenMP
> option ticked, due to trying to link with the base ld.  Can be fixed by
> setting CC, CXX, LD to a port-installed clang, clang++, lld.  The port
> should probably do this automatically.
> 
> * compat-linux-c7-base had a signal 11 when creating a package, which
> could be ignored without any apparent problem.  Might be related.
> 
> * evolution-data-server: build process apparently ends up linking
> against installed evolution libs.  This led to a build failure, due to
> missing libssl.so.8.  Trying to deinstall then reinstall.
> 
> Currently a little over halfway through.
> 
> On 10/14/18 9:18 AM, Eric McCorkle wrote:
>> I'm currently in the process of updating my laptop, rebuilding world,
>> then rebuilding *all* ports.  I have a large number of ports installed
>> (around 1200), and I tend to select a lot of build options.
>>
>> This report is intended to help shake out issues relating to OpenSSL
>> 1.1.1.  I'll be adding to this report as things progress.
>>
>> So far:
>>
>> * I'd seen issues with some C++ files not including string.h.  I can't
>> seem to reproduce this on my other laptop, so I'm going to assume it's
>> fixed.
>>
>> * Base libpmc jevents issue: buildworld fails for me when building
>> libpmc.  It appears to be missing a dependency link to build the jevents
>> executable in the pmu-events subdirectory.  I was able to work around
>> this by manually running make in that directory, then copying the result
>> to the object directory for LIB32 builds
>>
>> * librtmp: C compile errors directly attributable to OpenSSL 1.1.1
>> (missing defs, etc)
>>
>> * graphics/graphviz circular deps (unrelated to OpenSSL 1.1.1): graphviz
>> with gnomeui and/or librsvg options ticked ends up depending on vala,
>> which depends on graphviz.  Unrelated to OpenSSL 1.1.1, but warrants
>> reporting.
>>
>>
>> That's all so far.  Will send more info as it comes.
>>
> 



signature.asc
Description: OpenPGP digital signature

Re: OpenSSL 1.1.1 libssl.so version number

2018-10-14 Thread Konstantin Belousov 
On Sun, Oct 14, 2018 at 05:45:30PM +0200, Dirk Meyer wrote:
> Don Lewis schrieb:,
> 
> > It looks to me like the base libssl.so version needs to get moved to a
> > value that doesn't collide with ports, perhaps 12.  These are the
> > library version numbers currently used by the various ssl ports:
> > 
> > boringssl   1
> > openssl 9
> > openssl-devel   10
> > openssl111  11
> > libressl43
> > libressl-devel  44
> 
> The linker will always pick the highest so version of a lib (e.g. libssl.so).
> I the past the base version must be smaller then the port version,

This is simply not true, both static and dynamic ELF linkers do not care
about version numbers at all.

Static linker ld(1) only looks for libXXX.so file and records DT_SONAME from
the shared library into the linked binary.  Dynamic linker ld-elf.so.1
looks for exact match of the library filename and DT_SONAME.

So for instance we have
libc.so -> libc.so.7
symlink and libc.so.7 has DT_SONAME libc.so.7.  Then -lc causes recording
the dependency libc.so.7 and dynamic linker loads it when activating the
image. 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

RE: OpenSSL 1.1.1 libssl.so version number

2018-10-14 Thread Cy Schubert 
Not necessarily 12. ports/openssl111 should have the same ABI as HEAD so they 
should share the same version number. The fact that openssl111 in HEAD and 
openssl 1.0.2 in ports share the same version number but do not share the same 
ABI is the problem.

---
Sent using a tiny phone keyboard.
Apologies for any typos and autocorrect.
Also, this old phone only supports top post. Apologies.

Cy Schubert
 or 
The need of the many outweighs the greed of the few.
---

-Original Message-
From: Don Lewis
Sent: 14/10/2018 09:06
To: FreeBSD current
Cc: r...@freebsd.org
Subject: Re: OpenSSL 1.1.1 libssl.so version number

On 12 Oct, Don Lewis wrote:
> Prior to the OpenSSL 1.1.1 import, the base OpenSSL library was
> /usr/lib/libssl.so.8.  The security/openssl port (1.0.2p) installed
> ${LOCALBASE}/lib/ilbssl.so.9 and the security/openssl-devel port
> (1.1.0i) installed ${LOCALBASE}/lib/libssl.so.11.  After the import, the
> base OpenSSL library is /usr/lib/libssl.so.9.  Now if you build ports
> with DEFAULT_VERSIONS+=ssl=openssl, the library that actually gets used
> is ambiguous because there are now two different versions of libssl.so
> (1.0.2p and 1.1.1) with the same shared library version number.
> 
> I stumbled across this when debugging a virtualbox-ose configure
> failure.  The test executable was linked to the ports version of
> libssl.so but rtld chose the base libssl.so at run time.

It looks to me like the base libssl.so version needs to get moved to a
value that doesn't collide with ports, perhaps 12.  These are the
library version numbers currently used by the various ssl ports:

boringssl   1
openssl 9
openssl-devel   10
openssl111  11
libressl43
libressl-devel  44

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Problem compiling rust: observations on swap

2018-10-14 Thread Graham Perrin 
On 06/10/2018 23:41, Rebecca Cran wrote:

> On 10/6/18 6:40 AM, Greg V wrote:
>
>> BTW, this error message doesn't say much, but if cargo fails, you
>> might be out of memory.
>
> I was going to suggest being out of memory too. I've seen the rust build
> cause my system to run out of all 32GB RAM and 2GB swap.

Yeah, by coincidence I mentioned this yesterday in IRC:



– whilst poudriere built both llvm and rust. Around 6 of 8 GB swap used on a 
system with 16 GB real memory.

(I allowed a larger than usual partition for swap when I first installed the 
system.)
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSL 1.1.1 libssl.so version number

2018-10-14 Thread Daniel Eischen 


> On Oct 14, 2018, at 2:00 AM, Don Lewis  wrote:
> 
>> On 12 Oct, Don Lewis wrote:
>> Prior to the OpenSSL 1.1.1 import, the base OpenSSL library was
>> /usr/lib/libssl.so.8.  The security/openssl port (1.0.2p) installed
>> ${LOCALBASE}/lib/ilbssl.so.9 and the security/openssl-devel port
>> (1.1.0i) installed ${LOCALBASE}/lib/libssl.so.11.  After the import, the
>> base OpenSSL library is /usr/lib/libssl.so.9.  Now if you build ports
>> with DEFAULT_VERSIONS+=ssl=openssl, the library that actually gets used
>> is ambiguous because there are now two different versions of libssl.so
>> (1.0.2p and 1.1.1) with the same shared library version number.
>> 
>> I stumbled across this when debugging a virtualbox-ose configure
>> failure.  The test executable was linked to the ports version of
>> libssl.so but rtld chose the base libssl.so at run time.
> 
> It looks to me like the base libssl.so version needs to get moved to a
> value that doesn't collide with ports, perhaps 12.  These are the
> library version numbers currently used by the various ssl ports:

Even if base OpenSSL used 12, don't you potentially have the same problem if 
the port bumps their version sometime later?

And do you have a problem if a port library is built against a port OpenSSL, 
and another port library is built against base OpenSSL, then an app links to 
both libraries, getting both base and port OpenSSL's linked in the same image?  
It seems like you have to ensure that when you specify WITH_OPENSSL, that all 
your ports are [re]built this way, no?  I guess base OpenSSL is really no 
different, all ports need to be built using the same library, whether it's base 
or some other port version.

--
DE
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Strange panic at boot with vmm in loader.conf vs manually loading it

2018-10-14 Thread Mateusz Guzik 
On 10/14/18, Mike Tancsa  wrote:
> On 10/13/2018 12:48 PM, Allan Jude wrote:
>>
>> Strange that your crash is in ZFS here...
>>
>> Can you take a crash dump?
>>
>> It looks like something is trying to write to uninitialized memory here.
>
> I will need to pop in another drive or can I do a netdump at this point ?
>

This should be fixed with https://svnweb.freebsd.org/changeset/base/339355
i.e. just update.

-- 
Mateusz Guzik 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSL 1.1.1 libssl.so version number

2018-10-14 Thread Cy Schubert 
In message <0f7eb379-8c52-478a-aa5a-ac4257e5b...@freebsd.org>, Daniel 
Eischen w
rites:
> 
>
> > On Oct 12, 2018, at 10:58 PM, Cy Schubert  wrote
> :
> > 
> > In message , Don Lewis writes:
> >> Prior to the OpenSSL 1.1.1 import, the base OpenSSL library was
> >> /usr/lib/libssl.so.8.  The security/openssl port (1.0.2p) installed
> >> ${LOCALBASE}/lib/ilbssl.so.9 and the security/openssl-devel port
> >> (1.1.0i) installed ${LOCALBASE}/lib/libssl.so.11.  After the import, the
> >> base OpenSSL library is /usr/lib/libssl.so.9.  Now if you build ports
> >> with DEFAULT_VERSIONS+=ssl=openssl, the library that actually gets used
> >> is ambiguous because there are now two different versions of libssl.so
> >> (1.0.2p and 1.1.1) with the same shared library version number.
> >> 
> >> I stumbled across this when debugging a virtualbox-ose configure
> >> failure.  The test executable was linked to the ports version of
> >> libssl.so but rtld chose the base libssl.so at run time.
> > 
> > This is also the issue with ports-mgmt/pkg on a system that still 
> > requires OpenSSL 1.0.2 from ports in order to support an old client.
> > 
> > cwfw# pkg info
> > ld-elf.so.1: /usr/local/lib/libcrypto.so.9: version OPENSSL_1_1_0 
> > required by /usr/local/lib/libpkg.so.4 not defined
> > cwfw# 
> > 
> > If I remove security/openssl, the above issue is resolved however the 
> > old client, which should be replaced next year, fails to communicate 
> > with the server. The classic rock & a hard place scenario.
>
> Not saying this is a real solution for the general problem, but can you use a
>  libmap.conf entry to work around this?

I tried using the path1 path2 form. No joy there.


-- 
Cheers,
Cy Schubert 
FreeBSD UNIX: Web:  http://www.FreeBSD.org

The need of the many outweighs the greed of the few.


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: OpenSSL 1.1.1 Update report (ongoing)

2018-10-14 Thread Eric McCorkle 
More:

* ImageMagick (unrelated to OpenSSL 1.1.1): This fails with the OpenMP
option ticked, due to trying to link with the base ld.  Can be fixed by
setting CC, CXX, LD to a port-installed clang, clang++, lld.  The port
should probably do this automatically.

* compat-linux-c7-base had a signal 11 when creating a package, which
could be ignored without any apparent problem.  Might be related.

* evolution-data-server: build process apparently ends up linking
against installed evolution libs.  This led to a build failure, due to
missing libssl.so.8.  Trying to deinstall then reinstall.

Currently a little over halfway through.

On 10/14/18 9:18 AM, Eric McCorkle wrote:
> I'm currently in the process of updating my laptop, rebuilding world,
> then rebuilding *all* ports.  I have a large number of ports installed
> (around 1200), and I tend to select a lot of build options.
> 
> This report is intended to help shake out issues relating to OpenSSL
> 1.1.1.  I'll be adding to this report as things progress.
> 
> So far:
> 
> * I'd seen issues with some C++ files not including string.h.  I can't
> seem to reproduce this on my other laptop, so I'm going to assume it's
> fixed.
> 
> * Base libpmc jevents issue: buildworld fails for me when building
> libpmc.  It appears to be missing a dependency link to build the jevents
> executable in the pmu-events subdirectory.  I was able to work around
> this by manually running make in that directory, then copying the result
> to the object directory for LIB32 builds
> 
> * librtmp: C compile errors directly attributable to OpenSSL 1.1.1
> (missing defs, etc)
> 
> * graphics/graphviz circular deps (unrelated to OpenSSL 1.1.1): graphviz
> with gnomeui and/or librsvg options ticked ends up depending on vala,
> which depends on graphviz.  Unrelated to OpenSSL 1.1.1, but warrants
> reporting.
> 
> 
> That's all so far.  Will send more info as it comes.
> 



signature.asc
Description: OpenPGP digital signature

Re: Strange panic at boot with vmm in loader.conf vs manually loading it

2018-10-14 Thread Mike Tancsa 
On 10/13/2018 12:48 PM, Allan Jude wrote:
>
> Strange that your crash is in ZFS here...
>
> Can you take a crash dump?
>
> It looks like something is trying to write to uninitialized memory here. 

I will need to pop in another drive or can I do a netdump at this point ?

    ---Mike

-- 
---
Mike Tancsa, tel +1 519 651 3400 x203
Sentex Communications, m...@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada   


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: nginx introduces extra delay when talking to slow backend (probably FreeBSD kevent specific)

2018-10-14 Thread Maxim Dounin 
Hello!

On Fri, Oct 12, 2018 at 11:31:26PM +0300, Dmitry Marakasov wrote:

> * Dmitry Marakasov (amd...@amdmi3.ru) wrote:
> 
> I've gathered ktrace dumps for both cases, and it really looks that
> the problem is related to kevent. After nginx sends response back
> to client, it calls kevent(2) on client fd (which is 5).
> 
> When there is a bug (FreeBSD 11.2), the following happens:
> 
>  49365 nginx3.099362 CALL  
> kevent(0x7,0x8022a2000,0,0x8023005c0,0x200,0x7fffe598)
>  49365 nginx3.099419 STRU  struct kevent[] = {  }
>  49365 nginx3.194695 STRU  struct kevent[] = { { ident=5, 
> filter=EVFILT_WRITE, flags=0x20, fflags=0, data=0xbf88, 
> udata=0x8023633d1 } }
>  49365 nginx3.194733 RET   kevent 1
> ...
>  49365 nginx3.194858 CALL  
> kevent(0x7,0x8022a2000,0,0x8023005c0,0x200,0x7fffe598)
>  49365 nginx3.194875 STRU  struct kevent[] = {  }
>  49365 nginx3.835259 STRU  struct kevent[] = { { ident=5, 
> filter=EVFILT_READ, flags=0x8020, fflags=0, data=0, 
> udata=0x802346111 } }
>  49365 nginx3.835299 RET   kevent 1
> 
> E.g. read and write events come separately, both with huge delays.
> 
> On FreeBSD-CURRENT which doesn't experience the problem, kdump looks this way:
> 
>   8049 nginx3.081367 CALL  
> kevent(0x7,0x8012d1b40,0,0x8012da040,0x200,0x7fffe598)
>   8049 nginx3.081371 STRU  struct kevent[] = {  }
>   8049 nginx3.081492 STRU  struct kevent[] = { { ident=5, 
> filter=EVFILT_WRITE, flags=0x20, fflags=0, data=0xbf88, 
> udata=0x801341f11 }
>  { ident=5, filter=EVFILT_READ, flags=0x8020, 
> fflags=0, data=0, udata=0x801324e51 } }
>   8049 nginx3.081498 RET   kevent 2
> 
> E.g. both events come immediately and at the same time.
> 
> Not sure if that's problem of kevent or something it relies on or the
> way nginx uses it.

Have you tried looking into what happens in the client?  These 
events are client-related, and seems to match what client does as 
per tcpdump of traffic between nginx and the client.

Also, at least on my box (FreeBSD 10.4) this issue can be 
reproduced with curl, but not with fetch or wget.  Seems to be 
something curl-specific.  I'm not familiar with curl source code, 
but it seems to be sitting in a poll() call without any file 
descriptors for some reason:

  8862 curl 0.013972 GIO   fd 3 wrote 78 bytes
   "GET / HTTP/1.1\r
Host: localhost:8080\r
User-Agent: curl/7.61.0\r
Accept: */*\r
\r
   "
  8862 curl 0.013977 RET   sendto 78/0x4e
  8862 curl 0.013984 CALL  poll(0xbfbfe610,0x1,0)
  8862 curl 0.013987 RET   poll 0
  8862 curl 0.013992 CALL  poll(0xbfbfe768,0x1,0x1)
  8862 curl 0.016042 RET   poll 0
  8862 curl 0.016118 CALL  poll(0xbfbfe610,0x1,0)
  8862 curl 0.016137 RET   poll 0
  8862 curl 0.016197 CALL  poll(0xbfbfe768,0x1,0xc5)
  8862 curl 0.228557 RET   poll 0
  8862 curl 0.228605 CALL  poll(0xbfbfe610,0x1,0)
  8862 curl 0.228617 RET   poll 0
  8862 curl 0.228631 CALL  poll(0xbfbfe768,0x1,0x3e8)
  8862 curl 1.246374 RET   poll 0
  8862 curl 1.246420 CALL  poll(0,0,0x3e8)
  8862 curl 2.298297 RET   poll 0
  8862 curl 2.298410 CALL  poll(0xbfbfe610,0x1,0)
  8862 curl 2.298452 RET   poll 1
  8862 curl 2.298517 CALL  recvfrom(0x3,0x28ca,0x19000,0,0,0)
  8862 curl 2.298584 GIO   fd 3 read 171 bytes
   "HTTP/1.1 200 OK\r
Server: nginx/1.15.5\r

Note these lines:

  8862 curl 1.246420 CALL  poll(0,0,0x3e8)
  8862 curl 2.298297 RET   poll 0

This is a call without any file descriptors and with 1000 
millisecond timeout, so it will result in an unconditional 1 
second delay.

Not sure why you are seeing the problem with some FreeBSD version 
but not others, but different curl versions or curl compilation 
flags may explain things.  In my case curl version is as follows:

$ curl --version
curl 7.61.0 (i386-portbld-freebsd10.4) libcurl/7.61.0 OpenSSL/1.0.1u 
zlib/1.2.11 nghttp2/1.32.0
Release-Date: 2018-07-11
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smtp 
smtps telnet tftp 
Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL 
libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy 

Upgrading curl to 7.61.1 doesn't fix things.

-- 
Maxim Dounin
http://mdounin.ru/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

OpenSSL 1.1.1 Update report (ongoing)

2018-10-14 Thread Eric McCorkle 
I'm currently in the process of updating my laptop, rebuilding world,
then rebuilding *all* ports.  I have a large number of ports installed
(around 1200), and I tend to select a lot of build options.

This report is intended to help shake out issues relating to OpenSSL
1.1.1.  I'll be adding to this report as things progress.

So far:

* I'd seen issues with some C++ files not including string.h.  I can't
seem to reproduce this on my other laptop, so I'm going to assume it's
fixed.

* Base libpmc jevents issue: buildworld fails for me when building
libpmc.  It appears to be missing a dependency link to build the jevents
executable in the pmu-events subdirectory.  I was able to work around
this by manually running make in that directory, then copying the result
to the object directory for LIB32 builds

* librtmp: C compile errors directly attributable to OpenSSL 1.1.1
(missing defs, etc)

* graphics/graphviz circular deps (unrelated to OpenSSL 1.1.1): graphviz
with gnomeui and/or librsvg options ticked ends up depending on vala,
which depends on graphviz.  Unrelated to OpenSSL 1.1.1, but warrants
reporting.


That's all so far.  Will send more info as it comes.



signature.asc
Description: OpenPGP digital signature

Re: OpenSSL 1.1.1 libssl.so version number

2018-10-14 Thread Don Lewis 
On 12 Oct, Don Lewis wrote:
> Prior to the OpenSSL 1.1.1 import, the base OpenSSL library was
> /usr/lib/libssl.so.8.  The security/openssl port (1.0.2p) installed
> ${LOCALBASE}/lib/ilbssl.so.9 and the security/openssl-devel port
> (1.1.0i) installed ${LOCALBASE}/lib/libssl.so.11.  After the import, the
> base OpenSSL library is /usr/lib/libssl.so.9.  Now if you build ports
> with DEFAULT_VERSIONS+=ssl=openssl, the library that actually gets used
> is ambiguous because there are now two different versions of libssl.so
> (1.0.2p and 1.1.1) with the same shared library version number.
> 
> I stumbled across this when debugging a virtualbox-ose configure
> failure.  The test executable was linked to the ports version of
> libssl.so but rtld chose the base libssl.so at run time.

It looks to me like the base libssl.so version needs to get moved to a
value that doesn't collide with ports, perhaps 12.  These are the
library version numbers currently used by the various ssl ports:

boringssl   1
openssl 9
openssl-devel   10
openssl111  11
libressl43
libressl-devel  44

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"