poudriere extract failures with TAR=/usr/local/bin/gtar with gtar from ports

2018-10-15 Thread Graham Perrin 
For example:

> [00:01:10] [02] [00:00:19] Finished www/qt5-webkit | 
> qt5-webkit-5.212.0.a2_13: Failed: extract

In full:



root@momh167-gjp4-hpelitebook8570p-freebsd:~ # date ; uname -v ; pkg upgrade -f 
-r poudriere archivers/gtar
Tue 16 Oct 2018 06:03:24 BST
FreeBSD 12.0-ALPHA9 r339356 GENERIC-NODEBUG
Updating poudriere repository catalogue...
poudriere repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
    gtar-1.30 [poudriere]

Number of packages to be reinstalled: 1

Proceed with this action? [y/N]: y
[1/1] Reinstalling gtar-1.30...
[1/1] Extracting gtar-1.30: 100%
root@momh167-gjp4-hpelitebook8570p-freebsd:~ # nano 
/usr/local/etc/poudriere.d/make.conf




  GNU nano 3.1 
/usr/local/etc/poudriere.d/make.conf
   

ICA_CERTS=/usr/ports/distfiles/QuoVadisRootCA2.crt
DEFAULT_VERSIONS+= samba=4.8
# 
TAR=/usr/local/bin/gtar




root@momh167-gjp4-hpelitebook8570p-freebsd:~ # date ; poudriere ports -u ; 
poudriere bulk -j current www/otter-browser
Tue 16 Oct 2018 06:04:04 BST
[00:00:00] Updating portstree "default" with portsnap...Looking up 
portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching snapshot tag from ec2-eu-west-1.portsnap.freebsd.org... done.
Ports tree hasn't changed since last snapshot.
No updates needed.
Ports tree is already up to date.
 done
[00:00:00] Creating the reference jail... done
[00:00:01] Mounting system devices for current-default
[00:00:01] Mounting ports/packages/distfiles
[00:00:01] Stashing existing package repository
[00:00:01] Mounting ccache from: /var/cache/ccache
[00:00:01] Mounting packages from: 
/usr/local/poudriere/data/packages/current-default
[00:00:01] Copying /var/db/ports from: 
/usr/local/etc/poudriere.d/current-options
[00:00:01] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
/etc/resolv.conf -> 
/usr/local/poudriere/data/.m/current-default/ref/etc/resolv.conf
[00:00:01] Starting jail current-default
[00:00:04] Logs: 
/usr/local/poudriere/data/logs/bulk/current-default/2018-10-16_06h04m05s
[00:00:05] Loading MOVED for 
/usr/local/poudriere/data/.m/current-default/ref/usr/ports
[00:00:06] Ports supports: FLAVORS SELECTED_OPTIONS
[00:00:06] Gathering ports metadata
[00:00:14] Calculating ports order and dependencies
[00:00:16] Sanity checking the repository
[00:00:16] Checking packages for incremental rebuild needs
[00:00:43] Deleting stale symlinks... done
[00:00:43] Deleting empty directories... done
[00:00:43] Cleaning the build queue
[00:00:43] Sanity checking build queue
[00:00:43] Processing PRIORITY_BOOST
[00:00:43] Balancing pool
[00:00:43] Recording filesystem state for prepkg... done
[00:00:49] Building 3 packages using 2 builders
[00:00:49] Starting/Cloning builders
[00:00:51] Hit CTRL+t at any time to see build progress and stats
[00:00:51] [01] [00:00:00] Building www/qt5-webengine | qt5-webengine-5.9.5_8
[00:00:51] [02] [00:00:00] Building www/qt5-webkit | qt5-webkit-5.212.0.a2_13
[00:01:10] [02] [00:00:19] Finished www/qt5-webkit | qt5-webkit-5.212.0.a2_13: 
Failed: extract
[00:01:10] [02] [00:00:19] Skipping www/otter-browser | otter-browser-0.9.99.3: 
Dependent port www/qt5-webkit | qt5-webkit-5.212.0.a2_13 failed
[00:01:29] [01] [00:00:38] Finished www/qt5-webengine | qt5-webengine-5.9.5_8: 
Failed: extract
[00:01:30] Stopping 2 builders
[00:01:36] No package built, no need to update the repository
[00:01:36] Committing packages to repository
[00:01:37] Removing old packages
[00:01:37] Failed ports: www/qt5-webkit:extract www/qt5-webengine:extract
[00:01:37] Skipped ports: www/otter-browser
[current-default] [2018-10-16_06h04m05s] [committing:] Queued: 3  Built: 0  
Failed: 2  Skipped: 1  Ignored: 0  Tobuild: 0   Time: 00:01:33
[00:01:37] Logs: 
/usr/local/poudriere/data/logs/bulk/current-default/2018-10-16_06h04m05s
[00:01:37] Cleaning up
[00:01:37] Unmounting file systems
root@momh167-gjp4-hpelitebook8570p-freebsd:~ #
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Problem compiling rust: observations on swap: GNU tar

2018-10-15 Thread Cy Schubert 
In message <97680f3e-fb00-abc7-81d5-c61c9cae7...@gmail.com>, Graham 
Perrin writ
es:
> On 14/10/2018 22:34, Cy Schubert wrote:
>
> > Set TAR in make.conf to gnu tar from ports. Some tarballs will cause bsdtar
>  to exhaust memory and swap. There was discussion a while ago suggesting this
>  is a bug in vmm.
>
> Thanks!
>
> My make.conf for poudriere:
>
> root@momh167-gjp4-hpelitebook8570p-freebsd:~ # cat /usr/local/etc/poudriere.d
> /make.conf
> ICA_CERTS=/usr/ports/distfiles/QuoVadisRootCA2.crt
> DEFAULT_VERSIONS+= samba=4.8
> GNUTAR=on
> root@momh167-gjp4-hpelitebook8570p-freebsd:~ #
>
> – like so (the third line of the file), yes?

No, like this.

TAR=/usr/local/bin/gtar


-- 
Cheers,
Cy Schubert 
FreeBSD UNIX: Web:  http://www.FreeBSD.org

The need of the many outweighs the greed of the few.


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Problem compiling rust: observations on swap: GNU tar

2018-10-15 Thread Graham Perrin 
On 14/10/2018 22:34, Cy Schubert wrote:

> Set TAR in make.conf to gnu tar from ports. Some tarballs will cause bsdtar 
> to exhaust memory and swap. There was discussion a while ago suggesting this 
> is a bug in vmm.

Thanks!

My make.conf for poudriere:

root@momh167-gjp4-hpelitebook8570p-freebsd:~ # cat 
/usr/local/etc/poudriere.d/make.conf
ICA_CERTS=/usr/ports/distfiles/QuoVadisRootCA2.crt
DEFAULT_VERSIONS+= samba=4.8
GNUTAR=on
root@momh167-gjp4-hpelitebook8570p-freebsd:~ #

– like so (the third line of the file), yes?
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Relatively deterministic panic with sendfile(2) when running tests in the sxlock code

2018-10-15 Thread Enji Cooper (yaneurabeya) 

> On Oct 15, 2018, at 6:10 AM, Gleb Smirnoff  wrote:
> 
>  Enji,
> 
> can you please check that with this patch all your tests pass?

Hi Gleb!
It almost compiled. I just needed to dereference the `so` pointer:

$ git diff /usr/src/sys/kern/kern_sendfile.c
diff --git a/sys/kern/kern_sendfile.c b/sys/kern/kern_sendfile.c
index 438069aa721..50404ce5745 100644
--- a/sys/kern/kern_sendfile.c
+++ b/sys/kern/kern_sendfile.c
@@ -526,6 +526,8 @@ sendfile_getsock(struct thread *td, int s, struct file 
**sock_fp,
*so = (*sock_fp)->f_data;
if ((*so)->so_type != SOCK_STREAM)
return (EINVAL);
+   if (SOLISTENING(*so))
+   return (ENOTCONN);
return (0);
 }


After I applied that and rebuilt the kernel, it doesn’t panic anymore 
(and it fails with the correct errno).
Thank you so very much :)!
-Enji


signature.asc
Description: Message signed with OpenPGP

Re: OpenSSL 1.1.1 Update report (ongoing)

2018-10-15 Thread Eric McCorkle 
* gnome-vfs: C compile errors related to openssl, no viable mitigation

Almost done now, though ptlib and gnome-vfs may cause runtime trouble

On 10/14/18 7:13 PM, Eric McCorkle wrote:
> * ptlib; Fails to build, due to C compiler errors arising from
> source-level incompatibilities.  This gets dragged in by opal, which
> ends up being a dependency of ekiga, which is a dependency of gnome3.
> Resolved by adding it to the exclude list, which actually does not seem
> to be breaking the build of opal.
> 
> * ffmpeg: autoconf fails to detect openssl.  Probably easily fixable,
> but the trivial workaround is to tick the GNUTLS option (emacs ends
> up dragging in GNUTLS anyway, so it doesn't add more packages)
> 
> On 10/14/18 1:31 PM, Eric McCorkle wrote:
>> More:
>>
>> * ImageMagick (unrelated to OpenSSL 1.1.1): This fails with the OpenMP
>> option ticked, due to trying to link with the base ld.  Can be fixed by
>> setting CC, CXX, LD to a port-installed clang, clang++, lld.  The port
>> should probably do this automatically.
>>
>> * compat-linux-c7-base had a signal 11 when creating a package, which
>> could be ignored without any apparent problem.  Might be related.
>>
>> * evolution-data-server: build process apparently ends up linking
>> against installed evolution libs.  This led to a build failure, due to
>> missing libssl.so.8.  Trying to deinstall then reinstall.
>>
>> Currently a little over halfway through.
>>
>> On 10/14/18 9:18 AM, Eric McCorkle wrote:
>>> I'm currently in the process of updating my laptop, rebuilding world,
>>> then rebuilding *all* ports.  I have a large number of ports installed
>>> (around 1200), and I tend to select a lot of build options.
>>>
>>> This report is intended to help shake out issues relating to OpenSSL
>>> 1.1.1.  I'll be adding to this report as things progress.
>>>
>>> So far:
>>>
>>> * I'd seen issues with some C++ files not including string.h.  I can't
>>> seem to reproduce this on my other laptop, so I'm going to assume it's
>>> fixed.
>>>
>>> * Base libpmc jevents issue: buildworld fails for me when building
>>> libpmc.  It appears to be missing a dependency link to build the jevents
>>> executable in the pmu-events subdirectory.  I was able to work around
>>> this by manually running make in that directory, then copying the result
>>> to the object directory for LIB32 builds
>>>
>>> * librtmp: C compile errors directly attributable to OpenSSL 1.1.1
>>> (missing defs, etc)
>>>
>>> * graphics/graphviz circular deps (unrelated to OpenSSL 1.1.1): graphviz
>>> with gnomeui and/or librsvg options ticked ends up depending on vala,
>>> which depends on graphviz.  Unrelated to OpenSSL 1.1.1, but warrants
>>> reporting.
>>>
>>>
>>> That's all so far.  Will send more info as it comes.
>>>
>>
> 



signature.asc
Description: OpenPGP digital signature

Re: vm_fault on boot with NVMe/nda

2018-10-15 Thread Warner Losh 
At Netflix we have our OCA firmware based on FreeBSD -current (we take a
snapshot every 5 weeks or so). We've been booting thousands of machines off
nda for over a year... It absolutely works and is one of the things that
lets us deliver the content we do...

I have some patches in my queue waiting for the freeze to lift that do much
better trim shaping to the drives. You might also want to turn on
vfs.ffs.dotrimcons=1 which is a new feature that eliminates many of the
BIO_DELETE requests that come down from UFS that are turned into trims. nvd
has no queueing policy at all: it shot-guns all requests to the drive w/o
collapsing or any moderation at all... This isn't so good for most drives
out there today...

Warner

On Mon, Oct 15, 2018 at 8:38 AM Yuri Pankov  wrote:

> Daniel Nebdal wrote:
> > Hi. I have a 12-ALPHA9 / r339331 amd64 system (a HPE ProLiant ML30 G9),
> > with a Kingston NVMe SSD ("KINGSTON SKC1000480G") on a PCIe card.
> >
> > By default, it shows up as /dev/nvd0, and this is how I installed the
> > system. It has a single large UFS2 (with SJ and TRIM support) partition
> > mounted as /. (There's also a few other partitions on it that should be
> > irrelevant for this.) This works, but it does sometimes slow down for
> > minutes at the time with disturbing queue lengths in gstat; on the order
> of
> > tens of thousands. As I understand it, this is due to how TRIM operations
> > take precedence over everything else when using nvd ?
> >
> > Looking around, I noticed the nda driver for NVMe-through-CAM. To test
> it,
> > I added hw.nvme.use_nvd=0 to loader.conf. On one level, this works: The
> > drive shows up as /dev/nda0 . On the other hand, trying to mount nda0p2
> as
> > / floods the console with "vm_fault: pager read error, pid 1 (init)", and
> > never finishes booting.
> >
> > What is more interesting is that if I boot from the drive, but mount an
> > alpha9 usb stick as /, I can then mount the nda device just fine, and the
> > very minimal testing I did (using bin/cat and COPYRIGHT on the NVMe
> drive)
> > seems to work.
> >
> > So - is nda meant to be bootable, or am I a bit over-eager in trying to
> do
> > so?
> > If not, is there anything smart I can do to get better performance out of
> > nvd?
> > (Or have I just overlooked something obvious?)
> >
> > Dmesg from a normal nvd boot here:
> > https://openbenchmarking.org/system/1810159-RA-SSD30089593/SSD/dmesg
>
> FWIW, I set hw.nvme.use_nvd=0 in the installer, got 12-ALPHA8 installed
> on nda0, and it's happily booting from it (using ZFS, though), so it's
> certainly meant to be bootable.
>
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: vm_fault on boot with NVMe/nda

2018-10-15 Thread Yuri Pankov 
Daniel Nebdal wrote:
> Hi. I have a 12-ALPHA9 / r339331 amd64 system (a HPE ProLiant ML30 G9),
> with a Kingston NVMe SSD ("KINGSTON SKC1000480G") on a PCIe card.
> 
> By default, it shows up as /dev/nvd0, and this is how I installed the
> system. It has a single large UFS2 (with SJ and TRIM support) partition
> mounted as /. (There's also a few other partitions on it that should be
> irrelevant for this.) This works, but it does sometimes slow down for
> minutes at the time with disturbing queue lengths in gstat; on the order of
> tens of thousands. As I understand it, this is due to how TRIM operations
> take precedence over everything else when using nvd ?
> 
> Looking around, I noticed the nda driver for NVMe-through-CAM. To test it,
> I added hw.nvme.use_nvd=0 to loader.conf. On one level, this works: The
> drive shows up as /dev/nda0 . On the other hand, trying to mount nda0p2 as
> / floods the console with "vm_fault: pager read error, pid 1 (init)", and
> never finishes booting.
> 
> What is more interesting is that if I boot from the drive, but mount an
> alpha9 usb stick as /, I can then mount the nda device just fine, and the
> very minimal testing I did (using bin/cat and COPYRIGHT on the NVMe drive)
> seems to work.
> 
> So - is nda meant to be bootable, or am I a bit over-eager in trying to do
> so?
> If not, is there anything smart I can do to get better performance out of
> nvd?
> (Or have I just overlooked something obvious?)
> 
> Dmesg from a normal nvd boot here:
> https://openbenchmarking.org/system/1810159-RA-SSD30089593/SSD/dmesg

FWIW, I set hw.nvme.use_nvd=0 in the installer, got 12-ALPHA8 installed
on nda0, and it's happily booting from it (using ZFS, though), so it's
certainly meant to be bootable.



signature.asc
Description: OpenPGP digital signature

vm_fault on boot with NVMe/nda

2018-10-15 Thread Daniel Nebdal 
Hi. I have a 12-ALPHA9 / r339331 amd64 system (a HPE ProLiant ML30 G9),
with a Kingston NVMe SSD ("KINGSTON SKC1000480G") on a PCIe card.

By default, it shows up as /dev/nvd0, and this is how I installed the
system. It has a single large UFS2 (with SJ and TRIM support) partition
mounted as /. (There's also a few other partitions on it that should be
irrelevant for this.) This works, but it does sometimes slow down for
minutes at the time with disturbing queue lengths in gstat; on the order of
tens of thousands. As I understand it, this is due to how TRIM operations
take precedence over everything else when using nvd ?

Looking around, I noticed the nda driver for NVMe-through-CAM. To test it,
I added hw.nvme.use_nvd=0 to loader.conf. On one level, this works: The
drive shows up as /dev/nda0 . On the other hand, trying to mount nda0p2 as
/ floods the console with "vm_fault: pager read error, pid 1 (init)", and
never finishes booting.

What is more interesting is that if I boot from the drive, but mount an
alpha9 usb stick as /, I can then mount the nda device just fine, and the
very minimal testing I did (using bin/cat and COPYRIGHT on the NVMe drive)
seems to work.

So - is nda meant to be bootable, or am I a bit over-eager in trying to do
so?
If not, is there anything smart I can do to get better performance out of
nvd?
(Or have I just overlooked something obvious?)

Dmesg from a normal nvd boot here:
https://openbenchmarking.org/system/1810159-RA-SSD30089593/SSD/dmesg

-- 
Daniel Nebdal
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Relatively deterministic panic with sendfile(2) when running tests in the sxlock code

2018-10-15 Thread Gleb Smirnoff 
  Enji,

can you please check that with this patch all your tests pass?

-- 
Gleb Smirnoff
Index: sys/kern/kern_sendfile.c
===
--- sys/kern/kern_sendfile.c	(revision 339098)
+++ sys/kern/kern_sendfile.c	(working copy)
@@ -526,6 +526,8 @@ sendfile_getsock(struct thread *td, int s, struct
 	*so = (*sock_fp)->f_data;
 	if ((*so)->so_type != SOCK_STREAM)
 		return (EINVAL);
+	if (SOLISTENING(so))
+		return (ENOTCONN);
 	return (0);
 }
 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Relatively deterministic panic with sendfile(2) when running tests in the sxlock code

2018-10-15 Thread Gleb Smirnoff 
On Sun, Oct 14, 2018 at 10:17:28PM -0700, Enji Cooper (yaneurabeya) wrote:
E> Oh yipes. I guess passing in a server socket (a bound and listening socket) 
instead of a client socket (connect’ed to a server socket) for `s` will result 
in a crash?

Oh, thanks enough info. Thanks! Isn't related to sendfile but definitely
is related to my other changes. Will fix.

-- 
Gleb Smirnoff
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Strange panic at boot with vmm in loader.conf vs manually loading it

2018-10-15 Thread Mike Tancsa 
On 10/14/2018 2:19 PM, Mateusz Guzik wrote:
> On 10/14/18, Mike Tancsa  wrote:
>> On 10/13/2018 12:48 PM, Allan Jude wrote:
>>> Strange that your crash is in ZFS here...
>>>
>>> Can you take a crash dump?
>>>
>>> It looks like something is trying to write to uninitialized memory here.
>> I will need to pop in another drive or can I do a netdump at this point ?
>>
> This should be fixed with https://svnweb.freebsd.org/changeset/base/339355
> i.e. just update.
>
Thanks, just tried and all is good!


    ---Mike


-- 
---
Mike Tancsa, tel +1 519 651 3400 x203
Sentex Communications, m...@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada   

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"