Re: usr.bin/xinstall r366697 versus: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
I also have this breaking builds. Do we have plans to revert this until a
variant that doesn't break builds in some corner cases is completed?
On Tue, Dec 15, 2020 at 12:22 PM Ryan Moeller wrote:
>
> On 12/12/20 2:15 AM, Graham Perrin wrote:
> > On 23/11/2020 12:18, Graham Perrin wrote:
> >
> >> On 22/11/2020 12:00, Dimitry Andric wrote:
> >>> …
> >>> I'd guess it's an unintended side-effect of
> >>> https://svnweb.freebsd.org/base?view=revision=366697
> >>> ("install(1): Avoid unncessary fstatfs() calls and use mmap() based on
> >>> size").
> >>>
> >>> If you only revert usr.bin/xinstall to r366696, and then rebuild it,
> >>> does it still occur?
> >>>
> >>> -Dimitry
> >>
> >> Thank you!
> >>
> >> Success with r366696:
> >>
> >>
> >>
> >> …
> >
> >
> > Re:
> > <
> https://lists.freebsd.org/pipermail/freebsd-current/2020-November/077634.html>
>
> > please, should I open a bug for this?
> >
> >
> > Is the issue exposed by my choice of gzip-9 for e.g. /usr/ and if so,
> > can I avoid it by (now) reverting to lz4 for that part, and/or any
> > other part, of the file system?
> >
> >
>
> Unfortunately I don't have an answer for you at this time, but ACK.
>
>
> -Ryan
>
>
> >
> >
> >
> > root@mowa219-gjp4-8570p:~ # zfs get mountpoint,compression
> > NAME PROPERTY VALUE SOURCE
> > Transcend mountpoint /Volumes/t500 local
> > Transcend compression zstd
> > local
> > Transcend/VirtualBox mountpoint /Volumes/t500/VirtualBox inherited
> > from Transcend
> > Transcend/VirtualBox compression zstd inherited from Transcend
> > copperbowl mountpoint /copperbowl
> > local
> > copperbowl compression lz4
> > local
> > copperbowl/ROOT mountpoint
> > none local
> > copperbowl/ROOT compression lz4 inherited from copperbowl
> > copperbowl/ROOT/Waterfox mountpoint
> > / local
> > copperbowl/ROOT/Waterfox compression lz4 inherited from copperbowl
> > copperbowl/ROOT/r367081f mountpoint
> > / local
> > copperbowl/ROOT/r367081f compression lz4 inherited from copperbowl
> > copperbowl/ROOT/r367936e mountpoint
> > / local
> > copperbowl/ROOT/r367936e compression lz4 inherited from copperbowl
> > copperbowl/ROOT/r367936f mountpoint
> > / local
> > copperbowl/ROOT/r367936f compression lz4 inherited from copperbowl
> > copperbowl/ROOT/r367936f@2020-03-20-06:19:45 mountpoint
> > - -
> > copperbowl/ROOT/r367936f@2020-03-20-06:19:45 compression
> > - -
> > copperbowl/ROOT/r367936f@2020-11-22-23:18:47 mountpoint
> > - -
> > copperbowl/ROOT/r367936f@2020-11-22-23:18:47 compression
> > - -
> > copperbowl/ROOT/r367936f@2020-12-06-16:23:14 mountpoint
> > - -
> > copperbowl/ROOT/r367936f@2020-12-06-16:23:14 compression
> > - -
> > copperbowl/VirtualBox mountpoint
> > /usr/local/VirtualBox local
> > copperbowl/VirtualBox compression
> > gzip-9 local
> > copperbowl/iocage mountpoint /copperbowl/iocage inherited from
> > copperbowl
> > copperbowl/iocage compression
> > gzip-9 local
> > copperbowl/iocage/download mountpoint /copperbowl/iocage/download
> > inherited from copperbowl
> > copperbowl/iocage/download compression
> > lz4local
> > copperbowl/iocage/download/12.0-RELEASE mountpoint
> > /copperbowl/iocage/download/12.0-RELEASE inherited from copperbowl
> > copperbowl/iocage/download/12.0-RELEASE compression
> > lz4local
> > copperbowl/iocage/images mountpoint /copperbowl/iocage/images
> > inherited from copperbowl
> > copperbowl/iocage/images compression
> > lz4local
> > copperbowl/iocage/jails mountpoint /copperbowl/iocage/jails
> > inherited from copperbowl
> > copperbowl/iocage/jails compression
> > lz4local
> > copperbowl/iocage/jails/jbrowsers mountpoint
> > /copperbowl/iocage/jails/jbrowsers inherited from copperbowl
> > copperbowl/iocage/jails/jbrowsers compression lz4 inherited from
> > copperbowl/iocage/jails
> > copperbowl/iocage/jails/jbrowsers/root mountpoint
> > /copperbowl/iocage/jails/jbrowsers/root inherited from copperbowl
> > copperbowl/iocage/jails/jbrowsers/root compression lz4 inherited from
> > copperbowl/iocage/jails
> > copperbowl/iocage/log mountpoint /copperbowl/iocage/log inherited
> > from copperbowl
> > copperbowl/iocage/log compression
> > lz4
Re: usr.bin/xinstall r366697 versus: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On 12/12/20 2:15 AM, Graham Perrin wrote:
On 23/11/2020 12:18, Graham Perrin wrote:
On 22/11/2020 12:00, Dimitry Andric wrote:
…
I'd guess it's an unintended side-effect of
https://svnweb.freebsd.org/base?view=revision=366697
("install(1): Avoid unncessary fstatfs() calls and use mmap() based on
size").
If you only revert usr.bin/xinstall to r366696, and then rebuild it,
does it still occur?
-Dimitry
Thank you!
Success with r366696:
…
Re:
please, should I open a bug for this?
Is the issue exposed by my choice of gzip-9 for e.g. /usr/ and if so,
can I avoid it by (now) reverting to lz4 for that part, and/or any
other part, of the file system?
Unfortunately I don't have an answer for you at this time, but ACK.
-Ryan
root@mowa219-gjp4-8570p:~ # zfs get mountpoint,compression
NAME PROPERTY VALUE SOURCE
Transcend mountpoint /Volumes/t500 local
Transcend compression zstd
local
Transcend/VirtualBox mountpoint /Volumes/t500/VirtualBox inherited
from Transcend
Transcend/VirtualBox compression zstd inherited from Transcend
copperbowl mountpoint /copperbowl
local
copperbowl compression lz4
local
copperbowl/ROOT mountpoint
none local
copperbowl/ROOT compression lz4 inherited from copperbowl
copperbowl/ROOT/Waterfox mountpoint
/ local
copperbowl/ROOT/Waterfox compression lz4 inherited from copperbowl
copperbowl/ROOT/r367081f mountpoint
/ local
copperbowl/ROOT/r367081f compression lz4 inherited from copperbowl
copperbowl/ROOT/r367936e mountpoint
/ local
copperbowl/ROOT/r367936e compression lz4 inherited from copperbowl
copperbowl/ROOT/r367936f mountpoint
/ local
copperbowl/ROOT/r367936f compression lz4 inherited from copperbowl
copperbowl/ROOT/r367936f@2020-03-20-06:19:45 mountpoint
- -
copperbowl/ROOT/r367936f@2020-03-20-06:19:45 compression
- -
copperbowl/ROOT/r367936f@2020-11-22-23:18:47 mountpoint
- -
copperbowl/ROOT/r367936f@2020-11-22-23:18:47 compression
- -
copperbowl/ROOT/r367936f@2020-12-06-16:23:14 mountpoint
- -
copperbowl/ROOT/r367936f@2020-12-06-16:23:14 compression
- -
copperbowl/VirtualBox mountpoint
/usr/local/VirtualBox local
copperbowl/VirtualBox compression
gzip-9 local
copperbowl/iocage mountpoint /copperbowl/iocage inherited from
copperbowl
copperbowl/iocage compression
gzip-9 local
copperbowl/iocage/download mountpoint /copperbowl/iocage/download
inherited from copperbowl
copperbowl/iocage/download compression
lz4 local
copperbowl/iocage/download/12.0-RELEASE mountpoint
/copperbowl/iocage/download/12.0-RELEASE inherited from copperbowl
copperbowl/iocage/download/12.0-RELEASE compression
lz4 local
copperbowl/iocage/images mountpoint /copperbowl/iocage/images
inherited from copperbowl
copperbowl/iocage/images compression
lz4 local
copperbowl/iocage/jails mountpoint /copperbowl/iocage/jails
inherited from copperbowl
copperbowl/iocage/jails compression
lz4 local
copperbowl/iocage/jails/jbrowsers mountpoint
/copperbowl/iocage/jails/jbrowsers inherited from copperbowl
copperbowl/iocage/jails/jbrowsers compression lz4 inherited from
copperbowl/iocage/jails
copperbowl/iocage/jails/jbrowsers/root mountpoint
/copperbowl/iocage/jails/jbrowsers/root inherited from copperbowl
copperbowl/iocage/jails/jbrowsers/root compression lz4 inherited from
copperbowl/iocage/jails
copperbowl/iocage/log mountpoint /copperbowl/iocage/log inherited
from copperbowl
copperbowl/iocage/log compression
lz4 local
copperbowl/iocage/releases mountpoint /copperbowl/iocage/releases
inherited from copperbowl
copperbowl/iocage/releases compression
lz4 local
copperbowl/iocage/releases/12.0-RELEASE mountpoint
/copperbowl/iocage/releases/12.0-RELEASE inherited from copperbowl
copperbowl/iocage/releases/12.0-RELEASE compression lz4 inherited
from copperbowl/iocage/releases
copperbowl/iocage/releases/12.0-RELEASE/root mountpoint
Re: usr.bin/xinstall r366697 versus: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On 23/11/2020 12:18, Graham Perrin wrote:
On 22/11/2020 12:00, Dimitry Andric wrote:
…
I'd guess it's an unintended side-effect of
https://svnweb.freebsd.org/base?view=revision=366697
("install(1): Avoid unncessary fstatfs() calls and use mmap() based on
size").
If you only revert usr.bin/xinstall to r366696, and then rebuild it,
does it still occur?
-Dimitry
Thank you!
Success with r366696:
…
Re:
please, should I open a bug for this?
Is the issue exposed by my choice of gzip-9 for e.g. /usr/ and if so,
can I avoid it by (now) reverting to lz4 for that part, and/or any other
part, of the file system?
root@mowa219-gjp4-8570p:~ # zfs get mountpoint,compression
NAME PROPERTY VALUE SOURCE
Transcend mountpoint /Volumes/t500 local
Transcend compression zstd local
Transcend/VirtualBox mountpoint /Volumes/t500/VirtualBox inherited
from Transcend
Transcend/VirtualBox compression zstd inherited from Transcend
copperbowl mountpoint /copperbowl local
copperbowl compression lz4 local
copperbowl/ROOT mountpoint
none local
copperbowl/ROOT compression lz4 inherited from copperbowl
copperbowl/ROOT/Waterfox mountpoint
/ local
copperbowl/ROOT/Waterfox compression lz4 inherited from copperbowl
copperbowl/ROOT/r367081f mountpoint
/ local
copperbowl/ROOT/r367081f compression lz4 inherited from copperbowl
copperbowl/ROOT/r367936e mountpoint
/ local
copperbowl/ROOT/r367936e compression lz4 inherited from copperbowl
copperbowl/ROOT/r367936f mountpoint
/ local
copperbowl/ROOT/r367936f compression lz4 inherited from copperbowl
copperbowl/ROOT/r367936f@2020-03-20-06:19:45 mountpoint
- -
copperbowl/ROOT/r367936f@2020-03-20-06:19:45 compression
- -
copperbowl/ROOT/r367936f@2020-11-22-23:18:47 mountpoint
- -
copperbowl/ROOT/r367936f@2020-11-22-23:18:47 compression
- -
copperbowl/ROOT/r367936f@2020-12-06-16:23:14 mountpoint
- -
copperbowl/ROOT/r367936f@2020-12-06-16:23:14 compression
- -
copperbowl/VirtualBox mountpoint
/usr/local/VirtualBox local
copperbowl/VirtualBox compression
gzip-9 local
copperbowl/iocage mountpoint /copperbowl/iocage inherited from copperbowl
copperbowl/iocage compression
gzip-9 local
copperbowl/iocage/download mountpoint /copperbowl/iocage/download
inherited from copperbowl
copperbowl/iocage/download compression
lz4 local
copperbowl/iocage/download/12.0-RELEASE mountpoint
/copperbowl/iocage/download/12.0-RELEASE inherited from copperbowl
copperbowl/iocage/download/12.0-RELEASE compression
lz4 local
copperbowl/iocage/images mountpoint /copperbowl/iocage/images
inherited from copperbowl
copperbowl/iocage/images compression
lz4 local
copperbowl/iocage/jails mountpoint /copperbowl/iocage/jails inherited
from copperbowl
copperbowl/iocage/jails compression
lz4 local
copperbowl/iocage/jails/jbrowsers mountpoint
/copperbowl/iocage/jails/jbrowsers inherited from copperbowl
copperbowl/iocage/jails/jbrowsers compression lz4 inherited from
copperbowl/iocage/jails
copperbowl/iocage/jails/jbrowsers/root mountpoint
/copperbowl/iocage/jails/jbrowsers/root inherited from copperbowl
copperbowl/iocage/jails/jbrowsers/root compression lz4 inherited from
copperbowl/iocage/jails
copperbowl/iocage/log mountpoint /copperbowl/iocage/log inherited from
copperbowl
copperbowl/iocage/log compression
lz4 local
copperbowl/iocage/releases mountpoint /copperbowl/iocage/releases
inherited from copperbowl
copperbowl/iocage/releases compression
lz4 local
copperbowl/iocage/releases/12.0-RELEASE mountpoint
/copperbowl/iocage/releases/12.0-RELEASE inherited from copperbowl
copperbowl/iocage/releases/12.0-RELEASE compression lz4 inherited from
copperbowl/iocage/releases
copperbowl/iocage/releases/12.0-RELEASE/root mountpoint
/copperbowl/iocage/releases/12.0-RELEASE/root inherited from copperbowl
Re: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On 22/11/2020 18:06, Kyle Evans wrote:
> On Sun, Nov 22, 2020 at 6:00 AM Dimitry Andric wrote:
>> I'd guess it's an unintended side-effect of
>> https://svnweb.freebsd.org/base?view=revision=366697
>> ("install(1): Avoid unncessary fstatfs() calls and use mmap() based on
>> size").
>>
>
> Almost certainly -- before, we would never attempt to mmap() on ZFS.
>
> Tossing arichardson@ into CC as the committer
> Tossing mmacy@ and freqlabs@ into CC as ZFS folks
>
> Looking through sys/contrib/openzfs, there's special handling for mmap
> on linux because it bypasses the page cache and relies on caching in
> ARC. AFAICT the FreeBSD side seems to handle write() to mmap'd
> regions, but doesn't do anything with VOP_MMAPPED which might be
> needed to sync the file when it's mmap'd for reading like this. My
> understanding of how this is supposed to actually work on FreeBSD is
> limited, though, so I defer...
Last time I checked mmap worked correctly with ZFS, that was before the switch.
Perhaps, there was an undetected issue -- this can be tested, e.g., by applying
the install change to stable/12.
Perhaps, the ZFS switch came with a regression.
--
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
usr.bin/xinstall r366697 versus: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On 22/11/2020 12:00, Dimitry Andric wrote:
…
I'd guess it's an unintended side-effect of
https://svnweb.freebsd.org/base?view=revision=366697
("install(1): Avoid unncessary fstatfs() calls and use mmap() based on
size").
If you only revert usr.bin/xinstall to r366696, and then rebuild it,
does it still occur?
-Dimitry
Thank you!
Success with r366696:
root@mowa219-gjp4-8570p:~ # date ; uname -v
Sun Nov 22 14:36:10 GMT 2020
FreeBSD 13.0-CURRENT #71 r367081: Tue Oct 27 23:56:28 GMT 2020
root@momh167-gjp4-8570p:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG
root@mowa219-gjp4-8570p:~ # svn up /usr/src && portsnap auto
Updating '/usr/src':
At revision 367936.
Looking up portsnap.FreeBSD.org mirrors... 4 mirrors found.
Fetching snapshot tag from ipv4.aws.portsnap.freebsd.org... done.
Latest snapshot on server matches what we already have.
No updates needed.
Ports tree is already up to date.
root@mowa219-gjp4-8570p:~ # svn update -r366696 /usr/src/usr.bin/xinstall
Updating '/usr/src/usr.bin/xinstall':
U /usr/src/usr.bin/xinstall/xinstall.c
Updated to revision 366696.
root@mowa219-gjp4-8570p:~ # cd /usr/src/usr.bin/xinstall/
root@mowa219-gjp4-8570p:/usr/src/usr.bin/xinstall # make
echo xinstall.full: /usr/lib/libc.a /usr/lib/libmd.a >> .depend
cc -O2 -pipe -fno-common -I/usr/src/contrib/mtree
-I/usr/src/lib/libnetbsd -g -MD -MF.depend.xinstall.o -MTxinstall.o
-std=gnu99 -Wno-format-zero-length -fstack-protector-strong
-Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter
-Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type
-Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter
-Wcast-align -Wchar-subscripts -Winline -Wnested-externs
-Wredundant-decls -Wold-style-definition -Wno-pointer-sign
-Wmissing-variable-declarations -Wthread-safety -Wno-empty-body
-Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c
/usr/src/usr.bin/xinstall/xinstall.c -o xinstall.o
cc -O2 -pipe -fno-common -I/usr/src/contrib/mtree
-I/usr/src/lib/libnetbsd -g -MD -MF.depend.getid.o -MTgetid.o
-std=gnu99 -Wno-format-zero-length -fstack-protector-strong
-Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter
-Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type
-Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter
-Wcast-align -Wchar-subscripts -Winline -Wnested-externs
-Wredundant-decls -Wold-style-definition -Wno-pointer-sign
-Wmissing-variable-declarations -Wthread-safety -Wno-empty-body
-Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c
/usr/src/contrib/mtree/getid.c -o getid.o
cc -O2 -pipe -fno-common -I/usr/src/contrib/mtree
-I/usr/src/lib/libnetbsd -g -std=gnu99 -Wno-format-zero-length
-fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k
-W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes
-Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch
-Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline
-Wnested-externs -Wredundant-decls -Wold-style-definition
-Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety
-Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable
-Qunused-arguments -o xinstall.full xinstall.o getid.o
-L/usr/obj/usr/src/amd64.amd64/lib/libmd -lmd
objcopy --only-keep-debug xinstall.full install.debug
objcopy --strip-debug --add-gnu-debuglink=install.debug xinstall.full
xinstall
gzip -cn /usr/src/usr.bin/xinstall/install.1 > install.1.gz
===> tests (all)
echo '#! /usr/libexec/atf-sh' > install_test.tmp
cat /usr/src/usr.bin/xinstall/tests/install_test.sh >>install_test.tmp
chmod +x install_test.tmp
mv install_test.tmp install_test
root@mowa219-gjp4-8570p:/usr/src/usr.bin/xinstall # make install
install -s -o root -g wheel -m 555 xinstall /usr/bin/install
install -o root -g wheel -m 444 install.debug
/usr/lib/debug/usr/bin/install.debug
install -o root -g wheel -m 444 install.1.gz /usr/share/man/man1/
===> tests (install)
install -o root -g wheel -m 555 install_test
/usr/tests/usr.bin/xinstall/install_test
installing DIRS testsFILESDIR
install -d -m 0755 -o root -g wheel /usr/tests/usr.bin/xinstall
install -o root -g wheel -m 444 Kyuafile
/usr/tests/usr.bin/xinstall/Kyuafile
root@mowa219-gjp4-8570p:/usr/src/usr.bin/xinstall # cd /usr/src && make
-j3 buildworld && make -j3 buildkernel KERNCONF=GENERIC-NODEBUG
…
--
>>> Kernel build for GENERIC-NODEBUG completed on Sun Nov 22 22:43:02
GMT 2020
--
>>> Kernel(s) GENERIC-NODEBUG built in 4973 seconds, ncpu: 4, make -j3
--
root@mowa219-gjp4-8570p:/usr/src # beadm create r367936a
Created successfully
root@mowa219-gjp4-8570p:/usr/src # beadm activate r367936a
Activated successfully
Re: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On Sun, 22 Nov 2020 at 16:06, Kyle Evans wrote:
>
> On Sun, Nov 22, 2020 at 6:00 AM Dimitry Andric wrote:
> >
> > On 22 Nov 2020, at 08:06, Graham Perrin wrote:
> > >
> > > On 20/11/2020 09:57, Graham Perrin wrote:
> > >> On 16/11/2020 09:27, Graham Perrin wrote:
> > >>> Attempting to build r367615 on Friday 13th:
> > >>>
> > >>> …
> > >>>
> > >>> ===> lib/libprocstat/zfs (install)
> > >>> install -U -C -o root -g wheel -m 444
> > >>> /usr/src/lib/libprocstat/libprocstat.h
> > >>> /usr/obj/usr/src/amd64.amd64/tmp/usr/include/
> > >>> ===> lib/libc (obj,all,install)
> > >>> install -U -C -o root -g wheel -m 444 libc.a
> > >>> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/
> > >>> install -U -s -o root -g wheel -m 444 -S libc.so.7
> > >>> /usr/obj/usr/src/amd64.amd64/tmp/lib/
> > >>> install -U -o root -g wheel -m 444libc.so.7.debug
> > >>> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/
> > >>> install: short write to
> > >>> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug:
> > >>> 393216 bytes written, 7462472 bytes asked to write
> > >>> *** [_libinstall] Error code 71
> > >>>
> > >>> make[4]: stopped in /usr/src/lib/libc
> > >>> 1 error
> > >>>
> > >>> make[4]: stopped in /usr/src/lib/libc
> > >>> root@mowa219-gjp4-8570p:/usr/src #
> > >>
> > >> The same problem – short write to
> > >> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug –
> > >> when attempting buildworld of r367847.
> > >>
> > >> If it's relevant: I'm using r367081 for these attempts.
> > >
> > > The same problem when attempting buildworld of r367923. Prior to this I
> > > performed a fresh checkout of /usr/src
> > >
> > > What might cause these failures?
> > >
> > > If it's relevant: I have compression set to gzip-9 for the ZFS dataset
> > > that mounts at /usr
> >
> > I'd guess it's an unintended side-effect of
> > https://svnweb.freebsd.org/base?view=revision=366697
> > ("install(1): Avoid unncessary fstatfs() calls and use mmap() based on
> > size").
> >
>
> Almost certainly -- before, we would never attempt to mmap() on ZFS.
>
> Tossing arichardson@ into CC as the committer
> Tossing mmacy@ and freqlabs@ into CC as ZFS folks
>
> Looking through sys/contrib/openzfs, there's special handling for mmap
> on linux because it bypasses the page cache and relies on caching in
> ARC. AFAICT the FreeBSD side seems to handle write() to mmap'd
> regions, but doesn't do anything with VOP_MMAPPED which might be
> needed to sync the file when it's mmap'd for reading like this. My
> understanding of how this is supposed to actually work on FreeBSD is
> limited, though, so I defer...
>
I'm quite surprised by this, I simply re-used the logic that bin/cp also has.
However, it's possible that this is no longer used in cp due to copy_file_range?
To be honest I'm not sure whether mmap() even improves performance
compared to read/write since the all the MMU setup might be quite
expensive and the data might not even be available so we end up
copying anyway. Maybe we should change install to use read/write
unconditionally? It might also make sense to factor out the
copy_file_range+read/write fallback logic in bin/cp to a library that
can be used in usr.bin/install.
Alex
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On Sun, Nov 22, 2020 at 6:00 AM Dimitry Andric wrote:
>
> On 22 Nov 2020, at 08:06, Graham Perrin wrote:
> >
> > On 20/11/2020 09:57, Graham Perrin wrote:
> >> On 16/11/2020 09:27, Graham Perrin wrote:
> >>> Attempting to build r367615 on Friday 13th:
> >>>
> >>> …
> >>>
> >>> ===> lib/libprocstat/zfs (install)
> >>> install -U -C -o root -g wheel -m 444
> >>> /usr/src/lib/libprocstat/libprocstat.h
> >>> /usr/obj/usr/src/amd64.amd64/tmp/usr/include/
> >>> ===> lib/libc (obj,all,install)
> >>> install -U -C -o root -g wheel -m 444 libc.a
> >>> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/
> >>> install -U -s -o root -g wheel -m 444 -S libc.so.7
> >>> /usr/obj/usr/src/amd64.amd64/tmp/lib/
> >>> install -U -o root -g wheel -m 444libc.so.7.debug
> >>> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/
> >>> install: short write to
> >>> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug:
> >>> 393216 bytes written, 7462472 bytes asked to write
> >>> *** [_libinstall] Error code 71
> >>>
> >>> make[4]: stopped in /usr/src/lib/libc
> >>> 1 error
> >>>
> >>> make[4]: stopped in /usr/src/lib/libc
> >>> root@mowa219-gjp4-8570p:/usr/src #
> >>
> >> The same problem – short write to
> >> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug – when
> >> attempting buildworld of r367847.
> >>
> >> If it's relevant: I'm using r367081 for these attempts.
> >
> > The same problem when attempting buildworld of r367923. Prior to this I
> > performed a fresh checkout of /usr/src
> >
> > What might cause these failures?
> >
> > If it's relevant: I have compression set to gzip-9 for the ZFS dataset that
> > mounts at /usr
>
> I'd guess it's an unintended side-effect of
> https://svnweb.freebsd.org/base?view=revision=366697
> ("install(1): Avoid unncessary fstatfs() calls and use mmap() based on
> size").
>
Almost certainly -- before, we would never attempt to mmap() on ZFS.
Tossing arichardson@ into CC as the committer
Tossing mmacy@ and freqlabs@ into CC as ZFS folks
Looking through sys/contrib/openzfs, there's special handling for mmap
on linux because it bypasses the page cache and relies on caching in
ARC. AFAICT the FreeBSD side seems to handle write() to mmap'd
regions, but doesn't do anything with VOP_MMAPPED which might be
needed to sync the file when it's mmap'd for reading like this. My
understanding of how this is supposed to actually work on FreeBSD is
limited, though, so I defer...
Thanks,
Kyle Evans
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On 22 Nov 2020, at 08:06, Graham Perrin wrote:
>
> On 20/11/2020 09:57, Graham Perrin wrote:
>> On 16/11/2020 09:27, Graham Perrin wrote:
>>> Attempting to build r367615 on Friday 13th:
>>>
>>> …
>>>
>>> ===> lib/libprocstat/zfs (install)
>>> install -U -C -o root -g wheel -m 444
>>> /usr/src/lib/libprocstat/libprocstat.h
>>> /usr/obj/usr/src/amd64.amd64/tmp/usr/include/
>>> ===> lib/libc (obj,all,install)
>>> install -U -C -o root -g wheel -m 444 libc.a
>>> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/
>>> install -U -s -o root -g wheel -m 444 -S libc.so.7
>>> /usr/obj/usr/src/amd64.amd64/tmp/lib/
>>> install -U -o root -g wheel -m 444libc.so.7.debug
>>> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/
>>> install: short write to
>>> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug: 393216
>>> bytes written, 7462472 bytes asked to write
>>> *** [_libinstall] Error code 71
>>>
>>> make[4]: stopped in /usr/src/lib/libc
>>> 1 error
>>>
>>> make[4]: stopped in /usr/src/lib/libc
>>> root@mowa219-gjp4-8570p:/usr/src #
>>
>> The same problem – short write to
>> /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug – when
>> attempting buildworld of r367847.
>>
>> If it's relevant: I'm using r367081 for these attempts.
>
> The same problem when attempting buildworld of r367923. Prior to this I
> performed a fresh checkout of /usr/src
>
> What might cause these failures?
>
> If it's relevant: I have compression set to gzip-9 for the ZFS dataset that
> mounts at /usr
I'd guess it's an unintended side-effect of
https://svnweb.freebsd.org/base?view=revision=366697
("install(1): Avoid unncessary fstatfs() calls and use mmap() based on
size").
If you only revert usr.bin/xinstall to r366696, and then rebuild it,
does it still occur?
-Dimitry
signature.asc
Description: Message signed with OpenPGP
Re: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On 17/11/2020 00:39, Konstantin Belousov wrote: On Mon, Nov 16, 2020 at 10:56:06AM +, Graham Perrin wrote: On 16/11/2020 09:32, Benjamin Kaduk wrote: On Mon, Nov 16, 2020 at 09:27:29AM +, Graham Perrin wrote: Attempting to build r367615 on Friday 13th: … ===> lib/libprocstat/zfs (install) install -U -C -o root -g wheel -m 444 /usr/src/lib/libprocstat/libprocstat.h /usr/obj/usr/src/amd64.amd64/tmp/usr/include/ ===> lib/libc (obj,all,install) install -U -C -o root -g wheel -m 444 libc.a /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/ install -U -s -o root -g wheel -m 444 -S libc.so.7 /usr/obj/usr/src/amd64.amd64/tmp/lib/ install -U -o root -g wheel -m 444 libc.so.7.debug /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/ install: short write to /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug: 393216 bytes written, 7462472 bytes asked to write *** [_libinstall] Error code 71 Is your disk/filesystem full? -Ben AVAIL: 204G And what is the type of filesystem you are _installing_ to ? Also what is the type of filesystem for /tmp, and how much space do you have there ? root@mowa219-gjp4-8570p:~ # zfs get available,compression copperbowl/tmp copperbowl/usr NAME PROPERTY VALUE SOURCE copperbowl/tmp available 221G - copperbowl/tmp compression off local copperbowl/usr available 221G - copperbowl/usr compression gzip-9 local root@mowa219-gjp4-8570p:~ # zfs list NAME USED AVAIL REFER MOUNTPOINT copperbowl 213G 221G 88K /copperbowl copperbowl/ROOT 97.5G 221G 88K none copperbowl/ROOT/Waterfox 348M 221G 59.3G / copperbowl/ROOT/r367081e 1.61M 221G 38.0G / copperbowl/ROOT/r367081f 97.2G 221G 39.2G / copperbowl/VirtualBox 288K 221G 288K /usr/local/VirtualBox copperbowl/iocage 4.35G 221G 3.68M /copperbowl/iocage copperbowl/iocage/download 359M 221G 88K /copperbowl/iocage/download copperbowl/iocage/download/12.0-RELEASE 359M 221G 359M /copperbowl/iocage/download/12.0-RELEASE copperbowl/iocage/images 88K 221G 88K /copperbowl/iocage/images copperbowl/iocage/jails 2.75G 221G 88K /copperbowl/iocage/jails copperbowl/iocage/jails/jbrowsers 2.75G 221G 92K /copperbowl/iocage/jails/jbrowsers copperbowl/iocage/jails/jbrowsers/root 2.75G 221G 3.99G /copperbowl/iocage/jails/jbrowsers/root copperbowl/iocage/log 100K 221G 100K /copperbowl/iocage/log copperbowl/iocage/releases 1.24G 221G 88K /copperbowl/iocage/releases copperbowl/iocage/releases/12.0-RELEASE 1.24G 221G 88K /copperbowl/iocage/releases/12.0-RELEASE copperbowl/iocage/releases/12.0-RELEASE/root 1.24G 221G 1.24G /copperbowl/iocage/releases/12.0-RELEASE/root copperbowl/iocage/templates 88K 221G 88K /copperbowl/iocage/templates copperbowl/poudriere 35.9G 221G 88K /copperbowl/poudriere copperbowl/poudriere/data 1.73G 221G 96K /usr/local/poudriere/data copperbowl/poudriere/data/.m 88K 221G 88K /usr/local/poudriere/data/.m copperbowl/poudriere/data/cache 224K 221G 224K /usr/local/poudriere/data/cache copperbowl/poudriere/data/logs 1.71G 221G 1.71G /usr/local/poudriere/data/logs copperbowl/poudriere/data/packages 24.3M 221G 24.3M /usr/local/poudriere/data/packages copperbowl/poudriere/data/wrkdirs 88K 221G 88K /usr/local/poudriere/data/wrkdirs copperbowl/poudriere/jails 2.05G 221G 88K /copperbowl/poudriere/jails copperbowl/poudriere/jails/head 2.05G 221G 2.05G /usr/local/poudriere/jails/head copperbowl/poudriere/ports 32.1G 221G 88K /copperbowl/poudriere/ports copperbowl/poudriere/ports/default 32.1G 221G 32.1G /usr/local/poudriere/ports/default copperbowl/tmp 808K 221G 808K /tmp copperbowl/usr 70.4G 221G 88K /usr copperbowl/usr/home 67.5G 221G 61.9G /usr/home copperbowl/usr/ports 1.83G 221G 1.83G /usr/ports copperbowl/usr/src 1.06G 221G 1.06G /usr/src copperbowl/var 4.14G 221G 88K /var copperbowl/var/audit 88K 221G 88K /var/audit copperbowl/var/crash 3.81G 221G 3.81G /var/crash copperbowl/var/log 15.7M 221G 15.7M /var/log copperbowl/var/mail 136K 221G 136K /var/mail copperbowl/var/tmp
Re: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On 20/11/2020 09:57, Graham Perrin wrote: On 16/11/2020 09:27, Graham Perrin wrote: Attempting to build r367615 on Friday 13th: … ===> lib/libprocstat/zfs (install) install -U -C -o root -g wheel -m 444 /usr/src/lib/libprocstat/libprocstat.h /usr/obj/usr/src/amd64.amd64/tmp/usr/include/ ===> lib/libc (obj,all,install) install -U -C -o root -g wheel -m 444 libc.a /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/ install -U -s -o root -g wheel -m 444 -S libc.so.7 /usr/obj/usr/src/amd64.amd64/tmp/lib/ install -U -o root -g wheel -m 444 libc.so.7.debug /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/ install: short write to /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug: 393216 bytes written, 7462472 bytes asked to write *** [_libinstall] Error code 71 make[4]: stopped in /usr/src/lib/libc 1 error make[4]: stopped in /usr/src/lib/libc root@mowa219-gjp4-8570p:/usr/src # The same problem – short write to /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug – when attempting buildworld of r367847. If it's relevant: I'm using r367081 for these attempts. The same problem when attempting buildworld of r367923. Prior to this I performed a fresh checkout of /usr/src What might cause these failures? If it's relevant: I have compression set to gzip-9 for the ZFS dataset that mounts at /usr ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On 16/11/2020 09:27, Graham Perrin wrote: Attempting to build r367615 on Friday 13th: … ===> lib/libprocstat/zfs (install) install -U -C -o root -g wheel -m 444 /usr/src/lib/libprocstat/libprocstat.h /usr/obj/usr/src/amd64.amd64/tmp/usr/include/ ===> lib/libc (obj,all,install) install -U -C -o root -g wheel -m 444 libc.a /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/ install -U -s -o root -g wheel -m 444 -S libc.so.7 /usr/obj/usr/src/amd64.amd64/tmp/lib/ install -U -o root -g wheel -m 444 libc.so.7.debug /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/ install: short write to /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug: 393216 bytes written, 7462472 bytes asked to write *** [_libinstall] Error code 71 make[4]: stopped in /usr/src/lib/libc 1 error make[4]: stopped in /usr/src/lib/libc root@mowa219-gjp4-8570p:/usr/src # The same problem – short write to /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug – when attempting buildworld of r367847. If it's relevant: I'm using r367081 for these attempts. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On Mon, Nov 16, 2020 at 10:56:06AM +, Graham Perrin wrote: > On 16/11/2020 09:32, Benjamin Kaduk wrote: > > On Mon, Nov 16, 2020 at 09:27:29AM +, Graham Perrin wrote: > > > Attempting to build r367615 on Friday 13th: > > > > > > … > > > > > > ===> lib/libprocstat/zfs (install) > > > install -U -C -o root -g wheel -m 444 > > > /usr/src/lib/libprocstat/libprocstat.h > > > /usr/obj/usr/src/amd64.amd64/tmp/usr/include/ > > > ===> lib/libc (obj,all,install) > > > install -U -C -o root -g wheel -m 444 libc.a > > > /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/ > > > install -U -s -o root -g wheel -m 444 -S libc.so.7 > > > /usr/obj/usr/src/amd64.amd64/tmp/lib/ > > > install -U -o root -g wheel -m 444 libc.so.7.debug > > > /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/ > > > install: short write to > > > /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug: > > > 393216 bytes written, 7462472 bytes asked to write > > > *** [_libinstall] Error code 71 > > Is your disk/filesystem full? > > > > -Ben > AVAIL: 204G And what is the type of filesystem you are _installing_ to ? Also what is the type of filesystem for /tmp, and how much space do you have there ? ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On 16/11/2020 09:32, Benjamin Kaduk wrote: On Mon, Nov 16, 2020 at 09:27:29AM +, Graham Perrin wrote: Attempting to build r367615 on Friday 13th: … ===> lib/libprocstat/zfs (install) install -U -C -o root -g wheel -m 444 /usr/src/lib/libprocstat/libprocstat.h /usr/obj/usr/src/amd64.amd64/tmp/usr/include/ ===> lib/libc (obj,all,install) install -U -C -o root -g wheel -m 444 libc.a /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/ install -U -s -o root -g wheel -m 444 -S libc.so.7 /usr/obj/usr/src/amd64.amd64/tmp/lib/ install -U -o root -g wheel -m 444 libc.so.7.debug /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/ install: short write to /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug: 393216 bytes written, 7462472 bytes asked to write *** [_libinstall] Error code 71 Is your disk/filesystem full? -Ben AVAIL: 204G ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
On Mon, Nov 16, 2020 at 09:27:29AM +, Graham Perrin wrote: > Attempting to build r367615 on Friday 13th: > > … > > ===> lib/libprocstat/zfs (install) > install -U -C -o root -g wheel -m 444 > /usr/src/lib/libprocstat/libprocstat.h > /usr/obj/usr/src/amd64.amd64/tmp/usr/include/ > ===> lib/libc (obj,all,install) > install -U -C -o root -g wheel -m 444 libc.a > /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/ > install -U -s -o root -g wheel -m 444 -S libc.so.7 > /usr/obj/usr/src/amd64.amd64/tmp/lib/ > install -U -o root -g wheel -m 444 libc.so.7.debug > /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/ > install: short write to > /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug: > 393216 bytes written, 7462472 bytes asked to write > *** [_libinstall] Error code 71 Is your disk/filesystem full? -Ben ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
buildworld: lib/libc: install: short write to libc.so.7.debug: [_libinstall] Error code 71
Attempting to build r367615 on Friday 13th: … ===> lib/libprocstat/zfs (install) install -U -C -o root -g wheel -m 444 /usr/src/lib/libprocstat/libprocstat.h /usr/obj/usr/src/amd64.amd64/tmp/usr/include/ ===> lib/libc (obj,all,install) install -U -C -o root -g wheel -m 444 libc.a /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/ install -U -s -o root -g wheel -m 444 -S libc.so.7 /usr/obj/usr/src/amd64.amd64/tmp/lib/ install -U -o root -g wheel -m 444 libc.so.7.debug /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/ install: short write to /usr/obj/usr/src/amd64.amd64/tmp/usr/lib/debug/lib/libc.so.7.debug: 393216 bytes written, 7462472 bytes asked to write *** [_libinstall] Error code 71 make[4]: stopped in /usr/src/lib/libc 1 error make[4]: stopped in /usr/src/lib/libc root@mowa219-gjp4-8570p:/usr/src # ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
