Re: nvidia-driver build error (last ports, FreeBSD-HEAD)

2018-08-21 Thread Alan Cox
On 08/21/2018 23:29, Manfred Antar wrote:
>
>> On Aug 21, 2018, at 7:23 PM, Alexey Dokuchaev  wrote:
>>
>> On Tue, Aug 21, 2018 at 11:22:56PM +0700, Alex V. Petrov wrote:
>>>  Перенаправленное сообщение 
>>> Тема: nvidia-driver build error (last ports, FreeBSD-HEAD)
>>> Дата: Tue, 21 Aug 2018 16:41:42 +0700
>>> От: Alex V. Petrov 
>>> Кому: FreeBSD Ports 
>> Should be fixed as of r477761.
>>
>> ./danfe
> emulators/open-vm-tools is also broken from the recent changes to sys/vm:
> cc  -O2 -pipe -isystem /usr/local/include -fno-strict-aliasing  -Werror 
> -D_KERNEL -DKLD_MODULE -nostdinc  
> -I/usr/ports/emulators/open-vm-tools/work/open-vm-tools-stable-10.2.5/open-vm-tools/lib/include
>  
> -I/usr/ports/emulators/open-vm-tools/work/open-vm-tools-stable-10.2.5/open-vm-tools/modules/shared/vmxnet
>  -I. -I/usr/src/sys -I/usr/src/sys/contrib/ck/include -fno-common  
> -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer   -MD  
> -MF.depend.if_vxn.o -MTif_vxn.o -mcmodel=kernel -mno-red-zone -mno-mmx 
> -mno-sse -msoft-float  -fno-asynchronous-unwind-tables -ffreestanding -fwrapv 
> -fstack-protector -Wall -Wredundant-decls -Wnested-externs 
> -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wcast-qual -Wundef 
> -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs 
> -fdiagnostics-show-option -Wno-unknown-pragmas 
> -Wno-error-tautological-compare -Wno-error-empty-body 
> -Wno-error-parentheses-equality -Wno-error-unused-function 
> -Wno-error-pointer-sign -Wno-error-shift-negative-value 
> -Wno-address-of-packed-member  -mno-aes -mno-avx  -std=iso9899:1999 -c 
> if_vxn.c -o if_vxn.o
> --- vmmemctl ---
> os.c:410:68: error: too many arguments to function call, expected 2, have 3
>p->bitmap = (unsigned long *)kmem_malloc(kernel_arena, p->size, M_WAITOK | 
> M_ZERO);
> ~~~
> ^
> /usr/src/sys/sys/malloc.h:55:18: note: expanded from macro 'M_WAITOK'
> #define M_WAITOK0x0002  /* ok to block */
> ^
> /usr/src/sys/vm/vm_extern.h:67:1: note: 'kmem_malloc' declared here
> vm_offset_t kmem_malloc(vm_size_t size, int flags);
> ^
> 1 error generated.
>
> I also had to rebuild kde-workspace-kde4 and xorg-server before i could start 
> x without open-vm-tools.
> This is on a FreeBSD-12-Alpha2-current as of today.the old 
> open-vm-tools/modules/freebsd/vmmemctl
> will hang,so i needed to uninstall it to get x.
>
> if these lines are removed from open-vm-tools/modules/freebsd/vmmemctl/os.h 
> open-vm-tools will compile and work:
>
> 407,411d406
> < #if __FreeBSD_version < 100
> bitmap = (unsigned long *)kmem_alloc(kernel_map, p->size);
> < #else
> bitmap = (unsigned long *)kmem_malloc(kernel_arena, p->size, M_WAITOK 
> | M_ZERO);
> < #endif
>
> Not sure if this is the right fix but it enabled me to use the vm-tools again 
> and the associated modules

Change this to

#if __FreeBSD_version < 100
p->bitmap = (unsigned long *)kmem_alloc(kernel_map, p->size);
#elif  __FreeBSD_version < 1200080
p->bitmap = (unsigned long *)kmem_malloc(kernel_arena, p->size,
M_WAITOK | M_ZERO);
#else
p->bitmap = (unsigned long *)kmem_malloc(p->size, M_WAITOK | M_ZERO);
#endif

That said, it's not clear to me why this allocation doesn't use
malloc(9), then no #if's would be required across different versions of
FreeBSD.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: nvidia-driver build error (last ports, FreeBSD-HEAD)

2018-08-21 Thread Manfred Antar


> On Aug 21, 2018, at 7:23 PM, Alexey Dokuchaev  wrote:
> 
> On Tue, Aug 21, 2018 at 11:22:56PM +0700, Alex V. Petrov wrote:
>> 
>>  Перенаправленное сообщение 
>> Тема: nvidia-driver build error (last ports, FreeBSD-HEAD)
>> Дата: Tue, 21 Aug 2018 16:41:42 +0700
>> От: Alex V. Petrov 
>> Кому: FreeBSD Ports 
> 
> Should be fixed as of r477761.
> 
> ./danfe

emulators/open-vm-tools is also broken from the recent changes to sys/vm:
cc  -O2 -pipe -isystem /usr/local/include -fno-strict-aliasing  -Werror 
-D_KERNEL -DKLD_MODULE -nostdinc  
-I/usr/ports/emulators/open-vm-tools/work/open-vm-tools-stable-10.2.5/open-vm-tools/lib/include
 
-I/usr/ports/emulators/open-vm-tools/work/open-vm-tools-stable-10.2.5/open-vm-tools/modules/shared/vmxnet
 -I. -I/usr/src/sys -I/usr/src/sys/contrib/ck/include -fno-common  
-fno-omit-frame-pointer -mno-omit-leaf-frame-pointer   -MD  -MF.depend.if_vxn.o 
-MTif_vxn.o -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float  
-fno-asynchronous-unwind-tables -ffreestanding -fwrapv -fstack-protector -Wall 
-Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes 
-Wpointer-arith -Wcast-qual -Wundef -Wno-pointer-sign 
-D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs 
-fdiagnostics-show-option -Wno-unknown-pragmas -Wno-error-tautological-compare 
-Wno-error-empty-body -Wno-error-parentheses-equality 
-Wno-error-unused-function -Wno-error-pointer-sign 
-Wno-error-shift-negative-value -Wno-address-of-packed-member  -mno-aes 
-mno-avx  -std=iso9899:1999 -c if_vxn.c -o if_vxn.o
--- vmmemctl ---
os.c:410:68: error: too many arguments to function call, expected 2, have 3
   p->bitmap = (unsigned long *)kmem_malloc(kernel_arena, p->size, M_WAITOK | 
M_ZERO);
~~~
^
/usr/src/sys/sys/malloc.h:55:18: note: expanded from macro 'M_WAITOK'
#define M_WAITOK0x0002  /* ok to block */
^
/usr/src/sys/vm/vm_extern.h:67:1: note: 'kmem_malloc' declared here
vm_offset_t kmem_malloc(vm_size_t size, int flags);
^
1 error generated.

I also had to rebuild kde-workspace-kde4 and xorg-server before i could start x 
without open-vm-tools.
This is on a FreeBSD-12-Alpha2-current as of today.the old 
open-vm-tools/modules/freebsd/vmmemctl
will hang,so i needed to uninstall it to get x.

if these lines are removed from open-vm-tools/modules/freebsd/vmmemctl/os.h 
open-vm-tools will compile and work:

407,411d406
< #if __FreeBSD_version < 100
bitmap = (unsigned long *)kmem_alloc(kernel_map, p->size);
< #else
bitmap = (unsigned long *)kmem_malloc(kernel_arena, p->size, M_WAITOK | 
M_ZERO);
< #endif

Not sure if this is the right fix but it enabled me to use the vm-tools again 
and the associated modules

Manfred



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Native Encryption for ZFS on FreeBSD CFT

2018-08-21 Thread Sean Fagan
On Aug 21, 2018, at 8:11 PM, Alan Somers  wrote:
> The last time I looked (which was a long time ago), Oracle's ZFS encryption 
> looked extremely vulnerable to watermarking attacks.  Did anybody ever fix 
> that?

This isn’t Oracle’s implementation, but I don’t know how compatible or not it 
is with it.

Sean.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Native Encryption for ZFS on FreeBSD CFT

2018-08-21 Thread Allan Jude
On 2018-08-21 23:16, Alan Somers wrote:
> On Tue, Aug 21, 2018 at 9:13 PM Sean Fagan  wrote:
> 
>> On Aug 21, 2018, at 8:11 PM, Alan Somers  wrote:
>>> The last time I looked (which was a long time ago), Oracle's ZFS
>> encryption looked extremely vulnerable to watermarking attacks.  Did
>> anybody ever fix that?
>>
>> This isn’t Oracle’s implementation, but I don’t know how compatible or not
>> it is with it.
>>
>> Sean.
>>
> 
> It wasn't just an implementation problem, it was in the design.  IIRC,
> Oracle's encryption allowed encrypted blocks to be deduplicated.  There's
> pretty much no way to defend against watermarking attacks with such a
> design.  Does the new encryption design have the same flaw?
> 
> -Alan
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> 

There is a presentation from the OpenZFS developers summit that walks
through the design. It is not the same as the Oracle version, although
relatively similar.

Video: https://youtu.be/frnLiXclAMo
Slides:
https://drive.google.com/file/d/0B5hUzsxe4cdmU3ZTRXNxa2JIaDQ/view?usp=sharing

It says dedup only works within the same 'clone family', and uses a
unique IV for every block, except when the data is identical (when it
gets deduped)

It isn't clear to me from the presentation if this issue is mitigated or
not. Slide #26 suggests they have done more than Oracle did.


-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature


Re: Native Encryption for ZFS on FreeBSD CFT

2018-08-21 Thread Matthew Macy
On Tue, Aug 21, 2018 at 20:22 Alan Somers  wrote:

> On Tue, Aug 21, 2018 at 9:13 PM Sean Fagan  wrote:
>
>> On Aug 21, 2018, at 8:11 PM, Alan Somers  wrote:
>> > The last time I looked (which was a long time ago), Oracle's ZFS
>> encryption looked extremely vulnerable to watermarking attacks.  Did
>> anybody ever fix that?
>>
>> This isn’t Oracle’s implementation, but I don’t know how compatible or
>> not it is with it.
>>
>> Sean.
>>
>
> It wasn't just an implementation problem, it was in the design.  IIRC,
> Oracle's encryption allowed encrypted blocks to be deduplicated.  There's
> pretty much no way to defend against watermarking attacks with such a
> design.  Does the new encryption design have the same flaw?
>

I would ask the original developer that question (see the commit I linked
to). The current dedup  Implementation is terrible, so there are very few
users of it.

-M



>
> -Alan
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Native Encryption for ZFS on FreeBSD CFT

2018-08-21 Thread Alan Somers
On Tue, Aug 21, 2018 at 9:13 PM Sean Fagan  wrote:

> On Aug 21, 2018, at 8:11 PM, Alan Somers  wrote:
> > The last time I looked (which was a long time ago), Oracle's ZFS
> encryption looked extremely vulnerable to watermarking attacks.  Did
> anybody ever fix that?
>
> This isn’t Oracle’s implementation, but I don’t know how compatible or not
> it is with it.
>
> Sean.
>

It wasn't just an implementation problem, it was in the design.  IIRC,
Oracle's encryption allowed encrypted blocks to be deduplicated.  There's
pretty much no way to defend against watermarking attacks with such a
design.  Does the new encryption design have the same flaw?

-Alan
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Native Encryption for ZFS on FreeBSD CFT

2018-08-21 Thread Alan Somers
The last time I looked (which was a long time ago), Oracle's ZFS encryption
looked extremely vulnerable to watermarking attacks.  Did anybody ever fix
that?
-Alan

On Tue, Aug 21, 2018 at 8:28 PM Matthew Macy  wrote:

> On Tue, Aug 21, 2018 at 6:55 PM Matthew Macy  wrote:
>
> > To anyone with an interest in native encryption in ZFS please test the
> > projects/zfs-crypto-merge-0820 branch in my freebsd repo:
> > https://github.com/mattmacy/networking.git
> >
> >
> Oh and I neglected to state that this work is being supported by iX Systems
> and the tree is all built on work done by Sean Fagan at iX Systems. Please
> keep him in the loop on any problems encountered.
> Thanks.
>
>
>
> > ( git clone  https://github.com/mattmacy/networking.git -b
> > projects/zfs-crypto-merge-0820 )
> >
> > The UI is quite close to the Oracle Solaris ZFS crypto with minor
> > differences for specifying key location.
> >
> > Please note that once a feature is enabled on a pool it can't be
> > disabled. This means that if you enable encryption support on a pool
> > you will never be able to import it in to a ZFS without encryption
> > support. For this reason I would strongly advise against using this on
> > any pool that can't be easily replaced until this change has made its
> > way in to HEAD after the freeze has been lifted.
> >
> >
> > By way of background the original ZoL commit can be found at:
> >
> >
> https://github.com/zfsonlinux/zfs/pull/5769/commits/5aef9bedc801830264428c64cd2242d1b786fd49
> >
> > Thanks in advance.
> > -M
> >
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Native Encryption for ZFS on FreeBSD CFT

2018-08-21 Thread Matthew Macy
On Tue, Aug 21, 2018 at 6:55 PM Matthew Macy  wrote:

> To anyone with an interest in native encryption in ZFS please test the
> projects/zfs-crypto-merge-0820 branch in my freebsd repo:
> https://github.com/mattmacy/networking.git
>
>
Oh and I neglected to state that this work is being supported by iX Systems
and the tree is all built on work done by Sean Fagan at iX Systems. Please
keep him in the loop on any problems encountered.
Thanks.



> ( git clone  https://github.com/mattmacy/networking.git -b
> projects/zfs-crypto-merge-0820 )
>
> The UI is quite close to the Oracle Solaris ZFS crypto with minor
> differences for specifying key location.
>
> Please note that once a feature is enabled on a pool it can't be
> disabled. This means that if you enable encryption support on a pool
> you will never be able to import it in to a ZFS without encryption
> support. For this reason I would strongly advise against using this on
> any pool that can't be easily replaced until this change has made its
> way in to HEAD after the freeze has been lifted.
>
>
> By way of background the original ZoL commit can be found at:
>
> https://github.com/zfsonlinux/zfs/pull/5769/commits/5aef9bedc801830264428c64cd2242d1b786fd49
>
> Thanks in advance.
> -M
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Fwd: nvidia-driver build error (last ports, FreeBSD-HEAD)

2018-08-21 Thread Alexey Dokuchaev
On Tue, Aug 21, 2018 at 11:22:56PM +0700, Alex V. Petrov wrote:
> 
>  Перенаправленное сообщение 
> Тема: nvidia-driver build error (last ports, FreeBSD-HEAD)
> Дата: Tue, 21 Aug 2018 16:41:42 +0700
> От: Alex V. Petrov 
> Кому: FreeBSD Ports 

Should be fixed as of r477761.

./danfe
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: building LLVM threads gets killed

2018-08-21 Thread Mark Millard
Brooks Davis brooks at freebsd.org wrote on
Tue Aug 21 20:29:45 UTC 2018 :

> On Mon, Aug 20, 2018 at 07:33:32PM +0200, Dimitry Andric wrote:
> > . . .
> > 
> > I have attached a patch for most of the llvm ports, which sets the
> > LLVM_PARALLEL_LINK_JOBS CMake flag during the configure phase.
> 
> Committed in r477756.
> 

lld itself has --threads (default) and --no-threads
(non-default, single threaded link). (Mark Johnston
recently made me aware of this.) In my quick experiment
in just one context, --threads used 5 or so threads
in the lld process. (Likely helps with speed when
the hardware threads are simply available without
conflicts.)

Does LLVM_PARALLEL_LINK_JOBS contribute to which of
these is in use?



I've started experimenting with /etc/make.conf like
files having:

LDFLAGS.lld+= -Wl,--no-threads

in contexts with few cores and/or that do not have
lots of RAM. (But I'm unsure of the difference in
RAM usage across various contexts for the two styles
of lld use.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Native Encryption for ZFS on FreeBSD CFT

2018-08-21 Thread Matthew Macy
To anyone with an interest in native encryption in ZFS please test the
projects/zfs-crypto-merge-0820 branch in my freebsd repo:
https://github.com/mattmacy/networking.git

( git clone  https://github.com/mattmacy/networking.git -b
projects/zfs-crypto-merge-0820 )

The UI is quite close to the Oracle Solaris ZFS crypto with minor
differences for specifying key location.

Please note that once a feature is enabled on a pool it can't be
disabled. This means that if you enable encryption support on a pool
you will never be able to import it in to a ZFS without encryption
support. For this reason I would strongly advise against using this on
any pool that can't be easily replaced until this change has made its
way in to HEAD after the freeze has been lifted.


By way of background the original ZoL commit can be found at:
https://github.com/zfsonlinux/zfs/pull/5769/commits/5aef9bedc801830264428c64cd2242d1b786fd49

Thanks in advance.
-M
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


drm / drm2 removal in 12

2018-08-21 Thread Matthew Macy
Just in case anyone misses the change to UPDATING:

20180821:
drm and drm2 have been removed. Users on powerpc, 32-bit hardware,
or with GPUs predating Radeon and i915 will need to install the
graphics/drm-legacy-kmod. All other users should be able to use
one of the LinuxKPI-based ports: graphics/drm-stable-kmod,
graphics/drm-next-kmod, graphics/drm-devel-kmod.

Note that this applies only to 12.

-M
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CFT: TRIM Consolodation on UFS/FFS filesystems

2018-08-21 Thread Mark Millard
bob prohaska fbsd at www.zefox.net wrote on
Wed Aug 22 00:48:33 UTC 2018 :

> On Mon, Aug 20, 2018 at 12:40:56PM -0700, Kirk McKusick wrote:
> > . . .
> > 
> > To enable TRIM consolodation either use `sysctl vfs.ffs.dotrimcons=1'
> > or just set the `dotrimcons' variable in sys/ufs/ffs/ffs_alloc.c to 1.
> > 
> 
> Will the new feature be active  on a Raspberry Pi 3 using flash 
> on microSD and USB for file systems and swap? 

Even if a USB device contains appropriate storage in it, that does
not mean that the USB protocol in use has a way to request the
operation. (Similarly for other multiple stages of translation
than USB protocol being involved.)

For FreeBSD, UFS and ZFS have support if the requests can be sent
through all the stages. Swap partitions do not have support even
if the device does through all the stages.

(See https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206048 for
why I do not otherwise mention swap files.)

RPI3's use (some subset of?) USB 2.0 as I remember. I'm not aware
of the protocol supporting such. (I'm no expert, however.) Thus,
UFS and ZFS end up unable to do TRIM for such contexts as I
understand things.

> Can the feature be turned on using one of the conf files in /etc? 

At least for UFS there are commands for configuration, such as
tunefs and newfs that include control of such points. I do not
remember for ZFS.

As I remember if you enable it on UFS but it actually can not
do it for how the device is connected, FreeBSD reports the
issue at mount or some such.

I've used a SSD both directly via SATA and via a USB enclosure,
the same partitions/file systems across the uses. Only when it
was SATA-style-use did TRIM work.

> According to Sandisk, 
> "All microSD or USB drives are flash memory and does support the TRIM 
> command, however, 
> you will not  notice any difference after running TRIM command on memory 
> cards or USB 
> drives. TRIM command is basically used for SSD and Hard drives."

This gets back into what the protocols in use allow to be
requested when direct communication with the flash is not
in use. (More may be involved.)

> The "you will not notice any difference" qualification makes me slightly 
> uncertain
> the reply was well-informed, but if there's any hope of success I'd like to 
> try it.
> >From time to time there seem to be traffic jams among flash devices on the 
> >RPI3, it
> would a pleasant surprise if this feature helps.

I'll note that gstat with -d allows watching the "BIO_DELETE"
operations (in FreeBSD terms). One can see if they are what
time is being spent on.

Quoting g_bio(9) :

BIO_DELETE   Indicates that a certain range of data is no
 longer used and that it can be erased or
 freed as the underlying technology supports.
 Technologies like flash adaptation layers can
 arrange to erase the relevant blocks before
 they will become reassigned and cryptographic
 devices may want to fill random bits into the
 range to reduce the amount of data available
 for attack.

In your rpi3/2 experiments if you watch the column
sequence:

d/s kBps ms/d

I expect that you will find that they stay at:

  00  0.0

indicating lack of use.



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CFT: TRIM Consolodation on UFS/FFS filesystems

2018-08-21 Thread Greg Rivers
On Tuesday, August 21, 2018 17:48:43 Bob Prohaska wrote:
> Will the new feature be active  on a Raspberry Pi 3 using flash 
> on microSD and USB for file systems and swap? 
> 
It will work on any UFS file system, as long as the underlying medium supports 
TRIM. It will not work for swap because swap is not UFS.

> Can the feature be turned on using one of the conf files in /etc? 
> 
No, TRIM and other UFS features are controlled by parameters in the UFS 
superblock. See tunefs(8).

-- 
Greg
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CFT: TRIM Consolodation on UFS/FFS filesystems

2018-08-21 Thread bob prohaska
On Mon, Aug 20, 2018 at 12:40:56PM -0700, Kirk McKusick wrote:
> I have recently added TRIM consolodation support for the UFS/FFS
> filesystem. This feature consolodates large numbers of TRIM commands
> into a much smaller number of commands covering larger blocks of
> disk space. Best described by the commit message:
> 
>   Author: mckusick
>   Date: Sun Aug 19 16:56:42 2018
>   New Revision: 338056
>   URL: https://svnweb.freebsd.org/changeset/base/338056
> 
>   Log:
> Add consolodation of TRIM / BIO_DELETE commands to the UFS/FFS filesystem.
> 
> When deleting files on filesystems that are stored on flash-memory
> (solid-state) disk drives, the filesystem notifies the underlying
> disk of the blocks that it is no longer using. The notification
> allows the drive to avoid saving these blocks when it needs to
> flash (zero out) one of its flash pages. These notifications of
> no-longer-being-used blocks are referred to as TRIM notifications.
> In FreeBSD these TRIM notifications are sent from the filesystem
> to the drive using the BIO_DELETE command.
> 
> Until now, the filesystem would send a separate message to the drive
> for each block of the file that was deleted. Each Gigabyte of file
> size resulted in over 3000 TRIM messages being sent to the drive.
> This burst of messages can overwhelm the drive's task queue causing
> multiple second delays for read and write requests.
> 
> This implementation collects runs of contiguous blocks in the file
> and then consolodates them into a single BIO_DELETE command to the
> drive. The BIO_DELETE command describes the run of blocks as a
> single large block being deleted. Each Gigabyte of file size can
> result in as few as two BIO_DELETE commands and is typically less
> than ten.  Though these larger BIO_DELETE commands take longer to
> run, they do not clog the drive task queue, so read and write
> commands can intersperse effectively with them.
> 
> Though this new feature has been throughly reviewed and tested, it
> is being added disabled by default so as to minimize the possibility
> of disrupting the upcoming 12.0 release. It can be enabled by running
> ``sysctl vfs.ffs.dotrimcons=1''. Users are encouraged to test it.
> If no problems arise, we will consider requesting that it be enabled
> by default for 12.0.
> 
> Reviewed by:  kib
> Tested by:Peter Holm
> Sponsored by: Netflix
> 
> This support is off by default, but I am hoping that I can get enough
> testing to ensure that it (a) works, and (b) is helpful that it will
> be reasonable to have it turned on by default in 12.0. The cutoff for
> turning it on by default in 12.0 is September 19th. So I am requesting
> your testing feedback in the near-term. Please let me know if you have
> managed to use it successfully (or not) and also if it provided any
> performance difference (good or bad).
> 
> To enable TRIM consolodation either use `sysctl vfs.ffs.dotrimcons=1'
> or just set the `dotrimcons' variable in sys/ufs/ffs/ffs_alloc.c to 1.
> 

Will the new feature be active  on a Raspberry Pi 3 using flash 
on microSD and USB for file systems and swap? 

Can the feature be turned on using one of the conf files in /etc? 


According to Sandisk, 
"All microSD or USB drives are flash memory and does support the TRIM command, 
however, 
you will not  notice any difference after running TRIM command on memory cards 
or USB 
drives. TRIM command is basically used for SSD and Hard drives."

The "you will not notice any difference" qualification makes me slightly 
uncertain
the reply was well-informed, but if there's any hope of success I'd like to try 
it.
>From time to time there seem to be traffic jams among flash devices on the 
>RPI3, it
would a pleasant surprise if this feature helps.

Thanks for reading!

bob prohaska

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: panic excl->shared for an AF_LOCAL socket

2018-08-21 Thread Rick Macklem
Matthew Macy wrote:
[stuff snipped]
>I don't know what's special in this case, but I did revamp the locking there 
>several >months back so I'll take a look next weekend.

Thanks but don't worry about it for now. I think I figured out how the panic()
occurred. If the nfsd was accessing /var/run/nfsuserd.sock for a client and then
tried to soconnect() to it to do the upcall, the nfsd thread would already have
/var/run/nfsuserd.sock vnode locked.

The old way (and what FreeBSD-11 still does) was to use a UDP socket, which
isn't in the file system namespace. (I switched the default to AF_LOCAL so that
nfsuserd could be used in jails where 127.0.0.1 doesn't work, but I now think
it isn't safe to use an AF_LOCAL socket, since it is in the file system's 
namespace
and, therefore, can be accessed directly by the NFS code.

I think I'll revert the "switch to AF_LOCAL socket" patch.

Hopefully the reporter can help confirm this "theory", rick


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: building LLVM threads gets killed

2018-08-21 Thread Samy Mahmoudi
> And probably running ZFS, which ate all your memory and
> put you in a memory contrained environment.

That's exactly why I keep the following in my /boot/loader.rc:

# vfs.zfs.arc_max=1073741824
vfs.zfs.arc_max=2147483648
# vfs.zfs.arc_max=4294967296
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: building LLVM threads gets killed

2018-08-21 Thread Brooks Davis
On Mon, Aug 20, 2018 at 07:33:32PM +0200, Dimitry Andric wrote:
> On 20 Aug 2018, at 16:26, Rodney W. Grimes 
>  wrote:
> > 
> >> On 20 Aug 2018, at 05:01, blubee blubeeme  wrote:
> >>> 
> >>> I am running current compiling LLVM60 and when it comes to linking
> >>> basically all the processes on my computer gets killed; Chrome, Firefox 
> >>> and
> >>> some of the LLVM threads as well
> ...
> > 
> >> It is running out of RAM while running multiple parallel link jobs.  If
> >> you are building using WITH_DEBUG, turn that off, it consumes large
> >> amounts of memory.  If you must have debug info, try adding the
> >> following flag to the CMake command line:
> >> 
> >> -D LLVM_PARALLEL_LINK_JOBS:STRING="1"
> >> 
> >> That will limit the amount of parallel link jobs to 1, even if you
> >> specify -j 8 to gmake or ninja.
> >> 
> >> Brooks, it would not be a bad idea to always use this CMake flag in the
> >> llvm ports. :)
> > 
> > And this may also fix the issues that all the small
> > memory (aka, RPI*) buliders are facing when trying
> > to do -j4?
> 
> Possibly, as linking is usually the most memory-consuming part of the
> build process (and more so, if debugging is enabled).  Are there build
> logs available somewhere for those RPI builders?
> 
> I have attached a patch for most of the llvm ports, which sets the
> LLVM_PARALLEL_LINK_JOBS CMake flag during the configure phase.

Committed in r477756.

-- Brooks


signature.asc
Description: PGP signature


Re: buildworld failure: Do not include ${SRCTOP}/sys when building bootstrap tools

2018-08-21 Thread Alexander Richardson
In my testing 338129 fixed the issue. Seems like the problem is that
bsd.crunchgen.mk iterates over all directories to do a make obj when
it does the bootstrap-tools phase.
On Tue, 21 Aug 2018 at 14:49, Warner Losh  wrote:
>
>
>
> On Tue, Aug 21, 2018 at 12:38 AM, John Baldwin  wrote:
>>
>> On 8/20/18 9:00 PM, O. Hartmann wrote:
>> > -BEGIN PGP SIGNED MESSAGE-
>> > Hash: SHA512
>> >
>> > Am Mon, 20 Aug 2018 21:24:21 +0200
>> > "O. Hartmann"  schrieb:
>> >
>> >> -BEGIN PGP SIGNED MESSAGE-
>> >> Hash: SHA512
>> >>
>> >> Building NanoBSD world on CURRENT r338113 fails due to:
>> >>
>> >> [...]
>> >> cd /pool/sources/CURRENT/src/rescue/rescue/../../sbin/gbde &&  MK_TESTS=no
>> >> UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
>> >> MAKEOBJDIRPREFIX=/pool/nanobsd/amd64/ALERICH_amd64/pool/sources/CURRENT/src/amd64.amd64/rescue/rescue
>> >> make  MK_AUTO_OBJ=no  DIRPRFX=rescue/rescue/gbde/ -DRESCUE 
>> >> CRUNCH_CFLAGS=-DRESCUE
>> >> MK_AUTO_OBJ=no   obj make[5]: 
>> >> "/pool/sources/CURRENT/src/tools/build/mk/Makefile.boot"
>> >> line 18: Do not include ${SRCTOP}/sys when building bootstrap tools.  
>> >> Copy the header to
>> >> ${WORLDTMP}/legacy in tools/build/Makefile instead.  Error was caused by 
>> >> Makefile
>> >> in /pool/sources/CURRENT/src/sbin/gbde *** [obj_crunchdir_gbde] Error 
>> >> code 1
>> >>
>> >> make[4]: stopped in /pool/sources/CURRENT/src/rescue/rescue
>> >> [...]
>> >>
>> >>
>> >> This problem occured during today's source updates since I was able to 
>> >> build the NanoBSD
>> >> image I intend to build yesterday ~ r338060.
>> >>
>> >> What is going wrong?
>> >
>> > It seems the problem has been introduced after r338095, since r338095 
>> > builds ok, while
>> > r338096 doesn't.
>>
>> 338096 added this check to catch a kind of error in our Makefiles.  Alex 
>> (cc'd) can
>> help with figuring out what the error is.
>
>
> Except we're not building anything, we're making obj in rescue...  It looks 
> like a false positive...
>
> Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Fwd: nvidia-driver build error (last ports, FreeBSD-HEAD)

2018-08-21 Thread Alex V. Petrov



 Перенаправленное сообщение 
Тема: nvidia-driver build error (last ports, FreeBSD-HEAD)
Дата: Tue, 21 Aug 2018 16:41:42 +0700
От: Alex V. Petrov 
Кому: FreeBSD Ports 

cc  -O2 -pipe -fno-strict-aliasing -DNV_VERSION_STRING=\"390.77\"
-D__KERNEL__ -DNVRM -Wno-unused-function -Wuninitialized -O2
-fno-strict-aliasing -mno-red-zone -mcmodel=kernel -Wno-sign-co
mpare -Wno-format-extra-args -UDEBUG -U_DEBUG -DNDEBUG -Werror=undef
-Werror -D_KERNEL -DKLD_MODULE -nostdinc  -I. -I../common/inc -I.
-I/usr/src/sys -I/usr/src/sys/contrib/ck/include -fno-
common  -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer   -MD
-MF.depend.nvidia_subr.o -MTnvidia_subr.o -mcmodel=kernel -mno-red-zone
-mno-mmx -mno-sse -msoft-float  -fno-asynchronous-
unwind-tables -ffreestanding -fwrapv -fstack-protector -Wall
-Wredundant-decls -Wnested-externs -Wstrict-prototypes
-Wmissing-prototypes -Wpointer-arith -Wcast-qual -Wundef -Wno-pointer-sign
 -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs
-fdiagnostics-show-option -Wno-unknown-pragmas
-Wno-error-tautological-compare -Wno-error-empty-body
-Wno-error-parentheses-equality
-Wno-error-unused-function -Wno-error-pointer-sign
-Wno-error-shift-negative-value -Wno-address-of-packed-member  -mno-aes
-mno-avx  -std=iso9899:1999 -c nvidia_subr.c -o nvidia_subr.o
nvidia_subr.c:1131:41: error: too many arguments to function call,
expected 7, have 8
sc->dma_mask, PAGE_SIZE, 0, attr);
^~~~
/usr/src/sys/vm/vm_extern.h:61:1: note: 'kmem_alloc_contig' declared here
vm_offset_t kmem_alloc_contig(vm_size_t size, int flags,
^
nvidia_subr.c:1269:45: error: too many arguments to function call,
expected 7, have 8
sc->dma_mask, PAGE_SIZE, 0, attr);
^~~~
/usr/src/sys/vm/vm_extern.h:61:1: note: 'kmem_alloc_contig' declared here
vm_offset_t kmem_alloc_contig(vm_size_t size, int flags,
^
2 errors generated.
*** Error code 1

Stop.
make[4]: stopped in
/usr/ports/x11/nvidia-driver/work/NVIDIA-FreeBSD-x86_64-390.77/src/nvidia
*** Error code 1

Stop.
make[3]: stopped in
/usr/ports/x11/nvidia-driver/work/NVIDIA-FreeBSD-x86_64-390.77/src
*** Error code 1

Stop.
make[2]: stopped in
/usr/ports/x11/nvidia-driver/work/NVIDIA-FreeBSD-x86_64-390.77
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/x11/nvidia-driver
*** Error code 1

Stop.
make: stopped in /usr/ports/x11/nvidia-driver
-- 
-
Alex.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: buildworld failure: Do not include ${SRCTOP}/sys when building bootstrap tools

2018-08-21 Thread Alexander Richardson
I think relaxing the check to just avoid includes of "${SRCTOP}/sys" is
probably the best solution. It would be nice to also handle the
${.CURDIR}/../../sys case but since it's just there to prevent ABI issues
that's probably fine.

Alex

On Tue, 21 Aug 2018 at 15:19 Warner Losh  wrote:

> On Tue, Aug 21, 2018 at 8:16 AM, Warner Losh  wrote:
>
>> There's a half a dozen special targets, however. clean comes to mind...
>>
>>
>> However, this test is needlessly restrictive:
>>
>> .if
>> !empty(CFLAGS:M*${SRCTOP}/sys*:N*${SRCTOP}/sys/cddl/compat*:N*${SRCTOP}/sys/crypto*)
>>
>> since it matches
>>
>> CFLAGS+=-I${SRCTOP}/sys/sys/disk
>>
>> which is totally legit. It's designed to be legit everywhere for building
>> on Linux...
>>
>> .if
>> !empty(CFLAGS:M*${SRCTOP}/sys:N*${SRCTOP}/sys/cddl/compat:N*${SRCTOP}/sys/crypto)
>>
>> would be a better test, imho.
>>
>
> Although, I could passively agressively work around it with
>
> CFLAGS+=-I${.CURDIR}/../../sys/sys/disk
>
> which also kinda defeats its purpose...
>
> Warner
>
>
>> Warner
>>
>> On Tue, Aug 21, 2018 at 8:11 AM, Alexander Richardson <
>> arichard...@freebsd.org> wrote:
>>
>>> In my testing 338129 fixed the issue. Seems like the problem is that
>>> bsd.crunchgen.mk iterates over all directories to do a make obj when
>>> it does the bootstrap-tools phase.
>>> On Tue, 21 Aug 2018 at 14:49, Warner Losh  wrote:
>>> >
>>> >
>>> >
>>> > On Tue, Aug 21, 2018 at 12:38 AM, John Baldwin 
>>> wrote:
>>> >>
>>> >> On 8/20/18 9:00 PM, O. Hartmann wrote:
>>> >> > -BEGIN PGP SIGNED MESSAGE-
>>> >> > Hash: SHA512
>>> >> >
>>> >> > Am Mon, 20 Aug 2018 21:24:21 +0200
>>> >> > "O. Hartmann"  schrieb:
>>> >> >
>>> >> >> -BEGIN PGP SIGNED MESSAGE-
>>> >> >> Hash: SHA512
>>> >> >>
>>> >> >> Building NanoBSD world on CURRENT r338113 fails due to:
>>> >> >>
>>> >> >> [...]
>>> >> >> cd /pool/sources/CURRENT/src/rescue/rescue/../../sbin/gbde &&
>>> MK_TESTS=no
>>> >> >> UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
>>> >> >>
>>> MAKEOBJDIRPREFIX=/pool/nanobsd/amd64/ALERICH_amd64/pool/sources/CURRENT/src/amd64.amd64/rescue/rescue
>>> >> >> make  MK_AUTO_OBJ=no  DIRPRFX=rescue/rescue/gbde/ -DRESCUE
>>> CRUNCH_CFLAGS=-DRESCUE
>>> >> >> MK_AUTO_OBJ=no   obj make[5]:
>>> "/pool/sources/CURRENT/src/tools/build/mk/Makefile.boot"
>>> >> >> line 18: Do not include ${SRCTOP}/sys when building bootstrap
>>> tools.  Copy the header to
>>> >> >> ${WORLDTMP}/legacy in tools/build/Makefile instead.  Error was
>>> caused by Makefile
>>> >> >> in /pool/sources/CURRENT/src/sbin/gbde *** [obj_crunchdir_gbde]
>>> Error code 1
>>> >> >>
>>> >> >> make[4]: stopped in /pool/sources/CURRENT/src/rescue/rescue
>>> >> >> [...]
>>> >> >>
>>> >> >>
>>> >> >> This problem occured during today's source updates since I was
>>> able to build the NanoBSD
>>> >> >> image I intend to build yesterday ~ r338060.
>>> >> >>
>>> >> >> What is going wrong?
>>> >> >
>>> >> > It seems the problem has been introduced after r338095, since
>>> r338095 builds ok, while
>>> >> > r338096 doesn't.
>>> >>
>>> >> 338096 added this check to catch a kind of error in our Makefiles.
>>> Alex (cc'd) can
>>> >> help with figuring out what the error is.
>>> >
>>> >
>>> > Except we're not building anything, we're making obj in rescue...  It
>>> looks like a false positive...
>>> >
>>> > Warner
>>>
>>
>>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Loading carp module crash i386 (12-ALPHA2), seems VNET related

2018-08-21 Thread Marko Zec
On Tue, 21 Aug 2018 13:15:04 +
"Bjoern A. Zeeb"  wrote:
...
>  From the backtrace it seems like VNET_PCPUSTAT_SYSINIT() or the
> kernel linker, or possible pcpu things.  People have recently been
> touching almost all of this :(
> 
> Do you have a last-good revision?

Perhaps the PCPU area is overflowing?  Maybe increasing the
UMA_PCPU_ALLOC_SIZE could make the panic go away (this by no means
should be considered a proper fix)?

Marko
Index: sys/sys/pcpu.h
===
--- sys/sys/pcpu.h	(revision 338128)
+++ sys/sys/pcpu.h	(working copy)
@@ -219,7 +219,7 @@
 #endif
 #define	curvidata	PCPU_GET(vidata)
 
-#define UMA_PCPU_ALLOC_SIZE		PAGE_SIZE
+#define UMA_PCPU_ALLOC_SIZE		(PAGE_SIZE * 16)
 
 #ifdef CTASSERT
 #if defined(__i386__) || defined(__amd64__)
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: buildworld failure: Do not include ${SRCTOP}/sys when building bootstrap tools

2018-08-21 Thread Warner Losh
On Tue, Aug 21, 2018 at 8:16 AM, Warner Losh  wrote:

> There's a half a dozen special targets, however. clean comes to mind...
>
>
> However, this test is needlessly restrictive:
>
> .if !empty(CFLAGS:M*${SRCTOP}/sys*:N*${SRCTOP}/sys/cddl/compat*:
> N*${SRCTOP}/sys/crypto*)
>
> since it matches
>
> CFLAGS+=-I${SRCTOP}/sys/sys/disk
>
> which is totally legit. It's designed to be legit everywhere for building
> on Linux...
>
> .if !empty(CFLAGS:M*${SRCTOP}/sys:N*${SRCTOP}/sys/cddl/compat:N*
> ${SRCTOP}/sys/crypto)
>
> would be a better test, imho.
>

Although, I could passively agressively work around it with

CFLAGS+=-I${.CURDIR}/../../sys/sys/disk

which also kinda defeats its purpose...

Warner


> Warner
>
> On Tue, Aug 21, 2018 at 8:11 AM, Alexander Richardson <
> arichard...@freebsd.org> wrote:
>
>> In my testing 338129 fixed the issue. Seems like the problem is that
>> bsd.crunchgen.mk iterates over all directories to do a make obj when
>> it does the bootstrap-tools phase.
>> On Tue, 21 Aug 2018 at 14:49, Warner Losh  wrote:
>> >
>> >
>> >
>> > On Tue, Aug 21, 2018 at 12:38 AM, John Baldwin  wrote:
>> >>
>> >> On 8/20/18 9:00 PM, O. Hartmann wrote:
>> >> > -BEGIN PGP SIGNED MESSAGE-
>> >> > Hash: SHA512
>> >> >
>> >> > Am Mon, 20 Aug 2018 21:24:21 +0200
>> >> > "O. Hartmann"  schrieb:
>> >> >
>> >> >> -BEGIN PGP SIGNED MESSAGE-
>> >> >> Hash: SHA512
>> >> >>
>> >> >> Building NanoBSD world on CURRENT r338113 fails due to:
>> >> >>
>> >> >> [...]
>> >> >> cd /pool/sources/CURRENT/src/rescue/rescue/../../sbin/gbde &&
>> MK_TESTS=no
>> >> >> UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
>> >> >> MAKEOBJDIRPREFIX=/pool/nanobsd/amd64/ALERICH_amd64/pool/
>> sources/CURRENT/src/amd64.amd64/rescue/rescue
>> >> >> make  MK_AUTO_OBJ=no  DIRPRFX=rescue/rescue/gbde/ -DRESCUE
>> CRUNCH_CFLAGS=-DRESCUE
>> >> >> MK_AUTO_OBJ=no   obj make[5]: "/pool/sources/CURRENT/src/too
>> ls/build/mk/Makefile.boot"
>> >> >> line 18: Do not include ${SRCTOP}/sys when building bootstrap
>> tools.  Copy the header to
>> >> >> ${WORLDTMP}/legacy in tools/build/Makefile instead.  Error was
>> caused by Makefile
>> >> >> in /pool/sources/CURRENT/src/sbin/gbde *** [obj_crunchdir_gbde]
>> Error code 1
>> >> >>
>> >> >> make[4]: stopped in /pool/sources/CURRENT/src/rescue/rescue
>> >> >> [...]
>> >> >>
>> >> >>
>> >> >> This problem occured during today's source updates since I was able
>> to build the NanoBSD
>> >> >> image I intend to build yesterday ~ r338060.
>> >> >>
>> >> >> What is going wrong?
>> >> >
>> >> > It seems the problem has been introduced after r338095, since
>> r338095 builds ok, while
>> >> > r338096 doesn't.
>> >>
>> >> 338096 added this check to catch a kind of error in our Makefiles.
>> Alex (cc'd) can
>> >> help with figuring out what the error is.
>> >
>> >
>> > Except we're not building anything, we're making obj in rescue...  It
>> looks like a false positive...
>> >
>> > Warner
>>
>
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: buildworld failure: Do not include ${SRCTOP}/sys when building bootstrap tools

2018-08-21 Thread Warner Losh
There's a half a dozen special targets, however. clean comes to mind...


However, this test is needlessly restrictive:

.if
!empty(CFLAGS:M*${SRCTOP}/sys*:N*${SRCTOP}/sys/cddl/compat*:N*${SRCTOP}/sys/crypto*)

since it matches

CFLAGS+=-I${SRCTOP}/sys/sys/disk

which is totally legit. It's designed to be legit everywhere for building
on Linux...

.if
!empty(CFLAGS:M*${SRCTOP}/sys:N*${SRCTOP}/sys/cddl/compat:N*${SRCTOP}/sys/crypto)

would be a better test, imho.

Warner

On Tue, Aug 21, 2018 at 8:11 AM, Alexander Richardson <
arichard...@freebsd.org> wrote:

> In my testing 338129 fixed the issue. Seems like the problem is that
> bsd.crunchgen.mk iterates over all directories to do a make obj when
> it does the bootstrap-tools phase.
> On Tue, 21 Aug 2018 at 14:49, Warner Losh  wrote:
> >
> >
> >
> > On Tue, Aug 21, 2018 at 12:38 AM, John Baldwin  wrote:
> >>
> >> On 8/20/18 9:00 PM, O. Hartmann wrote:
> >> > -BEGIN PGP SIGNED MESSAGE-
> >> > Hash: SHA512
> >> >
> >> > Am Mon, 20 Aug 2018 21:24:21 +0200
> >> > "O. Hartmann"  schrieb:
> >> >
> >> >> -BEGIN PGP SIGNED MESSAGE-
> >> >> Hash: SHA512
> >> >>
> >> >> Building NanoBSD world on CURRENT r338113 fails due to:
> >> >>
> >> >> [...]
> >> >> cd /pool/sources/CURRENT/src/rescue/rescue/../../sbin/gbde &&
> MK_TESTS=no
> >> >> UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
> >> >> MAKEOBJDIRPREFIX=/pool/nanobsd/amd64/ALERICH_amd64/
> pool/sources/CURRENT/src/amd64.amd64/rescue/rescue
> >> >> make  MK_AUTO_OBJ=no  DIRPRFX=rescue/rescue/gbde/ -DRESCUE
> CRUNCH_CFLAGS=-DRESCUE
> >> >> MK_AUTO_OBJ=no   obj make[5]: "/pool/sources/CURRENT/src/
> tools/build/mk/Makefile.boot"
> >> >> line 18: Do not include ${SRCTOP}/sys when building bootstrap
> tools.  Copy the header to
> >> >> ${WORLDTMP}/legacy in tools/build/Makefile instead.  Error was
> caused by Makefile
> >> >> in /pool/sources/CURRENT/src/sbin/gbde *** [obj_crunchdir_gbde]
> Error code 1
> >> >>
> >> >> make[4]: stopped in /pool/sources/CURRENT/src/rescue/rescue
> >> >> [...]
> >> >>
> >> >>
> >> >> This problem occured during today's source updates since I was able
> to build the NanoBSD
> >> >> image I intend to build yesterday ~ r338060.
> >> >>
> >> >> What is going wrong?
> >> >
> >> > It seems the problem has been introduced after r338095, since r338095
> builds ok, while
> >> > r338096 doesn't.
> >>
> >> 338096 added this check to catch a kind of error in our Makefiles.
> Alex (cc'd) can
> >> help with figuring out what the error is.
> >
> >
> > Except we're not building anything, we're making obj in rescue...  It
> looks like a false positive...
> >
> > Warner
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: buildworld failure: Do not include ${SRCTOP}/sys when building bootstrap tools

2018-08-21 Thread Warner Losh
On Tue, Aug 21, 2018 at 7:49 AM, Warner Losh  wrote:

>
>
> On Tue, Aug 21, 2018 at 12:38 AM, John Baldwin  wrote:
>
>> On 8/20/18 9:00 PM, O. Hartmann wrote:
>> > -BEGIN PGP SIGNED MESSAGE-
>> > Hash: SHA512
>> >
>> > Am Mon, 20 Aug 2018 21:24:21 +0200
>> > "O. Hartmann"  schrieb:
>> >
>> >> -BEGIN PGP SIGNED MESSAGE-
>> >> Hash: SHA512
>> >>
>> >> Building NanoBSD world on CURRENT r338113 fails due to:
>> >>
>> >> [...]
>> >> cd /pool/sources/CURRENT/src/rescue/rescue/../../sbin/gbde &&
>> MK_TESTS=no
>> >> UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
>> >> MAKEOBJDIRPREFIX=/pool/nanobsd/amd64/ALERICH_amd64/pool/
>> sources/CURRENT/src/amd64.amd64/rescue/rescue
>> >> make  MK_AUTO_OBJ=no  DIRPRFX=rescue/rescue/gbde/ -DRESCUE
>> CRUNCH_CFLAGS=-DRESCUE
>> >> MK_AUTO_OBJ=no   obj make[5]: "/pool/sources/CURRENT/src/too
>> ls/build/mk/Makefile.boot"
>> >> line 18: Do not include ${SRCTOP}/sys when building bootstrap tools.
>> Copy the header to
>> >> ${WORLDTMP}/legacy in tools/build/Makefile instead.  Error was caused
>> by Makefile
>> >> in /pool/sources/CURRENT/src/sbin/gbde *** [obj_crunchdir_gbde] Error
>> code 1
>> >>
>> >> make[4]: stopped in /pool/sources/CURRENT/src/rescue/rescue
>> >> [...]
>> >>
>> >>
>> >> This problem occured during today's source updates since I was able to
>> build the NanoBSD
>> >> image I intend to build yesterday ~ r338060.
>> >>
>> >> What is going wrong?
>> >
>> > It seems the problem has been introduced after r338095, since r338095
>> builds ok, while
>> > r338096 doesn't.
>>
>> 338096 added this check to catch a kind of error in our Makefiles.  Alex
>> (cc'd) can
>> help with figuring out what the error is.
>>
>
> Except we're not building anything, we're making obj in rescue...  It
> looks like a false positive...
>

weird, I'm dying elsewhere.

You can recreate this with

cd tools/tools/nanobsd/embedded
sh -c ../nanobsd.sh -c qemu-amd64-uefi.cfg

it dies in mkimg. We have LOCAL_XTOOL_DIRS=usr.bin/mkimg so it's building
in stage 3. It's likely because of

CFLAGS+=-I${SRCTOP}/sys/sys/disk

which is 100% legit always by design, so the test in this case is a false
positive and must be relaxed.

I'm guessing that the rescue case above is similar: we're building an
different tool early.

Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: buildworld failure: Do not include ${SRCTOP}/sys when building bootstrap tools

2018-08-21 Thread Warner Losh
On Tue, Aug 21, 2018 at 12:38 AM, John Baldwin  wrote:

> On 8/20/18 9:00 PM, O. Hartmann wrote:
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA512
> >
> > Am Mon, 20 Aug 2018 21:24:21 +0200
> > "O. Hartmann"  schrieb:
> >
> >> -BEGIN PGP SIGNED MESSAGE-
> >> Hash: SHA512
> >>
> >> Building NanoBSD world on CURRENT r338113 fails due to:
> >>
> >> [...]
> >> cd /pool/sources/CURRENT/src/rescue/rescue/../../sbin/gbde &&
> MK_TESTS=no
> >> UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
> >> MAKEOBJDIRPREFIX=/pool/nanobsd/amd64/ALERICH_amd64/
> pool/sources/CURRENT/src/amd64.amd64/rescue/rescue
> >> make  MK_AUTO_OBJ=no  DIRPRFX=rescue/rescue/gbde/ -DRESCUE
> CRUNCH_CFLAGS=-DRESCUE
> >> MK_AUTO_OBJ=no   obj make[5]: "/pool/sources/CURRENT/src/
> tools/build/mk/Makefile.boot"
> >> line 18: Do not include ${SRCTOP}/sys when building bootstrap tools.
> Copy the header to
> >> ${WORLDTMP}/legacy in tools/build/Makefile instead.  Error was caused
> by Makefile
> >> in /pool/sources/CURRENT/src/sbin/gbde *** [obj_crunchdir_gbde] Error
> code 1
> >>
> >> make[4]: stopped in /pool/sources/CURRENT/src/rescue/rescue
> >> [...]
> >>
> >>
> >> This problem occured during today's source updates since I was able to
> build the NanoBSD
> >> image I intend to build yesterday ~ r338060.
> >>
> >> What is going wrong?
> >
> > It seems the problem has been introduced after r338095, since r338095
> builds ok, while
> > r338096 doesn't.
>
> 338096 added this check to catch a kind of error in our Makefiles.  Alex
> (cc'd) can
> help with figuring out what the error is.
>

Except we're not building anything, we're making obj in rescue...  It looks
like a false positive...

Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Loading carp module crash i386 (12-ALPHA2), seems VNET related

2018-08-21 Thread Bjoern A. Zeeb

On 21 Aug 2018, at 12:31, Olivier Cochard-Labbé wrote:


On Tue, Aug 21, 2018 at 11:29 AM Marko Zec  wrote:


On Mon, 20 Aug 2018 16:26:29 +0200
Olivier Cochard-Labbé  wrote:


Just loading carp kernel module is enough to panic it:

[root@router]~# uname -a
FreeBSD router.bsdrp.net 12.0-ALPHA2 FreeBSD 12.0-ALPHA2  r338100M
i386 [root@router]~# kldload carp
Fatal trap 12: page fault while in kernel mode


How many vnets were already instantiated at the time of kldloading
carp?  Does the panic ocur with no (non-default) vnets instantiated?



Hi,

None VNET were instantiated: Just start the latest 12-i386 image (
https://download.freebsd.org/ftp/snapshots/VM-IMAGES/12.0-CURRENT/i386/Latest/)
on a VM, and once logged as root enter 'kldload carp' to trigger a 
panic.


From the backtrace it seems like VNET_PCPUSTAT_SYSINIT() or the kernel 
linker, or possible pcpu things.  People have recently been touching 
almost all of this :(


Do you have a last-good revision?

/bz
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Loading carp module crash i386 (12-ALPHA2), seems VNET related

2018-08-21 Thread Olivier Cochard-Labbé
On Tue, Aug 21, 2018 at 11:29 AM Marko Zec  wrote:

> On Mon, 20 Aug 2018 16:26:29 +0200
> Olivier Cochard-Labbé  wrote:
>
> > Just loading carp kernel module is enough to panic it:
> >
> > [root@router]~# uname -a
> > FreeBSD router.bsdrp.net 12.0-ALPHA2 FreeBSD 12.0-ALPHA2  r338100M
> > i386 [root@router]~# kldload carp
> > Fatal trap 12: page fault while in kernel mode
>
> How many vnets were already instantiated at the time of kldloading
> carp?  Does the panic ocur with no (non-default) vnets instantiated?
>
>
Hi,

None VNET were instantiated: Just start the latest 12-i386 image (
https://download.freebsd.org/ftp/snapshots/VM-IMAGES/12.0-CURRENT/i386/Latest/)
on a VM, and once logged as root enter 'kldload carp' to trigger a panic.

Regards,

Olivier
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Loading carp module crash i386 (12-ALPHA2), seems VNET related

2018-08-21 Thread Marko Zec
On Mon, 20 Aug 2018 16:26:29 +0200
Olivier Cochard-Labbé  wrote:

> Just loading carp kernel module is enough to panic it:
> 
> [root@router]~# uname -a
> FreeBSD router.bsdrp.net 12.0-ALPHA2 FreeBSD 12.0-ALPHA2  r338100M
> i386 [root@router]~# kldload carp
> Fatal trap 12: page fault while in kernel mode

How many vnets were already instantiated at the time of kldloading
carp?  Does the panic ocur with no (non-default) vnets instantiated?

> cpuid = 0; apic id = 00
> fault virtual address   = 0x14de0f4c
> fault code  = supervisor write, page not present
> instruction pointer = 0x20:0x1201657c
> stack pointer   = 0x28:0x11f3e804
> frame pointer   = 0x28:0x11f3e80c
> code segment= base 0x0, limit 0xf, type 0x1b
> = DPL 0, pres 1, def32 1, gran 1
> processor eflags= interrupt enabled, resume, IOPL = 0
> current process = 55291 (kldload)
> [ thread pid 55291 tid 100070 ]
> Stopped at  vnet_carpstats_init+0x2c:   movl
> %eax,__stop_set_vnet(%ecx,%esi,1)
> 
> A backtrace on kgdb gives this output:
> 
> (kgdb) bt
> #0  __curthread () at ./machine/pcpu.h:226
> #1  doadump (textdump=) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_shutdown.c:366
> #2  0x009f69ee in db_dump (dummy=302081404, dummy2=false, dummy3=-1,
> dummy4=0x11f3e4fc "") at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/ddb/db_command.c:574
> #3  0x009f67c7 in db_command (last_cmdp=,
> cmd_table=, dopager=) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/ddb/db_command.c:481
> #4  0x009f6520 in db_command_loop () at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/ddb/db_command.c:534
> #5  0x009f978d in db_trap (type=12, code=0) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/ddb/db_main.c:252
> #6  0x0113e29e in kdb_trap (type=12, code=0, tf=0x11f3e7c4) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/subr_kdb.c:693
> #7  0x016d593c in trap_fatal (frame=0x11f3e7c4, eva=)
> at /usr/local/BSDRP/TESTING/FreeBSD/src/sys/i386/i386/trap.c:989
> #8  0x016d5a43 in trap_pfault (frame=, usermode=0,
> eva=350097228) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/i386/i386/trap.c:827
> #9  0x016d508f in trap (frame=0x11f3e7c4) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/i386/i386/trap.c:519
> #10 0xffc0315d in ?? ()
> #11 0x11f3e7c4 in ?? ()
> #12 0x0121c7ef in vnet_register_sysinit (arg=0x120182cc
> ) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/net/vnet.c:500
> #13 0x010c18f0 in linker_file_sysinit (lf=) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_linker.c:236
> #14 linker_load_file (filename=, result= out>)
> out>at /usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_linker.c:462
> #15 linker_load_module (kldname=, modname=0x11449c00
> "carp", parent=0x0, verinfo=0x0, lfpp=0x11f3ea6c) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_linker.c:2092
> #16 0x010c30d1 in kern_kldload (td=0x1157c380, file=0x11449c00 "carp",
> fileid=0x11f3ea98) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_linker.c:1071
> #17 0x010c31ee in sys_kldload (td=0x1157c380, uap=0x1157c604) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/kern/kern_linker.c:1097
> #18 0x016d6217 in syscallenter (td=) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/i386/i386/../../kern/subr_syscall.c:135
> #19 syscall (frame=0x11f3eba8) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/i386/i386/trap.c:1152
> #20 0xffc033a7 in ?? ()
> #21 0x11f3eba8 in ?? ()
> Backtrace stopped: Cannot access memory at address 0xffbfecec
> (kgdb) frame 12
> #12 0x0121c7ef in vnet_register_sysinit (arg=0x120182cc
> ) at
> /usr/local/BSDRP/TESTING/FreeBSD/src/sys/net/vnet.c:500
> 500 vs->func(vs->arg);
> 
> 
> Regards,
> 
> Olivier
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to
> "freebsd-current-unsubscr...@freebsd.org"

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Newly upgraded -CURRENT box does not boot

2018-08-21 Thread John Baldwin
On 8/21/18 4:19 AM, Kyle Evans wrote:
> On Mon, Aug 20, 2018 at 4:27 PM, Brett Gmoser
>  wrote:
>> Hi there,
>>
>> I was told to e-mail these addresses with this.
>>
>> I did an `svn update` on /usr/src last night, build world and kernel as
>> usual. This morning I installed the kernel, booted into single user,
>> installed world and did mergemaster -Ui as usual. The new kernel had booted
>> fine. Upon reboot, the machine will no longer boot:
>>
>> Startup error in /boot/lua/loader.lua:
>> LUA ERROR: cannot open /boot/lua/loader.lua: no such file or directory
>>
>> can't load 'kernel'
>>
>> Many things in the bootloader do not work, including "boot kernel.old", "ls
>> /boot", and various other things (most if not all just result in "Command
>> failed"). Interestingly, "ls /mnt" works, other directories do not. That's
>> the only clue I have.
>>
>> I'm able to reboot in an installer image and mount the drive just fine.
>> Everything is there and is as expected, including /boot/lua/loader.lua.
>>
>> I re-installed everything in /usr/src/stand (chroot'd on the installer
>> image, and "cd /usr/src/stand && make clean all install"). This did not fix
>> the problem.
>>
>> Does anybody happen to have any ideas?
>>
> 
> To briefly follow up and summarize the current standing here following
> some more discussion/attempts to fix on IRC:
> 
> 1.) x86 BIOS boot
> 2.) Problem appears for both forthloader and lualoader
> 3.) Early March loader works, recent loader does not [Only tried
> loader from the past ~day]
> 4.) ls / works, ls /mnt works, ls /boot and other directories fails
> 5.) However, /boot is confirmed intact and populated by booting via
> 11.2 install media and inspecting local disk
> 
> We'll hopefully be having a bisect session tomorrow to figure out
> where exactly this broke so that maybe Brett has a chance to upgrade
> to 12.0, unless this sounds familiar to someone and the cause is
> obvious. =)

I would start with bisecting the changes to libi386/biosdisk.c.  Also,
comparing 'lsdev -v' output between old and new loaders might be a useful
step before starting on the bisecting.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"