Re: filemon: weird full-time build although filemon enabled

[Simon J. Gerraty]

Hi,

> I build CURRENT on two technically similar systems on a almost daily basis. 
> Therefore, it
> was a great relief having WITH_META_MODE=yes set in /etc/src-env.conf for 
> incremental
> builds. To make my understanding of this clear (just in case I'm wrong): 
> setting
> WITH_META_MODE builds only portions that does not need to be build in the 
> make context.

I'm not sure I parsed that correctly, so let me just say;
With MK_META_MODE=yes, bmake should be put into meta mode.

This simply means that most targets get a .meta file, that records the
expanded command line, any command output and a syscall trace.
These can be very useful for debugging.

More important; on an update build bmake can use them to make targets
out-of-date that the normal rules miss.
Any change to the expanded command line, makes the target out-of-date.
Any change to any file read or executed makes the target out-of-date.
 
> Well, the reason writing this email is: on one system, I run almost every 
> reboot into a
> "full build" and this puzzles me a bit. The long-lasting and time exhausting 
> builds are

You can add -dM to your build flags to have meta_oodate() - the function
that does the above, explain why it decided the target is out-of-date.
The results can be enlightening.

Note, if a target rebuilds without -dM saying why - then the normal make
rules decided it was out-of-date.

> within the LLVM/CLANG tree. They consume a lot of time. The box in
> question does have a

Yes, building clang takes a while...

> Well, having said this, I need to mention that both systems use almost
> identical /etc/src.conf setting - except the order of appearance of
> the WITH_ tags.

That should not matter, bsd.mkopt.mk processes them in a specified
order.

> The problem: to make my point clear: the "weak" box starts compiling almost 
> everytime now
> the LLVM/CLANG tree while the XEON box does not. This is spooky.

Add -dM, and see if that tells you why.
Also confirm that filemon is getting loaded after each reboot.
If that comes and goes, that could cause the behavior you see.

> I think there is something missing an I'd like to ask what is the suggested 
> way to
> initially restart a full build to ensure that WITH_META_MODE gets initialised 
> correctly.

I think Bryan has set that up so that it should take care of itself
pretty well - even toggling bewteen non-meta and meta builds should
DTRT - which is why it matters if filemon was used that it is there for
next build.

> Well, I'm not a developer, so please be patient with my naive report.

No problem.  Adding -dM to your build (will add a fair bit of output),
will hopefully tell you what's up.

>From what you describe, I'd not be surprised if you see some
"a build command has changed" and it only takes a few to trigger a
cascade of targets out-of-date.

--sjg

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

more default uid/gid for NFS in mountd

[Rick Macklem]

Hi,

Five years ago (yea, it slipped through a crack;-), Slawa reported that files
created by root would end up owned by uid 2**32-2 (-2 as uint32_t).
This happens if there is no "-maproot=" in the /etc/exports line.

The cause is obvious. The value is set to -2 by default.

The question is... Should this be changed to 65534 (ie "nobody")?
- It would seem more consistent to make it the uid of nobody, but I can also see
  the argument that since it has been like this *forever*, that changing it 
would be
  a POLA violation.
What do others think?

It is also the case that mountd.c doesn't look "nobody" up in the password 
database
to set the default. It would be nice to do this, but it could result in the 
mountd daemon
getting "stuck" during a boot waiting for an unresponsive LDAP service or 
similar.
Does doing this sound like a good idea?

Thanks for any comments, rick
ps: Here's the original email thread, in case you are interested:
  https://lists.freebsd.org/pipermail/freebsd-stable/2012-March/066868.html

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: filemon: weird full-time build although filemon enabled

[Ngie Cooper (yaneurabeya)]

> On May 6, 2017, at 00:22, O. Hartmann  wrote:
> 
> I build CURRENT on two technically similar systems on a almost daily basis. 
> Therefore, it
> was a great relief having WITH_META_MODE=yes set in /etc/src-env.conf for 
> incremental
> builds. To make my understanding of this clear (just in case I'm wrong): 
> setting
> WITH_META_MODE builds only portions that does not need to be build in the 
> make context.
> 
> Well, the reason writing this email is: on one system, I run almost every 
> reboot into a
> "full build" and this puzzles me a bit. The long-lasting and time exhausting 
> builds are
> within the LLVM/CLANG tree. They consume a lot of time. The box in question 
> does have a
> weak CPU, only two physical cores, four threads, 8GB of RAM and builds the 
> /usr/obj
> residing on a SSD. The reference machine does have the same motherboard, also 
> a SSD, but
> has 16 GB RAM and a 4-core/8 threads XEON CPU - but both are "IvyBridge". The 
> XEON
> usually needs 30 - 40 minutes to compile a full world/kernel from a clean 
> /usr/obj, the
> "weak" box takes approximately 120 minutes - it is understandable that a 
> shortage of the
> build time is appreciated.
> 
> Well, having said this, I need to mention that both systems use almost
> identical /etc/src.conf setting - except the order of appearance of the WITH_ 
> tags. In
> fact, they are identical except the KERNCONF (naming of the kernel) and 
> PORTS_MODULES=,
> the "weak" box incorporates x11/nvidia-driver and 
> emulators/virtualbox-ose-kmod, so these
> modules are build every time the system gets rebuild, but the time taken by 
> those is
> negligible.
> 
> The problem: to make my point clear: the "weak" box starts compiling almost 
> everytime now
> the LLVM/CLANG tree while the XEON box does not. This is spooky.
> 
> I deleted on both  systems recently /usr/obj completely from its content and 
> restarted a
> buildworld again to hope, that the problem was introduced due to some files
> necessary for the BSD make environment to indicate the incremental build. But 
> no success.
> Even more spooky is the fact, that after a build on the "weak" box and a 
> build again, the
> box bevaves as expected not rebuilding everything again, but in some cases 
> after a
> reboot, a rebuild the hits again the build of LLVM/CLANG tree, while the XEON 
> box does
> not.
> 
> I think there is something missing an I'd like to ask what is the suggested 
> way to
> initially restart a full build to ensure that WITH_META_MODE gets initialised 
> correctly.
> 
> Well, I'm not a developer, so please be patient with my naive report.
> 
> Thanks in advance,

Dumb question: which kernel are you using on which machine (GENERIC, 
GENERIC-NODEBUG, a custom kernel with or without debug hooks, e.g., INVARIANTS, 
enabled)? Also, how are you building the system (locally using UFS or ZFS, 
remotely, e.g., over NFS, etc)?
Thanks,
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: filemon: weird full-time build although filemon enabled

[O. Hartmann]

Am Mon, 8 May 2017 11:23:01 -0700
"Ngie Cooper (yaneurabeya)"  schrieb:

> > On May 6, 2017, at 00:22, O. Hartmann  wrote:
> > 
> > I build CURRENT on two technically similar systems on a almost daily basis.
> > Therefore, it was a great relief having WITH_META_MODE=yes set in 
> > /etc/src-env.conf
> > for incremental builds. To make my understanding of this clear (just in 
> > case I'm
> > wrong): setting WITH_META_MODE builds only portions that does not need to 
> > be build in
> > the make context.
> > 
> > Well, the reason writing this email is: on one system, I run almost every 
> > reboot into
> > a "full build" and this puzzles me a bit. The long-lasting and time 
> > exhausting builds
> > are within the LLVM/CLANG tree. They consume a lot of time. The box in 
> > question does
> > have a weak CPU, only two physical cores, four threads, 8GB of RAM and 
> > builds
> > the /usr/obj residing on a SSD. The reference machine does have the same 
> > motherboard,
> > also a SSD, but has 16 GB RAM and a 4-core/8 threads XEON CPU - but both are
> > "IvyBridge". The XEON usually needs 30 - 40 minutes to compile a full 
> > world/kernel
> > from a clean /usr/obj, the "weak" box takes approximately 120 minutes - it 
> > is
> > understandable that a shortage of the build time is appreciated.
> > 
> > Well, having said this, I need to mention that both systems use almost
> > identical /etc/src.conf setting - except the order of appearance of the 
> > WITH_ tags. In
> > fact, they are identical except the KERNCONF (naming of the kernel) and
> > PORTS_MODULES=, the "weak" box incorporates x11/nvidia-driver and
> > emulators/virtualbox-ose-kmod, so these modules are build every time the 
> > system gets
> > rebuild, but the time taken by those is negligible.
> > 
> > The problem: to make my point clear: the "weak" box starts compiling almost 
> > everytime
> > now the LLVM/CLANG tree while the XEON box does not. This is spooky.
> > 
> > I deleted on both  systems recently /usr/obj completely from its content and
> > restarted a buildworld again to hope, that the problem was introduced due 
> > to some
> > files necessary for the BSD make environment to indicate the incremental 
> > build. But
> > no success. Even more spooky is the fact, that after a build on the "weak" 
> > box and a
> > build again, the box bevaves as expected not rebuilding everything again, 
> > but in some
> > cases after a reboot, a rebuild the hits again the build of LLVM/CLANG 
> > tree, while
> > the XEON box does not.
> > 
> > I think there is something missing an I'd like to ask what is the suggested 
> > way to
> > initially restart a full build to ensure that WITH_META_MODE gets 
> > initialised
> > correctly.
> > 
> > Well, I'm not a developer, so please be patient with my naive report.
> > 
> > Thanks in advance,  
> 
>   Dumb question: which kernel are you using on which machine (GENERIC,
> GENERIC-NODEBUG, a custom kernel with or without debug hooks, e.g., 
> INVARIANTS,
> enabled)? Also, how are you building the system (locally using UFS or ZFS, 
> remotely,
> e.g., over NFS, etc)? Thanks, -Ngie

- Custom kernel
- as far as I know, no debug hooks, no INVARIANTS (i tried to eliminate all of 
them)
- Build is performed on a Samsung SSD 850 PRO with UFS2, locally

-- 
O. Hartmann

Ich widerspreche der Nutzung oder Übermittlung meiner Daten für
Werbezwecke oder für die Markt- oder Meinungsforschung (§ 28 Abs. 4 BDSG).


pgp3vINIp8vSX.pgp
Description: OpenPGP digital signature

Re: ${src}/release/release.sh fails at makefs state

[Ngie Cooper (yaneurabeya)]

> On May 8, 2017, at 06:07, Alastair Hogge  wrote:
> 
> On Fri, 5 May 2017 11:26:44 PM Ngie Cooper wrote:
>> (CCing emaste)
>> 
>>> On May 5, 2017, at 21:55, Alastair Hogge  wrote:
>> …
>> 
>>> Calculated size of `memstick.img.part': 485474304 bytes, 9435 inodes
>>> Extent size set to 8192
>>> memstick.img.part: 463.0MB (948192 sectors) block size 8192, fragment size
>>> 1024
>>> 
>>>   using 9 cylinder groups of 54.38MB, 6960 blks, 1152 inodes.
>>> 
>>> super-block backups (for fsck -b #) at:
>>>32, 111392, 222752, 334112, 445472, 556832, 668192, 779552, 890912,
>>> 
>>> Populating `memstick.img.part'
>>> makefs: bread: read 8192 (684294144) returned 0: No error: 0
>>> makefs failed
>>> *** Error code 1
> 
> Thanks Ngie.
> 
> It look like r317744 is causing the problem:
> https://svnweb.freebsd.org/base?view=revision=317744
> 
> That is the checkout that breaks release building for me, prior revisions
> cause no problem.

Hi Alastair,
r317967 should allow you to build release images again.
Cheers!
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: bootcode capable of booting both UFS and ZFS?

[John Baldwin]

On Friday, May 05, 2017 11:01:03 PM Toomas Soome wrote:
> 
> > On 5. mai 2017, at 22:07, Julian Elischer  wrote:
> > 
> > Subject says it all really, is this an option at this time?
> > 
> > we'd like to try boot the main zfs root partition and then fall back to a 
> > small UFS based recovery partition.. is that possible?
> > 
> > I know we could use grub but I'd prefer keep it in the family.
> > 
> > 
> > 
> 
> 
> it is, sure. but there is an compromise to be made for it.
> 
> Lets start with what I have done in illumos port, as the idea there is 
> exactly about having as “universal” binaries as possible (just the binaries 
> are listed below to get the size):
> 
> -r-xr-xr-x   1 root sys   171008 apr 30 19:55 bootia32.efi
> -r-xr-xr-x   1 root sys   148992 apr 30 19:55 bootx64.efi
> -r--r--r--   1 root sys 1255 okt 25  2015 cdboot
> -r--r--r--   1 root sys   154112 apr 30 19:55 gptzfsboot
> -r-xr-xr-x   1 root sys   482293 mai  2 21:10 loader32.efi
> -r-xr-xr-x   1 root sys   499218 mai  2 21:10 loader64.efi
> -r--r--r--   1 root sys  512 okt 15  2015 pmbr
> -r--r--r--   1 root sys   377344 mai  2 21:10 pxeboot
> -r--r--r--   1 root sys   376832 mai  2 21:10 zfsloader
> 
> the loader (bios/efi) is built with full complement - zfs, ufs, dosfs, 
> cd9660, nfs, tftp + gzipfs. The cdboot is starting zfsloader (thats trivial 
> string change).
> 
> The gptzfsboot in illumos case is only built with zfs, dosfs and ufs - as it 
> has to support only disk based media to read out the loader. Also I am 
> building gptzfsboot with libstand and libi386 to get as much shared code as 
> possible - which has both good and bad sides, as usual;)
> 
> The gptzfsboot size means that with ufs the dedicated boot partition is 
> needed (freebsd-boot), with zfs the illumos port is always using the 3.5MB 
> boot area after first 2 labels (as there is no geli, the illumos does not 
> need dedicated boot partition with zfs).
> 
> As the freebsd-boot is currently created 512k, the size is not an issue. Also 
> using common code does allow the generic partition code to be used, so 
> GPT/MBR/BSD (VTOC in illumos case) labels are not problem.

The intention btw of the larger size for gptboot is so we could have a merged
gptboot / gptzfsboot.  I don't think ZFS was in FreeBSD when gptboot was first
written, but I would much rather have a merged gptboot binary that supports
both.  It just needs some logic for what to pick if it sees both.  (It would
also be nice to axe zfsloader and just pass a different KARGS_FLAG_FOO in to
select ZFS as the default boot device to /boot/loader, but zfsloader is probably
too baked into the system at this point.)

> Also note that we can still build the smaller dedicated blocks like boot2, 
> just that we can not use those blocks for more universal cases and eventually 
> those special cases will diminish.

Yes, the BSD label stuff is stuck with a smaller size, but GPT should support
unified bootstraps.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: filemon: weird full-time build although filemon enabled

[O. Hartmann]

Am Mon, 8 May 2017 10:17:05 -0700
"Simon J. Gerraty"  schrieb:

> Konstantin Belousov  wrote:
> > If I understand the motto of meta-mode, any file change is detected for any
> > file accessed during the build.  All dynamically-linked binary includes
> > the rtld into the process image, and rtld reads all config files in the
> > libmap.d subdirectories.  The end result is that everything must be rebuild
> > if any config file changed.  
> 
> Ah.
> 
> > Then, after the world build, according to OP, the nvidia driver port is
> > reinstalled, which installs the nvidia.conf anew, which triggers the
> > behaviour on the next build.  
> 
> Yes, that would account for it.
> 
> There are always exceptions - things we expect to be volatile and don't
> care - or for any other reason just want to ignore.
> 
> bmake has a set of knobs for telling it to ignore things.
> OP try
> 
> .MAKE.META.IGNORE_PATHS += /usr/local/etc/libmap.d
> 
> --sjg

I suppose I have to set this flag in 

/etc/src-env.conf

?

-- 
O. Hartmann

Ich widerspreche der Nutzung oder Übermittlung meiner Daten für
Werbezwecke oder für die Markt- oder Meinungsforschung (§ 28 Abs. 4 BDSG).


pgp9KiHwEgT3w.pgp
Description: OpenPGP digital signature

Re: filemon: weird full-time build although filemon enabled

[Konstantin Belousov]

On Mon, May 08, 2017 at 09:04:08AM -0700, Simon J. Gerraty wrote:
> O. Hartmann  wrote:
> > It is weird!
> > 
> > Today, after yesterday's built, I face the same 90 minutes build horror 
> > again, this time
> > I switched on "-dM" with the make command.
> > 
> > This happens:
> > 
> > [...]
> > /usr/obj/usr/src/lib/clang/libllvm/_usr_obj_usr_src_lib_clang_libllvm_CodeGen_SelectionDAG_LegalizeDAG.o.meta:
> > 15: file '/usr/local/etc/libmap.d/nvidia.conf' is newer than the target...
> > 
> 
> That does seem odd why anything involved in building clang should care
> about that file...
> 
> You can use the pid field in the syscall trace to show what process was
> looking at that file.
> 
> > This box has the following lines in /etc/src.conf to rebuild the nvidia 
> > kernel module
> > every time:
> > 
> > PORTS_MODULES+= x11/nvidia-driver
> > 
> > I do not know what is going on here ... :-(
> 
> well that might explain why nvidia.conf is updated, but not why clang
> build cares.

If I understand the motto of meta-mode, any file change is detected for any
file accessed during the build.  All dynamically-linked binary includes
the rtld into the process image, and rtld reads all config files in the
libmap.d subdirectories.  The end result is that everything must be rebuild
if any config file changed.

Then, after the world build, according to OP, the nvidia driver port is
reinstalled, which installs the nvidia.conf anew, which triggers the
behaviour on the next build.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: filemon: weird full-time build although filemon enabled

[O. Hartmann]

Am Mon, 8 May 2017 19:32:58 +0200
"O. Hartmann"  schrieb:

> Am Mon, 8 May 2017 10:17:05 -0700
> "Simon J. Gerraty"  schrieb:
> 
> > Konstantin Belousov  wrote:  
> > > If I understand the motto of meta-mode, any file change is detected for 
> > > any
> > > file accessed during the build.  All dynamically-linked binary includes
> > > the rtld into the process image, and rtld reads all config files in the
> > > libmap.d subdirectories.  The end result is that everything must be 
> > > rebuild
> > > if any config file changed.
> > 
> > Ah.
> >   
> > > Then, after the world build, according to OP, the nvidia driver port is
> > > reinstalled, which installs the nvidia.conf anew, which triggers the
> > > behaviour on the next build.
> > 
> > Yes, that would account for it.
> > 
> > There are always exceptions - things we expect to be volatile and don't
> > care - or for any other reason just want to ignore.
> > 
> > bmake has a set of knobs for telling it to ignore things.
> > OP try
> > 
> > .MAKE.META.IGNORE_PATHS += /usr/local/etc/libmap.d
> > 
> > --sjg  
> 
> I suppose I have to set this flag in 
> 
> /etc/src-env.conf
> 
> ?
> 

from empirical experience, it is neither of /etc/src.conf or /etc/src-env.conf 
nor does
setting via "env" do something ... :-(

-- 
O. Hartmann

Ich widerspreche der Nutzung oder Übermittlung meiner Daten für
Werbezwecke oder für die Markt- oder Meinungsforschung (§ 28 Abs. 4 BDSG).


pgpM6DXjd9kst.pgp
Description: OpenPGP digital signature

Re: filemon: weird full-time build although filemon enabled

[O. Hartmann]

Am Sun, 7 May 2017 23:24:55 -0700
"Simon J. Gerraty"  schrieb:

> Hi,
> 
> > I build CURRENT on two technically similar systems on a almost daily basis.
> > Therefore, it was a great relief having WITH_META_MODE=yes set in 
> > /etc/src-env.conf
> > for incremental builds. To make my understanding of this clear (just in 
> > case I'm
> > wrong): setting WITH_META_MODE builds only portions that does not need to 
> > be build in
> > the make context.  
> 
> I'm not sure I parsed that correctly, so let me just say;
> With MK_META_MODE=yes, bmake should be put into meta mode.
> 
> This simply means that most targets get a .meta file, that records the
> expanded command line, any command output and a syscall trace.
> These can be very useful for debugging.
> 
> More important; on an update build bmake can use them to make targets
> out-of-date that the normal rules miss.
> Any change to the expanded command line, makes the target out-of-date.
> Any change to any file read or executed makes the target out-of-date.
>  
> > Well, the reason writing this email is: on one system, I run almost every 
> > reboot into
> > a "full build" and this puzzles me a bit. The long-lasting and time 
> > exhausting builds
> > are  
> 
> You can add -dM to your build flags to have meta_oodate() - the function
> that does the above, explain why it decided the target is out-of-date.
> The results can be enlightening.
> 
> Note, if a target rebuilds without -dM saying why - then the normal make
> rules decided it was out-of-date.
> 
> > within the LLVM/CLANG tree. They consume a lot of time. The box in
> > question does have a  
> 
> Yes, building clang takes a while...
> 
> > Well, having said this, I need to mention that both systems use almost
> > identical /etc/src.conf setting - except the order of appearance of
> > the WITH_ tags.  
> 
> That should not matter, bsd.mkopt.mk processes them in a specified
> order.
> 
> > The problem: to make my point clear: the "weak" box starts compiling almost 
> > everytime
> > now the LLVM/CLANG tree while the XEON box does not. This is spooky.  
> 
> Add -dM, and see if that tells you why.
> Also confirm that filemon is getting loaded after each reboot.
> If that comes and goes, that could cause the behavior you see.
> 
> > I think there is something missing an I'd like to ask what is the suggested 
> > way to
> > initially restart a full build to ensure that WITH_META_MODE gets 
> > initialised
> > correctly.  
> 
> I think Bryan has set that up so that it should take care of itself
> pretty well - even toggling bewteen non-meta and meta builds should
> DTRT - which is why it matters if filemon was used that it is there for
> next build.
> 
> > Well, I'm not a developer, so please be patient with my naive report.  
> 
> No problem.  Adding -dM to your build (will add a fair bit of output),
> will hopefully tell you what's up.
> 
> From what you describe, I'd not be surprised if you see some
> "a build command has changed" and it only takes a few to trigger a
> cascade of targets out-of-date.
> 
> --sjg
> 


It is weird!

Today, after yesterday's built, I face the same 90 minutes build horror again, 
this time
I switched on "-dM" with the make command.

This happens:

[...]
/usr/obj/usr/src/lib/clang/libllvm/_usr_obj_usr_src_lib_clang_libllvm_CodeGen_SelectionDAG_LegalizeDAG.o.meta:
15: file '/usr/local/etc/libmap.d/nvidia.conf' is newer than the target...

This box has the following lines in /etc/src.conf to rebuild the nvidia kernel 
module
every time:

PORTS_MODULES+= x11/nvidia-driver

I do not know what is going on here ... :-(

-- 
O. Hartmann

Ich widerspreche der Nutzung oder Übermittlung meiner Daten für
Werbezwecke oder für die Markt- oder Meinungsforschung (§ 28 Abs. 4 BDSG).


pgpKi9FONw6or.pgp
Description: OpenPGP digital signature

Re: filemon: weird full-time build although filemon enabled

[Simon J. Gerraty]

O. Hartmann  wrote:
> It is weird!
> 
> Today, after yesterday's built, I face the same 90 minutes build horror 
> again, this time
> I switched on "-dM" with the make command.
> 
> This happens:
> 
> [...]
> /usr/obj/usr/src/lib/clang/libllvm/_usr_obj_usr_src_lib_clang_libllvm_CodeGen_SelectionDAG_LegalizeDAG.o.meta:
> 15: file '/usr/local/etc/libmap.d/nvidia.conf' is newer than the target...
> 

That does seem odd why anything involved in building clang should care
about that file...

You can use the pid field in the syscall trace to show what process was
looking at that file.

> This box has the following lines in /etc/src.conf to rebuild the nvidia 
> kernel module
> every time:
> 
> PORTS_MODULES+= x11/nvidia-driver
> 
> I do not know what is going on here ... :-(

well that might explain why nvidia.conf is updated, but not why clang
build cares.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: filemon: weird full-time build although filemon enabled

[Simon J. Gerraty]

Konstantin Belousov  wrote:
> If I understand the motto of meta-mode, any file change is detected for any
> file accessed during the build.  All dynamically-linked binary includes
> the rtld into the process image, and rtld reads all config files in the
> libmap.d subdirectories.  The end result is that everything must be rebuild
> if any config file changed.

Ah.

> Then, after the world build, according to OP, the nvidia driver port is
> reinstalled, which installs the nvidia.conf anew, which triggers the
> behaviour on the next build.

Yes, that would account for it.

There are always exceptions - things we expect to be volatile and don't
care - or for any other reason just want to ignore.

bmake has a set of knobs for telling it to ignore things.
OP try

.MAKE.META.IGNORE_PATHS += /usr/local/etc/libmap.d

--sjg
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: ${src}/release/release.sh fails at makefs state

[Alastair Hogge]

On Fri, 5 May 2017 11:26:44 PM Ngie Cooper wrote:
> (CCing emaste)
> 
> > On May 5, 2017, at 21:55, Alastair Hogge  wrote:
> …
> 
> > Calculated size of `memstick.img.part': 485474304 bytes, 9435 inodes
> > Extent size set to 8192
> > memstick.img.part: 463.0MB (948192 sectors) block size 8192, fragment size
> > 1024
> > 
> >using 9 cylinder groups of 54.38MB, 6960 blks, 1152 inodes.
> > 
> > super-block backups (for fsck -b #) at:
> > 32, 111392, 222752, 334112, 445472, 556832, 668192, 779552, 890912,
> > 
> > Populating `memstick.img.part'
> > makefs: bread: read 8192 (684294144) returned 0: No error: 0
> > makefs failed
> > *** Error code 1
> 
> Cheers,
> -Ngie

Thanks Ngie.

It look like r317744 is causing the problem:
https://svnweb.freebsd.org/base?view=revision=317744

That is the checkout that breaks release building for me, prior revisions 
cause no problem.

To good health,
Alastair
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: more default uid/gid for NFS in mountd

[Slawa Olhovchenkov]

On Mon, May 08, 2017 at 11:45:46AM +, Rick Macklem wrote:

> Hi,
> 
> Five years ago (yea, it slipped through a crack;-), Slawa reported that files
> created by root would end up owned by uid 2**32-2 (-2 as uint32_t).
> This happens if there is no "-maproot=" in the /etc/exports line.
> 
> The cause is obvious. The value is set to -2 by default.
> 
> The question is... Should this be changed to 65534 (ie "nobody")?
> - It would seem more consistent to make it the uid of nobody, but I can also 
> see
>   the argument that since it has been like this *forever*, that changing it 
> would be
>   a POLA violation.
> What do others think?

IMHO uid 2**32-2 is POLA violation.
Nobody expect this uid. Too much number. This is like bug.

> It is also the case that mountd.c doesn't look "nobody" up in the password 
> database
> to set the default. It would be nice to do this, but it could result in the 
> mountd daemon
> getting "stuck" during a boot waiting for an unresponsive LDAP service or 
> similar.
> Does doing this sound like a good idea?

This is (stuck at boot) already do for case of using NIS and nfsuserd.
I am regular see this for case of DNS failed at boot.
You offer don't impair current behaviour.

Thanks!

> Thanks for any comments, rick
> ps: Here's the original email thread, in case you are interested:
>   
> https://lists.freebsd.org/pipermail/freebsd-stable/2012-March/066868.html
> 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: more default uid/gid for NFS in mountd

[Eric van Gyzen]

On 05/08/2017 06:45, Rick Macklem wrote:
> Hi,
> 
> Five years ago (yea, it slipped through a crack;-), Slawa reported that files
> created by root would end up owned by uid 2**32-2 (-2 as uint32_t).
> This happens if there is no "-maproot=" in the /etc/exports line.
> 
> The cause is obvious. The value is set to -2 by default.
> 
> The question is... Should this be changed to 65534 (ie "nobody")?
> - It would seem more consistent to make it the uid of nobody, but I can also 
> see
>   the argument that since it has been like this *forever*, that changing it 
> would be
>   a POLA violation.
> What do others think?

Since the change is easily communicated in the release notes, I think it seems
quite reasonable, especially if you limit it to 12.0.

> It is also the case that mountd.c doesn't look "nobody" up in the password 
> database
> to set the default. It would be nice to do this, but it could result in the 
> mountd daemon
> getting "stuck" during a boot waiting for an unresponsive LDAP service or 
> similar.
> Does doing this sound like a good idea?

I imagine the lookup could be useful in heterogeneous networks.

You might consider adding a CLI flag to mountd to let the admin choose the user
by UID/GID, and possibly by username/groupname.  That would be a reasonable
workaround for networks that often hit the lookup problem.

Eric
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: bootcode capable of booting both UFS and ZFS?

[Warner Losh]

On Mon, May 8, 2017 at 12:10 PM, John Baldwin  wrote:
> On Friday, May 05, 2017 11:01:03 PM Toomas Soome wrote:
>>
>> > On 5. mai 2017, at 22:07, Julian Elischer  wrote:
>> >
>> > Subject says it all really, is this an option at this time?
>> >
>> > we'd like to try boot the main zfs root partition and then fall back to a 
>> > small UFS based recovery partition.. is that possible?
>> >
>> > I know we could use grub but I'd prefer keep it in the family.
>> >
>> >
>> >
>>
>>
>> it is, sure. but there is an compromise to be made for it.
>>
>> Lets start with what I have done in illumos port, as the idea there is 
>> exactly about having as “universal” binaries as possible (just the binaries 
>> are listed below to get the size):
>>
>> -r-xr-xr-x   1 root sys   171008 apr 30 19:55 bootia32.efi
>> -r-xr-xr-x   1 root sys   148992 apr 30 19:55 bootx64.efi
>> -r--r--r--   1 root sys 1255 okt 25  2015 cdboot
>> -r--r--r--   1 root sys   154112 apr 30 19:55 gptzfsboot
>> -r-xr-xr-x   1 root sys   482293 mai  2 21:10 loader32.efi
>> -r-xr-xr-x   1 root sys   499218 mai  2 21:10 loader64.efi
>> -r--r--r--   1 root sys  512 okt 15  2015 pmbr
>> -r--r--r--   1 root sys   377344 mai  2 21:10 pxeboot
>> -r--r--r--   1 root sys   376832 mai  2 21:10 zfsloader
>>
>> the loader (bios/efi) is built with full complement - zfs, ufs, dosfs, 
>> cd9660, nfs, tftp + gzipfs. The cdboot is starting zfsloader (thats trivial 
>> string change).
>>
>> The gptzfsboot in illumos case is only built with zfs, dosfs and ufs - as it 
>> has to support only disk based media to read out the loader. Also I am 
>> building gptzfsboot with libstand and libi386 to get as much shared code as 
>> possible - which has both good and bad sides, as usual;)
>>
>> The gptzfsboot size means that with ufs the dedicated boot partition is 
>> needed (freebsd-boot), with zfs the illumos port is always using the 3.5MB 
>> boot area after first 2 labels (as there is no geli, the illumos does not 
>> need dedicated boot partition with zfs).
>>
>> As the freebsd-boot is currently created 512k, the size is not an issue. 
>> Also using common code does allow the generic partition code to be used, so 
>> GPT/MBR/BSD (VTOC in illumos case) labels are not problem.
>
> The intention btw of the larger size for gptboot is so we could have a merged
> gptboot / gptzfsboot.  I don't think ZFS was in FreeBSD when gptboot was first
> written, but I would much rather have a merged gptboot binary that supports
> both.  It just needs some logic for what to pick if it sees both.  (It would
> also be nice to axe zfsloader and just pass a different KARGS_FLAG_FOO in to
> select ZFS as the default boot device to /boot/loader, but zfsloader is 
> probably
> too baked into the system at this point.)

I think this is a good idea, but we need to make sure that we can
build a smaller bootblocks that support only UFS so we can re-install
on older installations where we have a super-small boot partition.

Having said that, I'd love for there to be just one set of boot
blocks, assuming they fit into our ~540k boot block limit. Sadly, we
can't just create a 1MB or 10MB partition due to limitations in our
current MBR boot code, though I suppose we could fix that by just
loading the first part and make that cope with the rest. Not trivial,
but not impossible Likely something we won't need to do for some
time though.

>> Also note that we can still build the smaller dedicated blocks like boot2, 
>> just that we can not use those blocks for more universal cases and 
>> eventually those special cases will diminish.
>
> Yes, the BSD label stuff is stuck with a smaller size, but GPT should support
> unified bootstraps.

Agreed. We should also consider tossing a script into tools somewhere
to create a new slice that's bigger from the end of swap to allow
easier migrations...

Warner
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: filemon: weird full-time build although filemon enabled

[Simon J. Gerraty]

O. Hartmann  wrote:
> > .MAKE.META.IGNORE_PATHS += /usr/local/etc/libmap.d
> > 
> > --sjg
> 
> I suppose I have to set this flag in 
> 
> /etc/src-env.conf

That should work.
Let us know how it goes
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: ${src}/release/release.sh fails at makefs state

[Alastair Hogge]

On Mon, 8 May 2017 11:47:23 AM Ngie Cooper wrote:

[...]

> Hi Alastair,
>   r317967 should allow you to build release images again.

Thank you very muchly,
Alastair
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"