Re: -r325627 of head: mergemaster: Creating objdir after objdir after . . .

2017-11-16 Thread Bryan Drewery 
On 11/16/2017 1:15 PM, Mark Millard wrote:
> 
> ===
> Mark Millard
> mar...@dsl-only.net
> 
> On 2017-Nov-16, at 9:13 AM, Bryan Drewery  wrote:
> 
>>> . . .
>>
>> Can you test this patch please in context of this problem please?
>> It resolves read-only objdirs and should avoid more of the objdir
>> creations at mergemaster/installworld time.  It probably will still
>> create the etc/sendmail one.
>>
>> https://people.freebsd.org/~bdrewery/patches/top-level-objdirs.diff
>>
> . . .
> 
> I applied the patch.
> 
> The 3 experiments below only showed the sendmail related
> "Creating objdir" lines, no other "Creating objdir" lines.
> 
[...]
> 
> (So: None found in the typescript file.)
> 

Thanks!  I will commit it in a while.


-- 
Regards,
Bryan Drewery



signature.asc
Description: OpenPGP digital signature

Re: -r325627 of head: mergemaster: Creating objdir after objdir after . . .

2017-11-16 Thread Mark Millard 

===
Mark Millard
mar...@dsl-only.net

On 2017-Nov-16, at 9:13 AM, Bryan Drewery  wrote:

>> . . .
> 
> Can you test this patch please in context of this problem please?
> It resolves read-only objdirs and should avoid more of the objdir
> creations at mergemaster/installworld time.  It probably will still
> create the etc/sendmail one.
> 
> https://people.freebsd.org/~bdrewery/patches/top-level-objdirs.diff
> 
 . . .

I applied the patch.

The 3 experiments below only showed the sendmail related
"Creating objdir" lines, no other "Creating objdir" lines.



Based on using ( /tmp/usr/obj/armv7_clang not existing at the time):

# more ~/sys_build_scripts.armv7-host/mergemaster_armv7-armv7-host.sh 
kldload -n filemon && \
script 
~/sys_typescripts/typescript_mergemaster_armv7_clang_bootstrap_clang-armv7-host-$(date
 +%Y-%m-%d:%H:%M:%S) \
env __MAKE_CONF="/root/src.configs/make.conf" SRCCONF="/dev/null" 
SRC_ENV_CONF="/root/src.configs/src.conf.armv7-clang-bootstrap.armv7-host" \
MAKEOBJDIRPREFIX="/tmp/usr/obj/armv7_clang/arm.armv7" \
mergemaster -A armv7 $*

The merge master produced just the 3 objdir lines related to sendmail:

# ~/sys_build_scripts.armv7-host/mergemaster_armv7-armv7-host.sh -FUPi 
Script started, output file is 
/root/sys_typescripts/typescript_mergemaster_armv7_clang_bootstrap_clang-armv7-host-2017-11-16:12:31:48

*** Creating the temporary root environment in /var/tmp/temproot
 *** /var/tmp/temproot ready for use
 *** Creating and populating directory structure in /var/tmp/temproot

[Creating objdir /tmp/usr/obj/armv7_clang/arm.armv7/usr/src/arm.armv7...]
[Creating objdir /tmp/usr/obj/armv7_clang/arm.armv7/usr/src/arm.armv7/etc...]
[Creating objdir 
/tmp/usr/obj/armv7_clang/arm.armv7/usr/src/arm.armv7/etc/sendmail...]

. . . (other stuff) . . .



Based on ( /usr/obj/bpim3_clang not existing at the time [renamed]):

# more 
~/sys_build_scripts.armv7-host/make_bpim3_nodebug_clang_bootstrap-armv7-host.sh 
kldload -n filemon && \
script 
~/sys_typescripts/typescript_make_bpim3_nodebug_clang_bootstrap-armv7-host-$(date
 +%Y-%m-%d:%H:%M:%S) \
env __MAKE_CONF="/root/src.configs/make.conf" SRCCONF="/dev/null" 
SRC_ENV_CONF="/root/src.configs/src.conf.armv7-clang-bootstrap.armv7-host" \
WITH_META_MODE=yes \
WORLD_FLAGS="${WORLD_FLAGS} UBLDR_LOADADDR=0x4200" \
MAKEOBJDIRPREFIX="/usr/obj/bpim3_clang/arm.armv7" \
make $*

the check-old did not produce any objdir lines:

# 
~/sys_build_scripts.armv7-host/make_bpim3_nodebug_clang_bootstrap-armv7-host.sh 
check-old
Script started, output file is 
/root/sys_typescripts/typescript_make_bpim3_nodebug_clang_bootstrap-armv7-host-2017-11-16:12:44:43
>>> Checking for old files
>>> Checking for old libraries
>>> Checking for old directories
To remove old files and directories run 'make delete-old'.
To remove old libraries run 'make delete-old-libs'.

Script done, output file is 
/root/sys_typescripts/typescript_make_bpim3_nodebug_clang_bootstrap-armv7-host-2017-11-16:12:44:43


As for installworld installkernel creating a new directory tree. . .
(same .sh file for check-old above but with /usr/obj/bpim3_clang back in place)

# 
~/sys_build_scripts.armv7-host/make_bpim3_nodebug_clang_bootstrap-armv7-host.sh 
-j4 installworld installkernel 
DESTDIR=/usr/obj/DESTDIRs/clang-armv7-installworld-objdir-test
Script started, output file is 
/root/sys_typescripts/typescript_make_bpim3_nodebug_clang_bootstrap-armv7-host-2017-11-16:12:54:04
--- installworld ---
make[1]: 
"/usr/obj/bpim3_clang/arm.armv7/usr/src/arm.armv7/compiler-metadata.mk" line 1: 
Using cached compiler metadata from build at bpim3 on Tue Nov 14 22:25:57 PST 
2017
--- __installcheck_UGID ---
--- installworld ---
. . .
--- _kmodinstall ---
install -T release -o root -g wheel -m 555   g_mouse.ko 
/usr/obj/DESTDIRs/clang-armv7-installworld-objdir-test/boot/kernel/
install -T debug -o root -g wheel -m 555   g_mouse.ko.debug 
/usr/obj/DESTDIRs/clang-armv7-installworld-objdir-test/usr/lib/debug/boot/kernel/
--- afterinstall ---
kldxref /usr/obj/DESTDIRs/clang-armv7-installworld-objdir-test/boot/kernel

Script done, output file is 
/root/sys_typescripts/typescript_make_bpim3_nodebug_clang_bootstrap-armv7-host-2017-11-16:12:54:04

# grep "Creating objdir" 
/root/sys_typescripts/typescript_make_bpim3_nodebug_clang_bootstrap-armv7-host-2017-11-16:12:54:04
 | more
#

(So: None found in the typescript file.)

===
Mark Millard
markmi at dsl-only.net




___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"

Re: -r325627 of head: mergemaster: Creating objdir after objdir after . . .

2017-11-16 Thread Bryan Drewery 
On 11/11/2017 2:25 PM, Mark Millard wrote:
> 
> On 2017-Nov-11, at 8:47 AM, Bryan Drewery  wrote:
> 
>>> On Nov 11, 2017, at 00:51, Mark Millard  wrote:
>>>
 On 2017-Nov-10, at 5:16 PM, Bryan Drewery  wrote:

> On 11/10/2017 8:30 AM, Bryan Drewery wrote:
> . . . 
> In fact it's similar to my META_MODE whitelist in the top-level
> Makefile.  There's quite a few targets we don't care for AUTO_OBJ on,
> like distribute*, installworld, installkernel, etc.

 r325697 should fix it.
>>>
>>> Most of the messages are gone in -r325700 . But there was:
>>>
>>> *** Creating the temporary root environment in /var/tmp/temproot
>>> *** /var/tmp/temproot ready for use
>>> *** Creating and populating directory structure in /var/tmp/temproot
>>>
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/etc...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/etc/sendmail...]
>>>
>>> (No more objdir lines after that.)
>>
>> Yea this is expected. Mergemaster runs ‘make obj’ in etc/.
> 
> Hmm. I looking I see the:
> 
>   ${MM_MAKE} _obj SUBDIR_OVERRIDE=etc >/dev/null &&
>   ${MM_MAKE} everything SUBDIR_OVERRIDE=etc >/dev/null &&
> 
> Its too bad that the mergemaster man page makes no reference
> to depending on MAKEOBJDIRPREFIX (or its default) and the
> tree contents that it points to. If one has more than one
> tree around then one should be picking the right one --but
> nothing in the man page suggests that.
> 
> It also would not play well with not having that build tree
> available at the time of a mergemaster.

Can you test this patch please in context of this problem please?
It resolves read-only objdirs and should avoid more of the objdir
creations at mergemaster/installworld time.  It probably will still
create the etc/sendmail one.

https://people.freebsd.org/~bdrewery/patches/top-level-objdirs.diff

> 
> The 20130425 UPDATING entry does note use of
> MAKEOBJDIRPREFIX. It notes that mergemaster makes use of the
> specific, bootstrapped mtree and install. Does that mean
> that a cross-build needs mergemaster to be executed on
> the cross-build host instead of on the target system? I do
> not see the man page as well-covering such questions for
> the proper usage technique.
> 
> Dependency on /usr/src (by default or an alternate via -m) is
> clear from the man page. So, having a proper vintage of such
> and having mergemaster use it was clear. But such seems to not
> be sufficient, which was not clear.
> 
>> The top-level check-old objdir creation is unavoidable right now... you can 
>> use -DNO_OBJ if you want to avoid it.
>>
>>>
>
>> However from reading mergemaster.sh it seems that _at least_
>> /usr/obj/usr/src/etc/sendmail would be created before my changes.  Can
>> someone confirm that on stable/ or something?
>>
>>>
>>> (MAKEOBJDIRPREFIX= does control the path-prefix used
>>> if specified in the env list before mergemaster.)
>>>
>>>
> 
> 
> ===
> Mark Millard
> markmi at dsl-only.net
> 


-- 
Regards,
Bryan Drewery



signature.asc
Description: OpenPGP digital signature

Re: -r325627 of head: mergemaster: Creating objdir after objdir after . . .

2017-11-11 Thread Mark Millard 

On 2017-Nov-11, at 8:47 AM, Bryan Drewery  wrote:

>> On Nov 11, 2017, at 00:51, Mark Millard  wrote:
>> 
>>> On 2017-Nov-10, at 5:16 PM, Bryan Drewery  wrote:
>>> 
 On 11/10/2017 8:30 AM, Bryan Drewery wrote:
 . . . 
 In fact it's similar to my META_MODE whitelist in the top-level
 Makefile.  There's quite a few targets we don't care for AUTO_OBJ on,
 like distribute*, installworld, installkernel, etc.
>>> 
>>> r325697 should fix it.
>> 
>> Most of the messages are gone in -r325700 . But there was:
>> 
>> *** Creating the temporary root environment in /var/tmp/temproot
>> *** /var/tmp/temproot ready for use
>> *** Creating and populating directory structure in /var/tmp/temproot
>> 
>> [Creating objdir /usr/obj/usr/src/arm64.aarch64...]
>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/etc...]
>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/etc/sendmail...]
>> 
>> (No more objdir lines after that.)
> 
> Yea this is expected. Mergemaster runs ‘make obj’ in etc/.

Hmm. I looking I see the:

  ${MM_MAKE} _obj SUBDIR_OVERRIDE=etc >/dev/null &&
  ${MM_MAKE} everything SUBDIR_OVERRIDE=etc >/dev/null &&

Its too bad that the mergemaster man page makes no reference
to depending on MAKEOBJDIRPREFIX (or its default) and the
tree contents that it points to. If one has more than one
tree around then one should be picking the right one --but
nothing in the man page suggests that.

It also would not play well with not having that build tree
available at the time of a mergemaster.

The 20130425 UPDATING entry does note use of
MAKEOBJDIRPREFIX. It notes that mergemaster makes use of the
specific, bootstrapped mtree and install. Does that mean
that a cross-build needs mergemaster to be executed on
the cross-build host instead of on the target system? I do
not see the man page as well-covering such questions for
the proper usage technique.

Dependency on /usr/src (by default or an alternate via -m) is
clear from the man page. So, having a proper vintage of such
and having mergemaster use it was clear. But such seems to not
be sufficient, which was not clear.

> The top-level check-old objdir creation is unavoidable right now... you can 
> use -DNO_OBJ if you want to avoid it.
> 
>> 
 
> However from reading mergemaster.sh it seems that _at least_
> /usr/obj/usr/src/etc/sendmail would be created before my changes.  Can
> someone confirm that on stable/ or something?
> 
>> 
>> (MAKEOBJDIRPREFIX= does control the path-prefix used
>> if specified in the env list before mergemaster.)
>> 
>> 


===
Mark Millard
markmi at dsl-only.net

___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"

Re: -r325627 of head: mergemaster: Creating objdir after objdir after . . .

2017-11-11 Thread Mark Millard 

===
Mark Millard
mar...@dsl-only.net

On 2017-Nov-11, at 12:51 AM, Mark Millard  wrote:

> On 2017-Nov-10, at 5:16 PM, Bryan Drewery  wrote:
> 
>> On 11/10/2017 8:30 AM, Bryan Drewery wrote:
>>> On 11/10/17 7:52 AM, Bryan Drewery wrote:
 On 11/10/2017 12:46 AM, Mark Millard wrote:
> When I use the command:
> 
> ~/sys_build_scripts.aarch64-host/mergemaster_cortexA53-aarch64-host.sh 
> -FUPi -D/mnt
> 
> based on:
> 
> # more 
> ~/sys_build_scripts.aarch64-host/mergemaster_cortexA53-aarch64-host.sh
> kldload -n filemon && \
> script 
> ~/sys_typescripts/typescript_mergemaster_cortexA53_clang_bootstrap_clang-aarch64-host-$(date
>  +%Y-%m-%d:%H:%M:%S) \
> env __MAKE_CONF="/root/src.configs/make.conf" SRCCONF="/dev/null" 
> SRC_ENV_CONF="/root/src.configs/src.conf.cortexA53-clang-bootstrap.aarch64-host"
>  \
> mergemaster -A aarch64 $*
> 
> in a context where /usr/obj/usr does not exist
> (no local build tree present at the time), I get:
> 
> Script started, output file is 
> /root/sys_typescripts/typescript_mergemaster_cortexA53_clang_bootstrap_clang-aarch64-host-2017-11-09:23:57:04
> 
> *** Creating the temporary root environment in /var/tmp/temproot
> *** /var/tmp/temproot ready for use
> *** Creating and populating directory structure in /var/tmp/temproot
> 
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/share/termcap...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/etc/syslog.d...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/usr.sbin/rmt...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/etc/pam.d...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/csu...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/csu/aarch64...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libc...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libc_nonshared...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libcompiler_rt...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libclang_rt...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libc++...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libc++experimental...]
> [Creating nested objdir 
> /usr/obj/usr/src/arm64.aarch64/lib/libc++experimental/filesystem...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libcxxrt...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libelf...]
> [Creating nested objdir /usr/obj/usr/src/arm64.aarch64/lib/libelf/sys...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/msun...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libalias...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libalias/libalias...]
> . . . (long list) . . .
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/usr.sbin/wpa/hostapd...]
> [Creating objdir 
> /usr/obj/usr/src/arm64.aarch64/usr.sbin/wpa/hostapd_cli...]
> [Creating objdir 
> /usr/obj/usr/src/arm64.aarch64/usr.sbin/wpa/ndis_events...]
> 
> 
> 
> So a /usr/obj/usr/src/arm64.aarch64/ directory tree
> ends up being created.
 
 Hah, not what we want. I'll fix that.
 
>>> 
>>> In fact it's similar to my META_MODE whitelist in the top-level
>>> Makefile.  There's quite a few targets we don't care for AUTO_OBJ on,
>>> like distribute*, installworld, installkernel, etc.
>> 
>> r325697 should fix it.
> 
> Most of the messages are gone in -r325700 . But there was:
> 
> *** Creating the temporary root environment in /var/tmp/temproot
> *** /var/tmp/temproot ready for use
> *** Creating and populating directory structure in /var/tmp/temproot
> 
> [Creating objdir /usr/obj/usr/src/arm64.aarch64...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/etc...]
> [Creating objdir /usr/obj/usr/src/arm64.aarch64/etc/sendmail...]
> 
> (No more objdir lines after that.)
> 
>>> 
 However from reading mergemaster.sh it seems that _at least_
 /usr/obj/usr/src/etc/sendmail would be created before my changes.  Can
 someone confirm that on stable/ or something?
 
> 
> (MAKEOBJDIRPREFIX= does control the path-prefix used
> if specified in the env list before mergemaster.)
> 

FYI a check-old (with MAKEOBJDIRPREFIX="/usr/obj/cortexA53_clang/arm64.aarch64"
in use) got me a:

[Creating objdir 
/usr/obj/cortexA53_clang/arm64.aarch64/usr/src/arm64.aarch64...]

(Just the one objdir line.)

/usr/obj/cortexA53_clang/ did not exist when check-old
was started.

So, the activity does track MAKEOBJDIRPREFIX when it
is in use.

===
Mark Millard
markmi at dsl-only.net

___
freebsd-toolchain@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"

Re: -r325627 of head: mergemaster: Creating objdir after objdir after . . .

2017-11-10 Thread Bryan Drewery 
On 11/10/2017 8:30 AM, Bryan Drewery wrote:
> On 11/10/17 7:52 AM, Bryan Drewery wrote:
>> On 11/10/2017 12:46 AM, Mark Millard wrote:
>>> When I use the command:
>>>
>>> ~/sys_build_scripts.aarch64-host/mergemaster_cortexA53-aarch64-host.sh 
>>> -FUPi -D/mnt
>>>
>>> based on:
>>>
>>> # more 
>>> ~/sys_build_scripts.aarch64-host/mergemaster_cortexA53-aarch64-host.sh
>>> kldload -n filemon && \
>>> script 
>>> ~/sys_typescripts/typescript_mergemaster_cortexA53_clang_bootstrap_clang-aarch64-host-$(date
>>>  +%Y-%m-%d:%H:%M:%S) \
>>> env __MAKE_CONF="/root/src.configs/make.conf" SRCCONF="/dev/null" 
>>> SRC_ENV_CONF="/root/src.configs/src.conf.cortexA53-clang-bootstrap.aarch64-host"
>>>  \
>>> mergemaster -A aarch64 $*
>>>
>>> in a context where /usr/obj/usr does not exist
>>> (no local build tree present at the time), I get:
>>>
>>> Script started, output file is 
>>> /root/sys_typescripts/typescript_mergemaster_cortexA53_clang_bootstrap_clang-aarch64-host-2017-11-09:23:57:04
>>>
>>> *** Creating the temporary root environment in /var/tmp/temproot
>>>  *** /var/tmp/temproot ready for use
>>>  *** Creating and populating directory structure in /var/tmp/temproot
>>>
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/share/termcap...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/etc/syslog.d...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/usr.sbin/rmt...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/etc/pam.d...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/csu...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/csu/aarch64...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libc...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libc_nonshared...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libcompiler_rt...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libclang_rt...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libc++...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libc++experimental...]
>>> [Creating nested objdir 
>>> /usr/obj/usr/src/arm64.aarch64/lib/libc++experimental/filesystem...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libcxxrt...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libelf...]
>>> [Creating nested objdir /usr/obj/usr/src/arm64.aarch64/lib/libelf/sys...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/msun...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libalias...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/lib/libalias/libalias...]
>>> . . . (long list) . . .
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/usr.sbin/wpa/hostapd...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/usr.sbin/wpa/hostapd_cli...]
>>> [Creating objdir /usr/obj/usr/src/arm64.aarch64/usr.sbin/wpa/ndis_events...]
>>>
>>>
>>>
>>> So a /usr/obj/usr/src/arm64.aarch64/ directory tree
>>> ends up being created.
>>
>> Hah, not what we want. I'll fix that.
>>
> 
> In fact it's similar to my META_MODE whitelist in the top-level
> Makefile.  There's quite a few targets we don't care for AUTO_OBJ on,
> like distribute*, installworld, installkernel, etc.

r325697 should fix it.

> 
>> However from reading mergemaster.sh it seems that _at least_
>> /usr/obj/usr/src/etc/sendmail would be created before my changes.  Can
>> someone confirm that on stable/ or something?
>>
>>>
>>> (MAKEOBJDIRPREFIX= does control the path-prefix used
>>> if specified in the env list before mergemaster.)
>>>
>>> ===
>>> Mark Millard
>>> markmi at dsl-only.net
>>>
>>
>>
> 
> 


-- 
Regards,
Bryan Drewery



signature.asc
Description: OpenPGP digital signature