Re: svn commit: r368820 - head

[Mark Millard via svn-src-head]

On 2020-Dec-25, at 22:21, Mark Millard  wrote:

>> On 2020-Dec-24, at 21:17, Mark Millard  wrote:
>> 
>> Hartmann, O. o.hartmann at walstatt.org wrote on
>> Thu Dec 24 21:34:56 UTC 2020 :
> . . .

Looks like I was over specific about where I did "git fetch"
from and so made some sequences overcomplicated. So I adjust
that this time.

> I've done more exploring and so am more willing to be
> explicit about commands now that I've tried some of this.
> 
>>> I can not find (easily) any hints
>>> for those who were familiar with subversion and checking out /usr/src 
>>> either for
>>> 12-STABLE, 12.1-RELENG, 12.2-RELENG, CURRENT.
>> 
> 
> Presuming having each available at the same time in separate directory trees,
> but only one repository, and sticking to the HEAD commit for each at the time
> the local repository is updated from the remote one. Also presuming no locally
> updated sources so there is nothing to clean up or put to the side . . .
> 
> https://github.com/bsdimp/freebsd-git-docs/blob/main/faq.md has as its first 
> question:
> 
> "How do I track -current and -stable with only one copy of the repo?" So based
> on that text . . .
> 
> The notations for the git branches for what you list are (in order):
> 
> stable/12
> releng/12.1
> releng/12.2
> main
> 
> The later material will lead to there being 4 FreeBSD source trees (with my
> arbitrary example paths that you might not want to use):
> 
> /usr/fbsd/freebsd-current/
> /usr/fbsd/freebsd-stable-12/
> /usr/fbsd/freebsd-releng-12.1/
> /usr/fbsd/freebsd-releng-12.2/
> 
> When following the general structure that uses worktrees that is documented, 
> you
> can have at most one worktree for a branch. (Adding branches that you maintain
> to be related to those allows for more.) I'll indicate one worktree for each 
> but
> main. main gets no worktree: it already has a sufficient context.
> 
> The initial setup (I picked an example URL):
> 
> # mkdir -p /usr/fbsd
> # cd /usr/fbsd
> # git clone -o freebsd --config 
> remote.freebsd.fetch='+refs/notes/*:refs/notes/*' \
> ssh://anon...@git.freebsd.org/src.git freebsd-current
> # cd freebsd-current
> # git checkout main
> # git worktree add ../freebsd-stable-12 stable/12
> # git worktree add ../freebsd-releng-12.1 releng/12.1
> # git worktree add ../freebsd-releng-12.2 releng/12.2
> 
> Then, as an example of updating freebsd-current (the overall sequence follows
> the FAQ but I'll note a variation later):
> 
> # cd /usr/fbsd/freebsd-current
> # git checkout main
> # git pull --ff-only
> 
> Note: For the below, the above needs to have been
> done first: the pull involves the fetch of the
> remote material, including for the use in the
> below.
> 
> # cd ../freebsd-stable-12
> # git merge --ff_only freebsd/stable/12
> # cd ../freebsd-releng-12.1
> # git merge --ff_only freebsd/releng/12.1
> # cd ../freebsd-releng-12.2
> # git merge --ff_only freebsd/stable/12
> 
> I'll note that elsewhere it is recommended to do (once
> for each login using git for FreeBSD source activity):
> 
> # git config --global pull.ff only
> 
> because those --ff-only uses are important to keeping
> history as FreeBSD intends it (linear) and --ff-only
> can be a default. (This will not cover the below
> variation.)
> 
> 
> The variation that I mentioned follows . . .
> 
> You might not like needing to update freebsd-current in
> order to update, say, freebsd-stable-12 above. Avoiding
> the pull and using the analogous two commands in the
> proper order gives the following that only update the
> individual part of the fetch that was of interest.
> 
> I show binding to branches to directories explicitly
> (checkout) but such would not be needed unless the
> binding for the directory tree had been changed. Note
> that the below is not using pull and so the config
> above does not cause --ff-only defaults to be involved:
> Be reliably explicit.
> 
> 
> # cd /usr/fbsd/freebsd-current
> # git fetch freebsd
> # git checkout main
> # git merge --ff-only freebsd/main
> 
> vs.
> 

Improved:

# cd /usr/fbsd/freebsd-stable-12
# git fetch freebsd
# git checkout stable/12
# git merge --ff-only freebsd/stable/12

> vs.
> 

Improved:

# cd /usr/fbsd/freebsd-releng-12.1
# git fetch freebsd
# git checkout releng/12.1
# git merge --ff-only freebsd/releng/12.1

> vs.
> 

Improved:

# cd /usr/fbsd/freebsd-releng-12.2
# git fetch freebsd
# git checkout releng/12.2
# git merge --ff-only freebsd/releng/12.2

> 
> Note: After a fetch, one or more of the
> cd-checkout-merge sorts of sequences could
> be done without re-fetching.
> 
> Part of what a "merge --ff-only" does is to
> move what the active branch refers to, in
> the cases above, to a (potentially) new place
> in newly fetched material.
> 
> I hope that the above helps. It does not deal with
> picking out a specific commit out of the repository
> for a specific source directory tree. I do not know
> if you do such and it makes controlling the context
> more complicated to describe.
> 
> Not tied to 

Re: svn commit: r368820 - head

[Mark Millard via svn-src-head]

> On 2020-Dec-24, at 21:17, Mark Millard  wrote:
> 
> Hartmann, O. o.hartmann at walstatt.org wrote on
> Thu Dec 24 21:34:56 UTC 2020 :
. . .

I've done more exploring and so am more willing to be
explicit about commands now that I've tried some of this.

>> I can not find (easily) any hints
>> for those who were familiar with subversion and checking out /usr/src either 
>> for
>> 12-STABLE, 12.1-RELENG, 12.2-RELENG, CURRENT.
> 

Presuming having each available at the same time in separate directory trees,
but only one repository, and sticking to the HEAD commit for each at the time
the local repository is updated from the remote one. Also presuming no locally
updated sources so there is nothing to clean up or put to the side . . .

https://github.com/bsdimp/freebsd-git-docs/blob/main/faq.md has as its first 
question:

"How do I track -current and -stable with only one copy of the repo?" So based
on that text . . .

The notations for the git branches for what you list are (in order):

stable/12
releng/12.1
releng/12.2
main

The later material will lead to there being 4 FreeBSD source trees (with my
arbitrary example paths that you might not want to use):

/usr/fbsd/freebsd-current/
/usr/fbsd/freebsd-stable-12/
/usr/fbsd/freebsd-releng-12.1/
/usr/fbsd/freebsd-releng-12.2/

When following the general structure that uses worktrees that is documented, you
can have at most one worktree for a branch. (Adding branches that you maintain
to be related to those allows for more.) I'll indicate one worktree for each but
main. main gets no worktree: it already has a sufficient context.

The initial setup (I picked an example URL):

# mkdir -p /usr/fbsd
# cd /usr/fbsd
# git clone -o freebsd --config 
remote.freebsd.fetch='+refs/notes/*:refs/notes/*' \
ssh://anon...@git.freebsd.org/src.git freebsd-current
# cd freebsd-current
# git checkout main
# git worktree add ../freebsd-stable-12 stable/12
# git worktree add ../freebsd-releng-12.1 releng/12.1
# git worktree add ../freebsd-releng-12.2 releng/12.2

Then, as an example of updating freebsd-current (the overall sequence follows
the FAQ but I'll note a variation later):

# cd /usr/fbsd/freebsd-current
# git checkout main
# git pull --ff-only

Note: For the below, the above needs to have been
done first: the pull involves the fetch of the
remote material, including for the use in the
below.

# cd ../freebsd-stable-12
# git merge --ff_only freebsd/stable/12
# cd ../freebsd-releng-12.1
# git merge --ff_only freebsd/releng/12.1
# cd ../freebsd-releng-12.2
# git merge --ff_only freebsd/stable/12

I'll note that elsewhere it is recommended to do (once
for each login using git for FreeBSD source activity):

# git config --global pull.ff only

because those --ff-only uses are important to keeping
history as FreeBSD intends it (linear) and --ff-only
can be a default. (This will not cover the below
variation.)


The variation that I mentioned follows . . .

You might not like needing to update freebsd-current in
order to update, say, freebsd-stable-12 above. Avoiding
the pull and using the analogous two commands in the
proper order gives the following that only update the
individual part of the fetch that was of interest.

I show binding to branches to directories explicitly
(checkout) but such would not be needed unless the
binding for the directory tree had been changed. Note
that the below is not using pull and so the config
above does not cause --ff-only defaults to be involved:
Be reliably explicit.


# cd /usr/fbsd/freebsd-current
# git fetch freebsd
# git checkout main
# git merge --ff-only freebsd/main

vs.

# cd /usr/fbsd/freebsd-current
# git fetch freebsd
# cd /usr/fbsd/freebsd-stable-12
# git checkout stable/12
# git merge --ff-only freebsd/stable/12

vs.

# cd /usr/fbsd/freebsd-current
# git fetch freebsd
# cd /usr/fbsd/freebsd-releng-12.1
# git checkout releng/12.1
# git merge --ff-only freebsd/releng/12.1

vs.

# cd /usr/fbsd/freebsd-current
# git fetch freebsd
# cd /usr/fbsd/freebsd-releng-12.2
# git checkout releng/12.2
# git merge --ff-only freebsd/releng/12.2


Note: After a fetch, one or more of the
cd-checkout-merge sorts of sequences could
be done without re-fetching.

Part of what a "merge --ff-only" does is to
move what the active branch refers to, in
the cases above, to a (potentially) new place
in newly fetched material.

I hope that the above helps. It does not deal with
picking out a specific commit out of the repository
for a specific source directory tree. I do not know
if you do such and it makes controlling the context
more complicated to describe.

Not tied to that, you may want to look at:

https://github.com/bsdimp/freebsd-git-docs/blob/main/SUMMARY.md

that organizes the existing material and may make
it easier to pick out things to read.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list

Re: svn commit: r368820 - head

[Mark Millard via svn-src-head]

Hartmann, O. o.hartmann at walstatt.org wrote on
Thu Dec 24 21:34:56 UTC 2020 :

> According to the status of the wiki refered to by 
> https://wiki.freebsd.org/git,
>  most of
> the transition from svn to git has been done.
> According to the recent logs reported at 
> https://cgit.freebsd.org/src/,
>  the recent
> commits for "main" (I assume this is CURRENT?) are done 35 minutes ago, commit
> 322a188d45f17457f0de00a78c77228751dff03c.

"main" in git is is for what was CURRENT. git has its own use of
the text "HEAD" and so references to head require identification
of the context and possibly are best avoided. (I used to prefer
head to current, given head's use in svn.)

https://wiki.freebsd.org/git still lists "Finish git->svn exporter
for stable/11, stable/12, and supported releng/ branches" as
"In Q/A". The updates in svn should start again at some point.

12 and 11 via svn will still identify versions using text like
-r368999 and official builds will be based on such, if I
understand right. Official 13+ builds will not use such. If
one needs to match an official 12 or 11 build, using svn will
likely be easier to do that with.

If one can wait for the git -> svn delay before picking up
src changes in the ongoing future, continuing to use svn for
now may be easier.

It may (eventually) be better to use what official builds use.
But that still could mean svn for 12 and 11 until they are no
longer supported and only changing to git for 13+ (at some
point).


> The last message on this list is still this one I'm replying to.

FYI: there are (showing December threads style of access):

https://lists.freebsd.org/pipermail/freebsd-git/2020-December/thread.html

https://lists.freebsd.org/pipermail/dev-commits-src-main/2020-December/thread.html

They are probably better targets for your git related questions.

> I'm new to git, I usually do not switch between several code revisions 
> systems, I assume
> a lot of FreeBSD users are familiar with subversion and share my problems of 
> keeping
> pace. 
> 
> From the recent state of the list I assume its death.

svn is not dead for various versions of FreeBSD. It will not be
used for official 13+ builds, much like snapshots for 13 have not
been using svn (or its -r identifications) for some time
(despite svn for CURRENT being available for much of that time).

> I can not find (easily) any hints
> for those who were familiar with subversion and checking out /usr/src either 
> for
> 12-STABLE, 12.1-RELENG, 12.2-RELENG, CURRENT.

If one is going to use git for 12 and/or 11, one clones the same
repository as for access to 13.

Context matters for checkouts. I tend to identify a specific commit
that I want to use, not just the most recent on a branch. (historically
true for svn -rDD usage as well). But one does not clone such a
specific git repository version. One gets a clone or updates a
repository and then causes git to extract the desire version from it
into the file system.

This is messier than just taking the most recent commit for whatever
branch of interest from whatever repository one uses (that is at least
recent enough to contain what is of interest). But i've no clue if
this is relevant to your usage patterns or not.

So you may have to describe you intended usage criteria for picking
what to check out. Otherwise the instructions may be misleading.

It is also important to know if you need source trees for multiple
branches at the same time vs. only one at a time. One at a time
leads to only needing one copy of the repository but multiple at
a time could be either with just one repository (multiple
worktree's) or one repository each. There are disk space tradeoffs
involved in such choices. So: more for you to provide some
usage criteria for in order for any help to well match your
intended context.

(git has means of context switching what is seen in the file system,
allowing switching branches in place. This is part of how "one at a
time" can be relevant: one switches the context in the file system
instead.)

> For /usr/doc it was easy to "clone" the repository and, who would doubt it, 
> there were
> some hints how-to-do. But how do I checkout /usr/src for either CURRENT, 
> 12.{1,2}-RELENG
> and 12-STABLE? Where is the primer for those "not-so-development-involved" 
> people like
> me? And were to and how to report problems in themanner of this list 
> regarding to certain
> commits?

There are various preliminary materials at:

https://github.com/bsdimp/freebsd-git-docs/

including:

https://github.com/bsdimp/freebsd-git-docs/blob/main/mini-primer.md

and:

https://github.com/bsdimp/freebsd-git-docs/blob/main/src-cvt.md

But I've also been doing other explorations to find what fits my usage
patterns or if there are patterns that I likely should change. I've
done some "non-production" fiddling separate from my still-existing
svn based context to get an idea what various things would end up
being like to use. (I've even done some diff -r 

Re: svn commit: r364292 - in head: lib/libc/tests/sys lib/libfetch lib/libpam/libpam lib/msun/tests libexec/rtld-elf usr.bin/fetch usr.bin/unzip

[Mark Millard via svn-src-head]

> Author: delphij
> Date: Mon Aug 17 05:57:02 2020
> New Revision: 364292
> URL: 
> https://svnweb.freebsd.org/changeset/base/364292
> 
> 
> Log:
>   Don't explicitly specify c99 or gnu99 as the default is now gnu99.
>   
> . . .
> 
> Modified: head/lib/libc/tests/sys/Makefile
> ==
> --- head/lib/libc/tests/sys/Makefile  Sun Aug 16 23:55:23 2020
> (r364291)
> +++ head/lib/libc/tests/sys/Makefile  Mon Aug 17 05:57:02 2020
> (r364292)
> @@ -70,8 +70,6 @@ NETBSD_ATF_TESTS_C+=wait_noproc_test
>  NETBSD_ATF_TESTS_C+= wait_noproc_wnohang_test
>  NETBSD_ATF_TESTS_C+= write_test
>  
> -CSTD?=   c99
> -
>  LIBADD.getpid_test+= pthread
>  LIBADD.timer_create_test+=   rt
>  
> 
> Modified: head/lib/libfetch/Makefile
> ==
> --- head/lib/libfetch/MakefileSun Aug 16 23:55:23 2020
> (r364291)
> +++ head/lib/libfetch/MakefileMon Aug 17 05:57:02 2020
> (r364292)
> @@ -23,8 +23,6 @@ LIBADD+=md
>  
>  CFLAGS+= -DFTP_COMBINE_CWDS
>  
> -CSTD?=   c99
> -
>  SHLIB_MAJOR=6
>  
>  ftperr.h: ftp.errors ${.CURDIR}/Makefile
> 
> Modified: head/lib/libpam/libpam/Makefile
> ==
> --- head/lib/libpam/libpam/Makefile   Sun Aug 16 23:55:23 2020
> (r364291)
> +++ head/lib/libpam/libpam/Makefile   Mon Aug 17 05:57:02 2020
> (r364292)
> @@ -156,7 +156,6 @@ MAN?= openpam.3 \
>  
>  MLINKS?= pam.conf.5 pam.d.5
>  
> -CSTD?= c99
>  CFLAGS+= -I${OPENPAM}/include
>  CFLAGS+= -DLIB_MAJ=${SHLIB_MAJOR}
>  CFLAGS+= -DHAVE_DLFUNC=1
> 
> Modified: head/lib/msun/tests/Makefile
> ==
> --- head/lib/msun/tests/Makefile  Sun Aug 16 23:55:23 2020
> (r364291)
> +++ head/lib/msun/tests/Makefile  Mon Aug 17 05:57:02 2020
> (r364292)
> @@ -75,8 +75,6 @@ ATF_TESTS_C+=   trig_test
>  CFLAGS+= -O0
>  .endif
>  
> -CSTD=c99
> -
>  #COPTS+= -Wfloat-equal
>  
>  IGNORE_PRAGMA=
> 
> . . .
> 
> Modified: head/usr.bin/fetch/Makefile
> ==
> --- head/usr.bin/fetch/Makefile   Sun Aug 16 23:55:23 2020
> (r364291)
> +++ head/usr.bin/fetch/Makefile   Mon Aug 17 05:57:02 2020
> (r364292)
> @@ -1,7 +1,6 @@
>  # $FreeBSD$
>  
>  PROG=fetch
> -CSTD?=   c99
>  LIBADD=  fetch
>  
>  .include 
> 
> Modified: head/usr.bin/unzip/Makefile
> ==
> --- head/usr.bin/unzip/Makefile   Sun Aug 16 23:55:23 2020
> (r364291)
> +++ head/usr.bin/unzip/Makefile   Mon Aug 17 05:57:02 2020
> (r364292)
> @@ -1,7 +1,6 @@
>  # $FreeBSD$
>  
>  PROG = unzip
> -CSTD = c99
>  LIBADD=  archive
>  
>  .include 
> 

For the c99's that were removed: why were they not
gnu99's originally? If there is a reason, why is
it now okay to revert to gnu99 instead of sticking
with c99?

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r364053 - head/release/arm64

[Mark Millard via svn-src-head]

> Author: manu
> Date: Sat Aug  8 16:56:20 2020
> New Revision: 364053
> URL: 
> https://svnweb.freebsd.org/changeset/base/364053
> 
> 
> Log:
>   release: RPI3: Add the RPI2 DTB
>   
>   The RPI2 v1.2 is using the same SoC as the RPI3 so it can boot this image
>   but needs the RPI2 dtb.
>   
>   MFC after:  3 days
> 
> Modified:
>   head/release/arm64/RPI3.conf
> 
> Modified: head/release/arm64/RPI3.conf
> ==
> --- head/release/arm64/RPI3.conf  Sat Aug  8 16:41:28 2020
> (r364052)
> +++ head/release/arm64/RPI3.conf  Sat Aug  8 16:56:20 2020
> (r364053)
> @@ -4,7 +4,7 @@
>  #
>  
>  DTB_DIR="/usr/local/share/rpi-firmware"
> -DTB="bcm2710-rpi-3-b.dtb bcm2710-rpi-3-b-plus.dtb bcm2711-rpi-4-b.dtb"
> +DTB="bcm2709-rpi-2-b.dtb bcm2710-rpi-3-b.dtb bcm2710-rpi-3-b-plus.dtb 
> bcm2711-rpi-4-b.dtb"
>  EMBEDDED_TARGET_ARCH="aarch64"
>  EMBEDDED_TARGET="arm64"
>  EMBEDDEDBUILD=1

When I look at the likes of:

https://github.com/raspberrypi/firmware/tree/master/boot

I see two *-rpi-2-b.dtb links on the page, each referencing
one of:

https://github.com/raspberrypi/firmware/blob/master/boot/bcm2709-rpi-2-b.dtb
https://github.com/raspberrypi/firmware/blob/master/boot/bcm2710-rpi-2-b.dtb

It appears to me that bcm2709-rpi-2-b.dtb is for RPi2B V1.1 and before
(cortex-A7 based) and that bcm2710-rpi-2-b.dtb is for RPi2B v1.2 (the
cortex-A53 based variant). I base this on:

https://www.raspberrypi.org/forums/viewtopic.php?t=238262

and its table:

2708/2835: Pi Model A, Model B, A+, B+, Zero, Zero W, Compute Module
2709/2836: Pi 2B
2710/2837: Pi 3B, 3B+, Compute Module 3

mixed with: https://www.raspberrypi.org/documentation/hardware/raspberrypi/
and its outline:

• BCM2836
• The Broadcom processor used in Raspberry Pi 2
• BCM2837
• The Broadcom processor used in Raspberry Pi 3 (and later 
Raspberry Pi 2)

If that is correct, then the -r364053 is using the cortex-A7 variant's
dtb instead of the cortex-A53 variant's dtb. It looks like it should
be using bcm2710-rpi-2-b.dtb to be treating the RPI2B v1.2 in an
RPi3B like way.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: Introduce WITH(OUT)_LTO? (was: Re: svn commit: r362987 - in head: contrib/bc usr.bin/gh-bc) (LLVMgold.so and gnu's ld.gold)

[Mark Millard via svn-src-head]

On 2020-Jul-25, at 13:59, Mark Millard  wrote:

> On 2020-Jul-8, at 01:28, Stefan Eßer  wrote:
> 
>> Am 08.07.20 um 09:01 schrieb Mark Millard:
>>> The following is more informational than anything as far
>>> as I'm concerned. But there may be implications that I'm
>>> unaware of. (I sometimes experiment with toolchain use
>>> to see what the current status is for such use.)
>>> 
>>> I attempted to build a system for 32-bit powerpc using clang
>>> and binutils, building head -r363000 ( from -r363000 ). (This
>>> was a cross build, amd64 -> powerpc.) It got a new type of
>>> failure, compared to my past experience:
>> 
>> Hi Mark,
>> 
>> thank you for the report. I have tested with "make universe" (with
>> default settings) that this version builds on all architectures,
>> but Ed Maste has already disabled -flto for powerpc64, due to run
>> time issues (floating point exception, IIRC).
>> 
>> I know that you are actively working on PowerPC and I'd appreciate,
>> if you could provide me with information on which parameters cause
>> breakage and which work for you. The combination of CLANG with LTO
>> and GNU binutils cannot work - CLANG and GCC use incompatible file
>> formats to represent the intermediate object files.
> 
> Hmm. It looks a little more complicated than that . . .
> 
> Looks like the devel/llvm80 devel/llvm90 and devel/llvm10
> options for powerpc64 include one for:
> 
> GOLD=on: Build the LLVM Gold plugin for LTO
> 
> That produces a plugin (LLVMgold.so) for use with gnu's
> ld.gold ( from devel/binutils ).
> 
> . . .

Ignore those notes. It looks like I greatly misinterpreted. For
example doing some personal software builds with -flto in use
resulted in (using devel/llvm11 as an example context):
(powerpc64 context used)

"/usr/local/llvm11/bin/ld" . . . -plugin 
/usr/local/llvm11/bin/../lib/LLVMgold.so -plugin-opt=mcpu=ppc64 -plugin-opt=O3 
. . .

LLVMgold.so is for the llvm linker to use. I had built
llvm10 with the gold option selected and there is:

# ls -ldT /usr/local/llvm11/bin/../lib/LLVMgold.so
-rwxr-xr-x  1 root  wheel  94160 Jul 29 14:50:07 2020 
/usr/local/llvm11/bin/../lib/LLVMgold.so

But, for the system clang 10 with -flto involved:

"/usr/bin/ld" . . . -plugin /usr/bin/../lib/LLVMgold.so -plugin-opt=mcpu=ppc64 
-plugin-opt=O3 . . .

(yet there is no /usr/bin/../lib/LLVMgold.so present).

And for even the likes of just:

static volatile char big_area[67001] = "This is a test";

int main ()
{
big_area[67000] = '9';
}

commands like ( system clang and devel/llvm10 ):

cc -flto main.c
clang10 -flto main.c

for powerpc64 produce invalid a.out files that do
not even contain a main function when looked at
with the likes of objdump -d --prefix-addresses
and produce an a.out that does:

# ./a.out
Segmentation fault (core dumped)

Or when run inside gdb such builds produce things like:

Starting program: /root/c_tests/a.out 

Program received signal SIGSEGV, Segmentation fault.

(gdb) bt
#0  0x100412e8 in main ()
#1  0x10010718 in _start (argc=, argv=0xfbfffebb8, 
env=, obj=, cleanup=, 
ps_strings=)
at /usr/src/lib/csu/powerpc64/crt1_c.c:127
(gdb) disass
Dump of assembler code for function main:
=> 0x100412e8 <+0>: .long 0x0
   0x100412ec <+4>: mullhwu r0,r1,r1
   0x100412f0 <+8>: .long 0x0
   0x100412f4 <+12>:vmsumshm v0,v2,v17,v19
   0x100412f8 <+16>:.long 0x0
   0x100412fc <+20>:.long 0x0
   0x10041300 <+0>: .long 0xf
   0x10041304 <+4>: stmwr31,-4128(r31)
   0x10041308 <+0>: .long 0xf
   0x1004130c <+4>: stmwr31,-5176(r31)
   0x10041310 <+0>: .long 0x0
End of assembler dump.

For reference:

# uname -apKU
FreeBSD FBSDG5L2 13.0-CURRENT FreeBSD 13.0-CURRENT #13 r363590M: Sun Jul 26 
20:14:08 PDT 2020 
root@FBSDFHUGE:/usr/obj/powerpc64vtsc_clang/powerpc.powerpc64/usr/src/powerpc.powerpc64/sys/GENERIC64vtsc-NODBG
  powerpc powerpc64 1300102 1300102

# svnlite info /usr/ports/
Path: /usr/ports
Working Copy Root Path: /usr/ports
URL: svn://svn.freebsd.org/ports/head
Relative URL: ^/head
Repository Root: svn://svn.freebsd.org/ports
Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5
Revision: 543890
Node Kind: directory
Schedule: normal
Last Changed Author: gerald
Last Changed Rev: 543890
Last Changed Date: 2020-07-31 22:52:17 -0700 (Fri, 31 Jul 2020)



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: Introduce WITH(OUT)_LTO? (was: Re: svn commit: r362987 - in head: contrib/bc usr.bin/gh-bc) (LLVMgold.so and gnu's ld.gold)

[Mark Millard via svn-src-head]

On 2020-Jul-8, at 01:28, Stefan Eßer  wrote:

> Am 08.07.20 um 09:01 schrieb Mark Millard:
>> The following is more informational than anything as far
>> as I'm concerned. But there may be implications that I'm
>> unaware of. (I sometimes experiment with toolchain use
>> to see what the current status is for such use.)
>> 
>> I attempted to build a system for 32-bit powerpc using clang
>> and binutils, building head -r363000 ( from -r363000 ). (This
>> was a cross build, amd64 -> powerpc.) It got a new type of
>> failure, compared to my past experience:
> 
> Hi Mark,
> 
> thank you for the report. I have tested with "make universe" (with
> default settings) that this version builds on all architectures,
> but Ed Maste has already disabled -flto for powerpc64, due to run
> time issues (floating point exception, IIRC).
> 
> I know that you are actively working on PowerPC and I'd appreciate,
> if you could provide me with information on which parameters cause
> breakage and which work for you. The combination of CLANG with LTO
> and GNU binutils cannot work - CLANG and GCC use incompatible file
> formats to represent the intermediate object files.

Hmm. It looks a little more complicated than that . . .

Looks like the devel/llvm80 devel/llvm90 and devel/llvm10
options for powerpc64 include one for:

 GOLD=on: Build the LLVM Gold plugin for LTO

That produces a plugin (LLVMgold.so) for use with gnu's
ld.gold ( from devel/binutils ).

The system-clang/llvm materials do not seem to include
LLVMgold.so and so prevent use of that toolchain for
LTO via devel/binutils . (A FreeBSD choice rather than
a llvm* vs. binutils technically-blocking issue?)

devel/binutils builds and installs /usr/local/bin/ld.gold
for both 32-bit powerpc contexts and for powerpc64 contexts.

However, for the 32-bit powerpc context, the devel/llvm*
do not include an option to enable building LLVMgold.so .
I do not know if this is just a ports choice vs. if this
has a technically-blocking issue involved.


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r363510 - head/sys/dev/extres/clk

[Mark Millard via svn-src-head]

Michal Meloun mmel at FreeBSD.org wrote on
Sat Jul 25 06:32:24 UTC 2020 :

> Author: mmel
> Date: Sat Jul 25 06:32:23 2020
> New Revision: 363510
> URL: 
> https://svnweb.freebsd.org/changeset/base/363510
> 
> 
> Log:
>   Revert r363123.
>   As Emanuel poited me the Linux processes these clock assignments in forward
>   order, not in reversed. I misread the original code.
>   Tha problem with wrong order for assigned clocks found in tegra (and some 
> imx)
>   DT should be reanalyzed and solved by different way.
>   
>   MFC with:   r363123
>   Reported by;manu

This fixes the Rock64, allowing it to boot again. Thanks!

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r363031 - in head: contrib/bmake contrib/bmake/lst.lib contrib/bmake/mk contrib/bmake/mk/sys contrib/bmake/unit-tests usr.bin/bmake (make now broken)

[Mark Millard via svn-src-head]

On 2020-Jul-8, at 20:35, Yuri Pankov  wrote:

> Mark Millard wrote:
>> This seems to have broken doing buildworld buildkernel and
>> other things using make:
>> make[2]: "/usr/src/share/mk/bsd.compiler.mk" line 197: warning: String 
>> comparison operator should be either == or !=
>> make[2]: "/usr/src/share/mk/bsd.compiler.mk" line 197: warning: String 
>> comparison operator should be either == or !=
>> . . .
>> make[3]: "/usr/src/share/mk/bsd.sys.mk" line 100: warning: String comparison 
>> operator should be either == or !=
>> . . .
>> make[3]: "/usr/src/share/mk/bsd.sys.mk" line 81: warning: String comparison 
>> operator should be either == or !=
>> . . .
>> Using -d c shows the likes of:
>> . . .
>> lhs = "clang", rhs = "clang", op = ==
>> lhs = "${${:UCOMPILER_TYPE}__${${:U${_empty_var_}}_cc_hash}}", rhs = "gcc", 
>> op = ==
>> make[2]: "/usr/src/share/mk/bsd.compiler.mk" line 197: warning: String 
>> comparison operator should be either == or !=
>> lhs = "clang", rhs = "clang", op = ==
>> lhs = "LD", rhs = "LD", op = ==
>> . . .
>> left = 6.00, right = 2.00, op = <=
>> left = 6.00, right = 1.00, op = <=
>> lhs = "${${:UCOMPILER_TYPE}__${${:U${_empty_var_}}_cc_hash}}", rhs = 
>> "clang", op = ==
>> make[3]: "/usr/src/share/mk/bsd.sys.mk" line 100: warning: String comparison 
>> operator should be either == or !=
>> lhs = "${${:UCOMPILER_TYPE}__${${:U${_empty_var_}}_cc_hash}}", rhs = "gcc", 
>> op = ==
>> lhs = "clang", rhs = "gcc", op = ==
>> . . .
>> left = 0.00, right = 6.00, op = <=
>> left = 0.00, right = 3.00, op = <=
>> lhs = "clang", rhs = "gcc", op = ==
>> make[3]: "/usr/src/share/mk/bsd.sys.mk" line 81: warning: String comparison 
>> operator should be either == or !=
>> lhs = "clang", rhs = "clang", op = ==
>> left = 11.00, right = 7.00, op = >=
>> lhs = "amd64", rhs = "arm", op = ==
>> (Now I just need to figure out how to get back to a working context.)
> 
> For me, buildworld/buildkernel produced only warnings,

But, looking at the code in bmake, the expression is also
evaluated differently/incorrectly when it is classified as
having the problem of having a incorrect operator. In other
words: the behavior in make changes via misevaluated
expressions.


> though the one in ports is real issue:
> 
> $ make config
> make: "/usr/ports/Mk/bsd.port.mk" line 2096: warning: String comparison 
> operator should be either == or !=
> make: "/usr/ports/Mk/bsd.port.mk" line 2096: Malformed conditional 
> (defined(MAKE_JOBS_NUMBER_LIMIT) && ( ${MAKE_JOBS_NUMBER_LIMIT} < 
> ${_MAKE_JOBS_NUMBER} ))
> make: Fatal errors encountered -- cannot continue
> make: stopped in /usr/ports/devel/subversion

Not the only "real issue", I'm afraid.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r363031 - in head: contrib/bmake contrib/bmake/lst.lib contrib/bmake/mk contrib/bmake/mk/sys contrib/bmake/unit-tests usr.bin/bmake (make now broken)

[Mark Millard via svn-src-head]

This seems to have broken doing buildworld buildkernel and
other things using make:

make[2]: "/usr/src/share/mk/bsd.compiler.mk" line 197: warning: String 
comparison operator should be either == or !=
make[2]: "/usr/src/share/mk/bsd.compiler.mk" line 197: warning: String 
comparison operator should be either == or !=
. . .
make[3]: "/usr/src/share/mk/bsd.sys.mk" line 100: warning: String comparison 
operator should be either == or !=
. . .
make[3]: "/usr/src/share/mk/bsd.sys.mk" line 81: warning: String comparison 
operator should be either == or !=
. . .

Using -d c shows the likes of:

. . .
lhs = "clang", rhs = "clang", op = ==
lhs = "${${:UCOMPILER_TYPE}__${${:U${_empty_var_}}_cc_hash}}", rhs = "gcc", op 
= ==
make[2]: "/usr/src/share/mk/bsd.compiler.mk" line 197: warning: String 
comparison operator should be either == or !=
lhs = "clang", rhs = "clang", op = ==
lhs = "LD", rhs = "LD", op = ==
. . .
left = 6.00, right = 2.00, op = <=
left = 6.00, right = 1.00, op = <=
lhs = "${${:UCOMPILER_TYPE}__${${:U${_empty_var_}}_cc_hash}}", rhs = "clang", 
op = ==
make[3]: "/usr/src/share/mk/bsd.sys.mk" line 100: warning: String comparison 
operator should be either == or !=
lhs = "${${:UCOMPILER_TYPE}__${${:U${_empty_var_}}_cc_hash}}", rhs = "gcc", op 
= ==
lhs = "clang", rhs = "gcc", op = ==
. . .
left = 0.00, right = 6.00, op = <=
left = 0.00, right = 3.00, op = <=
lhs = "clang", rhs = "gcc", op = ==
make[3]: "/usr/src/share/mk/bsd.sys.mk" line 81: warning: String comparison 
operator should be either == or !=
lhs = "clang", rhs = "clang", op = ==
left = 11.00, right = 7.00, op = >=
lhs = "amd64", rhs = "arm", op = ==

(Now I just need to figure out how to get back to a working context.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: Introduce WITH(OUT)_LTO? (was: Re: svn commit: r362987 - in head: contrib/bc usr.bin/gh-bc)

[Mark Millard via svn-src-head]

On 2020-Jul-8, at 01:28, Stefan Eßer  wrote:

> Am 08.07.20 um 09:01 schrieb Mark Millard:
>> The following is more informational than anything as far
>> as I'm concerned. But there may be implications that I'm
>> unaware of. (I sometimes experiment with toolchain use
>> to see what the current status is for such use.)
>> 
>> I attempted to build a system for 32-bit powerpc using clang
>> and binutils, building head -r363000 ( from -r363000 ). (This
>> was a cross build, amd64 -> powerpc.) It got a new type of
>> failure, compared to my past experience:
> 
> Hi Mark,
> 
> thank you for the report. I have tested with "make universe" (with
> default settings) that this version builds on all architectures,
> but Ed Maste has already disabled -flto for powerpc64, due to run
> time issues (floating point exception, IIRC).

The builds I actually run are based on pure-llvm, not such
odd mixes --nor even gcc+binutils. Also, I've not even tried
a build since updating to head -r360311 --until now trying to
update things to head -r363000 (or so).

It is not obvious to me that FreeBSD intends for gcc based
builds or binutils based buildworld buildkernel to be
supported, even without mixing in LLVM. The ci.freebsd.org
builds via gcc6 for amd64 have not completed in a very
long time.

My standard build procedure historically builds combinations
like clang-with-binutils that used to completely build. (Not
that the result worked to boot and operate an old PowerMac in
recent times.) So I noticed this just by the build procedure
failing. There are other combinations that failed some time
ago that I turned off in my build procedure after sending a
notice to the lists.

As of the official switch to llvm-based for powerpc64 and
powerpc, I've not been able to boot and operate anything but
pure-llvm based builds, even for other types of builds that
did "complete". That includes not being able to build and
operate system based on modern gcc+binutils without LLVM.
It appears to me that it would be a fair sized effort to
get gcc+binutils working for powerpc64 or powerpc.

My history with odd combinations derived from experimenting
with more modern toolchains for powerpc64 and powerpc back
in the gcc 4.2.1 days when no llvm based linker was affectively
available. Although, I do like the idea of having code pass
through and the result operate based on multiple toolchains,
not just one, say, modern gcc+binutils as an alternate.

Still, if FreeBSD declared "gcc not supported" and "binutils
not supported" and "gcc+binutils not supported" for building
FreeBSD, I'd just stick to llvm based.

> I know that you are actively working on PowerPC and I'd appreciate,
> if you could provide me with information on which parameters cause
> breakage and which work for you.

I think that the answer is: only pure llvm-based has worked
for booting and operating since the switch away from gcc 4.2.1
for powerpc64 and powerpc.

Everything else either stops the build early or does not operate
correctly (as I remember anyway). Apparently, more combinations
are landing in the "stops the build early" category over time.

> The combination of CLANG with LTO
> and GNU binutils cannot work - CLANG and GCC use incompatible file
> formats to represent the intermediate object files.

To my knowledge, a modern gcc+binutils does not work as the
basis for a FreeBSD build for powerpc64 or powerpc. No need
to have a mixed toolchain involved or -flto involved to have
blocking issues.

> This version of bc uses advanced algorithms (compared to the one
> it replaced) for much reduced time complexity (a factor of more
> than 100 for large numbers has been observed). It uses layering
> for correctness, e.g. a set of "vector" management functions, that
> are built as a separate compile unit. Compiling with -flto lets the
> linker replace many function calls with constant parameters with
> much more efficient inline instructions, resulting in 30% higher
> performance. The code could be re-arranged to use inline functions
> instead of relying on -flto, but this would be a lot of effort and
> might make the code much harder to maintain.

I'm not objecting to -flto use, even if it constrains what
can be used as a basis for building and operating FreeBSD.

There are issues that well predate -flto use that block
other toolchains for powerpc64 and powerpc.

If more than llvm-based is to be supported, I'd just suggest
modern gcc+binutils, avoiding claiming support for mixes
such as llvm+binutils.

> If there is a condition that could be added to the Makefile to not
> enable LTO if CLANG+binutils or GCC+LLD are mixed, then I'm willing
> to add it.

If it happened that -flto use vs. not needs to be configurable
for other reasons and it happens to end up that turning it off
allows the mixes, that would be fine. (But there could be more
issues before any mixes would work.)

This would not be the same as targeting being sure that some
specific mix(es) would work (all issues handled for 

Re: svn commit: r362987 - in head: contrib/bc usr.bin/gh-bc

[Mark Millard via svn-src-head]

The following is more informational than anything as far
as I'm concerned. But there may be implications that I'm
unaware of. (I sometimes experiment with toolchain use
to see what the current status is for such use.)

I attempted to build a system for 32-bit powerpc using clang
and binutils, building head -r363000 ( from -r363000 ). (This
was a cross build, amd64 -> powerpc.) It got a new type of
failure, compared to my past experience:

--- gh-bc.full ---
/usr/local/powerpc-unknown-freebsd13.0/bin/ld: /usr/bin/../lib/LLVMgold.so: 
error loading plugin: Cannot open "/usr/bin/../lib/LLVMgold.so"
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** [gh-bc.full] Error code 1

Yep: /usr/lib/LLVMgold.so when -B/usr/local/powerpc-unknown-freebsd13.0/bin/
was in use.

I turns out that the link of gh-bc used -flto :

make[4]: stopped in /usr/src/usr.bin/gh-bc
.ERROR_TARGET='gh-bc.full'
.ERROR_META_FILE='/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc/usr.bin/gh-bc/gh-bc.full.meta'
.MAKE.LEVEL='4'
MAKEFILE=''
.MAKE.MODE='meta missing-filemon=yes missing-meta=yes silent=yes verbose'
_ERROR_CMD='cc -target powerpc-unknown-freebsd13.0 
--sysroot=/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc/tmp
 -B/usr/local/powerpc-unknown-freebsd13.0/bin/ -O2 -pipe -fno-common 
-B/usr/local/powerpc-unknown-freebsd13.0/bin/ -DMAINEXEC=bc 
-DNLSPATH=/usr/share/nls/%L/%N.cat -DBC_ENABLED -DBC_ENABLE_PROMPT 
-DBC_ENABLE_LONG_OPTIONS -DBC_ENABLE_EXTRA_MATH -DBC_ENABLE_HISTORY 
-DBC_ENABLE_RAND -DDC_ENABLED -DNDEBUG -DVERSION=3.1.1 
-I/usr/src/contrib/bc/include -DBC_ENABLE_NLS=1 -flto -g -std=gnu99 
-Wno-format-zero-length -fstack-protector-strong -Wsystem-headers -Wall 
-Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes 
-Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings 
-Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline 
-Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign 
-Wmissing-variable-declarations -Wthread-safety -Wno-empty-body 
-Wno-string-plus-int -Wno-unused-const-
 variable -Qunused-arguments  -Wl,--secure-plt  -o gh-bc.full args.o data.o 
file.o lang.o lex.o main.o num.o parse.o program.o read.o vector.o vm.o bc/bc.o 
bc/lex.o bc/parse.o dc/dc.o dc/lex.o dc/parse.o history/history.o bc_help.o 
dc_help.o lib.o lib2.o opt.o rand/rand.o  ;'
.CURDIR='/usr/src/usr.bin/gh-bc'
.MAKE='make'
.OBJDIR='/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc/usr.bin/gh-bc'
.TARGETS='all'
DESTDIR='/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc/tmp'
LD_LIBRARY_PATH=''
MACHINE='powerpc'
MACHINE_ARCH='powerpc'
MAKEOBJDIRPREFIX=''
MAKESYSPATH='/usr/src/share/mk'
MAKE_VERSION='20200606'
PATH='/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc/tmp/usr/sbin:/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc/tmp/usr/bin:/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc/tmp/legacy/usr/sbin:/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc/tmp/legacy/usr/bin:/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc/tmp/legacy/bin:/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc/tmp/legacy/usr/libexec::/sbin:/bin:/usr/sbin:/usr/bin'
SRCTOP='/usr/src'
OBJTOP='/usr/obj/powerpcvtsc_clang_altbinutils/powerpc.powerpc/usr/src/powerpc.powerpc'
.MAKE.MAKEFILES='/usr/src/share/mk/sys.mk /usr/src/share/mk/local.sys.env.mk 
/usr/src/share/mk/src.sys.env.mk 
/root/src.configs/src.conf.powerpc-clang_altbinutils-bootstrap.amd64-host 
/usr/src/share/mk/bsd.mkopt.mk /usr/src/share/mk/src.sys.obj.mk 
/usr/src/share/mk/auto.obj.mk /usr/src/share/mk/bsd.suffixes.mk 
/root/src.configs/make.conf /usr/src/share/mk/local.sys.mk 
/usr/src/share/mk/src.sys.mk /dev/null /usr/src/usr.bin/gh-bc/Makefile 
/usr/src/share/mk/src.opts.mk /usr/src/share/mk/bsd.own.mk 
/usr/src/share/mk/bsd.opts.mk /usr/src/share/mk/bsd.cpu.mk 
/usr/src/share/mk/bsd.compiler.mk /usr/src/share/mk/bsd.linker.mk 
/usr/src/share/mk/bsd.prog.mk /usr/src/share/mk/bsd.init.mk 
/usr/src/share/mk/local.init.mk /usr/src/share/mk/src.init.mk 
/usr/src/usr.bin/gh-bc/../Makefile.inc /usr/src/share/mk/bsd.libnames.mk 
/usr/src/share/mk/src.libnames.mk /usr/src/share/mk/bsd.nls.mk 
/usr/src/share/mk/bsd.confs.mk /usr/src/share/mk/bsd.files.mk 
/usr/src/share/mk/bsd.dirs.mk /usr/src/share/mk/bsd.
 incs.mk /usr/src/share/mk/bsd.links.mk /usr/src/share/mk/bsd.man.mk 
/usr/src/share/mk/bsd.dep.mk /usr/src/share/mk/bsd.clang-analyze.mk 
/usr/src/share/mk/bsd.obj.mk /usr/src/share/mk/bsd.subdir.mk 
/usr/src/share/mk/bsd.sys.mk'
.PATH='. /usr/src/usr.bin/gh-bc /usr/src/contrib/bc/src /usr/src/contrib/bc/gen 
/usr/src/contrib/bc/manuals 

Re: svn commit: r362998 - in head: stand/defaults sys/amd64/amd64 sys/powerpc/aim sys/vm

[Mark Millard via svn-src-head]

Beyond being a breaking change for existing /boot/blacklist.txt
files, there is the non-operation issue:

. . .
+ram_excludelist_type="ram_excludelist" # Required for the kernel to find
# the blacklist module
. . .

In other words, it is still using the term blacklist in one place
(and not explicitly as a reference to the prior naming convention
to help avoid any potential confusions).

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r362422 - head/sbin/dump

[Mark Millard via svn-src-head]

Hans Petter Selasky hps at selasky.org wrote on
Sat Jun 20 13:27:13 UTC 2020 :

> On 2020-06-20 13:10, Rodney W. Grimes wrote:
> >> Author: imp
> >> Date: Sat Jun 20 04:19:17 2020
> >> New Revision: 362422
> >> URL:https://svnweb.freebsd.org/changeset/base/362422
> >>
> >> Log:
> >>Increase the whimsy in this file by famring dump's work out to minions. 
> >> Adjust
> >>variables accordingly. Thankfully, we are able to do this without 
> >> additional
> >>banana expenditures.
> > This flys in the face of its intent and as a "commit" is more
> > racially biased than the code was!
> > 
> 
> Hi Warner,
> 
> Maybe a stupid question, but is this the correct meaning or description 
> of minion?
> 
> https://www.google.com/search?q=minions
> 
> minion; plural noun: minions
> 
>  a follower or underling of a powerful person, especially a servile 
> or unimportant one.
>  "he gets oppressed minions like me to fob them off"
> 
> "Minion" is still a person, like "slave" is, so I must say I agree with 
> Rodney about this, I don't see how this makes it any better? Can you 
> explain?

Collins | COBUILD Advanced Learner's Dictionary 9th edition :

QUOTE
min|ion
n-COUNT [usu pl, usu poss N]
If you refer to someone's minions, you are referring to
people who have to do what that person tells them to do,
especially unimportant or boring tasks.
[LITERARY, DISAPPROVAL]
She delegated the job to one of her minions.
END QUOTE

QUOTE (from descriptions of its COBUILD's terminology)
LITERARY: used mainly in novels, poetry, and other
forms of literature, e.g., plaintive

DISAPPROVAL: The label DISAPPROVAL indicates that you
use the word or expression to show that you dislike
the person or thing you are talking about. An example
of a word with this label is infantile.
END QUOTE

Minion is not listed as OFFENSIVE, VERY OFFENSIVE, RUDE,
or VERY RUDE. (I'll not quote the descriptions.)


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[Yet another oddity.]

On 2020-Jun-11, at 21:05, Mark Millard  wrote:
> 
> There is another oddity in the code structure, in
> that if pt was ever NULL the code would misuse the
> NULL before the test for non-NULL is made:
> 
>pt = moea_pvo_to_pte(pvo, -1);
> . . .
>old_pte = *pt;
> 
>/*
> * If the PVO is in the page table, update that pte as well.
> */
>if (pt != NULL) {
> 
> (I'm not claiming that this explains the panic.)

There is another NULL handling oddity that the
64-bit code does not have. I'll show 64
relevant bit code first:

pg = PHYS_TO_VM_PAGE(pvo->pvo_pte.pa & LPTE_RPGN);

. . .

if (. . .&& pg != NULL &&
(pg->a.flags & PGA_EXECUTABLE) == 0 &&
 . . .) {
if ((pg->oflags & VPO_UNMANAGED) == 0)
vm_page_aflag_set(pg, PGA_EXECUTABLE);
. . .
}

. . .
if (pg != NULL && . . .) {
refchg |= atomic_readandclear_32(>md.mdpg_attrs);
if (refchg & LPTE_CHG)
vm_page_dirty(pg);
if (refchg & LPTE_REF)
vm_page_aflag_set(pg, PGA_REFERENCED);
}

Note: the 2nd outer-if tests for pg != NULL, just like the
first outer-if above does. It avoids potential abuse-of-NULL
activity.

This is not true of the 32-bit code for "m":

m = PHYS_TO_VM_PAGE(old_pte.pte_lo & PTE_RPGN);
if (. . . && m != NULL &&
(m->a.flags & PGA_EXECUTABLE) == 0 &&
. . .) {
if ((m->oflags & VPO_UNMANAGED) == 0)
vm_page_aflag_set(m, PGA_EXECUTABLE);
moea_syncicache(pvo->pvo_pte.pa & PTE_RPGN,
PAGE_SIZE);
}
. . .
if ((pvo->pvo_vaddr & PVO_MANAGED) &&
(pvo->pvo_pte.prot & VM_PROT_WRITE)) {
refchg = 
atomic_readandclear_32(>md.mdpg_attrs);
if (refchg & PTE_CHG)
vm_page_dirty(m);
if (refchg & PTE_REF)
vm_page_aflag_set(m, PGA_REFERENCED);
}

The >md.mdpg_attrs use apparently could be based on m
being NULL because there is no test for m != NULL in the
2nd outer-if. Similarly for the other uses of m in that
code block.

My guess that that the 2nd outer-if should test something
like:

. . .
if (m != NULL &&
(pvo->pvo_vaddr & PVO_MANAGED) &&
(pvo->pvo_pte.prot & VM_PROT_WRITE)) {
. . .

(Presumes that the pre-existing m != NULL tests are
necessary, something that I do not know.)


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

There is another oddity in the code structure, in
that if pt was ever NULL the code would misuse the
NULL before the test for non-NULL is made:

pt = moea_pvo_to_pte(pvo, -1);
. . .
old_pte = *pt;

/*
 * If the PVO is in the page table, update that pte as well.
 */
if (pt != NULL) {

(I'm not claiming that this explains the panic.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[Just a better panic backtrace text copy.]

On 2020-Jun-11, at 20:29, Mark Millard  wrote:

> On 2020-Jun-11, at 19:25, Justin Hibbits  wrote:
> 
>> On Thu, 11 Jun 2020 17:30:24 -0700
>> Mark Millard  wrote:
>> 
>>> On 2020-Jun-11, at 16:49, Mark Millard  wrote:
>>> 
 On 2020-Jun-11, at 14:42, Justin Hibbits 
 wrote:
 . . .
>> 
>> Can you try moving the assignment to 'm' to right below the
>> moea_pte_change() call?
> 
> Panics during boot. svnlite diff shown later.
> 
> That change got me a panic just after the lines about ada0
> and cd0 details. (Unknown what internal stage.) Hand
> translated from a picture of the screen:
> 
> panic: . . .

I forgot that 32-bit powerpc dump does partially work
for PowerMacs (or at least my context). After booting
with a non-debug kernel I've kept around, looking at
/var/crash/vmcore.3 shows:

panic: vm_page_free_prep: mapping flags set in page 0xd032a078
cpuid = 1
time = 1591931757
KDB: stack backtrace:
0xd2dc4340: at kdb_backtrace+0x64
0xd2dc43a0: at vpanic+0x208
0xd2dc4410: at panic+0x64
0xd2dc4450: at vm_page_free_prep+0x348
0xd2dc4470: at vm_page_free_toq+0x3c
0xd2dc4490: at vm_page_free+0x20
0xd2dc44a0: at vm_object_collapse+0x4ac
0xd2dc4510: at vm_object_deallocate+0x430
0xd2dc4550: at vm_map_process_deferred+0xec
0xd2dc4570: at vm_map_remove+0x12c
0xd2dc4590: at exec_new_vmspace+0x20c
0xd2dc45f0: at exec_elf32_imgact+0xa70
0xd2dc46a0: at kern_execve+0x600
0xd2dc4910: at sys_execve+0x84
0xd2dc4970: at trap+0x748
0xd2dc4a10: at powerpc_interrupt+0x178
0xd2dc4a40: user SC trap by 0x100d71f8: srr1=0xf032
r1=0xd810 cr=0x82000280 xer=0 ctr=0x10173810 frame=0xd2dc4a48
KDB: enter: panic

/wrkdirs/usr/ports/devel/gdb/work-py37/gdb-9.1/gdb/inferior.c:283: 
internal-error: struct inferior *find_inferior_pid(int): Assertion `pid != 0' 
failed.





===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

On 2020-Jun-11, at 19:25, Justin Hibbits  wrote:

> On Thu, 11 Jun 2020 17:30:24 -0700
> Mark Millard  wrote:
> 
>> On 2020-Jun-11, at 16:49, Mark Millard  wrote:
>> 
>>> On 2020-Jun-11, at 14:42, Justin Hibbits 
>>> wrote:
>>> 
>>> On Thu, 11 Jun 2020 14:36:37 -0700
>>> Mark Millard  wrote:
>>> 
 On 2020-Jun-11, at 13:55, Justin Hibbits 
 wrote:
 
> On Wed, 10 Jun 2020 18:56:57 -0700
> Mark Millard  wrote:  
>>> . . .  
 
 
> That said, the attached patch effectively copies
> what's done in OEA6464 into OEA pmap.  Can you test it?
 
 I'll try it once I get a chance, probably later
 today.
 . . .  
>>> 
>>> No luck at the change being a fix, I'm afraid.
>>> 
>>> I verified that the build ended up with
>>> 
>>> 00926cb0  bl  008e8dc8 
>>> 00926cb4  mr  r27,r3
>>> 00926cb8  addir3,r3,36
>>> 00926cbc  hwsync
>>> 00926cc0  lwarx   r25,0,r3
>>> 00926cc4  li  r4,0
>>> 00926cc8  stwcx.  r4,0,r3
>>> 00926ccc  bne-00926cc0 
>>> 00926cd0  andi.   r3,r25,128
>>> 00926cd4  beq 00926ce0 
>>> 00926cd8  mr  r3,r27
>>> 00926cdc  bl  008e9874 
>>> 
>>> in the installed kernel. So I doubt a
>>> mis-build would be involved. It is a
>>> head -r360311 based context still. World is
>>> without MALLOC_PRODUCTION so that jemalloc
>>> code executes its asserts, catching more
>>> and earlier than otherwise.
>>> 
>>> First test . . .
>>> 
>>> The only thing that the witness kernel reported was:
>>> 
>>> Jun 11 15:58:16 FBSDG4S2 kernel: lock order reversal:
>>> Jun 11 15:58:16 FBSDG4S2 kernel:  1st 0x216fb00 Mountpoints (UMA
>>> zone) @ /usr/src/sys/vm/uma_core.c:4387 Jun 11 15:58:16 FBSDG4S2
>>> kernel:  2nd 0x1192d2c kernelpmap (kernelpmap) @
>>> /usr/src/sys/powerpc/aim/mmu_oea.c:1524 Jun 11 15:58:16 FBSDG4S2
>>> kernel: stack backtrace: Jun 11 15:58:16 FBSDG4S2 kernel: #0
>>> 0x5ec164 at witness_debugger+0x94 Jun 11 15:58:16 FBSDG4S2 kernel:
>>> #1 0x5ebe3c at witness_checkorder+0xb50 Jun 11 15:58:16 FBSDG4S2
>>> kernel: #2 0x536d5c at __mtx_lock_flags+0xcc Jun 11 15:58:16
>>> FBSDG4S2 kernel: #3 0x92636c at moea_kextract+0x5c Jun 11 15:58:16
>>> FBSDG4S2 kernel: #4 0x965d30 at pmap_kextract+0x98 Jun 11 15:58:16
>>> FBSDG4S2 kernel: #5 0x8bfdbc at zone_release+0xf0 Jun 11 15:58:16
>>> FBSDG4S2 kernel: #6 0x8c7854 at bucket_drain+0x2f0 Jun 11 15:58:16
>>> FBSDG4S2 kernel: #7 0x8c728c at bucket_free+0x54 Jun 11 15:58:16
>>> FBSDG4S2 kernel: #8 0x8c74fc at bucket_cache_reclaim+0x1bc Jun 11
>>> 15:58:16 FBSDG4S2 kernel: #9 0x8c7004 at zone_reclaim+0x128 Jun 11
>>> 15:58:16 FBSDG4S2 kernel: #10 0x8c3a40 at uma_reclaim+0x170 Jun 11
>>> 15:58:16 FBSDG4S2 kernel: #11 0x8c3f70 at uma_reclaim_worker+0x68
>>> Jun 11 15:58:16 FBSDG4S2 kernel: #12 0x50fbac at fork_exit+0xb0 Jun
>>> 11 15:58:16 FBSDG4S2 kernel: #13 0x9684ac at fork_trampoline+0xc
>>> 
>>> The processes that were hit were listed as:
>>> 
>>> Jun 11 15:59:11 FBSDG4S2 kernel: pid 971 (cron), jid 0, uid 0:
>>> exited on signal 11 (core dumped) Jun 11 16:02:59 FBSDG4S2 kernel:
>>> pid  (stress), jid 0, uid 0: exited on signal 6 (core dumped)
>>> Jun 11 16:03:27 FBSDG4S2 kernel: pid 871 (mountd), jid 0, uid 0:
>>> exited on signal 6 (core dumped) Jun 11 16:03:40 FBSDG4S2 kernel:
>>> pid 1065 (su), jid 0, uid 0: exited on signal 6 Jun 11 16:04:13
>>> FBSDG4S2 kernel: pid 1088 (su), jid 0, uid 0: exited on signal 6
>>> Jun 11 16:04:28 FBSDG4S2 kernel: pid 968 (sshd), jid 0, uid 0:
>>> exited on signal 6
>>> 
>>> Jun 11 16:05:42 FBSDG4S2 kernel: pid 1028 (login), jid 0, uid 0:
>>> exited on signal 6
>>> 
>>> Jun 11 16:05:46 FBSDG4S2 kernel: pid 873 (nfsd), jid 0, uid 0:
>>> exited on signal 6 (core dumped)
>>> 
>>> 
>>> Rebooting and rerunning and showing the stress output and such
>>> (I did not capture copies during the first test, but the first
>>> test had similar messages at the same sort of points):
>>> 
>>> Second test . . .
>>> 
>>> # stress -m 2 --vm-bytes 1700M
>>> stress: info: [1166] dispatching hogs: 0 cpu, 0 io, 2 vm, 0 hdd
>>> :
>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258:
>>> Failed assertion: "slab == extent_slab_get(extent)" :
>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258:
>>> Failed assertion: "slab == extent_slab_get(extent)" ^C
>>> 
>>> # exit
>>> :
>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:200:
>>> Failed assertion: "ret == sz_index2size_compute(index)" Abort trap
>>> 
>>> The other stuff was similar to to first test, not repeated here.  
>> 
>> The updated code looks odd to me for how "m" is
>> handled (part of a egrep to ensure I show all the
>> usage of m):
>> 
>> moea_protect(mmu_t mmu, pmap_t pm, vm_offset_t sva, vm_offset_t eva,
>>vm_page_t   m;
>>if (pm != kernel_pmap && m != NULL &&
>>(m->a.flags & PGA_EXECUTABLE) == 0 &&
>>if ((m->oflags & VPO_UNMANAGED) == 0)
>>  

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

On 2020-Jun-11, at 16:49, Mark Millard  wrote:

> On 2020-Jun-11, at 14:42, Justin Hibbits  wrote:
> 
> On Thu, 11 Jun 2020 14:36:37 -0700
> Mark Millard  wrote:
> 
>> On 2020-Jun-11, at 13:55, Justin Hibbits 
>> wrote:
>> 
>>> On Wed, 10 Jun 2020 18:56:57 -0700
>>> Mark Millard  wrote:
> . . .
>> 
>> 
>>> That said, the attached patch effectively copies
>>> what's done in OEA6464 into OEA pmap.  Can you test it?  
>> 
>> I'll try it once I get a chance, probably later
>> today.
>> . . .
> 
> No luck at the change being a fix, I'm afraid.
> 
> I verified that the build ended up with
> 
> 00926cb0  bl  008e8dc8 
> 00926cb4  mr  r27,r3
> 00926cb8  addir3,r3,36
> 00926cbc  hwsync
> 00926cc0  lwarx   r25,0,r3
> 00926cc4  li  r4,0
> 00926cc8  stwcx.  r4,0,r3
> 00926ccc  bne-00926cc0 
> 00926cd0  andi.   r3,r25,128
> 00926cd4  beq 00926ce0 
> 00926cd8  mr  r3,r27
> 00926cdc  bl  008e9874 
> 
> in the installed kernel. So I doubt a
> mis-build would be involved. It is a
> head -r360311 based context still. World is
> without MALLOC_PRODUCTION so that jemalloc
> code executes its asserts, catching more
> and earlier than otherwise.
> 
> First test . . .
> 
> The only thing that the witness kernel reported was:
> 
> Jun 11 15:58:16 FBSDG4S2 kernel: lock order reversal:
> Jun 11 15:58:16 FBSDG4S2 kernel:  1st 0x216fb00 Mountpoints (UMA zone) @ 
> /usr/src/sys/vm/uma_core.c:4387
> Jun 11 15:58:16 FBSDG4S2 kernel:  2nd 0x1192d2c kernelpmap (kernelpmap) @ 
> /usr/src/sys/powerpc/aim/mmu_oea.c:1524
> Jun 11 15:58:16 FBSDG4S2 kernel: stack backtrace:
> Jun 11 15:58:16 FBSDG4S2 kernel: #0 0x5ec164 at witness_debugger+0x94
> Jun 11 15:58:16 FBSDG4S2 kernel: #1 0x5ebe3c at witness_checkorder+0xb50
> Jun 11 15:58:16 FBSDG4S2 kernel: #2 0x536d5c at __mtx_lock_flags+0xcc
> Jun 11 15:58:16 FBSDG4S2 kernel: #3 0x92636c at moea_kextract+0x5c
> Jun 11 15:58:16 FBSDG4S2 kernel: #4 0x965d30 at pmap_kextract+0x98
> Jun 11 15:58:16 FBSDG4S2 kernel: #5 0x8bfdbc at zone_release+0xf0
> Jun 11 15:58:16 FBSDG4S2 kernel: #6 0x8c7854 at bucket_drain+0x2f0
> Jun 11 15:58:16 FBSDG4S2 kernel: #7 0x8c728c at bucket_free+0x54
> Jun 11 15:58:16 FBSDG4S2 kernel: #8 0x8c74fc at bucket_cache_reclaim+0x1bc
> Jun 11 15:58:16 FBSDG4S2 kernel: #9 0x8c7004 at zone_reclaim+0x128
> Jun 11 15:58:16 FBSDG4S2 kernel: #10 0x8c3a40 at uma_reclaim+0x170
> Jun 11 15:58:16 FBSDG4S2 kernel: #11 0x8c3f70 at uma_reclaim_worker+0x68
> Jun 11 15:58:16 FBSDG4S2 kernel: #12 0x50fbac at fork_exit+0xb0
> Jun 11 15:58:16 FBSDG4S2 kernel: #13 0x9684ac at fork_trampoline+0xc
> 
> The processes that were hit were listed as:
> 
> Jun 11 15:59:11 FBSDG4S2 kernel: pid 971 (cron), jid 0, uid 0: exited on 
> signal 11 (core dumped)
> Jun 11 16:02:59 FBSDG4S2 kernel: pid  (stress), jid 0, uid 0: exited on 
> signal 6 (core dumped)
> Jun 11 16:03:27 FBSDG4S2 kernel: pid 871 (mountd), jid 0, uid 0: exited on 
> signal 6 (core dumped)
> Jun 11 16:03:40 FBSDG4S2 kernel: pid 1065 (su), jid 0, uid 0: exited on 
> signal 6
> Jun 11 16:04:13 FBSDG4S2 kernel: pid 1088 (su), jid 0, uid 0: exited on 
> signal 6
> Jun 11 16:04:28 FBSDG4S2 kernel: pid 968 (sshd), jid 0, uid 0: exited on 
> signal 6
> 
> Jun 11 16:05:42 FBSDG4S2 kernel: pid 1028 (login), jid 0, uid 0: exited on 
> signal 6
> 
> Jun 11 16:05:46 FBSDG4S2 kernel: pid 873 (nfsd), jid 0, uid 0: exited on 
> signal 6 (core dumped)
> 
> 
> Rebooting and rerunning and showing the stress output and such
> (I did not capture copies during the first test, but the first
> test had similar messages at the same sort of points):
> 
> Second test . . .
> 
> # stress -m 2 --vm-bytes 1700M
> stress: info: [1166] dispatching hogs: 0 cpu, 0 io, 2 vm, 0 hdd
> : 
> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258: 
> Failed assertion: "slab == extent_slab_get(extent)"
> : 
> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258: 
> Failed assertion: "slab == extent_slab_get(extent)"
> ^C
> 
> # exit
> : /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:200: 
> Failed assertion: "ret == sz_index2size_compute(index)"
> Abort trap
> 
> The other stuff was similar to to first test, not repeated here.

The updated code looks odd to me for how "m" is
handled (part of a egrep to ensure I show all the
usage of m):

moea_protect(mmu_t mmu, pmap_t pm, vm_offset_t sva, vm_offset_t eva,
vm_page_t   m;
if (pm != kernel_pmap && m != NULL &&
(m->a.flags & PGA_EXECUTABLE) == 0 &&
if ((m->oflags & VPO_UNMANAGED) == 0)
vm_page_aflag_set(m, PGA_EXECUTABLE);
m = PHYS_TO_VM_PAGE(old_pte.pte_lo & PTE_RPGN);
refchg = 
atomic_readandclear_32(>md.mdpg_attrs);
vm_page_dirty(m);
vm_page_aflag_set(m, PGA_REFERENCED);

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

On 2020-Jun-11, at 14:42, Justin Hibbits  wrote:

On Thu, 11 Jun 2020 14:36:37 -0700
Mark Millard  wrote:

> On 2020-Jun-11, at 13:55, Justin Hibbits 
> wrote:
> 
>> On Wed, 10 Jun 2020 18:56:57 -0700
>> Mark Millard  wrote:
. . .
> 
> 
>> That said, the attached patch effectively copies
>> what's done in OEA6464 into OEA pmap.  Can you test it?  
> 
> I'll try it once I get a chance, probably later
> today.
> . . .

No luck at the change being a fix, I'm afraid.

I verified that the build ended up with

00926cb0  bl  008e8dc8 
00926cb4  mr  r27,r3
00926cb8  addir3,r3,36
00926cbc  hwsync
00926cc0  lwarx   r25,0,r3
00926cc4  li  r4,0
00926cc8  stwcx.  r4,0,r3
00926ccc  bne-00926cc0 
00926cd0  andi.   r3,r25,128
00926cd4  beq 00926ce0 
00926cd8  mr  r3,r27
00926cdc  bl  008e9874 

in the installed kernel. So I doubt a
mis-build would be involved. It is a
head -r360311 based context still. World is
without MALLOC_PRODUCTION so that jemalloc
code executes its asserts, catching more
and earlier than otherwise.

First test . . .

The only thing that the witness kernel reported was:

Jun 11 15:58:16 FBSDG4S2 kernel: lock order reversal:
Jun 11 15:58:16 FBSDG4S2 kernel:  1st 0x216fb00 Mountpoints (UMA zone) @ 
/usr/src/sys/vm/uma_core.c:4387
Jun 11 15:58:16 FBSDG4S2 kernel:  2nd 0x1192d2c kernelpmap (kernelpmap) @ 
/usr/src/sys/powerpc/aim/mmu_oea.c:1524
Jun 11 15:58:16 FBSDG4S2 kernel: stack backtrace:
Jun 11 15:58:16 FBSDG4S2 kernel: #0 0x5ec164 at witness_debugger+0x94
Jun 11 15:58:16 FBSDG4S2 kernel: #1 0x5ebe3c at witness_checkorder+0xb50
Jun 11 15:58:16 FBSDG4S2 kernel: #2 0x536d5c at __mtx_lock_flags+0xcc
Jun 11 15:58:16 FBSDG4S2 kernel: #3 0x92636c at moea_kextract+0x5c
Jun 11 15:58:16 FBSDG4S2 kernel: #4 0x965d30 at pmap_kextract+0x98
Jun 11 15:58:16 FBSDG4S2 kernel: #5 0x8bfdbc at zone_release+0xf0
Jun 11 15:58:16 FBSDG4S2 kernel: #6 0x8c7854 at bucket_drain+0x2f0
Jun 11 15:58:16 FBSDG4S2 kernel: #7 0x8c728c at bucket_free+0x54
Jun 11 15:58:16 FBSDG4S2 kernel: #8 0x8c74fc at bucket_cache_reclaim+0x1bc
Jun 11 15:58:16 FBSDG4S2 kernel: #9 0x8c7004 at zone_reclaim+0x128
Jun 11 15:58:16 FBSDG4S2 kernel: #10 0x8c3a40 at uma_reclaim+0x170
Jun 11 15:58:16 FBSDG4S2 kernel: #11 0x8c3f70 at uma_reclaim_worker+0x68
Jun 11 15:58:16 FBSDG4S2 kernel: #12 0x50fbac at fork_exit+0xb0
Jun 11 15:58:16 FBSDG4S2 kernel: #13 0x9684ac at fork_trampoline+0xc

The processes that were hit were listed as:

Jun 11 15:59:11 FBSDG4S2 kernel: pid 971 (cron), jid 0, uid 0: exited on signal 
11 (core dumped)
Jun 11 16:02:59 FBSDG4S2 kernel: pid  (stress), jid 0, uid 0: exited on 
signal 6 (core dumped)
Jun 11 16:03:27 FBSDG4S2 kernel: pid 871 (mountd), jid 0, uid 0: exited on 
signal 6 (core dumped)
Jun 11 16:03:40 FBSDG4S2 kernel: pid 1065 (su), jid 0, uid 0: exited on signal 6
Jun 11 16:04:13 FBSDG4S2 kernel: pid 1088 (su), jid 0, uid 0: exited on signal 6
Jun 11 16:04:28 FBSDG4S2 kernel: pid 968 (sshd), jid 0, uid 0: exited on signal 
6

Jun 11 16:05:42 FBSDG4S2 kernel: pid 1028 (login), jid 0, uid 0: exited on 
signal 6

Jun 11 16:05:46 FBSDG4S2 kernel: pid 873 (nfsd), jid 0, uid 0: exited on signal 
6 (core dumped)


Rebooting and rerunning and showing the stress output and such
(I did not capture copies during the first test, but the first
test had similar messages at the same sort of points):

Second test . . .

# stress -m 2 --vm-bytes 1700M
stress: info: [1166] dispatching hogs: 0 cpu, 0 io, 2 vm, 0 hdd
: 
/usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258: 
Failed assertion: "slab == extent_slab_get(extent)"
: 
/usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258: 
Failed assertion: "slab == extent_slab_get(extent)"
^C

# exit
: /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:200: 
Failed assertion: "ret == sz_index2size_compute(index)"
Abort trap

The other stuff was similar to to first test, not repeated here.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

On 2020-Jun-11, at 14:41, Brandon Bergren  wrote:

> An update from my end: I now have the ability to test dual processor G4 as 
> well, now that mine is up and running.

Cool.

FYI:

Dual processors are not required for the
problem to happen: the stress based testing
showed the problem just as easily on the
single-socket/single-core contexts that I
tried.

> On Thu, Jun 11, 2020, at 4:36 PM, Mark Millard wrote:
>> 
>> How did you test?
>> 
>> In my context it was far easier to see the problem
>> with builds that did not use MALLOC_PRODUCTION. In
>> other words: jemalloc having its asserts tested.
>> 
>> The easiest way I found to get the asserts to fail
>> was to do (multiple processes (-m) and totaling to
>> more than enough to force paging/swapping):
>> 
>> stress -m 2 --vm-bytes 1700M &
>> 
>> (Possibly setting up some shells first
>> to potentially later exit.)
>> 
>> Normally stress itself would hit jemalloc
>> asserts. Apparently the asserts did not
>> stop the code and it ran until a failure
>> occurred (via dtv=0x0). I never had to
>> manually stop the stress processes.
>> 
>> If no failures during, then exit shells
>> that likely were swapped out or partially
>> paged out during the stress run. They
>> hit jemalloc asserts during their cleanup
>> activity in my testing.
>> 
>> 
>>> That said, the attached patch effectively copies
>>> what's done in OEA6464 into OEA pmap.  Can you test it?
>> 
>> I'll try it once I get a chance, probably later
>> today.
>> 
>> I gather from what I see that moea64_protect did not
>> need the changes that you originally thought might
>> be required? I only see moea_protect changes in the
>> patch.
> 

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

On 2020-Jun-11, at 13:55, Justin Hibbits  wrote:

> On Wed, 10 Jun 2020 18:56:57 -0700
> Mark Millard  wrote:
> 
>> On 2020-May-13, at 08:56, Justin Hibbits  wrote:
>> 
>>> Hi Mark,  
>> 
>> Hello Justin.
> 
> Hi Mark,

Hello again, Justin.

>> 
>>> On Wed, 13 May 2020 01:43:23 -0700
>>> Mark Millard  wrote:
>>> 
 [I'm adding a reference to an old arm64/aarch64 bug that had
 pages turning to zero, in case this 32-bit powerpc issue is
 somewhat analogous.]
 
> . . .  
>>> ...  
 . . .
 
 (Note: dsl-only.net closed down, so the E-mail
 address reference is no longer valid.)
 
 Author: kib
 Date: Mon Apr 10 15:32:26 2017
 New Revision: 316679
 URL: 
 https://svnweb.freebsd.org/changeset/base/316679
 
 
 Log:
 Do not lose dirty bits for removing PROT_WRITE on arm64.
 
 Arm64 pmap interprets accessed writable ptes as modified, since
 ARMv8.0 does not track Dirty Bit Modifier in hardware. If writable
 bit is removed, page must be marked as dirty for MI VM.
 
 This change is most important for COW, where fork caused losing
 content of the dirty pages which were not yet scanned by
 pagedaemon.
 
 Reviewed by:   alc, andrew
 Reported and tested by:Mark Millard >>> dsl-only.net> PR:  217138, 217239
 Sponsored by:  The FreeBSD Foundation
 MFC after: 2 weeks
 
 Modified:
 head/sys/arm64/arm64/pmap.c
 
 Modified: head/sys/arm64/arm64/pmap.c
 ==
 --- head/sys/arm64/arm64/pmap.cMon Apr 10 12:35:58
 2017   (r316678) +++ head/sys/arm64/arm64/pmap.c   Mon
 Apr 10 15:32:26 2017   (r316679) @@ -2481,6 +2481,11 @@
 pmap_protect(pmap_t pmap, vm_offset_t sv sva += L3_SIZE) {
l3 = pmap_load(l3p);
if (pmap_l3_valid(l3)) {
 +  if ((l3 & ATTR_SW_MANAGED) &&
 +  pmap_page_dirty(l3)) {
 +
 vm_page_dirty(PHYS_TO_VM_PAGE(l3 &
 +  ~ATTR_MASK));
 +  }
pmap_set(l3p, ATTR_AP(ATTR_AP_RO));
PTE_SYNC(l3p);
/* XXX: Use pmap_invalidate_range
 */
 
 . . .
 
>>> 
>>> Thanks for this reference.  I took a quick look at the 3 pmap
>>> implementations we have (haven't check the new radix pmap yet), and
>>> it looks like only mmu_oea.c (32-bit AIM pmap, for G3 and G4) is
>>> missing vm_page_dirty() calls in its pmap_protect() implementation,
>>> analogous to the change you posted right above. Given this, I think
>>> it's safe to say that this missing piece is necessary.  We'll work
>>> on a fix for this; looking at moea64_protect(), there may be
>>> additional work needed to support this as well, so it may take a
>>> few days.  
>> 
>> Ping? Any clue when the above might happen?
>> 
>> I've been avoiding the old PowerMacs and leaving
>> them at head -r360311 , pending an update that
>> would avoid the kernel zeroing pages that it
>> should not zero. But I've seen that you were busy
>> with more modern contexts this last about a month.
>> 
>> And, clearly, my own context has left pending
>> (for much longer) other more involved activities
>> (compared to just periodically updating to
>> more recent FreeBSD vintages).
>> 
>> . . .
>> 
> 
> Sorry for the delay, I got sidetracked with a bunch of other
> development.

> I did install a newer FreeBSD on my dual G4 and couldn't
> see the problem.

How did you test?

In my context it was far easier to see the problem
with builds that did not use MALLOC_PRODUCTION. In
other words: jemalloc having its asserts tested.

The easiest way I found to get the asserts to fail
was to do (multiple processes (-m) and totaling to
more than enough to force paging/swapping):

stress -m 2 --vm-bytes 1700M &

(Possibly setting up some shells first
to potentially later exit.)

Normally stress itself would hit jemalloc
asserts. Apparently the asserts did not
stop the code and it ran until a failure
occurred (via dtv=0x0). I never had to
manually stop the stress processes.

If no failures during, then exit shells
that likely were swapped out or partially
paged out during the stress run. They
hit jemalloc asserts during their cleanup
activity in my testing.


> That said, the attached patch effectively copies
> what's done in OEA6464 into OEA pmap.  Can you test it?

I'll try it once I get a chance, probably later
today.

I gather from what I see that moea64_protect did not
need the changes that you originally thought might
be required? I only see moea_protect changes in the
patch.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

On 2020-May-13, at 08:56, Justin Hibbits  wrote:

> Hi Mark,

Hello Justin.

> On Wed, 13 May 2020 01:43:23 -0700
> Mark Millard  wrote:
> 
>> [I'm adding a reference to an old arm64/aarch64 bug that had
>> pages turning to zero, in case this 32-bit powerpc issue is
>> somewhat analogous.]
>> 
>>> . . .
> ...
>> . . .
>> 
>> (Note: dsl-only.net closed down, so the E-mail
>> address reference is no longer valid.)
>> 
>> Author: kib
>> Date: Mon Apr 10 15:32:26 2017
>> New Revision: 316679
>> URL: 
>> https://svnweb.freebsd.org/changeset/base/316679
>> 
>> 
>> Log:
>> Do not lose dirty bits for removing PROT_WRITE on arm64.
>> 
>> Arm64 pmap interprets accessed writable ptes as modified, since
>> ARMv8.0 does not track Dirty Bit Modifier in hardware. If writable
>> bit is removed, page must be marked as dirty for MI VM.
>> 
>> This change is most important for COW, where fork caused losing
>> content of the dirty pages which were not yet scanned by pagedaemon.
>> 
>> Reviewed by: alc, andrew
>> Reported and tested by:  Mark Millard 
>> PR:  217138, 217239
>> Sponsored by:The FreeBSD Foundation
>> MFC after:   2 weeks
>> 
>> Modified:
>> head/sys/arm64/arm64/pmap.c
>> 
>> Modified: head/sys/arm64/arm64/pmap.c
>> ==
>> --- head/sys/arm64/arm64/pmap.c  Mon Apr 10 12:35:58
>> 2017 (r316678) +++ head/sys/arm64/arm64/pmap.c   Mon Apr
>> 10 15:32:26 2017 (r316679) @@ -2481,6 +2481,11 @@
>> pmap_protect(pmap_t pmap, vm_offset_t sv sva += L3_SIZE) {
>>  l3 = pmap_load(l3p);
>>  if (pmap_l3_valid(l3)) {
>> +if ((l3 & ATTR_SW_MANAGED) &&
>> +pmap_page_dirty(l3)) {
>> +
>> vm_page_dirty(PHYS_TO_VM_PAGE(l3 &
>> +~ATTR_MASK));
>> +}
>>  pmap_set(l3p, ATTR_AP(ATTR_AP_RO));
>>  PTE_SYNC(l3p);
>>  /* XXX: Use pmap_invalidate_range */
>> 
>> . . .
>> 
> 
> Thanks for this reference.  I took a quick look at the 3 pmap
> implementations we have (haven't check the new radix pmap yet), and it
> looks like only mmu_oea.c (32-bit AIM pmap, for G3 and G4) is missing
> vm_page_dirty() calls in its pmap_protect() implementation, analogous
> to the change you posted right above. Given this, I think it's safe to
> say that this missing piece is necessary.  We'll work on a fix for
> this; looking at moea64_protect(), there may be additional work needed
> to support this as well, so it may take a few days.

Ping? Any clue when the above might happen?

I've been avoiding the old PowerMacs and leaving
them at head -r360311 , pending an update that
would avoid the kernel zeroing pages that it
should not zero. But I've seen that you were busy
with more modern contexts this last about a month.

And, clearly, my own context has left pending
(for much longer) other more involved activities
(compared to just periodically updating to
more recent FreeBSD vintages).

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r361775 - in head/sys: dts/arm64/overlays modules/dtb/rpi

[Mark Millard via svn-src-head]

Warner Losh imp at bsdimp.com wrote on
Thu Jun 4 02:47:14 UTC 2020 :

> On Wed, Jun 3, 2020, 6:28 PM Rodney W. Grimes 
> wrote:
> 
> > [ Charset UTF-8 unsupported, converting... ]
> > > Author: gonzo
> > > Date: Wed Jun  3 22:18:15 2020
> > > New Revision: 361775
> > > URL: https://svnweb.freebsd.org/changeset/base/361775
> > >
> > > Log:
> > >   Add spigen overlay for Raspberry Pi 4
> > >
> > >   Submitted by:   gergely.czuczy at harmless.hu
> > >
> > > Added:
> > >   head/sys/dts/arm64/overlays/spigen-rpi4.dtso   (contents, props
> > changed)
> > > Modified:
> > >   head/sys/modules/dtb/rpi/Makefile
> > >
> > > Added: head/sys/dts/arm64/overlays/spigen-rpi4.dtso
> > >
> > ==
> > > --- /dev/null 00:00:00 1970   (empty, because file is newly added)
> > > +++ head/sys/dts/arm64/overlays/spigen-rpi4.dtso  Wed Jun  3
> > 22:18:15 2020(r361775)
> > > @@ -0,0 +1,30 @@
> > > +/* $FreeBSD$ */
> >
> > This file needs some form of copyright/license.
> >
> 
> Dts files are like database files: they likely have no copyright
> protection.
> 

The below is not making claims about status-in-law.
It just notes example file content.


Cross checking a vague memory of what I'd seen
sometime in the past, using my head -r360311
context.

# egrep -ri "SPDX|Copyright" /usr/src/sys/gnu/dts/ | more

shows lots of Copyright notices and SPDX notices, including
a lot of "GPL-2.0", "GPL-2.0-only", "GPL-2.0+", and
"GPL-2.0-or-later" SPDX notices in the .dts , .dtsi  , and
.h files. It also showed some license lines with the word
"copyright" in the lines (whatever the capitalization
status).


By contrast,

# egrep -r "SPDX|Copyright" /usr/src/sys/dts/ | more

showed lots of Copyright notices but no SPDX notices.
It did show license lines with the word "copyright" in
the line as well.


For both egrep's, no *.dtso files were found as far as
I could tell.



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r361568 - head/sys/powerpc/aim

[Mark Millard via svn-src-head]

Justin Hibbits chmeeedalf at gmail.com wrote on
Thu May 28 02:41:06 UTC 2020 :

> On Thu, 28 May 2020 00:49:03 + (UTC)
> Brandon Bergren  wrote:
> 
> > Author: bdragon
> > Date: Thu May 28 00:49:02 2020
> > New Revision: 361568
> > URL: https://svnweb.freebsd.org/changeset/base/361568
> > 
> > Log:
> >   [PowerPC] Fix radix crash when passing -1 from userspace
> >   
> >   Found by running libc tests with radix enabled.
> >   
> >   Detect unsigned integer wrapping with a postcondition.
> >   
> >   Note: Radix MMU is not enabled by default yet.
> >   
> >   Sponsored by: Tag1 Consulting, Inc.
> > 
> > Modified:
> >   head/sys/powerpc/aim/mmu_radix.c
> > 
> > Modified: head/sys/powerpc/aim/mmu_radix.c
> > ==
> > --- head/sys/powerpc/aim/mmu_radix.cWed May 27 23:20:35
> > 2020(r361567) +++ head/sys/powerpc/aim/mmu_radix.c  Thu
> > May 28 00:49:02 2020(r361568) @@ -6000,7 +6000,8 @@
> > mmu_radix_kremove(vm_offset_t va) int mmu_radix_map_user_ptr(pmap_t
> > pm, volatile const void *uaddr, void **kaddr, size_t ulen, size_t
> > *klen) {
> > -   if ((uintptr_t)uaddr + ulen >= VM_MAXUSER_ADDRESS)
> > +   if ((uintptr_t)uaddr + ulen >= VM_MAXUSER_ADDRESS ||
> > +   (uintptr_t)uaddr + ulen < (uintptr_t)uaddr)
> > return (EFAULT);
> >  
> > *kaddr = (void *)(uintptr_t)uaddr;
> 
> Wouldn't
> 
> if ((uintptr_t)uaddr >= VM_MAXUSER_ADDRESS ||
> (uintptr_t)uaddr + ulen >= VM_MAXUSER_ADDRESS)
> 
> be more appropriate?

Using:

#define  VM_MAXUSER_ADDRESS320xf000

as an example for 32-bit AIM powerpc.

Let (uintptr_t)uaddr==0xe000u
Let ulen==0x3000u

Then (uintptr_t)uaddr+ulen == 0x1000u
(wrapped/truncated: "Detect unsigned integer wrapping")

So (right hand sides forced unsigned
by left hand sides being so):

(uintptr_t)uaddr= VM_MAXUSER_ADDRESS
||
(uintptr_t)uaddr >= (uintptr_t)VM_MAXUSER_ADDRESS - ulen


(I've left equality handling as it was, despite, for
example, 0xe000u with length 0x2000u having a last
address of 0xefffu and 0xefffu < 0xf000u .
There may be reasons to disallow that for all I know.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[I'm adding a reference to an old arm64/aarch64 bug that had
pages turning to zero, in case this 32-bit powerpc issue is
somewhat analogous.]

On 2020-May-13, at 00:29, Mark Millard  wrote:

> [stress alone is sufficient to have jemalloc asserts fail
> in stress, no need for a multi-socket G4 either. No need
> to involve nfsd, mountd, rpcbind or the like. This is not
> a claim that I know all the problems to be the same, just
> that a jemalloc reported failure in this simpler context
> happens and zeroed pages are involved.]
> 
> Reminder: head -r360311 based context.
> 
> 
> First I show a single CPU/core PowerMac G4 context failing
> in stress. (I actually did this later, but it is the
> simpler context.) I simply moved the media from the
> 2-socket G4 to this slower, single-cpu/core one.
> 
> cpu0: Motorola PowerPC 7400 revision 2.9, 466.42 MHz
> cpu0: Features 9c00
> cpu0: HID0 8094c0a4
> real memory  = 1577857024 (1504 MB)
> avail memory = 1527508992 (1456 MB)
> 
> # stress -m 1 --vm-bytes 1792M
> stress: info: [1024] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
> : 
> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258: 
> Failed assertion: "slab == extent_slab_get(extent)"
> stress: FAIL: [1024] (415) <-- worker 1025 got signal 6
> stress: WARN: [1024] (417) now reaping child worker processes
> stress: FAIL: [1024] (451) failed run completed in 243s
> 
> (Note: 1792 is the biggest it allowed with M.)
> 
> The following still pages in and out and fails:
> 
> # stress -m 1 --vm-bytes 1290M
> stress: info: [1163] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
> : 
> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258: 
> Failed assertion: "slab == extent_slab_get(extent)"
> . . .
> 
> By contrast, the following had no problem for as
> long as I let it run --and did not page in or out:
> 
> # stress -m 1 --vm-bytes 1280M
> stress: info: [1181] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
> 
> 
> 
> 
> The 2 socket PowerMac G4 context with 2048 MiByte of RAM . . .
> 
> stress -m 1 --vm-bytes 1792M
> 
> did not (quickly?) fail or page. 1792
> is as large as it would allow with M.
> 
> The following also did not (quickly?) fail
> (and were not paging):
> 
> stress -m 2 --vm-bytes 896M
> stress -m 4 --vm-bytes 448M
> stress -m 8 --vm-bytes 224M
> 
> (Only 1 example was run at a time.)
> 
> But the following all did quickly fail (and were
> paging):
> 
> stress -m 8 --vm-bytes 225M
> stress -m 4 --vm-bytes 449M
> stress -m 2 --vm-bytes 897M
> 
> (Only 1 example was run at a time.)
> 
> I'll note that when I exited an su process
> I ended up with a:
> 
> : /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:200: 
> Failed assertion: "ret == sz_index2size_compute(index)"
> Abort trap (core dumped)
> 
> and a matching su.core file. It appears
> that stress's activity leads to other
> processes also seeing examples of the
> zeroed-page(s) problem (probably su had
> paged some or had been fully swapped
> out):
> 
> (gdb) bt
> #0  thr_kill () at thr_kill.S:4
> #1  0x503821d0 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52
> #2  0x502e1d20 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
> #3  0x502d6144 in sz_index2size_lookup (index=) at 
> /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:200
> #4  sz_index2size (index=) at 
> /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:207
> #5  ifree (tsd=0x5008b018, ptr=0x50041460, tcache=0x5008b138, 
> slow_path=) at jemalloc_jemalloc.c:2583
> #6  0x502d5cec in __je_free_default (ptr=0x50041460) at 
> jemalloc_jemalloc.c:2784
> #7  0x502d62d4 in __free (ptr=0x50041460) at jemalloc_jemalloc.c:2852
> #8  0x501050cc in openpam_destroy_chain (chain=0x50041480) at 
> /usr/src/contrib/openpam/lib/libpam/openpam_load.c:113
> #9  0x50105094 in openpam_destroy_chain (chain=0x500413c0) at 
> /usr/src/contrib/openpam/lib/libpam/openpam_load.c:111
> #10 0x50105094 in openpam_destroy_chain (chain=0x50041320) at 
> /usr/src/contrib/openpam/lib/libpam/openpam_load.c:111
> #11 0x50105094 in openpam_destroy_chain (chain=0x50041220) at 
> /usr/src/contrib/openpam/lib/libpam/openpam_load.c:111
> #12 0x50105094 in openpam_destroy_chain (chain=0x50041120) at 
> /usr/src/contrib/openpam/lib/libpam/openpam_load.c:111
> #13 0x50105094 in openpam_destroy_chain (chain=0x50041100) at 
> /usr/src/contrib/openpam/lib/libpam/openpam_load.c:111
> #14 0x50105014 in openpam_clear_chains (policy=0x5064) at 
> /usr/src/contrib/openpam/lib/libpam/openpam_load.c:130
> #15 0x50101230 in pam_end (pamh=0x5060, status=) at 
> /usr/src/contrib/openpam/lib/libpam/pam_end.c:83
> #16 0x1001225c in main (argc=, argv=0x0) at 
> /usr/src/usr.bin/su/su.c:477
> 
> (gdb) print/x __je_sz_size2index_tab
> $1 = {0x0 }
> 
> 
> Notes:
> 
> Given that the original problem did not involve
> paging to the swap partition, may be just making
> it to the Laundry list or some such is sufficient,
> something that is also involved when the swap
> space is 

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[stress alone is sufficient to have jemalloc asserts fail
in stress, no need for a multi-socket G4 either. No need
to involve nfsd, mountd, rpcbind or the like. This is not
a claim that I know all the problems to be the same, just
that a jemalloc reported failure in this simpler context
happens and zeroed pages are involved.]

Reminder: head -r360311 based context.


First I show a single CPU/core PowerMac G4 context failing
in stress. (I actually did this later, but it is the
simpler context.) I simply moved the media from the
2-socket G4 to this slower, single-cpu/core one.

cpu0: Motorola PowerPC 7400 revision 2.9, 466.42 MHz
cpu0: Features 9c00
cpu0: HID0 8094c0a4
real memory  = 1577857024 (1504 MB)
avail memory = 1527508992 (1456 MB)

# stress -m 1 --vm-bytes 1792M
stress: info: [1024] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
: 
/usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258: 
Failed assertion: "slab == extent_slab_get(extent)"
stress: FAIL: [1024] (415) <-- worker 1025 got signal 6
stress: WARN: [1024] (417) now reaping child worker processes
stress: FAIL: [1024] (451) failed run completed in 243s

(Note: 1792 is the biggest it allowed with M.)

The following still pages in and out and fails:

# stress -m 1 --vm-bytes 1290M
stress: info: [1163] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
: 
/usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258: 
Failed assertion: "slab == extent_slab_get(extent)"
. . .

By contrast, the following had no problem for as
long as I let it run --and did not page in or out:

# stress -m 1 --vm-bytes 1280M
stress: info: [1181] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd




The 2 socket PowerMac G4 context with 2048 MiByte of RAM . . .

stress -m 1 --vm-bytes 1792M

did not (quickly?) fail or page. 1792
is as large as it would allow with M.

The following also did not (quickly?) fail
(and were not paging):

stress -m 2 --vm-bytes 896M
stress -m 4 --vm-bytes 448M
stress -m 8 --vm-bytes 224M

(Only 1 example was run at a time.)

But the following all did quickly fail (and were
paging):

stress -m 8 --vm-bytes 225M
stress -m 4 --vm-bytes 449M
stress -m 2 --vm-bytes 897M

(Only 1 example was run at a time.)

I'll note that when I exited an su process
I ended up with a:

: /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:200: 
Failed assertion: "ret == sz_index2size_compute(index)"
Abort trap (core dumped)

and a matching su.core file. It appears
that stress's activity leads to other
processes also seeing examples of the
zeroed-page(s) problem (probably su had
paged some or had been fully swapped
out):

(gdb) bt
#0  thr_kill () at thr_kill.S:4
#1  0x503821d0 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52
#2  0x502e1d20 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
#3  0x502d6144 in sz_index2size_lookup (index=) at 
/usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:200
#4  sz_index2size (index=) at 
/usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:207
#5  ifree (tsd=0x5008b018, ptr=0x50041460, tcache=0x5008b138, 
slow_path=) at jemalloc_jemalloc.c:2583
#6  0x502d5cec in __je_free_default (ptr=0x50041460) at jemalloc_jemalloc.c:2784
#7  0x502d62d4 in __free (ptr=0x50041460) at jemalloc_jemalloc.c:2852
#8  0x501050cc in openpam_destroy_chain (chain=0x50041480) at 
/usr/src/contrib/openpam/lib/libpam/openpam_load.c:113
#9  0x50105094 in openpam_destroy_chain (chain=0x500413c0) at 
/usr/src/contrib/openpam/lib/libpam/openpam_load.c:111
#10 0x50105094 in openpam_destroy_chain (chain=0x50041320) at 
/usr/src/contrib/openpam/lib/libpam/openpam_load.c:111
#11 0x50105094 in openpam_destroy_chain (chain=0x50041220) at 
/usr/src/contrib/openpam/lib/libpam/openpam_load.c:111
#12 0x50105094 in openpam_destroy_chain (chain=0x50041120) at 
/usr/src/contrib/openpam/lib/libpam/openpam_load.c:111
#13 0x50105094 in openpam_destroy_chain (chain=0x50041100) at 
/usr/src/contrib/openpam/lib/libpam/openpam_load.c:111
#14 0x50105014 in openpam_clear_chains (policy=0x5064) at 
/usr/src/contrib/openpam/lib/libpam/openpam_load.c:130
#15 0x50101230 in pam_end (pamh=0x5060, status=) at 
/usr/src/contrib/openpam/lib/libpam/pam_end.c:83
#16 0x1001225c in main (argc=, argv=0x0) at 
/usr/src/usr.bin/su/su.c:477

(gdb) print/x __je_sz_size2index_tab
$1 = {0x0 }


Notes:

Given that the original problem did not involve
paging to the swap partition, may be just making
it to the Laundry list or some such is sufficient,
something that is also involved when the swap
space is partially in use (according to top). Or
sitting in the inactive list for a long time, if
that has some special status.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[Yet another new kind of experiment. But this looks
like I can cause problems in fairly sort order on
demand now. Finally! And with that I've much better
evidence for kernel vs. user-space process for making
the zeroed memory appear in, for example, nfsd.]

I've managed to get:

: 
/usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258: 
Failed assertion: "slab == extent_slab_get(extent)"
: 
/usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:258: 
Failed assertion: "slab == extent_slab_get(extent)"

and eventually:

[1]   Segmentation fault (core dumped) stress -m 2 --vm-bytes 1700M

from a user program (stress) while another machine was attempted an nfs
mount during the stress activity:

# mount -onoatime,soft ...:/ /mnt && umount /mnt && rpcinfo -s ...
[tcp] ...:/: RPCPROG_MNT: RPC: Timed out

(To get failure I may have to run the commands
multiple times. Timing details against stress's
activity seem to matter.)

That failure lead to:

# ls -ldT /*.core*
-rw---  1 root  wheel  3899392 May 12 19:52:26 2020 /mountd.core

# ls -ldT *.core*
-rw---  1 root  wheel  2682880 May 12 20:00:26 2020 stress.core

(Note which of nfsd, mountd, or rpcbind need not be
fully repeatable. stress.core seems to be written twice,
probably because of the -m 2 in use.)

The context that let me do this was to first (on the 2 socket
G4 with a full 2048 MiBYte RAM configuration):

stress -m 2 --vm-bytes 1700M &

Note that the stress command keeps the memory busy
and causes paging to the swap/page space. I've not
tried to make it just fit without paging or just
barely paging or such. The original context did not
involve paging or low RAM, so I do not expect paging
to be required but can be involved.

The stress program backtrace is different:

4827return (tls_get_addr_slow(dtvp, index, offset));
4828}
(gdb) bt -full
#0  0x41831b04 in tls_get_addr_common (dtvp=0x4186c010, index=2, 
offset=4294937444) at /usr/src/libexec/rtld-elf/rtld.c:4824
dtv = 0x0
#1  0x4182bfcc in __tls_get_addr (ti=) at 
/usr/src/libexec/rtld-elf/powerpc/reloc.c:848
tp = 
p = 
#2  0x41a83464 in __get_locale () at 
/usr/src/lib/libc/locale/xlocale_private.h:199
No locals.
#3  fprintf (fp=0x41b355f8, fmt=0x1804cbc "%s: FAIL: [%lli] (%d) ") at 
/usr/src/lib/libc/stdio/fprintf.c:57
ap = {{gpr = 2 '\002', fpr = 0 '\000', reserved = 20731, 
overflow_arg_area = 0xdb78, reg_save_area = 0xdae8}}
ret = 
#4  0x01802348 in main (argc=, argv=) at 
stress.c:415
status = 
ret = 6
do_dryrun = 0
retval = 0
children = 1
do_backoff = 
do_hdd_bytes = 
do_hdd = 
do_vm_keep = 0
do_vm_hang = -1
do_vm_stride = 4096
do_vm_bytes = 1782579200
do_vm = 108174317627375616
do_io = 
do_cpu = 
do_timeout = 108176117243859333
starttime = 1589338322
i = 
forks = 
pid = 6140
stoptime = 
runtime = 

Apparently the asserts did not stop the code
and it ran until a failure occurred (via
dtv=0x0).

Stress uses a mutex stored on a page that gets
the "turns into zeros" problem, preventing
the mprotect(ADDR,1,1) type of test because
stress will write on the page. (I've not tried
to find a minimal form of stress run.)

The the same sort of globals are again
zeroed, such as:

(gdb) print/x __je_sz_size2index_tab
$1 = {0x0 }


Another attempt lost rpcbind instead instead of
mountd:

# ls -ldT /*.core
-rw---  1 root  wheel  3899392 May 12 19:52:26 2020 /mountd.core
-rw---  1 root  wheel  3170304 May 12 20:03:00 2020 /rpcbind.core


I again find that when I use gdb 3 times
to:

attach ???
x/x __je_sz_size2index_tab
print (int)mprotext(ADDRESS,1,1)
quit

for each of rpcbind, mountd, and nfsd master that
those processes no longer fail during the
mount/umount/rpcinfo (or are far less likely to).

But it turns out that later when I "service nfsd
stop" nfsd does get the zeroed Memory based assert
and core dumps. (I'd done a bunch of the mount/umount/
rpcinfo sequences before the stop.)

That the failure is during SIGUSR1 based shutdown,
leads me to wonder if killing off some child
process(es) is involved in the problem.

There was *no* evidence of a signal for an attempt
to write the page from the user process. It appears
that the kernel is doing something that changes what
the process sees --instead of the user-space programs
stomping on its own memory content.

I've no clue how to track down the kernel activity
that changes what the process sees on some page(s)
of memory.

(Prior testing with a debug kernel did not report
problems, despite getting an example failure. So
that seems insufficient.)

At least a procedure is now known that does not
involved waiting hours or days.


The procedure (adjusted for how much RAM is present
and number of cpus/cores?) could be appropriate to
run in other contexts than the 32-bit powerpc 

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[A new kind of experiment and partial results.]

Given the zero'ed memory page(s) that for some of
the example contexts include a page that should not
be changing after initialization in my context
(jemalloc global variables), I have attempted the
following for such examples:

A) Run gdb
B) Attach to one of the live example processes.
C) Check that the page is not zeroed yet.
   (print/x __je_sz_size2index_tab)
D) protect the page containing the start
   of __je_sz_size2index_tab, using 0x1
   as the PROT_READ mask.
   (print (int)mprotect(ADDRESS,1,0x1))
E) detach.

The hope was to discover which of the following
was involved:

A) user-space code trying to write the page should
   get a SIGSEGV. In this case I'd likely be able
   to see what code was attempting the write.

B) kernel-code doing something odd to the content
   or mapping of memory would not (or need not)
   lead to SIGSEGV. In this case I'd be unlikely
   to see what code lead to the zeros on the page.

So far I've gotten only one failure example, nfsd
during its handling of a SIGUSR1. Previous nfs
mounts and dismounts worked fine, not asserting,
indicating that at the time the page was not
zeroed.

I got no evidence of SIGSEGV from an attempted user
space write to the page. But the nfsd.core shows the
page as zeroed and the assert having caused abort().
That suggests the kernel side of things for what
leads to the zeros.

It turns out that just before the "unregsiteration()"
activity is "killchildren()" activity:

(gdb) list
971 
972 static void
973 nfsd_exit(int status)
974 {
975 killchildren();
976 unregistration();
977 exit(status);
978 }

(frame #12) used via:

(gdb) list cleanup
954 /*
955  * Cleanup master after SIGUSR1.
956  */
957 static void
958 cleanup(__unused int signo)
959 {
960 nfsd_exit(0);
961 }
. . .

and (for master):

(void)signal(SIGUSR1, cleanup);

This suggests the possibility that the zero'd
pages could be associated with killing the
child processes. (I've had a past aarch64
context where forking had problems with
pages that were initially common to parent
and child processes. In that context having
the processes swap out [not just mostly
paged out] and then swap back in was
involved in showing the problem. The issue
was fixed and was aarch64 specific. But it
leaves me willing to consider fork-related
memory management as possibly odd in some
way for 32-bit powerpc.)


Notes . . .

Another possible kind of evidence: I've gone far
longer with the machine doing just normal background
processing with nothing failing on its own. This
suggests that the (int)mprotect(ADDRESS,1,0x1) might
be changing the context --or just doing the attach
and detach in gdb does. I've nothing solid in this
area so I'll ignore it, other than this note.



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[I caused nfsd to having things shifted in mmeory some to
see it it tracked content vs. page boundary for where the
zeros stop. Non-nfsd examples omitted.]

> . . .
>> nfsd hit an assert, failing ret == sz_size2index_compute(size)
> 
> [Correction: That should have referenced sz_index2size_lookup(index).]
> 
>> (also, but a different caller of sz_size2index):
> 
> [Correction: The "also" comment should be ignored:
> sz_index2size_lookup(index) is referenced below.]
> 
>> 
>> (gdb) bt
>> #0  thr_kill () at thr_kill.S:4
>> #1  0x502b2170 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52
>> #2  0x50211cc0 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
>> #3  0x50206104 in sz_index2size_lookup (index=) at 
>> /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:200
>> #4  sz_index2size (index=) at 
>> /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:207
>> #5  ifree (tsd=0x50094018, ptr=0x50041028, tcache=0x50094138, 
>> slow_path=) at jemalloc_jemalloc.c:2583
>> #6  0x50205cac in __je_free_default (ptr=0x50041028) at 
>> jemalloc_jemalloc.c:2784
>> #7  0x50206294 in __free (ptr=0x50041028) at jemalloc_jemalloc.c:2852
>> #8  0x50287ec8 in ns_src_free (src=0x50329004, srclistsize=) 
>> at /usr/src/lib/libc/net/nsdispatch.c:452
>> #9  ns_dbt_free (dbt=0x50329000) at /usr/src/lib/libc/net/nsdispatch.c:436
>> #10 vector_free (vec=0x50329000, count=, esize=12, 
>> free_elem=) at /usr/src/lib/libc/net/nsdispatch.c:253
>> #11 nss_atexit () at /usr/src/lib/libc/net/nsdispatch.c:578
>> #12 0x5028d958 in __cxa_finalize (dso=0x0) at 
>> /usr/src/lib/libc/stdlib/atexit.c:240
>> #13 0x502117f8 in exit (status=0) at /usr/src/lib/libc/stdlib/exit.c:74
>> #14 0x10013f9c in child_cleanup (signo=) at 
>> /usr/src/usr.sbin/nfsd/nfsd.c:969
>> #15 
>> #16 0x in ?? ()
>> 
>> (gdb) up 3
>> #3  0x50206104 in sz_index2size_lookup (index=) at 
>> /usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:200
>> 200  assert(ret == sz_index2size_compute(index));
>> 
>> (ret is optimized out.)
>> 
>> 197  JEMALLOC_ALWAYS_INLINE size_t
>> 198  sz_index2size_lookup(szind_t index) {
>> 199  size_t ret = (size_t)sz_index2size_tab[index];
>> 200  assert(ret == sz_index2size_compute(index));
>> 201  return ret;
>> 202  }
> 
> (gdb) print/x __je_sz_index2size_tab
> $3 = {0x0 }
> 
> Also:
> 
> (gdb) x/4x __je_arenas+16368/4
> 0x5030cab0 <__je_arenas+16368>:   0x  0x  
> 0x  0x
> (gdb) print/x __je_arenas_lock
>  
> $8 = {{{prof_data = {tot_wait_time = {ns = 0x0}, max_wait_time = {ns = 0x0}, 
> n_wait_times = 0x0, n_spin_acquired = 0x0, max_n_thds = 0x0, n_waiting_thds = 
> {repr = 0x0}, n_owner_switches = 0x0, 
>   prev_owner = 0x0, n_lock_ops = 0x0}, lock = 0x0, postponed_next = 0x0, 
> locked = {repr = 0x0}}}, witness = {name = 0x0, rank = 0x0, comp = 0x0, 
> opaque = 0x0, link = {qre_next = 0x0, 
> qre_prev = 0x0}}, lock_order = 0x0}
> (gdb) print/x __je_narenas_auto
> $9 = 0x0
> (gdb) print/x malloc_conf  
> $10 = 0x0
> (gdb) print/x __je_ncpus 
> $11 = 0x0
> (gdb) print/x __je_manual_arena_base
> $12 = 0x0
> (gdb) print/x __je_sz_pind2sz_tab   
> $13 = {0x0 }
> (gdb) print/x __je_sz_size2index_tab
> $1 = {0x0 , 0x1a, 0x1b , 0x1c  64 times>}
> 
>> Booting and immediately trying something like:
>> 
>> service nfsd stop
>> 
>> did not lead to a failure. But may be after
>> a while it would and be less drastic than a
>> reboot or power down.
> 
> More detail:
> 
> So, for rpcbind and nfds at some point a large part of
> __je_sz_size2index_tab is being stomped on, as is all of
> __je_sz_index2size_tab and more.
> 
> . . .
> 
> For nfsd, it is similar (again showing the partially
> non-zero live process context instead of the all-zeros
> from the .core file):
> 
> 0x5030cab0 <__je_arenas+16368>:   0x  0x  
> 0x  0x0009
> 0x5030cac0 <__je_arenas_lock>:0x  0x  
> 0x  0x
> 0x5030cad0 <__je_arenas_lock+16>: 0x  0x  
> 0x  0x
> 0x5030cae0 <__je_arenas_lock+32>: 0x  0x  
> 0x  0x
> 0x5030caf0 <__je_arenas_lock+48>: 0x  0x  
> 0x  0x
> 0x5030cb00 <__je_arenas_lock+64>: 0x  0x502ff070  
> 0x  0x
> 0x5030cb10 <__je_arenas_lock+80>: 0x500ebb04  0x0003  
> 0x  0x
> 0x5030cb20 <__je_arenas_lock+96>: 0x5030cb10  0x5030cb10  
> 0x  0x
> 
> Then the memory in the crash continues to be zero until:
> 
> 0x5030d000 <__je_sz_size2index_tab+384>:  0x1a1b1b1b  0x1b1b1b1b  
> 0x1b1b1b1b  0x1b1b1b1b
> 
> Notice the interesting page boundary for where non-zero
> is first available again!

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[More details for a sshd failure. The other
examples are omitted. The sshd failure also shows
a all-zeros-up-to-a-page-boundary issue, just for
a different address range.]

On 2020-May-7, at 12:06, Mark Millard  wrote:
> 
> [mountd failure example: also at sz_size2index_lookup assert
> for the same zero'd memory problem.]
> 
>> On 2020-May-7, at 00:46, Mark Millard  wrote:
>> 
>> [__je_sz_size2index_tab seems messed up in 2 of the
>> asserting contexts: first 384 are zero in both. More
>> before that is also messed up (all zero). I show the
>> details later below.]
>> 
>> On 2020-May-6, at 16:57, Mark Millard  wrote:
>> 
>>> [This explores process crashes that happen during system
>>> shutdown, in a context not having MALLOC_PRODUCTION= .
>>> So assert failures are reported as the stopping points.]
>>> 
>>> It looks like shutdown -p now, shutdown -r now, and the
>>> like can lead some processes to assert during their exit
>>> attempt, including a sshd failure (that I've not seen
>>> before), rpcbind, and nfsd. I show information about the
>>> observed asserts for those below.
>>> 
>>> 
>>> sshd hit an assert, failing slab == extent_slab_get(extent) :
>>> 
>>> (gdb) bt 
>>> #0  thr_kill () at thr_kill.S:4
>>> #1  0x50927170 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52
>>> #2  0x50886cc0 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
>>> #3  0x508834b0 in arena_dalloc (tsdn=, ptr=, 
>>> tcache=, alloc_ctx=, slow_path=>> out>)
>>>  at 
>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:315
>>> #4  idalloctm (tsdn=0x500dd040, ptr=0x5008a180, tcache=0x500dd160, 
>>> alloc_ctx=, is_internal=, 
>>> slow_path=)
>>>  at 
>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal_inlines_c.h:118
>>> #5  0x5087b0a4 in ifree (tsd=0x500dd040, ptr=0x5008a180, tcache=0x500dd160, 
>>> slow_path=) at jemalloc_jemalloc.c:2590
>>> #6  0x5087acac in __je_free_default (ptr=0x5008a180) at 
>>> jemalloc_jemalloc.c:2784
>>> #7  0x5087b294 in __free (ptr=0x5008a180) at jemalloc_jemalloc.c:2852
>>> #8  0x10029464 in server_accept_loop (config_s=, 
>>> sock_in=, sock_out=, newsock=) 
>>> at /usr/src/crypto/openssh/sshd.c:1185
>>> #9  main (ac=, av=0xde3c) at 
>>> /usr/src/crypto/openssh/sshd.c:2009
>>> 
>>> . . .
>>> (gdb) up
>>> #2  0x50886cc0 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
>>> 67  (void)raise(SIGABRT);
>>> (gdb) up
>>> #3  0x508834b0 in arena_dalloc (tsdn=, ptr=, 
>>> tcache=, alloc_ctx=, slow_path=>> out>)
>>>  at 
>>> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:315
>>> 315 assert(slab == extent_slab_get(extent));
>>> 
>>> (gdb) list
>>> 310 rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
>>> 311 extent_t *extent = rtree_extent_read(tsdn, 
>>> _rtree,
>>> 312 rtree_ctx, (uintptr_t)ptr, true);
>>> 313 assert(szind == extent_szind_get(extent));
>>> 314 assert(szind < SC_NSIZES);
>>> 315 assert(slab == extent_slab_get(extent));
>>> 316 }
>>> 317 
>>> 318 if (likely(slab)) {
>>> 319 /* Small allocation. */
>>> 
>>> More fully:
>>> 
>>> 285 JEMALLOC_ALWAYS_INLINE void
>>> 286 arena_dalloc(tsdn_t *tsdn, void *ptr, tcache_t *tcache,
>>> 287 alloc_ctx_t *alloc_ctx, bool slow_path) {
>>> 288 assert(!tsdn_null(tsdn) || tcache == NULL);
>>> 289 assert(ptr != NULL);
>>> 290 
>>> 291 if (unlikely(tcache == NULL)) {
>>> 292 arena_dalloc_no_tcache(tsdn, ptr);
>>> 293 return;
>>> 294 }
>>> 295 
>>> 296 szind_t szind;
>>> 297 bool slab;
>>> 298 rtree_ctx_t *rtree_ctx;
>>> 299 if (alloc_ctx != NULL) {
>>> 300 szind = alloc_ctx->szind;
>>> 301 slab = alloc_ctx->slab;
>>> 302 assert(szind != SC_NSIZES);
>>> 303 } else {
>>> 304 rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
>>> 305 rtree_szind_slab_read(tsdn, _rtree, rtree_ctx,
>>> 306 (uintptr_t)ptr, true, , );
>>> 307 }
>>> 308 
>>> 309 if (config_debug) {
>>> 310 rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
>>> 311 extent_t *extent = rtree_extent_read(tsdn, 
>>> _rtree,
>>> 312 rtree_ctx, (uintptr_t)ptr, true);
>>> 313 assert(szind == extent_szind_get(extent));
>>> 314 assert(szind < SC_NSIZES);
>>> 315 assert(slab == extent_slab_get(extent));
>>> 316 }
>>> 317 
>>> 318 if (likely(slab)) {
>>> 319 /* Small allocation. */
>>> 320 tcache_dalloc_small(tsdn_tsd(tsdn), tcache, ptr, szind,
>>> 321 slow_path);
>>> 322 } else {
>>> 323 arena_dalloc_large(tsdn, ptr, tcache, szind, slow_path);
>>> 324 }
>>> 325 }
>> 
>> . . .

The machine code for:

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[mountd failure example: also at sz_size2index_lookup assert
for the same zero'd memory problem.]

> On 2020-May-7, at 00:46, Mark Millard  wrote:
> 
> [__je_sz_size2index_tab seems messed up in 2 of the
> asserting contexts: first 384 are zero in both. More
> before that is also messed up (all zero). I show the
> details later below.]
> 
> On 2020-May-6, at 16:57, Mark Millard  wrote:
> 
>> [This explores process crashes that happen during system
>> shutdown, in a context not having MALLOC_PRODUCTION= .
>> So assert failures are reported as the stopping points.]
>> 
>> It looks like shutdown -p now, shutdown -r now, and the
>> like can lead some processes to assert during their exit
>> attempt, including a sshd failure (that I've not seen
>> before), rpcbind, and nfsd. I show information about the
>> observed asserts for those below.
>> 
>> 
>> sshd hit an assert, failing slab == extent_slab_get(extent) :
>> 
>> (gdb) bt 
>> #0  thr_kill () at thr_kill.S:4
>> #1  0x50927170 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52
>> #2  0x50886cc0 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
>> #3  0x508834b0 in arena_dalloc (tsdn=, ptr=, 
>> tcache=, alloc_ctx=, slow_path=)
>>   at 
>> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:315
>> #4  idalloctm (tsdn=0x500dd040, ptr=0x5008a180, tcache=0x500dd160, 
>> alloc_ctx=, is_internal=, slow_path=> out>)
>>   at 
>> /usr/src/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal_inlines_c.h:118
>> #5  0x5087b0a4 in ifree (tsd=0x500dd040, ptr=0x5008a180, tcache=0x500dd160, 
>> slow_path=) at jemalloc_jemalloc.c:2590
>> #6  0x5087acac in __je_free_default (ptr=0x5008a180) at 
>> jemalloc_jemalloc.c:2784
>> #7  0x5087b294 in __free (ptr=0x5008a180) at jemalloc_jemalloc.c:2852
>> #8  0x10029464 in server_accept_loop (config_s=, 
>> sock_in=, sock_out=, newsock=) 
>> at /usr/src/crypto/openssh/sshd.c:1185
>> #9  main (ac=, av=0xde3c) at 
>> /usr/src/crypto/openssh/sshd.c:2009
>> 
>> . . .
>> (gdb) up
>> #2  0x50886cc0 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
>> 67   (void)raise(SIGABRT);
>> (gdb) up
>> #3  0x508834b0 in arena_dalloc (tsdn=, ptr=, 
>> tcache=, alloc_ctx=, slow_path=)
>>   at 
>> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:315
>> 315  assert(slab == extent_slab_get(extent));
>> 
>> (gdb) list
>> 310  rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
>> 311  extent_t *extent = rtree_extent_read(tsdn, 
>> _rtree,
>> 312  rtree_ctx, (uintptr_t)ptr, true);
>> 313  assert(szind == extent_szind_get(extent));
>> 314  assert(szind < SC_NSIZES);
>> 315  assert(slab == extent_slab_get(extent));
>> 316  }
>> 317  
>> 318  if (likely(slab)) {
>> 319  /* Small allocation. */
>> 
>> More fully:
>> 
>> 285  JEMALLOC_ALWAYS_INLINE void
>> 286  arena_dalloc(tsdn_t *tsdn, void *ptr, tcache_t *tcache,
>> 287  alloc_ctx_t *alloc_ctx, bool slow_path) {
>> 288  assert(!tsdn_null(tsdn) || tcache == NULL);
>> 289  assert(ptr != NULL);
>> 290  
>> 291  if (unlikely(tcache == NULL)) {
>> 292  arena_dalloc_no_tcache(tsdn, ptr);
>> 293  return;
>> 294  }
>> 295  
>> 296  szind_t szind;
>> 297  bool slab;
>> 298  rtree_ctx_t *rtree_ctx;
>> 299  if (alloc_ctx != NULL) {
>> 300  szind = alloc_ctx->szind;
>> 301  slab = alloc_ctx->slab;
>> 302  assert(szind != SC_NSIZES);
>> 303  } else {
>> 304  rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
>> 305  rtree_szind_slab_read(tsdn, _rtree, rtree_ctx,
>> 306  (uintptr_t)ptr, true, , );
>> 307  }
>> 308  
>> 309  if (config_debug) {
>> 310  rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
>> 311  extent_t *extent = rtree_extent_read(tsdn, 
>> _rtree,
>> 312  rtree_ctx, (uintptr_t)ptr, true);
>> 313  assert(szind == extent_szind_get(extent));
>> 314  assert(szind < SC_NSIZES);
>> 315  assert(slab == extent_slab_get(extent));
>> 316  }
>> 317  
>> 318  if (likely(slab)) {
>> 319  /* Small allocation. */
>> 320  tcache_dalloc_small(tsdn_tsd(tsdn), tcache, ptr, szind,
>> 321  slow_path);
>> 322  } else {
>> 323  arena_dalloc_large(tsdn, ptr, tcache, szind, slow_path);
>> 324  }
>> 325  }
> 
> The following are only shown for contrast with the later
> cases of lots of zeros showing up where below shows
> non-zero values (taken from sshd.core, which failed
> differently):
> 
> (gdb) x/4x __je_arenas+16368/4
> 0x50981ab0 <__je_arenas+16368>:   0x  0x  
> 0x  0x0009
> (gdb) 

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[__je_sz_size2index_tab seems messed up in 2 of the
asserting contexts: first 384 are zero in both. More
before that is also messed up (all zero). I show the
details later below.]

On 2020-May-6, at 16:57, Mark Millard  wrote:

> [This explores process crashes that happen during system
> shutdown, in a context not having MALLOC_PRODUCTION= .
> So assert failures are reported as the stopping points.]
> 
> It looks like shutdown -p now, shutdown -r now, and the
> like can lead some processes to assert during their exit
> attempt, including a sshd failure (that I've not seen
> before), rpcbind, and nfsd. I show information about the
> observed asserts for those below.
> 
> 
> sshd hit an assert, failing slab == extent_slab_get(extent) :
> 
> (gdb) bt 
> #0  thr_kill () at thr_kill.S:4
> #1  0x50927170 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52
> #2  0x50886cc0 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
> #3  0x508834b0 in arena_dalloc (tsdn=, ptr=, 
> tcache=, alloc_ctx=, slow_path=)
>at 
> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:315
> #4  idalloctm (tsdn=0x500dd040, ptr=0x5008a180, tcache=0x500dd160, 
> alloc_ctx=, is_internal=, slow_path= out>)
>at 
> /usr/src/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal_inlines_c.h:118
> #5  0x5087b0a4 in ifree (tsd=0x500dd040, ptr=0x5008a180, tcache=0x500dd160, 
> slow_path=) at jemalloc_jemalloc.c:2590
> #6  0x5087acac in __je_free_default (ptr=0x5008a180) at 
> jemalloc_jemalloc.c:2784
> #7  0x5087b294 in __free (ptr=0x5008a180) at jemalloc_jemalloc.c:2852
> #8  0x10029464 in server_accept_loop (config_s=, 
> sock_in=, sock_out=, newsock=) 
> at /usr/src/crypto/openssh/sshd.c:1185
> #9  main (ac=, av=0xde3c) at 
> /usr/src/crypto/openssh/sshd.c:2009
> 
> . . .
> (gdb) up
> #2  0x50886cc0 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
> 67(void)raise(SIGABRT);
> (gdb) up
> #3  0x508834b0 in arena_dalloc (tsdn=, ptr=, 
> tcache=, alloc_ctx=, slow_path=)
>at 
> /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:315
> 315   assert(slab == extent_slab_get(extent));
> 
> (gdb) list
> 310   rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
> 311   extent_t *extent = rtree_extent_read(tsdn, 
> _rtree,
> 312   rtree_ctx, (uintptr_t)ptr, true);
> 313   assert(szind == extent_szind_get(extent));
> 314   assert(szind < SC_NSIZES);
> 315   assert(slab == extent_slab_get(extent));
> 316   }
> 317   
> 318   if (likely(slab)) {
> 319   /* Small allocation. */
> 
> More fully:
> 
> 285   JEMALLOC_ALWAYS_INLINE void
> 286   arena_dalloc(tsdn_t *tsdn, void *ptr, tcache_t *tcache,
> 287   alloc_ctx_t *alloc_ctx, bool slow_path) {
> 288   assert(!tsdn_null(tsdn) || tcache == NULL);
> 289   assert(ptr != NULL);
> 290   
> 291   if (unlikely(tcache == NULL)) {
> 292   arena_dalloc_no_tcache(tsdn, ptr);
> 293   return;
> 294   }
> 295   
> 296   szind_t szind;
> 297   bool slab;
> 298   rtree_ctx_t *rtree_ctx;
> 299   if (alloc_ctx != NULL) {
> 300   szind = alloc_ctx->szind;
> 301   slab = alloc_ctx->slab;
> 302   assert(szind != SC_NSIZES);
> 303   } else {
> 304   rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
> 305   rtree_szind_slab_read(tsdn, _rtree, rtree_ctx,
> 306   (uintptr_t)ptr, true, , );
> 307   }
> 308   
> 309   if (config_debug) {
> 310   rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
> 311   extent_t *extent = rtree_extent_read(tsdn, 
> _rtree,
> 312   rtree_ctx, (uintptr_t)ptr, true);
> 313   assert(szind == extent_szind_get(extent));
> 314   assert(szind < SC_NSIZES);
> 315   assert(slab == extent_slab_get(extent));
> 316   }
> 317   
> 318   if (likely(slab)) {
> 319   /* Small allocation. */
> 320   tcache_dalloc_small(tsdn_tsd(tsdn), tcache, ptr, szind,
> 321   slow_path);
> 322   } else {
> 323   arena_dalloc_large(tsdn, ptr, tcache, szind, slow_path);
> 324   }
> 325   }

The following are only shown for contrast with the later
cases of lots of zeros showing up where below shows
non-zero values (taken from sshd.core, which failed
differently):

(gdb) x/4x __je_arenas+16368/4
0x50981ab0 <__je_arenas+16368>: 0x  0x  0x  
0x0009
(gdb) print/x __je_arenas_lock 
$1 = {{{prof_data = {tot_wait_time = {ns = 0x0}, max_wait_time = {ns = 0x0}, 
n_wait_times = 0x0, n_spin_acquired = 0x0, max_n_thds = 0x0, n_waiting_thds = 
{repr = 0x0}, n_owner_switches = 0x0, 
prev_owner 

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[This explores process crashes that happen during system
shutdown, in a context not having MALLOC_PRODUCTION= .
So assert failures are reported as the stopping points.]

It looks like shutdown -p now, shutdown -r now, and the
like can lead some processes to assert during their exit
attempt, including a sshd failure (that I've not seen
before), rpcbind, and nfsd. I show information about the
observed asserts for those below.


sshd hit an assert, failing slab == extent_slab_get(extent) :

(gdb) bt 
#0  thr_kill () at thr_kill.S:4
#1  0x50927170 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52
#2  0x50886cc0 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
#3  0x508834b0 in arena_dalloc (tsdn=, ptr=, 
tcache=, alloc_ctx=, slow_path=)
at /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:315
#4  idalloctm (tsdn=0x500dd040, ptr=0x5008a180, tcache=0x500dd160, 
alloc_ctx=, is_internal=, slow_path=)
at 
/usr/src/contrib/jemalloc/include/jemalloc/internal/jemalloc_internal_inlines_c.h:118
#5  0x5087b0a4 in ifree (tsd=0x500dd040, ptr=0x5008a180, tcache=0x500dd160, 
slow_path=) at jemalloc_jemalloc.c:2590
#6  0x5087acac in __je_free_default (ptr=0x5008a180) at jemalloc_jemalloc.c:2784
#7  0x5087b294 in __free (ptr=0x5008a180) at jemalloc_jemalloc.c:2852
#8  0x10029464 in server_accept_loop (config_s=, 
sock_in=, sock_out=, newsock=) at 
/usr/src/crypto/openssh/sshd.c:1185
#9  main (ac=, av=0xde3c) at 
/usr/src/crypto/openssh/sshd.c:2009

. . .
(gdb) up
#2  0x50886cc0 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
67  (void)raise(SIGABRT);
(gdb) up
#3  0x508834b0 in arena_dalloc (tsdn=, ptr=, 
tcache=, alloc_ctx=, slow_path=)
at /usr/src/contrib/jemalloc/include/jemalloc/internal/arena_inlines_b.h:315
315 assert(slab == extent_slab_get(extent));

(gdb) list
310 rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
311 extent_t *extent = rtree_extent_read(tsdn, 
_rtree,
312 rtree_ctx, (uintptr_t)ptr, true);
313 assert(szind == extent_szind_get(extent));
314 assert(szind < SC_NSIZES);
315 assert(slab == extent_slab_get(extent));
316 }
317 
318 if (likely(slab)) {
319 /* Small allocation. */

More fully:

285 JEMALLOC_ALWAYS_INLINE void
286 arena_dalloc(tsdn_t *tsdn, void *ptr, tcache_t *tcache,
287 alloc_ctx_t *alloc_ctx, bool slow_path) {
288 assert(!tsdn_null(tsdn) || tcache == NULL);
289 assert(ptr != NULL);
290 
291 if (unlikely(tcache == NULL)) {
292 arena_dalloc_no_tcache(tsdn, ptr);
293 return;
294 }
295 
296 szind_t szind;
297 bool slab;
298 rtree_ctx_t *rtree_ctx;
299 if (alloc_ctx != NULL) {
300 szind = alloc_ctx->szind;
301 slab = alloc_ctx->slab;
302 assert(szind != SC_NSIZES);
303 } else {
304 rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
305 rtree_szind_slab_read(tsdn, _rtree, rtree_ctx,
306 (uintptr_t)ptr, true, , );
307 }
308 
309 if (config_debug) {
310 rtree_ctx = tsd_rtree_ctx(tsdn_tsd(tsdn));
311 extent_t *extent = rtree_extent_read(tsdn, 
_rtree,
312 rtree_ctx, (uintptr_t)ptr, true);
313 assert(szind == extent_szind_get(extent));
314 assert(szind < SC_NSIZES);
315 assert(slab == extent_slab_get(extent));
316 }
317 
318 if (likely(slab)) {
319 /* Small allocation. */
320 tcache_dalloc_small(tsdn_tsd(tsdn), tcache, ptr, szind,
321 slow_path);
322 } else {
323 arena_dalloc_large(tsdn, ptr, tcache, szind, slow_path);
324 }
325 }


rpcbind hit an assert, failing ret == sz_size2index_compute(size) :

(gdb) bt
#0  thr_kill () at thr_kill.S:4
#1  0x502f2170 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52
#2  0x50251d04 in abort () at /usr/src/lib/libc/stdlib/abort.c:79
#3  0x5024f260 in sz_size2index_lookup (size=) at 
/usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:159
#4  sz_size2index (size=) at 
/usr/src/contrib/jemalloc/include/jemalloc/internal/sz.h:166
#5  imalloc_body (sopts=0xb360, dopts=0xb340, tsd=0x5009a018) at 
jemalloc_jemalloc.c:2066
#6  0x50244874 in imalloc (sopts=0xb360, dopts=0xb340) at 
/usr/src/contrib/jemalloc/include/jemalloc/internal/tsd.h:331
#7  0x50244fe8 in __calloc (num=1, size=96) at jemalloc_jemalloc.c:2498
#8  0x50265690 in svc_xprt_alloc () at /usr/src/lib/libc/rpc/svc.c:541
#9  0x502635f4 in makefd_xprt (fd=14, sendsize=9000, 

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[This report just shows an interesting rpcbind crash:
a pointer was filled with part of a string instead,
leading to a failed memory access attempt from the junk
address produced.]

Core was generated by `/usr/sbin/rpcbind'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x5024405c in rendezvous_request (xprt=, msg=) at /usr/src/lib/libc/rpc/svc_vc.c:335
335 cd->recvsize = r->recvsize;

(gdb) list
330 _setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, , 
sizeof (len));
331 }
332 
333 cd = (struct cf_conn *)newxprt->xp_p1;
334 
335 cd->recvsize = r->recvsize;
336 cd->sendsize = r->sendsize;
337 cd->maxrec = r->maxrec;
338 
339 if (cd->maxrec != 0) {

(gdb) print/c *cd
Cannot access memory at address 0x2d202020

FYI:

. . .
   0x50244050 <+452>:   bl  0x502e3404 <.plt_pic32._setsockopt>
   0x50244054 <+456>:   lwz r27,80(r29)
   0x50244058 <+460>:   lwz r3,4(r24)
=> 0x5024405c <+464>:   stw r3,436(r27)

Note the 80(r29) use.

(gdb) info reg
r0 0x50244020  1344552992
r1 0xb400  4294947840
r2 0x500a1018  1342836760
r3 0x2328  9000
r4 0x32ef559c  854545820
r5 0x0 0
r6 0xb360  4294947680
r7 0xb364  4294947684
r8 0x5004733c  1342468924
r9 0x0 0
r100x2032
r110x50252ea0  1344614048
r120x24200ca0  606080160
r130x0 0
r140x0 0
r150xbc28  4294949928
r160x10002848  268445768
r170x1004  268697600
r180x2 2
r190x0 0
r200x1 1
r210x5004c044  1342488644
r220xb63c  4294948412
r230x80128
r240x50048010  1342472208
r250x1420
r260xb630  4294948400
r270x2d202020  757080096
r280xf 15
r290x50047308  1342468872
r300x5030112c  1345327404
r310x1004  268697600
pc 0x5024405c  0x5024405c 
msr
cr 0x842000a0  2216689824
lr 0x50244020  0x50244020 
ctr0x50252ea0  1344614048
xer0x0 0
fpscr  0x0 0
vscr   
vrsave 

(gdb) x/s 0x50047308+72
0x50047350: " -  -   -\n"

So it tried to use "-   " as a pointer value.

It appears that the r29 value was from:

   0x50243f90 <+260>:   mr  r28,r3
   0x50243f94 <+264>:   lwz r4,0(r24)
   0x50243f98 <+268>:   lwz r5,4(r24)
   0x50243f9c <+272>:   mr  r3,r28
   0x50243fa0 <+276>:   bl  0x5024308c 
   0x50243fa4 <+280>:   lwz r27,36(r1)
   0x50243fa8 <+284>:   mr  r29,r3

The makefd_xprt being used as part of:

/*
 * make a new transporter (re-uses xprt)
 */
newxprt = makefd_xprt(sock, r->sendsize, r->recvsize);
newxprt->xp_rtaddr.buf = mem_alloc(len);
if (newxprt->xp_rtaddr.buf == NULL)
return (FALSE);
memcpy(newxprt->xp_rtaddr.buf, , len);
newxprt->xp_rtaddr.len = len;
#ifdef PORTMAP
if (addr.ss_family == AF_INET || addr.ss_family == AF_LOCAL) {
newxprt->xp_raddr = *(struct sockaddr_in 
*)newxprt->xp_rtaddr.buf;
newxprt->xp_addrlen = sizeof (struct sockaddr_in);
}
#endif  /* PORTMAP */
if (__rpc_fd2sockinfo(sock, ) && si.si_proto == IPPROTO_TCP) {
len = 1;
/* XXX fvdl - is this useful? */
_setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, , sizeof (len));
}
 
cd = (struct cf_conn *)newxprt->xp_p1;
 
cd->recvsize = r->recvsize;
cd->sendsize = r->sendsize;
cd->maxrec = r->maxrec;

FYI:

(gdb) print *r
$5 = {sendsize = 9000, recvsize = 9000, maxrec = 9000}

There is more evidence of strings in pointers in *newxprt
(xp_tp, oa_base, xp_p1, xp_p2, xp_p3):

(gdb) print *newxprt
$7 = {xp_fd = 15, xp_port = 0, xp_ops = 0x50329e1c, xp_addrlen = 16, xp_raddr = 
{sin_len = 16 '\020', sin_family = 1 '\001', sin_port = 0, sin_addr = {s_addr = 
0}, 
sin_zero = "\000\000\000\000\000\000\000"}, xp_ops2 = 0x756e6978, xp_tp = 
0x2020 , 
  xp_netid = 0x1001 , 
xp_ltaddr = {maxlen = 0, len = 0, buf = 0x0}, xp_rtaddr = {maxlen = 539828256, 
len = 16, buf = 0x50047330}, xp_verf = {
oa_flavor = 0, oa_base = 0x202d2020 , oa_length = 538976288}, xp_p1 = 0x2d202020, xp_p2 = 0x20202020, 
xp_p3 = 

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[This report just shows some material for the
sendmail SIGSEGV's, based on truss output.]

I've returned to using the modern jemalloc because
it seems to show problems more, after having
caught the earlier reported dhclient example under
the older jemalloc context. (Again: jemalloc may
be exposing a problem elsewhere.)

I had truss monitor sendmail, including following
child processes. A child process start and end
normally looks like:

  963: 3092.707439595 0.33895 sigprocmask(SIG_SETMASK,{ SIGCHLD },0x0) = 0 
(0x0)
 2432: 3092.708024959 0.0 
  963: 3092.708136462 0.000470115 fork() = 2432 (0x980)
 2432: 3092.708441039 0.33319 thr_self(0x5012) = 0 (0x0)
. . .
 2432: 3092.717283761 0.36008 sigaction(SIGQUIT,{ SIG_IGN SA_RESTART ss_t 
},{ SIG_IGN SA_RESTART ss_t }) = 0 (0x0)
 2432: 3092.717544288 0.34352 sigprocmask(SIG_SETMASK,{ },0x0) = 0 (0x0)
 2432: 3092.717799894 0.35768 close(0)   = 0 (0x0)
 2432: 3092.718174733 0.000103726 openat(AT_FDCWD,"/dev/null",O_RDONLY,00) = 0 
(0x0)
 2432: 3092.718480437 0.52091 openat(AT_FDCWD,"/dev/null",O_WRONLY,00) = 4 
(0x4)
 2432: 3092.718778028 0.37856 dup2(4,1)  = 1 (0x1)
 2432: 3092.719003051 0.34255 dup2(4,2)  = 2 (0x2)
 2432: 3092.719225122 0.33655 close(4)   = 0 (0x0)
 2432: 3092.719437735 0.47626 fstat(0,{ mode=crw-rw-rw- 
,inode=22,size=0,blksize=4096 }) = 0 (0x0)
 2432: 3092.719679274 0.37400 fstat(1,{ mode=crw-rw-rw- 
,inode=22,size=0,blksize=4096 }) = 0 (0x0)
 2432: 3092.719908859 0.35816 fstat(2,{ mode=crw-rw-rw- 
,inode=22,size=0,blksize=4096 }) = 0 (0x0)
 2432: 3092.720138299 0.33727 clock_gettime(13,{ 1588570204.0 }) = 
0 (0x0)
 2432: 3092.720658945 0.35360 getpid()   = 2432 (0x980)
 2432: 3092.720931594 0.48730 
__sysctl("kern.proc.args.2432",4,0x0,0x0,0x508ae000,49) = 0 (0x0)
 2432: 3092.721572338 0.62318 
open(".",O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC,00) = 4 (0x4)
 2432: 3092.721962132 0.37808 fcntl(4,F_ISUNIONSTACK,0x0) = 0 (0x0)
 2432: 3092.722323792 0.98613 
getdirentries(4,"\0\0\0\0\0\t\M-I\M^Q\0\0\0\0\0\0"...,4096,{ 0x0 }) = 144 (0x90)
 2432: 3092.722944875 0.36944 getdirentries(4,0x50859000,4096,{ 0x200 }) = 
0 (0x0)
 2432: 3092.723294461 0.45250 close(4)   = 0 (0x0)
 2432: 3092.723576400 0.41144 clock_gettime(13,{ 1588570204.0 }) = 
0 (0x0)
 2432: 3092.723828718 0.37928 setitimer(0,{ 0.00, 0.00 },0x0) = 0 
(0x0)
 2432: 3092.724092245 0.35815 sigprocmask(SIG_UNBLOCK,{ SIGALRM },{ }) = 0 
(0x0)
 2432: 3092.724591527 0.38024 getpid()   = 2432 (0x980)
 2432: 3092.724952323 0.52955 setuid(0x0)ERR#1 'Operation not permitted'
 2432: 3092.727852159 0.42969 clock_gettime(4,{ 21633.960374942 }) = 0 (0x0)
 2432: 3092.728508193 0.34327 clock_gettime(4,{ 21633.961033929 }) = 0 (0x0)
 2432: 3092.729146608 0.36872 clock_gettime(4,{ 21633.961670903 }) = 0 (0x0)
 2432: 3092.729824967 0.38360 clock_gettime(4,{ 21633.962349071 }) = 0 (0x0)
 2432: 3092.732435446 0.001634793 exit(0x0) 
 2432: 3092.732555855 0.001755202 process exit, rval = 0
  963: 3092.732638865 0.018571063 SIGNAL 20 (SIGCHLD) code=CLD_EXITED pid=2432 
uid=25 status=0
  963: 3092.732822864 0.018755062 sigsuspend({ }) ERR#4 'Interrupted system 
call'
  963: 3092.733076525 0.39968 sigprocmask(SIG_SETMASK,{ SIGCHLD },0x0) = 0 
(0x0)
  963: 3092.733447788 0.76601 wait4(-1,{ EXITED,val=0 },WNOHANG,0x0) = 2432 
(0x980)
  963: 3092.733781410 0.34783 wait4(-1,0xbe60,WNOHANG,0x0) ERR#10 'No 
child processes'
  963: 3092.734065366 0.37328 sigreturn(0xbe90) EJUSTRETURN
  963: 3092.734263408 0.33295 sigprocmask(SIG_BLOCK,0x0,{ }) = 0 (0x0)
(No activity in 963 for about 1800 seconds.)

But once it starts failing, the SIGSEGV happens just after the:

setuid(0x0)ERR#1 'Operation not permitted'

For example:

. . .
 4745: 37293.335510778 0.35023 clock_gettime(13,{ 1588604405.0 }) = 
0 (0x0)
 4745: 37293.335754863 0.37593 setitimer(0,{ 0.00, 0.00 },0x0) = 0 
(0x0)
 4745: 37293.336010253 0.36200 sigprocmask(SIG_UNBLOCK,{ SIGALRM },{ }) = 0 
(0x0)
 4745: 37293.336488027 0.33823 getpid()  = 4745 (0x1289)
 4745: 37293.336836797 0.51995 setuid(0x0)   ERR#1 'Operation not permitted'
 4745: 37293.338022675 0.001237873 SIGNAL 11 (SIGSEGV) code=SEGV_MAPERR 
trapno=768 addr=0x506a11f0
 4745: 37293.339546520 0.002761718 process killed, signal = 11
  963: 37293.339627249 0.050797919 SIGNAL 20 (SIGCHLD) code=CLD_KILLED pid=4745 
uid=25 status=11
  963: 37293.339794781 0.050965451 sigsuspend({ }) ERR#4 'Interrupted system 
call'
  963: 37293.340038313 0.37544 sigprocmask(SIG_SETMASK,{ SIGCHLD },0x0) = 0 
(0x0)
  963: 37293.340329951 0.70215 wait4(-1,{ SIGNALED,sig=SIGSEGV 
},WNOHANG,0x0) = 4745 (0x1289)
  963: 37293.340634048 0.34903 wait4(-1,0xbe60,WNOHANG,0x0) ERR#10 'No 
child processes'
  963: 37293.340901417 0.37615 

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

This note just reports things from looking at 2
.core files (mountd and rpcbind) from the new
jemalloc context's failures. May be someone that
knows more can get something out of it. I've
not included any of the prior message history.


For mountd:

Core was generated by `/usr/sbin/mountd -r'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x50235df0 in cache_bin_dalloc_easy (bin=, 
bin_info=, ptr=0x50049160) at 
/usr/src/contrib/jemalloc/include/jemalloc/internal/cache_bin.h:121

warning: Source file is more recent than executable.
121 if (unlikely(bin->ncached == bin_info->ncached_max)) {

It turns out that bin_info traces back to:

cache_bin_t *bin = tcache_small_bin_get(tcache, alloc_ctx.szind);
cache_bin_info_t *bin_info = _bin_info[alloc_ctx.szind];
if (!cache_bin_dalloc_easy(bin, bin_info, ptr)) {
return false;
}

based on:

#define tcache_bin_info JEMALLOC_N(tcache_bin_info)
and:
#  define JEMALLOC_N(n) __je_##n

But gdb reports:

(gdb) print __je_tcache_bin_info
$3 = (cache_bin_info_t *) 0x0

(gdb) print alloc_ctx
$1 = {szind = 0, slab = }

so bin_info = NULL and bin_info->ncached_max would
fail (and does).

By contrast, bin->ncached seems to be from:

(gdb) print *(cache_bin_t*)0x50094018
$6 = {low_water = 65536, ncached = 1, tstats = {nrequests = 
4796680610075595179}, avail = 0x0}


For rpcbind:

Core was generated by `/usr/sbin/rpcbind'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x50243fec in rendezvous_request (xprt=, msg=) at /usr/src/lib/libc/rpc/svc_vc.c:323
323 newxprt->xp_raddr = *(struct sockaddr_in 
*)newxprt->xp_rtaddr.buf;

(gdb) print *newxprt
$5 = {xp_fd = 14, xp_port = 0, xp_ops = 0x50329e1c, xp_addrlen = 0, xp_raddr = 
{sin_len = 0 '\000', sin_family = 0 '\000', sin_port = 0, sin_addr = {s_addr = 
0}, 
sin_zero = "\000\000\000\000P1O\374"}, xp_ops2 = 0x50329e34, xp_tp = 0x0, 
xp_netid = 0x50047310 "unix", xp_ltaddr = {maxlen = 1345322064, len = 
1970170232, buf = 0x2020}, xp_rtaddr = {
maxlen = 268500992, len = 16, buf = 0x0}, xp_verf = {oa_flavor = 0, oa_base 
= 0x202d2020 , oa_length = 
538976364}, xp_p1 = 0x6f6f7000, 
  xp_p2 = 0x0, xp_p3 = 0x202d2020, xp_type = 538976288}

so newxprt->xp_rtaddr.buf == 0x0 . But taht is very odd . . .

/*
 * make a new transporter (re-uses xprt)
 */
newxprt = makefd_xprt(sock, r->sendsize, r->recvsize);
newxprt->xp_rtaddr.buf = mem_alloc(len);
if (newxprt->xp_rtaddr.buf == NULL)
return (FALSE);
memcpy(newxprt->xp_rtaddr.buf, , len);
newxprt->xp_rtaddr.len = len;
#ifdef PORTMAP
if (addr.ss_family == AF_INET || addr.ss_family == AF_LOCAL) {
newxprt->xp_raddr = *(struct sockaddr_in 
*)newxprt->xp_rtaddr.buf;
newxprt->xp_addrlen = sizeof (struct sockaddr_in);
}
#endif  /* PORTMAP */

Or, in machine code terms:

   0x50243f90 <+260>:   mr  r28,r3
   0x50243f94 <+264>:   lwz r4,0(r24)
   0x50243f98 <+268>:   lwz r5,4(r24)
   0x50243f9c <+272>:   mr  r3,r28
   0x50243fa0 <+276>:   bl  0x5024308c 
   0x50243fa4 <+280>:   lwz r27,36(r1)
   0x50243fa8 <+284>:   mr  r29,r3
   0x50243fac <+288>:   li  r3,1
   0x50243fb0 <+292>:   mr  r4,r27
   0x50243fb4 <+296>:   bl  0x502e3214 <.plt_pic32.calloc>
   0x50243fb8 <+300>:   cmplwi  r3,0
   0x50243fbc <+304>:   stw r3,64(r29)
   0x50243fc0 <+308>:   beq 0x50244160 

Note: getting here means that newxprt->xp_rtaddr.buf
ws not NULL . Also the value was stored to 64(r29).

   0x50243fc4 <+312>:   addir4,r1,296
   0x50243fc8 <+316>:   mr  r5,r27
   0x50243fcc <+320>:   bl  0x502e3264 <.plt_pic32.memcpy>

Note: getting here means that memcpy was able to
store where ewxprt->xp_rtaddr.buf indicated (as
the r3 value).

   0x50243fd0 <+324>:   lbz r3,297(r1)
   0x50243fd4 <+328>:   stw r27,60(r29)
   0x50243fd8 <+332>:   addir3,r3,-1
   0x50243fdc <+336>:   clrlwi  r3,r3,24
   0x50243fe0 <+340>:   cmplwi  r3,1
   0x50243fe4 <+344>:   bgt 0x50244014 
   0x50243fe8 <+348>:   lwz r3,64(r29)
=> 0x50243fec <+352>:   lwz r4,0(r3)

Note: the failure was above with r3==0x0:

(gdb) info reg
r0 0x50243fb8  1344552888
r1 0xb400  4294947840
r2 0x500a1018  1342836760
r3 0x0 0
r4 0xb538  4294948152
r5 0x1016
r6 0x50047328  1342468904
r7 0x0 0
r8 0x50047324  1342468900
r9 0x0 0
r100x2032
r110x502d691c  1345153308
r120x24200880  606079104
r130x0 0
r140x0 0
r150xbc28  4294949928

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[The bit argument ot bitmap_unset seems to be way
too large.]

On 2020-May-3, at 11:08, Mark Millard  wrote:

> [At around 4AM local time dhcient got a signal 11,
> despite the jemalloc revert. The other exmaples
> have not happened.]
> 
> On 2020-May-2, at 18:46, Mark Millard  wrote:
> 
>> [I'm only claiming the new jemalloc is involved and that
>> reverting avoids the problem.]
>> 
>> I've been reporting to some lists problems with:
>> 
>> dhclient
>> sendmail
>> rpcbind
>> mountd
>> nfsd
>> 
>> getting SIGSEGV (signal 11) crashes and some core
>> dumps on the old 2-socket (1 core per socket) 32-bit
>> PowerMac G4 running head -r360311.
>> 
>> Mikaël Urankar sent a note suggesting that I try
>> testing reverting head -r360233 for my head -r360311
>> context. He got it right . . .
>> 
>> 
>> Context:
>> 
>> The problem was noticed by an inability to have
>> other machines do a:
>> 
>> mount -onoatime,soft OLDPOWERMAC-LOCAL-IP:/... /mnt
>> 
>> sort of operation and to have succeed. By contrast, on
>> the old PowerMac G4 I could initiate mounts against
>> other machines just fine.
>> 
>> I do not see any such problems on any of (all based
>> on head -r360311):
>> 
>> powerpc64 (old PowerMac G5 2-sockets with 2 cores each)
>> armv7 (OrangePi+ 2ed)
>> aarch64 (Rock64, RPi4, RPi3,
>>OverDrive 1000,
>>Macchiatobin Double Shot)
>> amd64 (ThreadRipper 1950X)
>> 
>> So I expect something 32-bit powerpc specific
>> is somehow involved, even if jemalloc is only
>> using whatever it is.
>> 
>> (A kyua run with a debug kernel did not find other
>> unexpected signal 11 sources on the 32-bit PowerMac
>> compared to past kyua runs, at least that I noticed.
>> There were a few lock order reversals that I do not
>> know if they are expected or known-safe or not.
>> I've reported those reversals to the lists as well.)
>> 
>> 
>> Recent experiments based on the suggestion:
>> 
>> Doing the buildworld, buildkernel and installing just
>> the new kernel and rebooting made no difference.
>> 
>> But then installing the new world and rebooting did
>> make things work again: I no longer get core files
>> for the likes of (old cores from before the update):
>> 
>> # find / -name "*.core" -print
>> /var/spool/clientmqueue/sendmail.core
>> /rpcbind.core
>> /mountd.core
>> /nfsd.core
>> 
>> Nor do I see the various notices for sendmail
>> signal 11's that did not leave behind a core file
>> --or for dhclient (no core file left behind).
>> And I can mount the old PowerMac's drive from
>> other machines just fine.
>> 
>> 
>> Other notes:
>> 
>> I do not actively use sendmail but it was left
>> to do its default things, partially to test if
>> such default things are working. Unfortunately,
>> PowerMacs have a problematical status under
>> FreeBSD and my context has my historical
>> experiments with avoiding various problems.
> 
> Looking, I see that I got a:
> 
> pid 572 (dhclient), jid 0, uid 0: exited on signal 11 (core dumped)
> 
> notice under the reverted build. No instances
> of the other examples. This is the first that a
> dhclient example has produced a .core file.
> 
> gdb indicates 0x5180936c for r7 in:
> 
> lwz r8,36(r7)
> 
> as leading to the failure. This was in
> arena_dalloc_bin_locked_impl (where
> arena_slab_reg_dalloc and bitmap_unset
> were apparently inlined).
> 
> The chain for the example seems to be:
> fork_privchld -> dispatch_imsg -> jemalloc
> 
> For reference . . .
> 
> # gdb dhclient /dhclient.core 
> GNU gdb (GDB) 9.1 [GDB v9.1 for FreeBSD]
> Copyright (C) 2020 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later 
> . . .
> Reading symbols from dhclient...
> Reading symbols from /usr/lib/debug//sbin/dhclient.debug...
> [New LWP 100089]
> Core was generated by `dhclient: gem0 [priv]'.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0  bitmap_unset (bitmap=0x50407164, binfo=, bit=167842154) at 
> /usr/powerpc32_src/contrib/jemalloc/include/jemalloc/internal/bitmap.h:341
> 341   /usr/powerpc32_src/contrib/jemalloc/include/jemalloc/internal/bitmap.h: 
> No such file or directory.
> (gdb) bt -full
> #0  bitmap_unset (bitmap=0x50407164, binfo=, bit=167842154) at 
> /usr/powerpc32_src/contrib/jemalloc/include/jemalloc/internal/bitmap.h:341
>goff = 
>gp = 0x51809390
>propagate = 
>g = 
>i = 
> #1  arena_slab_reg_dalloc (slab=0x50407140, slab_data=0x50407164, 
> ptr=0x50088b50) at jemalloc_arena.c:273
>bin_info = 
>binind = 0
>regind = 167842154
> #2  arena_dalloc_bin_locked_impl (tsdn=0x5009f018, arena=, 
> slab=, ptr=, junked=) at 
> jemalloc_arena.c:1540
>slab_data = 
>binind = 
>bin_info = 
>bin = 
>nfree = 
> #3  0x502916a8 in __je_arena_dalloc_bin_junked_locked (tsdn=, 
> arena=, extent=, ptr=) at 
> jemalloc_arena.c:1559
> No locals.
> #4  0x50250d2c in __je_tcache_bin_flush_small (tsd=0x5009f018, 
> tcache=, 

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[At around 4AM local time dhcient got a signal 11,
despite the jemalloc revert. The other exmaples
have not happened.]

On 2020-May-2, at 18:46, Mark Millard  wrote:

> [I'm only claiming the new jemalloc is involved and that
> reverting avoids the problem.]
> 
> I've been reporting to some lists problems with:
> 
> dhclient
> sendmail
> rpcbind
> mountd
> nfsd
> 
> getting SIGSEGV (signal 11) crashes and some core
> dumps on the old 2-socket (1 core per socket) 32-bit
> PowerMac G4 running head -r360311.
> 
> Mikaël Urankar sent a note suggesting that I try
> testing reverting head -r360233 for my head -r360311
> context. He got it right . . .
> 
> 
> Context:
> 
> The problem was noticed by an inability to have
> other machines do a:
> 
> mount -onoatime,soft OLDPOWERMAC-LOCAL-IP:/... /mnt
> 
> sort of operation and to have succeed. By contrast, on
> the old PowerMac G4 I could initiate mounts against
> other machines just fine.
> 
> I do not see any such problems on any of (all based
> on head -r360311):
> 
> powerpc64 (old PowerMac G5 2-sockets with 2 cores each)
> armv7 (OrangePi+ 2ed)
> aarch64 (Rock64, RPi4, RPi3,
> OverDrive 1000,
> Macchiatobin Double Shot)
> amd64 (ThreadRipper 1950X)
> 
> So I expect something 32-bit powerpc specific
> is somehow involved, even if jemalloc is only
> using whatever it is.
> 
> (A kyua run with a debug kernel did not find other
> unexpected signal 11 sources on the 32-bit PowerMac
> compared to past kyua runs, at least that I noticed.
> There were a few lock order reversals that I do not
> know if they are expected or known-safe or not.
> I've reported those reversals to the lists as well.)
> 
> 
> Recent experiments based on the suggestion:
> 
> Doing the buildworld, buildkernel and installing just
> the new kernel and rebooting made no difference.
> 
> But then installing the new world and rebooting did
> make things work again: I no longer get core files
> for the likes of (old cores from before the update):
> 
> # find / -name "*.core" -print
> /var/spool/clientmqueue/sendmail.core
> /rpcbind.core
> /mountd.core
> /nfsd.core
> 
> Nor do I see the various notices for sendmail
> signal 11's that did not leave behind a core file
> --or for dhclient (no core file left behind).
> And I can mount the old PowerMac's drive from
> other machines just fine.
> 
> 
> Other notes:
> 
> I do not actively use sendmail but it was left
> to do its default things, partially to test if
> such default things are working. Unfortunately,
> PowerMacs have a problematical status under
> FreeBSD and my context has my historical
> experiments with avoiding various problems.

Looking, I see that I got a:

pid 572 (dhclient), jid 0, uid 0: exited on signal 11 (core dumped)

notice under the reverted build. No instances
of the other examples. This is the first that a
dhclient example has produced a .core file.

gdb indicates 0x5180936c for r7 in:

lwz r8,36(r7)

as leading to the failure. This was in
arena_dalloc_bin_locked_impl (where
arena_slab_reg_dalloc and bitmap_unset
were apparently inlined).

The chain for the example seems to be:
fork_privchld -> dispatch_imsg -> jemalloc

For reference . . .

# gdb dhclient /dhclient.core 
GNU gdb (GDB) 9.1 [GDB v9.1 for FreeBSD]
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
. . .
Reading symbols from dhclient...
Reading symbols from /usr/lib/debug//sbin/dhclient.debug...
[New LWP 100089]
Core was generated by `dhclient: gem0 [priv]'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  bitmap_unset (bitmap=0x50407164, binfo=, bit=167842154) at 
/usr/powerpc32_src/contrib/jemalloc/include/jemalloc/internal/bitmap.h:341
341 /usr/powerpc32_src/contrib/jemalloc/include/jemalloc/internal/bitmap.h: 
No such file or directory.
(gdb) bt -full
#0  bitmap_unset (bitmap=0x50407164, binfo=, bit=167842154) at 
/usr/powerpc32_src/contrib/jemalloc/include/jemalloc/internal/bitmap.h:341
goff = 
gp = 0x51809390
propagate = 
g = 
i = 
#1  arena_slab_reg_dalloc (slab=0x50407140, slab_data=0x50407164, 
ptr=0x50088b50) at jemalloc_arena.c:273
bin_info = 
binind = 0
regind = 167842154
#2  arena_dalloc_bin_locked_impl (tsdn=0x5009f018, arena=, 
slab=, ptr=, junked=) at 
jemalloc_arena.c:1540
slab_data = 
binind = 
bin_info = 
bin = 
nfree = 
#3  0x502916a8 in __je_arena_dalloc_bin_junked_locked (tsdn=, 
arena=, extent=, ptr=) at 
jemalloc_arena.c:1559
No locals.
#4  0x50250d2c in __je_tcache_bin_flush_small (tsd=0x5009f018, 
tcache=, tbin=0x5009f1c0, binind=, rem=24) at 
jemalloc_tcache.c:149
ptr = 
i = 0
extent = 0x50407140
bin_arena = 0x50400380
bin = 
ndeferred = 0
merged_stats = 
arena = 0x50400380
nflush = 75
__vla_expr0 = 
item_extent = 0xd1f0
#5  

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

On 2020-May-3, at 01:26, nonameless at ukr.net wrote:




> --- Original message ---
> From: "Mark Millard" 
> Date: 3 May 2020, 04:47:14
> 
> 
> 
>> [I'm only claiming the new jemalloc is involved and that
>> reverting avoids the problem.]
>> 
>> I've been reporting to some lists problems with:
>> 
>> dhclient
>> sendmail
>> rpcbind
>> mountd
>> nfsd
>> 
>> getting SIGSEGV (signal 11) crashes and some core
>> dumps on the old 2-socket (1 core per socket) 32-bit
>> PowerMac G4 running head -r360311.
>> 
>> Mikaël Urankar sent a note suggesting that I try
>> testing reverting head -r360233 for 
>> my head -r360311
>> context. He got it right . . .
>> 
>> 
>> Context:
>> 
>> The problem was noticed by an inability to have
>> other machines do a:
>> 
>> mount -onoatime,soft OLDPOWERMAC-LOCAL-IP:/... /mnt
>> 
>> sort of operation and to have succeed. By contrast, on
>> the old PowerMac G4 I could initiate mounts against
>> other machines just fine.
>> 
>> I do not see any such problems on any of (all based
>> on head -r360311):
>> 
>> powerpc64 (old PowerMac G5 2-sockets with 2 cores each)
>> armv7 (OrangePi+ 2ed)
>> aarch64 (Rock64, RPi4, RPi3,
>> OverDrive 1000,
>> Macchiatobin Double Shot)
>> amd64 (ThreadRipper 1950X)
>> 
>> So I expect something 32-bit powerpc specific
>> is somehow involved, even if jemalloc is only
>> using whatever it is.
>> 
>> (A kyua run with a debug kernel did not find other
>> unexpected signal 11 sources on the 32-bit PowerMac
>> compared to past kyua runs, at least that I noticed.
>> There were a few lock order reversals that I do not
>> know if they are expected or known-safe or not.
>> I've reported those reversals to the lists as well.)
>> 
>> 
>> Recent experiments based on the suggestion:
>> 
>> Doing the buildworld, buildkernel and installing just
>> the new kernel and rebooting made no difference.
>> 
>> But then installing the new world and rebooting did
>> make things work again: I no longer get core files
>> for the likes of (old cores from before the update):
>> 
>> # find / -name "*.core" -print
>> /var/spool/clientmqueue/sendmail.core
>> /rpcbind.core
>> /mountd.core
>> /nfsd.core
>> 
>> Nor do I see the various notices for sendmail
>> signal 11's that did not leave behind a core file
>> --or for dhclient (no core file left behind).
>> And I can mount the old PowerMac's drive from
>> other machines just fine.
>> 
>> 
>> Other notes:
>> 
>> I do not actively use sendmail but it was left
>> to do its default things, partially to test if
>> such default things are working. Unfortunately,
>> PowerMacs have a problematical status under
>> FreeBSD and my context has my historical
>> experiments with avoiding various problems.
>> 
>> ===
>> Mark Millard
>> marklmi at yahoo.com
>> ( dsl-only.net went
>> away in early 2018-Mar)
>> 
> 
> Hi Mark,
> 
> It should be fixed, but not by reverting to old version. We can't stuck on 
> old version because of ancient hardware. I think upstream is not interested 
> in support such hardware. So, it have to patched locally.

Observing and reporting the reverting result is an initial
part of problem isolation. I made no request for FreeBSD
to give up on using the updated jemalloc. (Unfortunately,
I'm not sure what a good next step of problem isolation
might be for the dual-socket PowerMac G4 context.)

Other than reverting, no patch is known for the issue at
this point. More problem isolation is needed first.

While I do not have access, https://wiki.freebsd.org/powerpc
lists more modern 32-bit powerpc hardware as supported:
MPC85XX evaluation boards and AmigaOne A1222 (powerpcspe).
(The AmigaOne A1222 seems to be dual-ore/single-socket.)

So folks with access to one of those may want to see
if they also see the problem(s) with head -r360233 or
later.

Another interesting context to test could be single-socket
with just one core. (I might be able to do that on another
old PowerMac, booting the same media after moving the
media.)

If I understand right, the most common 32-bit powerpc
tier 2 hardware platforms may still be old PowerMac's.
They are considered supported and "mature", instead of
just "stable". See https://wiki.freebsd.org/powerpc .
However, the reality is that there are various problems
for old PowerMacs (32-bit and 64-bit, at least when
there is more than one socket present). The wiki page
does not hint at such. (I'm not sure about
single socket/multi-core PowerMacs: no access to
such.)

It is certainly possible for some problem to happen
that would lead to dropping the supported-status
for some or all old 32-bit PowerMacs, even as tier 2.
But that has not happened yet and I'd have no say in
such a choice.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r360233 - in head: contrib/jemalloc . . . : This partially breaks a 2-socket 32-bit powerpc (old PowerMac G4) based on head -r360311

[Mark Millard via svn-src-head]

[I'm only claiming the new jemalloc is involved and that
reverting avoids the problem.]

I've been reporting to some lists problems with:

dhclient
sendmail
rpcbind
mountd
nfsd

getting SIGSEGV (signal 11) crashes and some core
dumps on the old 2-socket (1 core per socket) 32-bit
PowerMac G4 running head -r360311.

Mikaël Urankar sent a note suggesting that I try
testing reverting head -r360233 for my head -r360311
context. He got it right . . .


Context:

The problem was noticed by an inability to have
other machines do a:

mount -onoatime,soft OLDPOWERMAC-LOCAL-IP:/... /mnt

sort of operation and to have succeed. By contrast, on
the old PowerMac G4 I could initiate mounts against
other machines just fine.

I do not see any such problems on any of (all based
on head -r360311):

powerpc64 (old PowerMac G5 2-sockets with 2 cores each)
armv7 (OrangePi+ 2ed)
aarch64 (Rock64, RPi4, RPi3,
 OverDrive 1000,
 Macchiatobin Double Shot)
amd64 (ThreadRipper 1950X)

So I expect something 32-bit powerpc specific
is somehow involved, even if jemalloc is only
using whatever it is.

(A kyua run with a debug kernel did not find other
unexpected signal 11 sources on the 32-bit PowerMac
compared to past kyua runs, at least that I noticed.
There were a few lock order reversals that I do not
know if they are expected or known-safe or not.
I've reported those reversals to the lists as well.)


Recent experiments based on the suggestion:

Doing the buildworld, buildkernel and installing just
the new kernel and rebooting made no difference.

But then installing the new world and rebooting did
make things work again: I no longer get core files
for the likes of (old cores from before the update):

# find / -name "*.core" -print
/var/spool/clientmqueue/sendmail.core
/rpcbind.core
/mountd.core
/nfsd.core

Nor do I see the various notices for sendmail
signal 11's that did not leave behind a core file
--or for dhclient (no core file left behind).
And I can mount the old PowerMac's drive from
other machines just fine.


Other notes:

I do not actively use sendmail but it was left
to do its default things, partially to test if
such default things are working. Unfortunately,
PowerMacs have a problematical status under
FreeBSD and my context has my historical
experiments with avoiding various problems.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r359992 - head/sys/powerpc/aim broke i.freebsd.org/job/FreeBSD-head-powerpc-build/ builds

[Mark Millard via svn-src-head]

>From https://ci.freebsd.org/job/FreeBSD-head-powerpc-build/15810/console :

--- mmu_oea64.o ---

23:10:11 /usr/src/sys/powerpc/aim/mmu_oea64.c:933:7: error: format specifies 
type 'uintmax_t' (aka 'unsigned long long') but the argument has type 'unsigned 
int' [-Werror,-Wformat]

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r359829 - in head/sys: amd64/conf conf

[Mark Millard via svn-src-head]

> Author: cem
> Date: Sun Apr 12 18:04:20 2020
> New Revision: 359829
> URL: 
> https://svnweb.freebsd.org/changeset/base/359829
> 
> 
> Log:
>   Add queue(2) debug macros as build options
>   
>   Add QUEUE_MACRO_DEBUG_TRACE and QUEUE_MACRO_DEBUG_TRASH as proper kernel
>   options.  While here, alpha-sort the debug section of sys/conf/options.
>   
>   Enable QUEUE_MACRO_DEBUG_TRASH in amd64 GENERIC (but not GENERIC-NODEBUG)
>   kernels.  It is similar in nature and cost to other use-after-free pointer
>   trashing we do in GENERIC.  It is probably reasonable to enable in any arch
>   GENERIC kernel that defines INVARIANTS.

Going the other way: Is QUEUE_MACRO_DEBUG_TRASH unsafe when INVARIANTS
has been disabled?

I historically produce non-debug/less-debug kernels via including GENERIC
and then disabling various debugging options, including frequently
disabling INVARIANTS and INVARIANTS_SUPPORT.

Would the following be appropriate for the comments, with its extra
"required by" on the first line shown:

 optionsINVARIANTS  # Enable calls of extra sanity 
checking, required by QUEUE_MACRO_DEBUG_TRASH
 optionsINVARIANT_SUPPORT   # Extra sanity checks of internal 
structures, required by INVARIANTS
 optionsQUEUE_MACRO_DEBUG_TRASH # Trash queue(2) internal pointers on 
invalidation

Should head/sys/conf/NOTES have material about avoiding
QUEUE_MACRO_DEBUG_TRASH with INVARIANTS disabled?



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358658 - head

[Mark Millard via svn-src-head]

Gleb Smirnoff glebius at FreeBSD.org wrote on
Wed Mar 4 23:49:22 UTC 2020 :

> author: glebius
> Date: Wed Mar  4 23:49:20 2020
> New Revision: 358658
> URL: 
> https://svnweb.freebsd.org/changeset/base/358658
> 
> 
> Log:
>   Add a missing bktr header.
> 
> Modified:
>   head/ObsoleteFiles.inc
> 
> Modified: head/ObsoleteFiles.inc
> ==
> --- head/ObsoleteFiles.incWed Mar  4 22:32:40 2020(r358657)
> +++ head/ObsoleteFiles.incWed Mar  4 23:49:20 2020(r358658)
> @@ -39,6 +39,7 @@
>  # 20200301: bktr removed
>  OLD_DIRS+=usr/include/dev/bktr
>  OLD_FILES+=usr/include/dev/bktr/ioctl_bktr.h
> +OLD_FILES+=usr/include/dev/bktr/ioctl_bt848.h
>  OLD_FILES+=usr/include/dev/bktr/ioctl_meteor.h
>  .if ${TARGET_ARCH} == "i386"
>  OLD_FILES+=usr/include/machine/ioctl_bktr.h

I also see an include of dev/bktr/ioctl_bt848.h that
is still around:

# grep -r ioctl_bt848 /usr/src/* | more
/usr/src/usr.bin/setchannel/setchannel.c:#include 

Other than a SPDX licensing ID tag addition in late 2017,
the .c file has been unchanged since late 2010 ( head -r214886 ).

/usr/src/usr.bin/setchannel/ has 3 other files as well:

# ls -ldT /usr/src/usr.bin/setchannel/*
-rw-r--r--  1 root  wheel   124 Dec 20 20:06:09 2019 
/usr/src/usr.bin/setchannel/Makefile
-rw-r--r--  1 root  wheel   315 Dec 20 20:06:09 2019 
/usr/src/usr.bin/setchannel/Makefile.depend
-rw-r--r--  1 root  wheel  3258 Dec 20 20:06:10 2019 
/usr/src/usr.bin/setchannel/setchannel.1
-rw-r--r--  1 root  wheel  6825 Dec 20 20:06:09 2019 
/usr/src/usr.bin/setchannel/setchannel.c


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358439 - head/sys/amd64/include

[Mark Millard via svn-src-head]

Konstantin Belousov kostikbel at gmail.com wrote on
Mon Mar 2 18:27:05 UTC 2020 ":

> On Mon, Mar 02, 2020 at 09:13:53AM -0800, Ryan Libby wrote:
> > On Mon, Mar 2, 2020 at 12:45 AM Alexander V. Chernikov  > ipfw.ru> wrote:
> > >
> > > 28.02.2020, 18:32, "Ryan Libby" :
> > > > Author: rlibby
> > > > Date: Fri Feb 28 18:32:36 2020
> > > > New Revision: 358439
> > > > URL: https://svnweb.freebsd.org/changeset/base/358439
> > > >
> > > > Log:
> > > >   amd64 atomic.h: minor codegen optimization in flag access
> > > >
> > > >   Previously the pattern to extract status flags from inline assembly
> > > >   blocks was to use setcc in the block to write the flag to a register.
> > > >   This was suboptimal in a few ways:
> > > >- It would lead to code like: sete %cl; test %cl; jne, i.e. a flag
> > > >  would just be loaded into a register and then reloaded to a flag.
> > > >- The setcc would force the block to use an additional register.
> > > >- If the client code didn't care for the flag value then the setcc
> > > >  would be entirely pointless but could not be eliminated by the
> > > >  optimizer.
> > > >
> > > >   A more modern inline asm construct (since gcc 6 and clang 9) allows 
> > > > for
> > > This effectively restricts kernel builds by all older compilers.
> > > Is there any chance of making it conditional depending on the compiler 
> > > version/features?
> What is the older compiler you need to use for base system ?
> And why ?

[A copy of something I sent to svn-src-head and rlibby a on Feb 29
that showed the example of 2 ports failing to build on FreeBSD build
servers, where the server sent out the notice of the failure.]

head -r358439 breaks unmodified ports that use, for example, clang70 and
clang++70:

https://lists.freebsd.org/pipermail/freebsd-emulation/2020-February/017672.html
https://lists.freebsd.org/pipermail/freebsd-emulation/2020-February/017675.html

show things like . . .

kBuild: Compiling HGSMIGuestR0Lib - 
/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-5.2.34/src/VBox/GuestHost/HGSMI/HGSMICommon.cpp
clang-7: warning: argument unused during compilation: '--param 
inline-unit-growth=100' [-Wunused-command-line-argument]
clang-7: warning: argument unused during compilation: '--param 
large-function-growth=1000' [-Wunused-command-line-argument]
In file included from 
/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-5.2.34/src/VBox/GuestHost/HGSMI/HGSMICommon.cpp:38:
In file included from 
/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-5.2.34/include/VBox/Graphics/VBoxVideoIPRT.h:32:
In file included from 
/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-5.2.34/include/iprt/string.h:45:
In file included from /usr/src/sys/sys/libkern.h:41:
In file included from /usr/src/sys/sys/systm.h:44:
/usr/include/machine/atomic.h:230:1: error: invalid output constraint '=@cce' 
in asm
ATOMIC_CMPSET(char);
^
/usr/include/machine/atomic.h:205:4: note: expanded from macro 'ATOMIC_CMPSET'
   : "=@cce" (res),/* 0 */ \
 ^
/usr/include/machine/atomic.h:230:1: error: invalid output constraint '=@cce' 
in asm
/usr/include/machine/atomic.h:222:4: note: expanded from macro 'ATOMIC_CMPSET'
   : "=@cce" (res),/* 0 */ \
 ^
/usr/include/machine/atomic.h:231:1: error: invalid output constraint '=@cce' 
in asm
ATOMIC_CMPSET(short);
^
/usr/include/machine/atomic.h:205:4: note: expanded from macro 'ATOMIC_CMPSET'
   : "=@cce" (res),/* 0 */ \
 ^
/usr/include/machine/atomic.h:231:1: error: invalid output constraint '=@cce' 
in asm
/usr/include/machine/atomic.h:222:4: note: expanded from macro 'ATOMIC_CMPSET'
   : "=@cce" (res),/* 0 */ \
 ^
/usr/include/machine/atomic.h:232:1: error: invalid output constraint '=@cce' 
in asm
ATOMIC_CMPSET(int);
^
/usr/include/machine/atomic.h:205:4: note: expanded from macro 'ATOMIC_CMPSET'
   : "=@cce" (res),/* 0 */ \
 ^
/usr/include/machine/atomic.h:232:1: error: invalid output constraint '=@cce' 
in asm
/usr/include/machine/atomic.h:222:4: note: expanded from macro 'ATOMIC_CMPSET'
   : "=@cce" (res),/* 0 */ \
 ^
/usr/include/machine/atomic.h:233:1: error: invalid output constraint '=@cce' 
in asm
ATOMIC_CMPSET(long);
^
/usr/include/machine/atomic.h:205:4: note: expanded from macro 'ATOMIC_CMPSET'
   : "=@cce" (res),/* 0 */ \
 ^
/usr/include/machine/atomic.h:233:1: error: invalid output constraint '=@cce' 
in asm
/usr/include/machine/atomic.h:222:4: note: expanded from macro 'ATOMIC_CMPSET'
   : "=@cce" (res),/* 0 */ \
 ^
/usr/include/machine/atomic.h:280:4: error: invalid output constraint '=@ccc' 
in asm
   : "=@ccc" (res),/* 0 */
 ^
/usr/include/machine/atomic.h:296:4: error: 

Re: svn commit: r358500 - in head: . etc/mtree include sys/conf sys/i386/include sys/powerpc/conf

[Mark Millard via svn-src-head]

Warner Losh imp at FreeBSD.org wrote on
Sun Mar 1 20:37:45 UTC 2020 :

> +# 20200301: bktr removed
> +OLD_DIRS+=usr/include/dev/bktr
> +OLD_FILES+=usr/include/dev/bktr/ioctl_bktr.h
> +OLD_FILES+=usr/include/dev/bktr/ioctl_meteor.h
> +.if ${TARGET_ARCH} == "i386"
> +OLD_FILES+=usr/include/machine/ioctl_bktr.h
> +OLD_FILES+=usr/include/machine/ioctl_meteor.h
> +.endif
> +OLD_FILES+=usr/share/man/man4/bktr.4.gz

But I'm seeing rmdir failures in head -r358510 :
(It is an example from my cross builds. I replaced the
leading >>>'s with []'s.)

[] Removing old files (only deletes safe to delete libs)
/usr/obj/DESTDIRs/clang-cortexA72-installworld-poud/usr/include/dev/bktr/ioctl_meteor.h
/usr/obj/DESTDIRs/clang-cortexA72-installworld-poud/usr/share/man/man4/bktr.4.gz
/usr/obj/DESTDIRs/clang-cortexA72-installworld-poud/usr/share/zoneinfo/Etc/UCT
[] Old files removed
[] Removing old directories
rmdir: 
/usr/obj/DESTDIRs/clang-cortexA72-installworld-poud/usr/include/dev/bktr: 
Directory not empty
[] Old directories removed

Where afterwards:

# ls -laT 
/usr/obj/DESTDIRs/clang-cortexA72-installworld-poud/usr/include/dev/bktr/
total 20
drwxr-xr-x   2 root  wheel   512 Mar  1 19:32:47 2020 .
drwxr-xr-x  34 root  wheel   512 Jul 31 20:50:10 2019 ..
-r--r--r--   1 root  wheel  9780 Mar 27 01:33:24 2019 ioctl_bt848.h

But I also see (from the host environment this time):

# grep -r ioctl_bt848 /usr/src/* | more
/usr/src/usr.bin/setchannel/setchannel.c:#include 


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358439 - head/sys/amd64/include

[Mark Millard via svn-src-head]

head -r358439 breaks unmodified ports that use, for example, clang70 and
clang++70:

https://lists.freebsd.org/pipermail/freebsd-emulation/2020-February/017672.html
https://lists.freebsd.org/pipermail/freebsd-emulation/2020-February/017675.html

show things like . . .

kBuild: Compiling HGSMIGuestR0Lib - 
/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-5.2.34/src/VBox/GuestHost/HGSMI/HGSMICommon.cpp
clang-7: warning: argument unused during compilation: '--param 
inline-unit-growth=100' [-Wunused-command-line-argument]
clang-7: warning: argument unused during compilation: '--param 
large-function-growth=1000' [-Wunused-command-line-argument]
In file included from 
/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-5.2.34/src/VBox/GuestHost/HGSMI/HGSMICommon.cpp:38:
In file included from 
/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-5.2.34/include/VBox/Graphics/VBoxVideoIPRT.h:32:
In file included from 
/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-5.2.34/include/iprt/string.h:45:
In file included from /usr/src/sys/sys/libkern.h:41:
In file included from /usr/src/sys/sys/systm.h:44:
/usr/include/machine/atomic.h:230:1: error: invalid output constraint '=@cce' 
in asm
ATOMIC_CMPSET(char);
^
/usr/include/machine/atomic.h:205:4: note: expanded from macro 'ATOMIC_CMPSET'
: "=@cce" (res),/* 0 */ \
  ^
/usr/include/machine/atomic.h:230:1: error: invalid output constraint '=@cce' 
in asm
/usr/include/machine/atomic.h:222:4: note: expanded from macro 'ATOMIC_CMPSET'
: "=@cce" (res),/* 0 */ \
  ^
/usr/include/machine/atomic.h:231:1: error: invalid output constraint '=@cce' 
in asm
ATOMIC_CMPSET(short);
^
/usr/include/machine/atomic.h:205:4: note: expanded from macro 'ATOMIC_CMPSET'
: "=@cce" (res),/* 0 */ \
  ^
/usr/include/machine/atomic.h:231:1: error: invalid output constraint '=@cce' 
in asm
/usr/include/machine/atomic.h:222:4: note: expanded from macro 'ATOMIC_CMPSET'
: "=@cce" (res),/* 0 */ \
  ^
/usr/include/machine/atomic.h:232:1: error: invalid output constraint '=@cce' 
in asm
ATOMIC_CMPSET(int);
^
/usr/include/machine/atomic.h:205:4: note: expanded from macro 'ATOMIC_CMPSET'
: "=@cce" (res),/* 0 */ \
  ^
/usr/include/machine/atomic.h:232:1: error: invalid output constraint '=@cce' 
in asm
/usr/include/machine/atomic.h:222:4: note: expanded from macro 'ATOMIC_CMPSET'
: "=@cce" (res),/* 0 */ \
  ^
/usr/include/machine/atomic.h:233:1: error: invalid output constraint '=@cce' 
in asm
ATOMIC_CMPSET(long);
^
/usr/include/machine/atomic.h:205:4: note: expanded from macro 'ATOMIC_CMPSET'
: "=@cce" (res),/* 0 */ \
  ^
/usr/include/machine/atomic.h:233:1: error: invalid output constraint '=@cce' 
in asm
/usr/include/machine/atomic.h:222:4: note: expanded from macro 'ATOMIC_CMPSET'
: "=@cce" (res),/* 0 */ \
  ^
/usr/include/machine/atomic.h:280:4: error: invalid output constraint '=@ccc' 
in asm
: "=@ccc" (res),/* 0 */
  ^
/usr/include/machine/atomic.h:296:4: error: invalid output constraint '=@ccc' 
in asm
: "=@ccc" (res),/* 0 */
  ^
/usr/include/machine/atomic.h:312:4: error: invalid output constraint '=@ccc' 
in asm
: "=@ccc" (res),/* 0 */
  ^
/usr/include/machine/atomic.h:328:4: error: invalid output constraint '=@ccc' 
in asm
: "=@ccc" (res),/* 0 */
  ^



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358459 - in head/contrib: gcc gcclibs

[Mark Millard via svn-src-head]

Pedro Giffuni pfg at FreeBSD.org wrote on
Sat Feb 29 15:42:34 UTC 2020 :

> On 29/02/2020 07:40, Ed Maste wrote:
> > Author: emaste
> > Date: Sat Feb 29 12:40:27 2020
> > New Revision: 358459
> > URL: https://svnweb.freebsd.org/changeset/base/358459
> >
> > Log:
> >Remove contrib/gcc and contrib/gcclibs
> >
> >GCC 4.2.1 was disconnected from FreeBSD in r358454.
> >
> >Sponsored by:The FreeBSD Foundation
> >
> > Deleted:
> >head/contrib/gcc/
> >head/contrib/gcclibs/
> And now it's really gone!
> 
> While I am no doubt very happy to see this happen, I just have to thank 
> the FSF for keeping a competitive compiler for such a long time and 
> raising the bar. When the project started and we needed a compiler, they 
> were there.
> 
> We will also not miss it since we will continue to enjoy it as an 
> important package in FreeBSD ports collection :).


As of head -r358472 , it looks like contrib/libstdc++ has
survived so far. It may be another thing that goes.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358462 - head/share/mk

[Mark Millard via svn-src-head]

> Author: emaste
> Date: Sat Feb 29 13:15:01 2020
> New Revision: 358462
> URL: 
> https://svnweb.freebsd.org/changeset/base/358462
> 
> 
> Log:
>   src.opts.mk: simplify Clang and lld bootstrap defaults
>   
>   With the retirement of GCC 4.2.1 we can assume the host compiler supports
>   C++11, and can simplify the Clang and LLD defaults.  Clang and lld are now
>   enabled by default everywhere, and are used as the bootstrap compiler and
>   linker for all targets except MIPS.


lld vs. 32-bit powerpc?

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358454 - in head: . etc/mtree gnu/lib gnu/lib/libgcov gnu/lib/libgomp gnu/lib/libstdc++ gnu/lib/libsupc++ gnu/usr.bin gnu/usr.bin/cc gnu/usr.bin/gperf share/mk targets/pseudo/bootst

[Mark Millard via svn-src-head]

In cleaning out my old gcc 4.2.1 related material I have patches in places
that have not been deleted:

M   /usr/src/contrib/gcc/unwind-dw2.c
M   /usr/src/contrib/gcc/unwind-dw2.h

Is /head/contrib/gcc deliberately being kept? (binutils
needing it for ld for 32-bit powerpc? System gdb needing
it?)

I do not have patches below /head/contrib/gcclibs but it
exists as well. Similar questions, I guess.

(I've still got things back at head -r358132 but am cleaning
things out well before I update.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358392 - head/share/man/man9

[Mark Millard via svn-src-head]

[Ignoring llvm-project and libstdc++, I only find
about 24 other instances of "static_assert".]

On 2020-Feb-27, at 19:03, Mark Millard  wrote

> On 2020-Feb-27, at 16:37, John Baldwin  wrote:
> 
>> On 2/27/20 2:45 PM, Mark Millard wrote:
>>> John Baldwin jhb at FreeBSD.org wrote on
>>> Thu Feb 27 16:55:01 UTC 2020:
>>> 
 On 2/27/20 7:30 AM, Warner Losh wrote:
> Author: imp
> Date: Thu Feb 27 15:30:13 2020
> New Revision: 358392
> URL: https://svnweb.freebsd.org/changeset/base/358392
> 
> Log:
> _Static_assert is to be preferred to CTASSERT.
> 
> Document the existing prefernce that _Static_assert be used in preference 
> to the
> old CTASSERT we used to use for compile time assertions.
 
 Actually, I think what we want to use is static_assert().  The intention in
 userland C is that _Static_assert() is an internal keyword and 
 adds static_assert() as an alias, similar to  defining alignas,
 etc.  I think what we should do for the kernel is have  define
 map static_assert to _Static_assert and replace existing _Static_assert
 usage with the proper spelling.
 
>>> 
>>> Be warned static_assert is a C++ keyword as of C++11.
>>> 
>>> c++11 added: static_assert(bool_constexpr,message)
>>> c++17 added: static_assert(bool_constexpr)
>>> 
>>> C11 added _Static_assert(expression,message)
>>> C2x gets  _Static_assert(expression)
>>> 
>>> C11 added "#define static_assert _Static_assert" to 
>>> 
>>> It makes for a bit of a mess in code to be allowed
>>> to be processed by both C and C++.
>>> 
>>> The wording may need to specify enough to tell what to
>>> do for such code and the headers may need logic to
>>> cause that context to exist across both languages
>>> when the header is allowed for both.
>> 
>> The intent of the C11 changes is to permit equivalent use of static_assert()
>> in both languages.  The #define in  is guarded to not kick in for
>> C++.
> 
> Ahh. I had vague memories of past oddities in the area
> that I was involved in. So I looked and found (in head
> -r3578132 source) :
> 
> # more /usr/src/sys/sys/cdefs.h
> . . .
> #if !__has_extension(c_static_assert)
> #if (defined(__cplusplus) && __cplusplus >= 201103L) || \
>__has_extension(cxx_static_assert)
> #define _Static_assert(x, y)static_assert(x, y)
> #elif __GNUC_PREREQ__(4,6) && !defined(__cplusplus)
> /* Nothing, gcc 4.6 and higher has _Static_assert built-in */
> #elif defined(__COUNTER__)
> #define _Static_assert(x, y)__Static_assert(x, __COUNTER__)
> #define __Static_assert(x, y)   ___Static_assert(x, y)
> #define ___Static_assert(x, y)  typedef char __assert_ ## y[(x) ? 1 : -1] \
>__unused
> #else
> #define _Static_assert(x, y)struct __hack
> #endif
> #endif
> . . .
> 
> # more /usr/include/assert.h
> . . .
> /*
> * Static assertions.  In principle we could define static_assert for
> * C++ older than C++11, but this breaks if _Static_assert is
> * implemented as a macro.
> *
> * C++ template parameters may contain commas, even if not enclosed in
> * parentheses, causing the _Static_assert macro to be invoked with more
> * than two parameters.
> */
> #if __ISO_C_VISIBLE >= 2011 && !defined(__cplusplus)
> #define static_assert   _Static_assert
> #endif
> . . .
> 
> So one can define a _Static_assert macro, sometimes in terms of
> static_assert, and the other sometimes defines a static_assert
> macro in terms of _Static_assert.
> 
> Messy, but possibly localized.
> 
> That prompted a more general grep:
> 
> # grep -r "define[ _]*[Ss]tatic_assert" /usr/src/* | more
> /usr/src/contrib/llvm-project/libcxx/include/__config:# define 
> static_assert(...) _Static_assert(__VA_ARGS__)
> /usr/src/contrib/llvm-project/libunwind/src/config.h:  #define 
> static_assert(__b, __m) \
> /usr/src/include/assert.h: * Static assertions.  In principle we could define 
> static_assert for
> /usr/src/include/assert.h:#define   static_assert   _Static_assert
> /usr/src/sys/sys/cdefs.h:#define_Static_assert(x, y)
> static_assert(x, y)
> /usr/src/sys/sys/cdefs.h:#define_Static_assert(x, y)
> __Static_assert(x, __COUNTER__)
> /usr/src/sys/sys/cdefs.h:#define__Static_assert(x, y)   
> ___Static_assert(x, y)
> /usr/src/sys/sys/cdefs.h:#define___Static_assert(x, y)  typedef char 
> __assert_ ## y[(x) ? 1 : -1] \
> /usr/src/sys/sys/cdefs.h:#define_Static_assert(x, y)struct __hack
> /usr/src/usr.bin/dtc/util.hh:#define static_assert(x, y) ((void)0)
> 
> So not much else.

Just an FYI: Looks like static_assert is little used (outside
llvm-project and libstdc++):

# grep -r "\" /usr/src/* | grep -v llvm-project | grep -v 
"libstdc\+\+" | more
/usr/src/contrib/googletest/googlemock/include/gmock/gmock-matchers.h:  
static_assert(
/usr/src/contrib/googletest/googlemock/include/gmock/gmock-matchers.h:
static_assert(sizeof...(Args) > 0, "Must have at least one matcher.");

Re: svn commit: r358392 - head/share/man/man9

[Mark Millard via svn-src-head]

On 2020-Feb-27, at 16:37, John Baldwin  wrote:

> On 2/27/20 2:45 PM, Mark Millard wrote:
>> John Baldwin jhb at FreeBSD.org wrote on
>> Thu Feb 27 16:55:01 UTC 2020:
>> 
>>> On 2/27/20 7:30 AM, Warner Losh wrote:
 Author: imp
 Date: Thu Feb 27 15:30:13 2020
 New Revision: 358392
 URL: https://svnweb.freebsd.org/changeset/base/358392
 
 Log:
  _Static_assert is to be preferred to CTASSERT.
 
  Document the existing prefernce that _Static_assert be used in preference 
 to the
  old CTASSERT we used to use for compile time assertions.
>>> 
>>> Actually, I think what we want to use is static_assert().  The intention in
>>> userland C is that _Static_assert() is an internal keyword and 
>>> adds static_assert() as an alias, similar to  defining alignas,
>>> etc.  I think what we should do for the kernel is have  define
>>> map static_assert to _Static_assert and replace existing _Static_assert
>>> usage with the proper spelling.
>>> 
>> 
>> Be warned static_assert is a C++ keyword as of C++11.
>> 
>> c++11 added: static_assert(bool_constexpr,message)
>> c++17 added: static_assert(bool_constexpr)
>> 
>> C11 added _Static_assert(expression,message)
>> C2x gets  _Static_assert(expression)
>> 
>> C11 added "#define static_assert _Static_assert" to 
>> 
>> It makes for a bit of a mess in code to be allowed
>> to be processed by both C and C++.
>> 
>> The wording may need to specify enough to tell what to
>> do for such code and the headers may need logic to
>> cause that context to exist across both languages
>> when the header is allowed for both.
> 
> The intent of the C11 changes is to permit equivalent use of static_assert()
> in both languages.  The #define in  is guarded to not kick in for
> C++.

Ahh. I had vague memories of past oddities in the area
that I was involved in. So I looked and found (in head
-r3578132 source) :

# more /usr/src/sys/sys/cdefs.h
. . .
#if !__has_extension(c_static_assert)
#if (defined(__cplusplus) && __cplusplus >= 201103L) || \
__has_extension(cxx_static_assert)
#define _Static_assert(x, y)static_assert(x, y)
#elif __GNUC_PREREQ__(4,6) && !defined(__cplusplus)
/* Nothing, gcc 4.6 and higher has _Static_assert built-in */
#elif defined(__COUNTER__)
#define _Static_assert(x, y)__Static_assert(x, __COUNTER__)
#define __Static_assert(x, y)   ___Static_assert(x, y)
#define ___Static_assert(x, y)  typedef char __assert_ ## y[(x) ? 1 : -1] \
__unused
#else
#define _Static_assert(x, y)struct __hack
#endif
#endif
. . .

# more /usr/include/assert.h
. . .
/*
 * Static assertions.  In principle we could define static_assert for
 * C++ older than C++11, but this breaks if _Static_assert is
 * implemented as a macro.
 *
 * C++ template parameters may contain commas, even if not enclosed in
 * parentheses, causing the _Static_assert macro to be invoked with more
 * than two parameters.
 */
#if __ISO_C_VISIBLE >= 2011 && !defined(__cplusplus)
#define static_assert   _Static_assert
#endif
. . .

So one can define a _Static_assert macro, sometimes in terms of
static_assert, and the other sometimes defines a static_assert
macro in terms of _Static_assert.

Messy, but possibly localized.

That prompted a more general grep:

# grep -r "define[ _]*[Ss]tatic_assert" /usr/src/* | more
/usr/src/contrib/llvm-project/libcxx/include/__config:# define 
static_assert(...) _Static_assert(__VA_ARGS__)
/usr/src/contrib/llvm-project/libunwind/src/config.h:  #define 
static_assert(__b, __m) \
/usr/src/include/assert.h: * Static assertions.  In principle we could define 
static_assert for
/usr/src/include/assert.h:#define   static_assert   _Static_assert
/usr/src/sys/sys/cdefs.h:#define_Static_assert(x, y)
static_assert(x, y)
/usr/src/sys/sys/cdefs.h:#define_Static_assert(x, y)
__Static_assert(x, __COUNTER__)
/usr/src/sys/sys/cdefs.h:#define__Static_assert(x, y)   
___Static_assert(x, y)
/usr/src/sys/sys/cdefs.h:#define___Static_assert(x, y)  typedef char 
__assert_ ## y[(x) ? 1 : -1] \
/usr/src/sys/sys/cdefs.h:#define_Static_assert(x, y)struct __hack
/usr/src/usr.bin/dtc/util.hh:#define static_assert(x, y) ((void)0)

So not much else.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358392 - head/share/man/man9

[Mark Millard via svn-src-head]

John Baldwin jhb at FreeBSD.org wrote on
Thu Feb 27 16:55:01 UTC 2020:

> On 2/27/20 7:30 AM, Warner Losh wrote:
> > Author: imp
> > Date: Thu Feb 27 15:30:13 2020
> > New Revision: 358392
> > URL: https://svnweb.freebsd.org/changeset/base/358392
> > 
> > Log:
> >   _Static_assert is to be preferred to CTASSERT.
> >   
> >   Document the existing prefernce that _Static_assert be used in preference 
> > to the
> >   old CTASSERT we used to use for compile time assertions.
> 
> Actually, I think what we want to use is static_assert().  The intention in
> userland C is that _Static_assert() is an internal keyword and 
> adds static_assert() as an alias, similar to  defining alignas,
> etc.  I think what we should do for the kernel is have  define
> map static_assert to _Static_assert and replace existing _Static_assert
> usage with the proper spelling.
> 

Be warned static_assert is a C++ keyword as of C++11.

c++11 added: static_assert(bool_constexpr,message)
c++17 added: static_assert(bool_constexpr)

C11 added _Static_assert(expression,message)
C2x gets  _Static_assert(expression)

C11 added "#define static_assert _Static_assert" to 

It makes for a bit of a mess in code to be allowed
to be processed by both C and C++.

The wording may need to specify enough to tell what to
do for such code and the headers may need logic to
cause that context to exist across both languages
when the header is allowed for both.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r358164 - in head: . lib/ncurses/ncurses

[Mark Millard via svn-src-head]

[This is not an objection to the version upgrade.]

FYI: I happen to have updated to head -r358132 on multiple
environments, not having done anything to force rebuilds
of things tied to ncurses.

armv7 / 32-bit powerpc : poudriere options displays look normal.

aarch64 / powerpc64 / amd64: they look odd/poor.
(But I could still see what I needed too see to figure out what
to do.)

The breakage seems to be tied to 64-bit contexts in some way.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r356437 - head/lib/libprocstat

[Mark Millard via svn-src-head]

(Derived from O. Hartmann's report, despite his not
having the same source version, . . .)

In the code:

823 static struct filestat_list *
824 procstat_getfiles_sysctl(struct procstat *procstat, struct kinfo_proc 
*kp,
825 int mmapped)
826 {
827 struct kinfo_file *kif, *files;
. . .
841 switch(procstat->type) {
842 case PROCSTAT_SYSCTL:
843 files = kinfo_getfile(kp->ki_pid, );
844 break;
845 case PROCSTAT_CORE:
846 files = kinfo_getfile_core(procstat->core, );
847 break;
848 default:
849 assert(!"invalid type");
850 }
851 if (files == NULL && errno != EPERM) {
852

for the "default:" case, if assert is disabled to
be a no-op via NDEBUG, then files would be tested
in an uninitialized state.

The compiler complaint seems valid for this.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r356379 - head/share/mk

[Mark Millard via svn-src-head]

Justin Hibbits chmeeedalf at gmail.com  wrote on
Sun Jan 5 18:03:59 UTC 2020 :

> On Sun, 5 Jan 2020 12:21:53 -0500 Ed Maste  wrote: > 
> On Sun, 5 Jan 2020 at 11:59, Ed Maste  wrote: 
> > > 
> > > Author: emaste 
> > > Date: Sun Jan 5 16:59:24 2020 
> > > New Revision: 356379 
> > > URL: https://svnweb.freebsd.org/changeset/base/356379
> > > 
> > > Log: 
> > > src.opts.mk: disable BINUTILS_BOOTSTRAP on mips64* 
> > 
> > I should probably make this the case for all of mips* and sparc64. 
> > Using a decade-old binutils with a contemporary compiler (either Clang 
> > or GCC) isn't likely to be a combination used by anyone else and it's 
> > not going to be a good use of our time investigating and addressing 
> > any issues that arise. 
> powerpc and powerpcspe still use in-tree binutils (or external binutils... 
> but in-tree works well enough, until we start adding ifuncs), until such a 
> time that lld is fully functional. 

FYI:

I've tried the following combinations involving
devel/binutils@powerpc for 32-bit powerpc, either
the kernel crashed extremely early or the build
fails to complete.

system-clang and devel/binutils@powerpc (the crashing case)
devel/gcc9@powerpc and devel/binutils@powerpc (fails to build)

(These were cross builds.)

On the old 32-bit PowerMacs the crash is before
Apple's displayed material is replaced and it
gets back to Apple's prompt after the error
message.

The only combination that I've had success with
so far for 32-bit powerpc has been system-clang
for buildkernel with the in-tree-binutils used.

This may not matter much until ifuncs additions
are desired --or ever if lld becomes sufficient
first.

The build failure is for:

--- libc.so.7.full ---
/usr/local/bin/powerpc-unknown-freebsd13.0-ld: bss-plt forced due to 
/usr/obj/powerpcvtsc_xtoolchain-gcc/powerpc.powerpc/usr/src/powerpc.powerpc/tmp/usr/lib/crtbeginS.o
collect2: error: ld returned 1 exit status
*** [libc.so.7.full] Error code 1

As for the crashing case, looking at the kernel
produced show significant differences in where
ABS is used and some other differences. (I do
not claim anything like a complete comparison.)

Without work on one or both sides,
devel/binutils@powerpc does not seem to be
an alternative.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r356344 - head

[Mark Millard via svn-src-head]

> Author: jhb
> Date: Sat Jan  4 00:59:47 2020
> New Revision: 356344
> URL: 
> https://svnweb.freebsd.org/changeset/base/356344
> . . .
> +MAKE_PARAMS_powerpc?=CROSS_TOOLCHAIN=powerpc64-gcc6
> . . .
> +TOOLCHAINS_powerpc=  powerpc64-gcc6
> . . .

(I know that for now gcc9 is not in use but
I'm looking at the overall pattern going
forward vs. what is expected to work shorter
term, even if not yet fully.)

Given the likes of:

TARGET=powerpc TARGET_ARCH=powerpc
TARGET=powerpc TARGET_ARCH=powerpc64

Is this going to change at some point to use
powerpc-gcc9 and powerpc64-gcc9 (and the matching
devel/binutils@powerpc and devel/binutils@powerpc64
related commands), picking the matching commands?
(More TARGET_ARCH based?)

Until then, is it really restricted to powerpc64
because of the toolchain used (and lack of various
command parameters to possibly control all the
required behavioral differences for targeting
32-bit powerpc)?

(powerpcspe has been dropped from gcc, if I
understand right. So I do not expect that it
can be covered. So I did not list it as another
TARGET_ARCH value above.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r356289 - head

[Mark Millard via svn-src-head]

John Baldwin jhb at FreeBSD.org wrote on
Thu Jan 2 21:41:07 UTC 2020 :

> On 1/2/20 1:34 PM, John Baldwin wrote:
> > Author: jhb
> > Date: Thu Jan  2 21:34:44 2020
> > New Revision: 356289
> > URL: https://svnweb.freebsd.org/changeset/base/356289
> > 
> > Log:
> >   Look for cross toolchain makefiles in /usr/share/toolchains.
> >   
> >   The freebsd-binutils and freebsd-gcc* packages install toolchain
> >   makefiles to /usr/share/toolchains rather than LOCALBASE.
> 
> The short version is that you can do something like this to use GCC
> as the system compiler (/usr/bin/cc):
> 
> cd /usr/ports/base/binutils ; make install clean
> cd ../gcc6 ; make install clean
> 
> Then 'make CROSS_TOOLCHAIN=freebsd-gcc6 buildworld', etc.  If you aren't
> planning on doing cross-builds you can set CROSS_TOOLCHAIN in /etc/src.conf.
> 
> As described elsewhere, the base/* packages can be cross-built (along
> with pkg), so for any architectures not yet using clang we could fairly
> easily provide a cross-built package repo (though that architecture list
> is becoming rather small).  I will probably add a base/gcc9 port once we
> can build a full system with gcc9.

Just an FYI from some experiments that I did recently:

devel/binutils@powerpc and devel/binutils@powerpc64 and
base/binutils@powerpc and base_binutils@powerpc64 do
not put out the same content as lld or the old
toolchain's ld (still used normally for 32-bit powerpc).
buildkernel runs to completion but the result
crashes from the kernel sseflf-loading and what was
produced not matching. For example, binutils put out
very few ABS symbols compared to lld/old-ld and
classifies the kernel as ET_EXEC, not ET_DYN, because
of the non-zero VirtAddr in:

Program Headers:
  Type   Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
. . .
  LOAD   0x00 0x0010 0x0010 0xd9786c 0x1083648 RWE 0x1
. . .

(I'm not claiming that is all there is that matters.)


These points are true using system-clang as the
compiler. (I've only cross-built so far.)

I do not know if other architectures have similar issues
or not. But, it appears that in some cases, there is more
work to allowing the GNU linker to be used, for at least
buildkernel .

I do not know if there is an intent to support a (modern)
GNU binutils ld (in addition to lld) or not. So it may be
that the effort would not be put in. (I'm not claiming
which side(s) would change if the effort was put in.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r356054 - in head: share/mk stand

[Mark Millard via svn-src-head]

On 2019-Dec-24, at 21:26, Mark Millard  wrote:

> In:
> 
> +# Defines a variable for Binutils linker, to be used to workaround some
> +# issue with LLVM LLD (i.e. support for PowerPC32 bit on PowerPC64)
> +#
> +# This is an unavoidable cross coupling with Makefile.inc1 and
> +# normal builds works when CROSS_BINUTILS_PREFIX and could be removed
> +# when LLD PowerPC 32 bit support is completed
> +.if defined(CROSS_BINUTILS_PREFIX)
> +LD_BFD=${LOCALBASE}/bin/${CROSS_BINUTILS_PREFIX}-ld.bfd
> +.else
> +LD_BFD=${OBJTOP}/tmp/usr/bin/ld.bfd
> +.endif
> 
> I'm confused by the 2 lines:
> 
> +.if defined(CROSS_BINUTILS_PREFIX)
> +LD_BFD=${LOCALBASE}/bin/${CROSS_BINUTILS_PREFIX}-ld.bfd
> 
> Looking in /usr/local/share/toolchains/*.mk I see
> CROSS_BINUTILS_PREFIX as a full path in each case
> ( based on devel/freebsd-gcc[69] in my context):
> 
> # grep -r CROSS_BINU /usr/local/share/toolchains/ | more
> /usr/local/share/toolchains/aarch64-gcc6.mk:CROSS_BINUTILS_PREFIX=/usr/local/aarch64-unknown-freebsd13.0/bin/
> /usr/local/share/toolchains/amd64-gcc6.mk:CROSS_BINUTILS_PREFIX=/usr/local/x86_64-unknown-freebsd13.0/bin/
> /usr/local/share/toolchains/powerpc64-gcc6.mk:CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-unknown-freebsd13.0/bin/
> /usr/local/share/toolchains/aarch64-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/aarch64-unknown-freebsd13.0/bin/
> /usr/local/share/toolchains/amd64-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/x86_64-unknown-freebsd13.0/bin/
> /usr/local/share/toolchains/powerpc64-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-unknown-freebsd13.0/bin/
> /usr/local/share/toolchains/powerpc-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/powerpc-unknown-freebsd13.0/bin/
> 
> And when I look in /usr/srcv/Makefile.inc1 I see:
> 
> /usr/src/Makefile.inc1:CROSS_BINUTILS_PREFIX=/usr/local/${TARGET_TRIPLE}/bin/
> 
> Having an overall path structure like:
> 
> ${LOCALBASE}/bin/usr/local/powerpc64-unknown-freebsd13.0/bin/-ld.bfd
> 
> seems strange (using powerpc64 as an example).
> 

I got to the point of trying a build that happens to have
tried to use this code. It failed, reporting:

--- libc.so.7.full ---
building shared library libc.so.7
cc: error: invalid linker name in argument 
'-fuse-ld=/usr/local/bin//usr/local/powerpc64-unknown-freebsd13.0/bin/-ld.bfd'
*** [libc.so.7.full] Error code 1

So I switched the code to:

.if defined(CROSS_BINUTILS_PREFIX)
LD_BFD=${CROSS_BINUTILS_PREFIX}ld.bfd
.else
LD_BFD=${OBJTOP}/tmp/usr/bin/ld.bfd
.endif

With that in place, the build completed.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r356116 - head (powerpc families jumping to clang-based; powerpc64 also to ELFv2)

[Mark Millard via svn-src-head]

Ronald Klop ronald-lists at klop.ws wrote on
Fri Dec 27 09:39:22 UTC 2019 :

QUOTE
Do powerpc people need to do something special while updating?
Like a clean buildkernel/buildworld. Or is this just a note for historical  
bookkeeping?
END QUOTE


See https://lists.freebsd.org/pipermail/freebsd-ppc/2019-December/011042.html

But it was -r356118 before powerpc64 and 32-bit powerpc
started to complete builds on ci.freebsd.org . powerpcspe
still fails to build there.

I agree that UPDATING could use more material guiding
actions, especially for powerpc64 where the ELFv1->ELFv2
change is an ABI incompatibility. Quoting from that
message:

QUOTE
Therefore, it is recommended to perform a complete reinstall of the
system on powerpc64 platforms.
END QUOTE

There may well have been an implicit "from scratch"
intended in that quote. No use of old port builds
either for powerpc64.

Those using binary updates of ports via pkg for
powerpc64 may have mismatch issues around the
transition. I've no clue for binary system updates
(if there are such for powerpc64). (I've only
built from source for world, kernel, and ports
for my normal activity.)



Another issue for those that build from source is
the RAM use and time taken for buildworld buildkernel ,
at least on older or low end hardware, such as old
PowerMacs. The armv7 and small board aarch64 folks
experience with this may prove useful. Avoiding Out
Of Memory process kills from free RAM staying low for
long periods or from slow paging or other issues can
be a challenge. Unfortunately, the messages complain
about being "out of swap" even when that is not the
issue. It is not always clear just what criteria leads
to a OOM process kill, making it hard to figure out
how to avoid those happening.

Real "out of swap" conditions also normally report
console messages of the form:

. . . swap_pager_getswapspace . . .: failed

Otherwise it is unlikely it is actually out of
swap space.

For delaying how long free RAM staying low is
tolerated, one can increase vm.pageout_oom_seq from
12 to larger. The management of slow paging I've
less experience with. With low enough RAM it may
not be possible to have enough swap/page space to
allow llvm materials to all build, even for -j1
builds.

Examples follow that I use in contexts with
sufficient RAM that I do not have to worry about
out of swap/page space for how I buildworld
buildkernel . These I've set in /etc/sysctl.conf .

#
# Delay when persisstent low free RAM leads to
# Out Of Memory killing of processes:
vm.pageout_oom_seq=120
#
# For plunty of swap/paging space (will not
# run out), avoid pageout delays leading to
# Out Of Memory killing of processes:
vm.pfault_oom_attempts=-1

Note: In my context "plunty" really means
sufficient RAM that paging is rare for how
I build. But others have reported on using
the -1 in contexts where paging was heavy
at times and OOM kills had been happening
that were eliminated by the assignment
(more built).

I've no experience with the below alternative
to that -1 use:

#
# For possibly insufficient swap/paging space
# (might run out), increase the pageout delay
# that leads to Out Of Memory killing of
# processes:
#vm.pfault_oom_attempts= ???
#vm.pfault_oom_wait= ???
# (The multiplication is the total but there
# are other potential tradoffs in the factors
# multiplied, even for nearly the same total.)

There are folks that have used the likes
of:

vm.pageout_oom_seq=1200

(but I do not know if such was necessary
for their context.)



I'm not claiming that these 3 vm.??? figures
are always sufficient. Nor am I claiming
that tunables are always available that would
be sufficient. Nor that it is easy to find
the ones that do exist that might help for
specific OOM kill issues.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r356054 - in head: share/mk stand

[Mark Millard via svn-src-head]

In:

+# Defines a variable for Binutils linker, to be used to workaround some
+# issue with LLVM LLD (i.e. support for PowerPC32 bit on PowerPC64)
+#
+# This is an unavoidable cross coupling with Makefile.inc1 and
+# normal builds works when CROSS_BINUTILS_PREFIX and could be removed
+# when LLD PowerPC 32 bit support is completed
+.if defined(CROSS_BINUTILS_PREFIX)
+LD_BFD=${LOCALBASE}/bin/${CROSS_BINUTILS_PREFIX}-ld.bfd
+.else
+LD_BFD=${OBJTOP}/tmp/usr/bin/ld.bfd
+.endif

I'm confused by the 2 lines:

+.if defined(CROSS_BINUTILS_PREFIX)
+LD_BFD=${LOCALBASE}/bin/${CROSS_BINUTILS_PREFIX}-ld.bfd

Looking in /usr/local/share/toolchains/*.mk I see
CROSS_BINUTILS_PREFIX as a full path in each case
( based on devel/freebsd-gcc[69] in my context):

# grep -r CROSS_BINU /usr/local/share/toolchains/ | more
/usr/local/share/toolchains/aarch64-gcc6.mk:CROSS_BINUTILS_PREFIX=/usr/local/aarch64-unknown-freebsd13.0/bin/
/usr/local/share/toolchains/amd64-gcc6.mk:CROSS_BINUTILS_PREFIX=/usr/local/x86_64-unknown-freebsd13.0/bin/
/usr/local/share/toolchains/powerpc64-gcc6.mk:CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-unknown-freebsd13.0/bin/
/usr/local/share/toolchains/aarch64-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/aarch64-unknown-freebsd13.0/bin/
/usr/local/share/toolchains/amd64-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/x86_64-unknown-freebsd13.0/bin/
/usr/local/share/toolchains/powerpc64-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-unknown-freebsd13.0/bin/
/usr/local/share/toolchains/powerpc-gcc9.mk:CROSS_BINUTILS_PREFIX=/usr/local/powerpc-unknown-freebsd13.0/bin/

And when I look in /usr/srcv/Makefile.inc1 I see:

/usr/src/Makefile.inc1:CROSS_BINUTILS_PREFIX=/usr/local/${TARGET_TRIPLE}/bin/

Having an overall path structure like:

${LOCALBASE}/bin/usr/local/powerpc64-unknown-freebsd13.0/bin/-ld.bfd

seems strange (using powerpc64 as an example).

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r355819 - in head/sys: arm/arm arm64/arm64 i386/i386 mips/mips powerpc/powerpc riscv/riscv

[Mark Millard via svn-src-head]

> Author: jeff
> Date: Mon Dec 16 20:15:04 2019
> New Revision: 355819
> URL: 
> https://svnweb.freebsd.org/changeset/base/355819
> 
> 
> Log:
>   Repeat the spinlock_enter/exit pattern from amd64 on other architectures to
>   fix an assert violation introduced in r355784.  Without this spinlock_exit()
>   may see owepreempt and switch before reducing the spinlock count.  amd64
>   had been optimized to do a single critical enter/exit regardless of the
>   number of spinlocks which avoided the problem and this optimization had
>   not been applied elsewhere.
>   
>   Reported by:emaste
>   Suggested by:   rlibby
>   Discussed with: jhb, rlibby
>   Tested by:  manu (arm64)
> 
> Modified:
>   head/sys/arm/arm/machdep.c
>   head/sys/arm64/arm64/machdep.c
>   head/sys/i386/i386/machdep.c
>   head/sys/mips/mips/machdep.c
>   head/sys/powerpc/powerpc/machdep.c
>   head/sys/riscv/riscv/machdep.c

It looks like sparc64 still has the old code structure:

>From /usr/src/sys/sparc64/sparc64/machdep.c . . .

void
spinlock_enter(void)
{
struct thread *td;
register_t pil;

td = curthread;
if (td->td_md.md_spinlock_count == 0) {
pil = rdpr(pil);
wrpr(pil, 0, PIL_TICK);
td->td_md.md_spinlock_count = 1;
td->td_md.md_saved_pil = pil;
} else
td->td_md.md_spinlock_count++;
critical_enter();
}

void
spinlock_exit(void)
{
struct thread *td;
register_t pil;

td = curthread;
critical_exit();
pil = td->td_md.md_saved_pil;
td->td_md.md_spinlock_count--;
if (td->td_md.md_spinlock_count == 0)
wrpr(pil, pil, 0);
}


(Not that I ever do anything with sparc64 machines.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r355759 - in head: include sys/sys

[Mark Millard via svn-src-head]

Now it is clang builds that are broken, at least ones that use:

sys/compat/linuxkpi/common/include/linux/compiler.h

An example from:

https://ci.freebsd.org/job/FreeBSD-head-aarch64-build/14953/console

is:

14:32:22 --- all_subdir_lindebugfs ---
14:32:22 In file included from /usr/src/sys/compat/lindebugfs/lindebugfs.c:72:
14:32:22 In file included from 
/usr/src/sys/compat/linuxkpi/common/include/linux/debugfs.h:33:
14:32:22 In file included from 
/usr/src/sys/compat/linuxkpi/common/include/linux/fs.h:41:
14:32:22 In file included from 
/usr/src/sys/compat/linuxkpi/common/include/linux/types.h:38:
14:32:22 /usr/src/sys/compat/linuxkpi/common/include/linux/compiler.h:53:9: 
error: '__deprecated' macro redefined [-Werror,-Wmacro-redefined]
14:32:22 #define __deprecated
14:32:22 ^
14:32:22 /usr/src/sys/sys/cdefs.h:472:9: note: previous definition is here
14:32:22 #define __deprecated(m) __attribute__((__deprecated__(m)))
14:32:22 ^



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r352558 - head/usr.bin/top

[Mark Millard via svn-src-head]

> Author: daichi
> Date: Fri Sep 20 17:37:23 2019
> New Revision: 352558
> URL: 
> https://svnweb.freebsd.org/changeset/base/352558
> 
> 
> Log:
>   top(1): support multibyte characters in command names (ARGV array)
>   depending on locale.
>   
>- add setlocale()
>- remove printable() function
>- add VIS_OCTAL and VIS_SAFE to the flag of strvisx() to display
>  non-printable characters that do not use C-style backslash sequences
>  in three digit octal sequence, or remove it
>   
>   This change allows multibyte characters to be displayed according to
>   locale. If it is recognized as a non-display character according to the
>   locale, it is displayed in three digit octal sequence.
> 

Initially picking on tab characters as an example of what is
probably a somewhat broader issue . . .

Ever since this change, characters like tabs that do not fit
in the next character cell when output, but for which they
are !isprintable(...), now mess up the top display. Again
using tab as an example: line wrapping from the text having
been shifted over by more than one character cell. top does
not track the line wrapping result in how it decides what
to output for the following display updates.

As stands I type "hh" to referesh the display when I care
about clearing up the messy display.

The removed !isprintable handling functionality had its uses
and an alternative is needed for some types of characters if
the display is not to be messed up by what some command lines
that are echoed have in them.

I picked on tab because it was obviously involved in some
examples that I watched update. I'm not claiming tab is the
only character that ends up with e cursor in an unexpected
place now. In fact . . .

Looking around I found:

 VIS_SAFEOnly encode "unsafe" characters.  Unsafe means control char-
 acters which may cause common terminals to perform unexpected
 functions.  Currently this form allows space, tab, newline,
 backspace, bell, and return - in addition to all graphic
 characters - unencoded.

So tab, newline, backspace, bell, and return are not encoded
or replaced now but do not have top's expected cursor position
result.


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r351364 - in head/sys: crypto/blowfish crypto/chacha20 crypto/des opencrypto

[Mark Millard via svn-src-head]

Warner Losh imp at bsdimp.com wrote on
Fri Aug 23 22:05:39 UTC 2019 :

> On Fri, Aug 23, 2019 at 12:26 PM Conrad Meyer  wrote:
> 
> > At expected peril of wading into a thread >4 emails deep,
> >
> > @Warner, modern GCC reports a similar warning; it just doesn't become
> > an error (at least in CI?).  I'm not sure of the mechanism.  Maybe
> > CI-specific?  Old GCC didn't have a -Wno-error for -Wcast-qual, so
> > -Wcast-qual + -Werror there produced an error; that's why
> > $NO_WCAST_QUAL in conf/kern.mk is defined to -Wno-cast-qual for old
> > GCC but -Wno-error=cast-qual for newer compilers.  That said, this
> > file does not appear to be compiled with ${NO_WCAST_QUAL} either way.
> >
> 
> Yea. I'm unsure. It's an odd warning, and an odd way to get around it. In
> general, nobody cares about gcc 4.2.1, so pinning implementing that belief
> to this specific bug may have been unwise. I just assumed newer versions
> wouldn't warn on this, but I saw on IRC that the types are stupidly
> different...
. . .

While I know how to build for powerpc64 ( using WITHOUT_LIB32= )
without using gcc4.2.1 or the matching binutils, 32-bit powerpc
is not so clear for how to target without gcc 4.2.1, at least
based on what I'm familiar with. (I normally work with materials
from the official FreeBSD svn, not other development material
sources.)

(Locally reverting the secure-plt change would help for
amd64->powerpc cross builds based on system-clang and
devel/powerpc64-binutils or base/binutils . Even lld from
devel/llvm90 is not ready for 32-bit powerpc as yet, from what
I've seen, so llvm90 is not yet a self-contained solution.)

May be someone else does know a good way to build FreeBSD for
tracking head for targeting 32-bit powerpc as things are now?
May I the only one not knowing how to build for this target
as things are?

I'd been trying to avoid disabling the secure-plt change
that causes gnu's modern ld's to revert to bss-plt and return
an error code that stops the build when system-clang or
llvm90 is used. (The old FreeBSD ld does not return the error
code but does report reverting to bss-plt. But it fails for
buildkernel via a separate issue when system-clang is used.)

As stands I'm not aware of how to target 32-bit power pc
from some more modern gcc/binutils combination.

As stands it looks like reverting secure-plt's use is the
direction I'd have to go to avoid gcc 4.2.1 .

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r351253 - in head/contrib/libc++: include src

[Mark Millard via svn-src-head]

> Author: dim
> Date: Tue Aug 20 17:39:32 2019
> New Revision: 351253
> URL: 
> https://svnweb.freebsd.org/changeset/base/351253
> 
> 
> Log:
>   Pull in r368867 from upstream libc++ trunk (by Marshall Clow):
>   
> Rework recursive_timed_mutex so that it uses __thread_id instead of
> using the lower-level __libcpp_thread_id. This is prep for fixing
> PR42918. Reviewed as 
> https://reviews.llvm.org/D65895
> 
>   
>   Pull in r368916 from upstream libc++ trunk (by Marshall Clow):
>   
> Fix thread comparison by making sure we never pass our special 'not a
> thread' value to the underlying implementation. Fixes PR#42918.

FYI: llvm's -r368916 has a successor fix: -r329399 tied to:

https:://reviews.llvm.org/D66480

that is summarized with:

QUOTE
In fixing https://bugs.llvm.org/show_bug.cgi?id=42918 (r368916), I broke 
external threading support.
This should fix that
END QUOTE


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r346588 - head/lib/libc/powerpc64/string

[Mark Millard via svn-src-head]

[I did not deal with translating register usage correctly.]

> On 2019-Apr-27, at 01:50, Mark Millard  wrote:
> 
> Justin Hibbits jhibbits at FreeBSD.org wrote on
> Fri Apr 26 16:21:47 UTC 2019 :
> 
>> This actually uses 'cmpb' which is only available on PowerISA 2.05+, so
>> I'll need to pull it out for now, and re-enable it once we have
>> ifuncs.  As it stands, this commit broke the G5 and POWER4/POWER5.
> 
> As I understand the code like:
> 
>   xor %r8,%r8,%r8 /* %r8 <- Zero. */
>   xor %r0,%r5,%r6 /* Check if double words are different. */
>   cmpb%r7,%r5,%r8 /* Check if double words contain zero. */
> 
>   /*
>* If double words are different or contain zero,
>* find what byte is different or contains zero,
>* else load next double words.
>*/
>   or. %r9,%r7,%r0
>   bne .Lstrcmp_check_zeros_differences
> 
> (and similarly for the loop. . .):
> 
> A) Each byte of %r5 that is non-zero needs that byte of %r7 to be zero.
> B) Each byte of %r5 that is zero need that byte of %r7 to be non-zero.
> 
> (cmpb assigns 0xff for non-zero as I understand, but even one non-zero
> bit is sufficient for the overall code structure.)
> 
> If I've got that much correct, then the following might be an
> alternative to cmpb for now. I'll explain the code via commented
> c/c++-ish code and then show the assembler notation:
> 
> unsigned long ul_has_zero_byte(unsigned long b)
> {
>unsigned long constexpr low_7bits_of_bytes{0x7f7f7f7f'7f7f7f7ful};
> 
>   // Illustrating byte 
> transformations:
>unsigned long const x= b & low_7bits_of_bytes; // 0x00->0x00, 
> 0x80->0x00, other->ms-bit-in-byte==0
>unsigned long const y= x + low_7bits_of_bytes; // ->0x7f, 
> ->0x7f,  ->ms-bit-in-byte==1
>unsigned long const z= b | y | low_7bits_of_bytes; // ->0x7f, 
> ->0xff,  ->0xff
>return ~z; // ->0x80, 
> ->0x00,  ->0x00
> }
> 
> (used in a powerpc64 context, so unsigned long being 64 bits).
> 
> So, not using %r8 as zero but for a different value,
> each cmpb can be replaced by:
> 
> # Only once to set up the value in %r8 (Note: 32639=0x7f7f):
> lis r8,32639
> ori r8,r8,32639
> rldimi  r8,r8,32,0
> 
> # each "cmpb %r7,%r5,%r8" replaced by:
> and r7,r5,r8
> add r7,r7,r8
> nor r5,r7,r5
> andcr5,r5,r8

The above 4 lines are an incorrect match to the context's
register usage: only r7 of the 3 registers r5, r7, r8
should have been changed. It looks like another temporary
register (for the stage) is required to make a match:

and  %r9,%r5,%r8
add  %r9,%r9,%r8
nor  %r7,%r9,%r5
andc %r7,%r7,%r8

(%r9 later being replaced via: or. %r9,%r7,%r0)

> (The code is from compiler output, but with registers adjusted
> to match the context.)
> 
> 
> The c/c++-ish code came from thinking about material from Hacker's
> Delight Second Edition and the specific criteria needed here: it
> uses part of Figure 6-2 "Find First 0-Byte, branch-free code",
> adjusted for width and for returning something sufficient here.
> 



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r346588 - head/lib/libc/powerpc64/string

[Mark Millard via svn-src-head]

Justin Hibbits jhibbits at FreeBSD.org wrote on
Fri Apr 26 16:21:47 UTC 2019 :

> This actually uses 'cmpb' which is only available on PowerISA 2.05+, so
> I'll need to pull it out for now, and re-enable it once we have
> ifuncs.  As it stands, this commit broke the G5 and POWER4/POWER5.

As I understand the code like:

xor %r8,%r8,%r8 /* %r8 <- Zero. */
xor %r0,%r5,%r6 /* Check if double words are different. */
cmpb%r7,%r5,%r8 /* Check if double words contain zero. */

/*
 * If double words are different or contain zero,
 * find what byte is different or contains zero,
 * else load next double words.
 */
or. %r9,%r7,%r0
bne .Lstrcmp_check_zeros_differences

(and similarly for the loop. . .):

A) Each byte of %r5 that is non-zero needs that byte of %r7 to be zero.
B) Each byte of %r5 that is zero need that byte of %r7 to be non-zero.

(cmpb assigns 0xff for non-zero as I understand, but even one non-zero
bit is sufficient for the overall code structure.)

If I've got that much correct, then the following might be an
alternative to cmpb for now. I'll explain the code via commented
c/c++-ish code and then show the assembler notation:

unsigned long ul_has_zero_byte(unsigned long b)
{
unsigned long constexpr low_7bits_of_bytes{0x7f7f7f7f'7f7f7f7ful};

   // Illustrating byte 
transformations:
unsigned long const x= b & low_7bits_of_bytes; // 0x00->0x00, 
0x80->0x00, other->ms-bit-in-byte==0
unsigned long const y= x + low_7bits_of_bytes; // ->0x7f, 
->0x7f,  ->ms-bit-in-byte==1
unsigned long const z= b | y | low_7bits_of_bytes; // ->0x7f, 
->0xff,  ->0xff
return ~z; // ->0x80, 
->0x00,  ->0x00
}

(used in a powerpc64 context, so unsigned long being 64 bits).

So, not using %r8 as zero but for a different value,
each cmpb can be replaced by:

# Only once to set up the value in %r8 (Note: 32639=0x7f7f):
lis r8,32639
ori r8,r8,32639
rldimi  r8,r8,32,0

# each "cmpb %r7,%r5,%r8" replaced by:
and r7,r5,r8
add r7,r7,r8
nor r5,r7,r5
andcr5,r5,r8

(The code is from compiler output, but with registers adjusted
to match the context.)


The c/c++-ish code came from thinking about material from Hacker's
Delight Second Edition and the specific criteria needed here: it
uses part of Figure 6-2 "Find First 0-Byte, branch-free code",
adjusted for width and for returning something sufficient here.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r346441 - in head/sys/modules: em fusefs iavf

[Mark Millard via svn-src-head]

Using if_igb.ko and if_em.ko as examples:

-r324406 (2017-Oct-7) used relative symbolic links (via a different technique) 
[sbruno]
-r324500 (3 days later) used hard links (ian, with sbruno submitting)

-r346441 (2019-Apr-20) is back to relative symbolic links. (asomers)

Sbruno or Ian might want to comment on why relative links only lasted
3 days last time they were tried.



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r340676 - in head/sys: kern sys

[Mark Millard via svn-src-head]

Mateusz Guzik mjg at FreeBSD.org wrote on
Tue Nov 20 14:58:42 UTC 2018 :

> +#if defined(__mips__) || defined(__powerpc__)
> +#define UNR64_LOCKED
> +#endif

But on powerpc64 ( system clang from head -r339076 ):

# clang -dM -E -x c /dev/null | grep -i __power
#define __POWERPC__ 1
#define __powerpc64__ 1
#define __powerpc__ 1

and (I only have the one gcc* vintage around):

# gcc8 -dM -E -x c /dev/null | grep -i __power
#define __powerpc64__ 1
#define __powerpc__ 1

so the following are used on powerpc64:

+#ifdef UNR64_LOCKED
+uint64_t alloc_unr64(struct unrhdr64 *);
+#else
. . .

and:

+#ifdef UNR64_LOCKED
+uint64_t
+alloc_unr64(struct unrhdr64 *unr64)
+{
+   uint64_t item;
+
+   mtx_lock();
+   item = unr64->counter++;
+   mtx_unlock();
+   return (item);
+}
+#endif

Was that the intent for powerpc64?


Notes about the powerpc64 context used:

# uname -apKU
FreeBSD FBSDG5L 12.0-ALPHA8 FreeBSD 12.0-ALPHA8 #4 r339076M: Mon Oct 15 
13:19:35 PDT 2018 
markmi@FBSDG5L:/usr/obj/powerpc64vtsc_xtoolchain-gcc/powerpc.powerpc64/usr/src/powerpc.powerpc64/sys/GENERIC64vtsc-NODBG
  powerpc powerpc64 1200084 1200084

buildworld buildkernel was via devel/powerpc64-xtoolchain-gcc .

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r340664 - head/sys/sys

[Mark Millard via svn-src-head]

Warner Losh imp at FreeBSD.org wrote on
Tue Nov 20 07:11:24 UTC 2018 :

> Instead, used fixed constants because there's no way to say ceil(X)
> for integer math. . . .

For a ratio of unsigned integers, with 0

// ceil_x_div_y(x,y), given 0https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

[I note what I've failed to find a way to investigate.]

On 2018-Nov-7, at 11:53, Mark Millard  wrote:

> [I trace code associated with bl <1322.plt_call.getenv>
> in the two contexts and extend the range over which things
> appear to match: up to some point after the branch
> b <__glink_PLTresolve> .]
> 
> On 2018-Nov-6, at 19:12, Mark Millard  wrote:
> 
>> [I've present a little information about the longer-existing
>> failure's odd backtrace for /libexec/ld-elf.so.1 /bin/ls
>> --but on powerpc64 FreeBSD instead of 32-bit powerpc FreeBSD.]
>> 
>> On 2018-Nov-2, at 11:50, Konstantin Belousov  wrote:
>> 
>>> On Fri, Nov 02, 2018 at 10:38:08AM -0700, Mark Millard wrote:
 On 2018-Nov-2, at 8:52 AM, Konstantin Belousov  
 wrote:
 
> . . .
 
 That seems better. But it crashes during /bin/ls execution
 ( 0x0180 addresses ), apparently in a library routine
 ( 0x41?? addresses ):
 
 Program received signal SIGSEGV, Segmentation fault.
 0x411220b4 in ?? ()
 (gdb) bt
 #0  0x411220b4 in ?? ()
 #1  0x4112200c in ?? ()
 #2  0x01803c84 in ?? ()
 #3  0x018023b4 in ?? ()
 #4  0x010121a0 in .rtld_start () at 
 /usr/src/libexec/rtld-elf/powerpc/rtld_start.S:112
 
 Using a normal gdb run of /bin/ls suggests:
 
 #2  0x01803c84 in ?? () should be in main and seems to be: bl 0x1818914 
 
 #3  0x018023b4 in ?? () should be in _start
 
 Looking in the test context:
 
 0x1803c80: bl  0x1818914
 0x1803c84: cmpwi   cr7,r3,-1
 
 and:
 
 0x1818914: li  r11,59
 0x1818918: b   0x18186f4
 
 and:
 
 0x18186f4: rlwinm  r11,r11,2,0,29
 0x18186f8: addis   r11,r11,386
 0x18186fc: lwz r11,-30316(r11)
 0x1818700: mtctr   r11
 0x1818704: bctr
 
 Breaking at the bctr and using info reg:
 
 r110x4125ffa0  1093009312
 
 It looks like there is some amount of
 activity before the traceback addresses
 show up.
 
 I've not found a good way to fill in the "in ??()"
 (or analogous) information. The addresses 0x411220??
 do not match up with a normal run of /bin/ls from
 gdb: the addresses can not be accessed.
 
 
 
 It does appear that the code is in /lib/libc.so.7 in the
 test context:
 
 Breakpoint 2, reloc_non_plt (obj=0x41041600, obj_rtld=0x41104b57, flags=4, 
 lockstate=0x0) at /usr/src/libexec/rtld-elf/powerpc/reloc.c:338
 . . .
 
>>> There seems to be an issue with the direct execution mode on ppc.
>>> Even otherwise working ld-elf.so.1 segfaults if I try to use it as
>>> standalone binary.
>>> 
>>> But if I specify patched ld-elf.so.1 as the interpreter for some program,
>>> using 'cc -Wl,-I,/ld-elf.so.1' it works.  So I see there two bugs,
>>> one is regression due to textsize calculation, which should be fixed by
>>> my patch.  Another is the direct exec problem.
>> 
>> I've got a little more information about the odd backtrace
>> from the /libexec/ld-elf.so.1 /bin/ls failure that the
>> prior patch allowed getting to, although for a powerpc64
>> example context.
>> 
>> The information is only identifying where the code was
>> in /bin/ls and /lib/libc.so.1 in the backtrace. For
>> libc.so.1 I found the same code sequences in a gdb of
>> /bin/ls directly, matching one first, using the addresses
>> vs. in the /libexec/ld-elf.so.1 /bin/ls process to
>> find offsets for going back and forth, and then used
>> that two find the 2nd backtrace addresses material.
>> 
>> Overall it suggests to me that (in somewhat 
>> symbolic terms):
>> 
>> bl <1322.plt_call.getenv>
>> 
>> eventually lead to executing the wrong code.
>> 
>> 
>> The supporting detail is as follows.
>> 
>> The /libexec/ld-elf.so.1 part of the backtrace was
>> easy to find where the code was:
>> 
>> (gdb) run /bin/ls
>> Starting program: /libexec/ld-elf.so.1 /bin/ls
>> 
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x00080118d81c in ?? ()
>> (gdb) bt
>> #0  0x00080118d81c in ?? ()
>> #1  0x00080118d920 in ?? ()
>> #2  0x10002558 in ?? ()
>> #3  0x100037b0 in ?? ()
>> #4  0x01018450 in ._rtld_start () at 
>> /usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104
>> Backtrace stopped: frame did not save the PC
>> 
>> (gdb) 
>> 101  ld  %r7,128(%r1)/* exit proc */
>> 102  ld  %r8,136(%r1)/* ps_strings */
>> 103  
>> 104  blrl/* _start(argc, argv, envp, obj, cleanup, ps_strings) */
>> 105  
>> 106  li  %r0,1   /* _exit() */
>> 107  sc
>> 
>> 
>> The /bin/ls part of the backtrace was easy to find
>> were the code was:
>> 
>> (gdb) symbol-file /bin/ls
>> Load new symbol table from "/bin/ls"? (y or n) y
>> Reading symbols from /bin/ls...Reading symbols from 
>> /usr/lib/debug//bin/ls.debug...done.
>> done.
>> (gdb) bt
>> #0  0x00080118d81c in ?? ()
>> #1  

Re: svn commit: r340231 - head/sys/kern

[Mark Millard via svn-src-head]

John Baldwin jhb at FreeBSD.org wrote on
Wed Nov 7 21:36:02 UTC 2018 :

> On 11/7/18 1:01 PM, Ed Schouten wrote:
> > Op wo 7 nov. 2018 om 19:32 schreef John Baldwin :
> >> Modified: head/sys/kern/imgact_elf.c
> >> ==
> >> --- head/sys/kern/imgact_elf.c  Wed Nov  7 18:29:54 2018(r340230)
> >> +++ head/sys/kern/imgact_elf.c  Wed Nov  7 18:32:02 2018(r340231)
> >> @@ -120,7 +120,8 @@ SYSCTL_INT(_debug, OID_AUTO, __elfN(legacy_coredump),
> >>
> >>  int __elfN(nxstack) =
> >>  #if defined(__amd64__) || defined(__powerpc64__) /* both 64 and 32 bit */ 
> >> || \
> >> -(defined(__arm__) && __ARM_ARCH >= 7) || defined(__aarch64__)
> >> +(defined(__arm__) && __ARM_ARCH >= 7) || defined(__aarch64__) || \
> >> +defined(__riscv)
> >> 1;
> >>  #else
> >> 0;
> > 
> > Are we getting to the point that it might make sense to invert this
> > logic, i.e., just list the architectures that require executable
> > stacks?
> 
> It's not clear.  The remaining set is i386 (should be able to use nxstack
> when using PAE and PG_NX is supported), MIPS (no X permission in PTEs),
> 32-bit powerpc (no X permissions in PTEs AFAICT), and sparc64 (no X
> permissions in PTEs AFAICT).  For architectures without X ptes, removing
> VM_PROT_EXECUTE from the stack permissions is a no-op and would be
> harmless, so we could perhaps just default this to always on at this
> point?


FreeBSD for 32-bit powerpc can also run on powerpc64 hardware,
something I commonly do with 32-bit powerpc FreeBSD.

So I'm not sure if some of the powerpc64 context sometimes
applies for such use or not. May be the mode of use is as-if
it was limited like 32-bit powerpc hardware for what you
propose?

I do not now if i386 or mips have the same sort of
question relative to use on 64-bit capable hardware.


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

[I trace code associated with bl <1322.plt_call.getenv>
in the two contexts and extend the range over which things
appear to match: up to some point after the branch
b <__glink_PLTresolve> .]

On 2018-Nov-6, at 19:12, Mark Millard  wrote:

> [I've present a little information about the longer-existing
> failure's odd backtrace for /libexec/ld-elf.so.1 /bin/ls
> --but on powerpc64 FreeBSD instead of 32-bit powerpc FreeBSD.]
> 
> On 2018-Nov-2, at 11:50, Konstantin Belousov  wrote:
> 
>> On Fri, Nov 02, 2018 at 10:38:08AM -0700, Mark Millard wrote:
>>> On 2018-Nov-2, at 8:52 AM, Konstantin Belousov  
>>> wrote:
>>> 
 . . .
>>> 
>>> That seems better. But it crashes during /bin/ls execution
>>> ( 0x0180 addresses ), apparently in a library routine
>>> ( 0x41?? addresses ):
>>> 
>>> Program received signal SIGSEGV, Segmentation fault.
>>> 0x411220b4 in ?? ()
>>> (gdb) bt
>>> #0  0x411220b4 in ?? ()
>>> #1  0x4112200c in ?? ()
>>> #2  0x01803c84 in ?? ()
>>> #3  0x018023b4 in ?? ()
>>> #4  0x010121a0 in .rtld_start () at 
>>> /usr/src/libexec/rtld-elf/powerpc/rtld_start.S:112
>>> 
>>> Using a normal gdb run of /bin/ls suggests:
>>> 
>>> #2  0x01803c84 in ?? () should be in main and seems to be: bl 0x1818914 
>>> 
>>> #3  0x018023b4 in ?? () should be in _start
>>> 
>>> Looking in the test context:
>>> 
>>>  0x1803c80: bl  0x1818914
>>>  0x1803c84: cmpwi   cr7,r3,-1
>>> 
>>> and:
>>> 
>>>  0x1818914: li  r11,59
>>>  0x1818918: b   0x18186f4
>>> 
>>> and:
>>> 
>>>  0x18186f4: rlwinm  r11,r11,2,0,29
>>>  0x18186f8: addis   r11,r11,386
>>>  0x18186fc: lwz r11,-30316(r11)
>>>  0x1818700: mtctr   r11
>>>  0x1818704: bctr
>>> 
>>> Breaking at the bctr and using info reg:
>>> 
>>> r110x4125ffa0   1093009312
>>> 
>>> It looks like there is some amount of
>>> activity before the traceback addresses
>>> show up.
>>> 
>>> I've not found a good way to fill in the "in ??()"
>>> (or analogous) information. The addresses 0x411220??
>>> do not match up with a normal run of /bin/ls from
>>> gdb: the addresses can not be accessed.
>>> 
>>> 
>>> 
>>> It does appear that the code is in /lib/libc.so.7 in the
>>> test context:
>>> 
>>> Breakpoint 2, reloc_non_plt (obj=0x41041600, obj_rtld=0x41104b57, flags=4, 
>>> lockstate=0x0) at /usr/src/libexec/rtld-elf/powerpc/reloc.c:338
>>> . . .
>>> 
>> There seems to be an issue with the direct execution mode on ppc.
>> Even otherwise working ld-elf.so.1 segfaults if I try to use it as
>> standalone binary.
>> 
>> But if I specify patched ld-elf.so.1 as the interpreter for some program,
>> using 'cc -Wl,-I,/ld-elf.so.1' it works.  So I see there two bugs,
>> one is regression due to textsize calculation, which should be fixed by
>> my patch.  Another is the direct exec problem.
> 
> I've got a little more information about the odd backtrace
> from the /libexec/ld-elf.so.1 /bin/ls failure that the
> prior patch allowed getting to, although for a powerpc64
> example context.
> 
> The information is only identifying where the code was
> in /bin/ls and /lib/libc.so.1 in the backtrace. For
> libc.so.1 I found the same code sequences in a gdb of
> /bin/ls directly, matching one first, using the addresses
> vs. in the /libexec/ld-elf.so.1 /bin/ls process to
> find offsets for going back and forth, and then used
> that two find the 2nd backtrace addresses material.
> 
> Overall it suggests to me that (in somewhat 
> symbolic terms):
> 
> bl <1322.plt_call.getenv>
> 
> eventually lead to executing the wrong code.
> 
> 
> The supporting detail is as follows.
> 
> The /libexec/ld-elf.so.1 part of the backtrace was
> easy to find where the code was:
> 
> (gdb) run /bin/ls
> Starting program: /libexec/ld-elf.so.1 /bin/ls
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0x00080118d81c in ?? ()
> (gdb) bt
> #0  0x00080118d81c in ?? ()
> #1  0x00080118d920 in ?? ()
> #2  0x10002558 in ?? ()
> #3  0x100037b0 in ?? ()
> #4  0x01018450 in ._rtld_start () at 
> /usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104
> Backtrace stopped: frame did not save the PC
> 
> (gdb) 
> 101   ld  %r7,128(%r1)/* exit proc */
> 102   ld  %r8,136(%r1)/* ps_strings */
> 103   
> 104   blrl/* _start(argc, argv, envp, obj, cleanup, ps_strings) */
> 105   
> 106   li  %r0,1   /* _exit() */
> 107   sc
> 
> 
> The /bin/ls part of the backtrace was easy to find
> were the code was:
> 
> (gdb) symbol-file /bin/ls
> Load new symbol table from "/bin/ls"? (y or n) y
> Reading symbols from /bin/ls...Reading symbols from 
> /usr/lib/debug//bin/ls.debug...done.
> done.
> (gdb) bt
> #0  0x00080118d81c in ?? ()
> #1  0x00080118d920 in ?? ()
> #2  0x10002558 in main (argc=, argv=0x80134bdb0) at 
> /usr/src/bin/ls/ls.c:268
> #3  0x100037b0 in _start (argc=, 
> argv=0x3fffdb70, env=0x3fffdb88, obj=, 
> cleanup=, ps_strings=)
>at 

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

[I've present a little information about the longer-existing
failure's odd backtrace for /libexec/ld-elf.so.1 /bin/ls
--but on powerpc64 FreeBSD instead of 32-bit powerpc FreeBSD.]

On 2018-Nov-2, at 11:50, Konstantin Belousov  wrote:

> On Fri, Nov 02, 2018 at 10:38:08AM -0700, Mark Millard wrote:
>> On 2018-Nov-2, at 8:52 AM, Konstantin Belousov  
>> wrote:
>> 
>>> . . .
>> 
>> That seems better. But it crashes during /bin/ls execution
>> ( 0x0180 addresses ), apparently in a library routine
>> ( 0x41?? addresses ):
>> 
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x411220b4 in ?? ()
>> (gdb) bt
>> #0  0x411220b4 in ?? ()
>> #1  0x4112200c in ?? ()
>> #2  0x01803c84 in ?? ()
>> #3  0x018023b4 in ?? ()
>> #4  0x010121a0 in .rtld_start () at 
>> /usr/src/libexec/rtld-elf/powerpc/rtld_start.S:112
>> 
>> Using a normal gdb run of /bin/ls suggests:
>> 
>> #2  0x01803c84 in ?? () should be in main and seems to be: bl 0x1818914 
>> 
>> #3  0x018023b4 in ?? () should be in _start
>> 
>> Looking in the test context:
>> 
>>   0x1803c80: bl  0x1818914
>>   0x1803c84: cmpwi   cr7,r3,-1
>> 
>> and:
>> 
>>   0x1818914: li  r11,59
>>   0x1818918: b   0x18186f4
>> 
>> and:
>> 
>>   0x18186f4: rlwinm  r11,r11,2,0,29
>>   0x18186f8: addis   r11,r11,386
>>   0x18186fc: lwz r11,-30316(r11)
>>   0x1818700: mtctr   r11
>>   0x1818704: bctr
>> 
>> Breaking at the bctr and using info reg:
>> 
>> r110x4125ffa01093009312
>> 
>> It looks like there is some amount of
>> activity before the traceback addresses
>> show up.
>> 
>> I've not found a good way to fill in the "in ??()"
>> (or analogous) information. The addresses 0x411220??
>> do not match up with a normal run of /bin/ls from
>> gdb: the addresses can not be accessed.
>> 
>> 
>> 
>> It does appear that the code is in /lib/libc.so.7 in the
>> test context:
>> 
>> Breakpoint 2, reloc_non_plt (obj=0x41041600, obj_rtld=0x41104b57, flags=4, 
>> lockstate=0x0) at /usr/src/libexec/rtld-elf/powerpc/reloc.c:338
>> . . .
>> 
> There seems to be an issue with the direct execution mode on ppc.
> Even otherwise working ld-elf.so.1 segfaults if I try to use it as
> standalone binary.
> 
> But if I specify patched ld-elf.so.1 as the interpreter for some program,
> using 'cc -Wl,-I,/ld-elf.so.1' it works.  So I see there two bugs,
> one is regression due to textsize calculation, which should be fixed by
> my patch.  Another is the direct exec problem.

I've got a little more information about the odd backtrace
from the /libexec/ld-elf.so.1 /bin/ls failure that the
prior patch allowed getting to, although for a powerpc64
example context.

The information is only identifying where the code was
in /bin/ls and /lib/libc.so.1 in the backtrace. For
libc.so.1 I found the same code sequences in a gdb of
/bin/ls directly, matching one first, using the addresses
vs. in the /libexec/ld-elf.so.1 /bin/ls process to
find offsets for going back and forth, and then used
that two find the 2nd backtrace addresses material.

Overall it suggests to me that (in somewhat 
symbolic terms):

bl <1322.plt_call.getenv>

eventually lead to executing the wrong code.


The supporting detail is as follows.

The /libexec/ld-elf.so.1 part of the backtrace was
easy to find where the code was:

(gdb) run /bin/ls
Starting program: /libexec/ld-elf.so.1 /bin/ls

Program received signal SIGSEGV, Segmentation fault.
0x00080118d81c in ?? ()
(gdb) bt
#0  0x00080118d81c in ?? ()
#1  0x00080118d920 in ?? ()
#2  0x10002558 in ?? ()
#3  0x100037b0 in ?? ()
#4  0x01018450 in ._rtld_start () at 
/usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104
Backtrace stopped: frame did not save the PC

(gdb) 
101 ld  %r7,128(%r1)/* exit proc */
102 ld  %r8,136(%r1)/* ps_strings */
103 
104 blrl/* _start(argc, argv, envp, obj, cleanup, ps_strings) */
105 
106 li  %r0,1   /* _exit() */
107 sc


The /bin/ls part of the backtrace was easy to find
were the code was:

(gdb) symbol-file /bin/ls
Load new symbol table from "/bin/ls"? (y or n) y
Reading symbols from /bin/ls...Reading symbols from 
/usr/lib/debug//bin/ls.debug...done.
done.
(gdb) bt
#0  0x00080118d81c in ?? ()
#1  0x00080118d920 in ?? ()
#2  0x10002558 in main (argc=, argv=0x80134bdb0) at 
/usr/src/bin/ls/ls.c:268
#3  0x100037b0 in _start (argc=, 
argv=0x3fffdb70, env=0x3fffdb88, obj=, 
cleanup=, ps_strings=)
at /usr/src/lib/csu/powerpc64/crt1.c:96
#4  0x01018450 in ?? ()
#5  0x in ?? ()

(gdb) fr 3 
#3  0x100037b0 in _start (argc=, 
argv=0x3fffdb70, env=0x3fffdb88, obj=, 
cleanup=, ps_strings=)
at /usr/src/lib/csu/powerpc64/crt1.c:96
96  exit(main(argc, argv, env));
(gdb) down
#2  0x10002558 in main (argc=, argv=0x80134bdb0) at 
/usr/src/bin/ls/ls.c:268
268 while 

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Nov-3, at 12:51 PM, Konstantin Belousov  wrote:

> On Sat, Nov 03, 2018 at 12:04:53PM -0700, Mark Millard wrote:
>> 80903 ld-elf.so.1 CALL  
>> mmap(0x1,0xb000,0,0x6010,0x,0x1,0,0)
>> 80903 ld-elf.so.1 RET   mmap -1 errno 12 Cannot allocate memory
> 
> This is the allocation of VA for the whole binary.  I guess that the normal
> non-PIE binary virtual base on arm is 0x1.
> 
> I put the arm PIE base at 0x12000, which obviously causes the conflict
> between pre-loaded ld-elf.so.1 and later attempt to mmap the binary at
> the linked address.  In fact I am happy that we added MAP_EXCL and it
> catched the situation.
> 
> You can try to change ET_DYN_LOAD_ADDR on arm:
> 
> diff --git a/sys/arm/include/elf.h b/sys/arm/include/elf.h
> index 7424b45e78f..28fc14e3dae 100644
> --- a/sys/arm/include/elf.h
> +++ b/sys/arm/include/elf.h
> @@ -84,7 +84,7 @@ __ElfType(Auxinfo);
>  */
> #define   MAGIC_TRAMP_NUMBER  0x5c03
> 
> -#define  ET_DYN_LOAD_ADDR0x12000
> +#define  ET_DYN_LOAD_ADDR0x50
> 
> /* Flags passed in AT_HWCAP. */
> #define   HWCAP_SWP   0x0001  /* Unsupported, never 
> set.*/

I finally got to building-for, installing-on, booting-the, and
testing-the armv7 issue. The change worked:

# /libexec/ld-elf.so.1 /bin/ls -c1
.cshrc
.history
.k5login
.lesshst
.lldb
.login
.profile
.shrc
.subversion

(and so on).

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Nov-3, at 12:04 PM, Mark Millard  wrote:

> On 2018-Nov-3, at 8:49 AM, Konstantin Belousov  wrote:
> 
>> On Fri, Nov 02, 2018 at 05:51:25PM -0700, Mark Millard wrote:
>>> On 2018-Nov-2, at 11:50 AM, Konstantin Belousov  
>>> wrote:
>>> 
 On Fri, Nov 02, 2018 at 10:38:08AM -0700, Mark Millard wrote:
> . . .
 
 There seems to be an issue with the direct execution mode on ppc.
 Even otherwise working ld-elf.so.1 segfaults if I try to use it as
 standalone binary.
 
 But if I specify patched ld-elf.so.1 as the interpreter for some program,
 using 'cc -Wl,-I,/ld-elf.so.1' it works.  So I see there two bugs,
 one is regression due to textsize calculation, which should be fixed by
 my patch.  Another is the direct exec problem.
>>> 
>>> My head -r339076 based powerpc64 and armv7 contexts also
>>> fail for:
>>> 
>>> # /libexec/ld-elf.so.1 /bin/ls
>>> 
>>> The armv7 (a Cortext-A7 context) is interestingly different
>>> in how it fails:
>>> 
>>> # /libexec/ld-elf.so.1 /bin/ls
>>> ld-elf.so.1: /bin/ls: mmap of entire address space failed: Cannot allocate 
>>> memory
>> Can you show the ktrace/kdump for this ?
> 
> Sure, in the Cortex-A7 context . . .
> 
> . . .
> 
> 
>>> My aarch64 context (a Cortext-A53 context) had no problem.
>>> 
>>> (All 3 examples are without any of the the recent updates
>>> or patches to ld-elf.so.1 source code.)
>> 
>> And still, does the patch for isync range works ?  You can test the new
>> ld-elf.so.1 standalone by hard-coding its path into the binary.  Build e.g.
>> only ls(1) by using make in its directory, then re-issue the linking
>> command with the additional flag '-Wl,-I,
>> and try to run ls.full.
> 
> Looks like the old PowerMac is available again for
> such activity. So I'll see about testing.

I tried installing a buildworld that has the patches
via using the make arguments:

installworld distrib-dirs distribution DB_FROM_SRC=1 
DESTDIR=/usr/obj/DESTDIRs/gcc421-powerpc-installworld-poud

(I normally have poudriere-devel use this gcc421-powerpc-installworld-poud
in this environment.)

Then:

# chroot /usr/obj/DESTDIRs/gcc421-powerpc-installworld-poud
# pwd
/
# ls
.cshrc  COPYRIGHT   bootetc libexec 
mnt procroottmp var
.profilebin dev lib media   
net rescue  sbinusr
# df -m
Filesystem 1M-blocks  Used Avail Capacity  Mounted on
/dev/ufs/FBSDG4Srootfs 95203 38217 4936944%/
devfs  0 0 0   100%/dev
# ls -lTd /bin/ls
-r-xr-xr-x  1 root  wheel  36816 Nov  3 19:19:26 2018 /bin/ls
# ls -lTd /libexec/ld-elf.so.1*
-r-xr-xr-x  1 root  wheel  136100 Nov  3 19:17:53 2018 /libexec/ld-elf.so.1
-r-xr-xr-x  1 root  wheel  135932 Nov  2 02:10:34 2018 /libexec/ld-elf.so.1.old
# /usr/libexec/gdb /libexec/ld-elf.so.1
. . .
(gdb) disass reloc_non_plt
Dump of assembler code for function reloc_non_plt:
. . .
0x2b48 : bl  0xef78 
0x2b4c : lwz r31,64(r29)
0x2b50 : lwz r9,68(r29)
0x2b54 : mr  r11,r31
0x2b58 : add r0,r31,r9
0x2b5c : cmplw   cr7,r31,r0
0x2b60 : blt+cr7,0x2b78 
0x2b64 : b   0x2bbc 
0x2b68 : addir31,r31,32
0x2b6c : add r0,r11,r9
0x2b70 : cmplw   cr7,r0,r31
0x2b74 : ble-cr7,0x2bbc 
0x2b78 : lwz r0,0(r31)
0x2b7c : cmpwi   cr7,r0,1
0x2b80 : bne+cr7,0x2b68 
0x2b84 : lwz r0,24(r31)
0x2b88 : andi.   r10,r0,1
---Type  to continue, or q  to quit---
0x2b8c : beq+0x2b68 
0x2b90 : lwz r0,52(r29)
0x2b94 : lwz r3,8(r31)
0x2b98 : lwz r4,20(r31)
0x2b9c : addir31,r31,32
0x2ba0 : add r3,r0,r3
0x2ba4 : bl  0x12858 <__syncicache>
0x2ba8 : lwz r11,64(r29)
0x2bac : lwz r9,68(r29)
0x2bb0 : add r0,r11,r9
0x2bb4 : cmplw   cr7,r0,r31
0x2bb8 : bgt+cr7,0x2b78 
. . .


So, unless the chroot somehow bypasses use of that local
/libexec/ld-elf.so.1 , the __synicache in a loop seems to
be working under the chroot use.

Let me know if this test is insufficient for some reason.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Nov-3, at 8:49 AM, Konstantin Belousov  wrote:

> On Fri, Nov 02, 2018 at 05:51:25PM -0700, Mark Millard wrote:
>> On 2018-Nov-2, at 11:50 AM, Konstantin Belousov  
>> wrote:
>> 
>>> On Fri, Nov 02, 2018 at 10:38:08AM -0700, Mark Millard wrote:
 . . .
>>> 
>>> There seems to be an issue with the direct execution mode on ppc.
>>> Even otherwise working ld-elf.so.1 segfaults if I try to use it as
>>> standalone binary.
>>> 
>>> But if I specify patched ld-elf.so.1 as the interpreter for some program,
>>> using 'cc -Wl,-I,/ld-elf.so.1' it works.  So I see there two bugs,
>>> one is regression due to textsize calculation, which should be fixed by
>>> my patch.  Another is the direct exec problem.
>> 
>> My head -r339076 based powerpc64 and armv7 contexts also
>> fail for:
>> 
>> # /libexec/ld-elf.so.1 /bin/ls
>> 
>> The armv7 (a Cortext-A7 context) is interestingly different
>> in how it fails:
>> 
>> # /libexec/ld-elf.so.1 /bin/ls
>> ld-elf.so.1: /bin/ls: mmap of entire address space failed: Cannot allocate 
>> memory
> Can you show the ktrace/kdump for this ?

Sure, in the Cortex-A7 context . . .

# ktrace -t+ /libexec/ld-elf.so.1 /bin/ls
ld-elf.so.1: /bin/ls: mmap of entire address space failed: Cannot allocate 
memory

# kdump | less
 80903 ktrace   RET   ktrace 0
 80903 ktrace   CALL  execve(0xbfbfee23,0xbfbfecf0,0xbfbfecfc)
 80903 ktrace   NAMI  "/libexec/ld-elf.so.1"
 80903 ld-elf.so.1 RET   execve JUSTRETURN
 80903 ld-elf.so.1 CALL  
mmap(0,0x2,0x3,0x1002,0x,0,0,0)
 80903 ld-elf.so.1 RET   mmap 537071616/0x20031000
 80903 ld-elf.so.1 CALL  issetugid
 80903 ld-elf.so.1 RET   issetugid 0
 80903 ld-elf.so.1 CALL  
openat(AT_FDCWD,0xbfbfee2d,0x30)
 80903 ld-elf.so.1 NAMI  "/bin/ls"
 80903 ld-elf.so.1 RET   openat 3
 80903 ld-elf.so.1 CALL  fstat(0x3,0xbfbfe638)
 80903 ld-elf.so.1 STRU  struct stat {dev=95, ino=2568217, mode=0100555, 
nlink=1, uid=0, gid=0, rdev=5140776, atime=1538464078.957949000, 
mtime=1538464078.958055000, ctime=1538464078.95881, 
birthtime=1538464078.957947000, size=39440, blksize=32768, blocks=80, flags=0x0 
}
 80903 ld-elf.so.1 RET   fstat 0
 80903 ld-elf.so.1 CALL  geteuid
 80903 ld-elf.so.1 RET   geteuid 0
 80903 ld-elf.so.1 CALL  
mmap(0,0x1000,0x1,0x40002,0x3,0,0,0)
 80903 ld-elf.so.1 RET   mmap 537202688/0x20051000
 80903 ld-elf.so.1 CALL  
mmap(0x1,0xb000,0,0x6010,0x,0x1,0,0)
 80903 ld-elf.so.1 RET   mmap -1 errno 12 Cannot allocate memory
 80903 ld-elf.so.1 CALL  munmap(0x20051000,0x1000)
 80903 ld-elf.so.1 RET   munmap 0
 80903 ld-elf.so.1 CALL  close(0x3)
 80903 ld-elf.so.1 RET   close 0
 80903 ld-elf.so.1 CALL  write(0x2,0x12e38,0xd)
 80903 ld-elf.so.1 GIO   fd 2 wrote 13 bytes
   "ld-elf.so.1: "
 80903 ld-elf.so.1 RET   write 13/0xd
 80903 ld-elf.so.1 CALL  write(0x2,0x33238,0x44)
 80903 ld-elf.so.1 GIO   fd 2 wrote 68 bytes
   "/bin/ls: mmap of entire address space failed: Cannot allocate memory"
 80903 ld-elf.so.1 RET   write 68/0x44
 80903 ld-elf.so.1 CALL  write(0x2,0xbfbfe1e7,0x1)
 80903 ld-elf.so.1 GIO   fd 2 wrote 1 byte
   "
   "
 80903 ld-elf.so.1 RET   write 1
 80903 ld-elf.so.1 CALL  exit(0x1)


>> My aarch64 context (a Cortext-A53 context) had no problem.
>> 
>> (All 3 examples are without any of the the recent updates
>> or patches to ld-elf.so.1 source code.)
> 
> And still, does the patch for isync range works ?  You can test the new
> ld-elf.so.1 standalone by hard-coding its path into the binary.  Build e.g.
> only ls(1) by using make in its directory, then re-issue the linking
> command with the additional flag '-Wl,-I,
> and try to run ls.full.

Looks like the old PowerMac is available again for
such activity. So I'll see about testing.



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Nov-2, at 11:50 AM, Konstantin Belousov  wrote:

> On Fri, Nov 02, 2018 at 10:38:08AM -0700, Mark Millard wrote:
>> . . .
> 
> There seems to be an issue with the direct execution mode on ppc.
> Even otherwise working ld-elf.so.1 segfaults if I try to use it as
> standalone binary.
> 
> But if I specify patched ld-elf.so.1 as the interpreter for some program,
> using 'cc -Wl,-I,/ld-elf.so.1' it works.  So I see there two bugs,
> one is regression due to textsize calculation, which should be fixed by
> my patch.  Another is the direct exec problem.

My head -r339076 based powerpc64 and armv7 contexts also
fail for:

# /libexec/ld-elf.so.1 /bin/ls

The armv7 (a Cortext-A7 context) is interestingly different
in how it fails:

# /libexec/ld-elf.so.1 /bin/ls
ld-elf.so.1: /bin/ls: mmap of entire address space failed: Cannot allocate 
memory

My aarch64 context (a Cortext-A53 context) had no problem.

(All 3 examples are without any of the the recent updates
or patches to ld-elf.so.1 source code.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Nov-2, at 8:52 AM, Konstantin Belousov  wrote:

> On Fri, Nov 02, 2018 at 08:30:17AM -0700, Mark Millard wrote:
>> Breakpoint 4, reloc_non_plt (obj=0x41041000, obj_rtld=0x1801cc7, flags=4, 
>> lockstate=0x0) at /usr/src/libexec/rtld-elf/powerpc/reloc.c:338
>> 338  __syncicache(obj->mapbase + phdr->p_vaddr,
>> 1: x/i $pc
>> => 0x1012b90 :lwz r0,36(r29)
>> (gdb) print/x obj->mapbase+phdr->p_vaddr
>> $17 = 0x300
>> (gdb) print/x obj->mapbase  
>> $18 = 0x180
>> (gdb) print/x phdr->p_vaddr
>> $19 = 0x180
>> (gdb) c
>> Continuing.
>> 
>> Program received signal SIGSEGV, Segmentation fault.
>> __syncicache (from=0x300, len=34112) at 
>> /usr/src/lib/libc/powerpc/gen/syncicache.c:94
>> 94   __asm __volatile ("dcbst 0,%0" :: "r"(p));
>> 1: x/i $pc
>> => 0x10228b8 <__syncicache+96>:  dcbst   0,r11
>> 
>> 
>> 
>> It looks to me like the 0x180 component of the
>> overall figure was double counted. ( phdr->p_vaddr
>> would vary but obj->mapbase would not. )
>> 
>> Omit "obj->mapbase + "?
> I used the wrong addend.  Try this fix.
> 
> diff --git a/libexec/rtld-elf/powerpc/reloc.c 
> b/libexec/rtld-elf/powerpc/reloc.c
> index e921a4dc7d1..fae28dd9224 100644
> --- a/libexec/rtld-elf/powerpc/reloc.c
> +++ b/libexec/rtld-elf/powerpc/reloc.c
> @@ -294,6 +294,7 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int 
> flags,
> {
>   const Elf_Rela *relalim;
>   const Elf_Rela *rela;
> + const Elf_Phdr *phdr;
>   SymCache *cache;
>   int r = -1;
> 
> @@ -327,8 +328,18 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int 
> flags,
>   if (cache != NULL)
>   free(cache);
> 
> - /* Synchronize icache for text seg in case we made any changes */
> - __syncicache(obj->mapbase, obj->textsize);
> + /*
> +  * Synchronize icache for executable segments in case we made
> +  * any changes.
> +  */
> + for (phdr = obj->phdr;
> + (const char *)phdr < (const char *)obj->phdr + obj->phsize;
> + phdr++) {
> + if (phdr->p_type == PT_LOAD && (phdr->p_flags & PF_X) != 0) {
> + __syncicache(obj->relocbase + phdr->p_vaddr,
> + phdr->p_memsz);
> + }
> + }
> 
>   return (r);
> }
> diff --git a/libexec/rtld-elf/powerpc64/reloc.c 
> b/libexec/rtld-elf/powerpc64/reloc.c
> index c2d6dac13b1..15f31738ab7 100644
> --- a/libexec/rtld-elf/powerpc64/reloc.c
> +++ b/libexec/rtld-elf/powerpc64/reloc.c
> @@ -291,6 +291,7 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int 
> flags,
> {
>   const Elf_Rela *relalim;
>   const Elf_Rela *rela;
> + const Elf_Phdr *phdr;
>   SymCache *cache;
>   int bytes = obj->dynsymcount * sizeof(SymCache);
>   int r = -1;
> @@ -327,8 +328,18 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int 
> flags,
>   if (cache)
>   munmap(cache, bytes);
> 
> - /* Synchronize icache for text seg in case we made any changes */
> - __syncicache(obj->mapbase, obj->textsize);
> + /*
> +  * Synchronize icache for executable segments in case we made
> +  * any changes.
> +  */
> + for (phdr = obj->phdr;
> + (const char *)phdr < (const char *)obj->phdr + obj->phsize;
> + phdr++) {
> + if (phdr->p_type == PT_LOAD && (phdr->p_flags & PF_X) != 0) {
> + __syncicache(obj->relocbase + phdr->p_vaddr,
> + phdr->p_memsz);
> + }
> + }
> 
>   return (r);
> }

That seems better. But it crashes during /bin/ls execution
( 0x0180 addresses ), apparently in a library routine
( 0x41?? addresses ):

Program received signal SIGSEGV, Segmentation fault.
0x411220b4 in ?? ()
(gdb) bt
#0  0x411220b4 in ?? ()
#1  0x4112200c in ?? ()
#2  0x01803c84 in ?? ()
#3  0x018023b4 in ?? ()
#4  0x010121a0 in .rtld_start () at 
/usr/src/libexec/rtld-elf/powerpc/rtld_start.S:112

Using a normal gdb run of /bin/ls suggests:

#2  0x01803c84 in ?? () should be in main and seems to be: bl 0x1818914 

#3  0x018023b4 in ?? () should be in _start

Looking in the test context:

   0x1803c80:   bl  0x1818914
   0x1803c84:   cmpwi   cr7,r3,-1

and:

   0x1818914:   li  r11,59
   0x1818918:   b   0x18186f4

and:

   0x18186f4:   rlwinm  r11,r11,2,0,29
   0x18186f8:   addis   r11,r11,386
   0x18186fc:   lwz r11,-30316(r11)
   0x1818700:   mtctr   r11
   0x1818704:   bctr

Breaking at the bctr and using info reg:

r110x4125ffa0   1093009312

It looks like there is some amount of
activity before the traceback addresses
show up.

I've not found a good way to fill in the "in ??()"
(or analogous) information. The addresses 0x411220??
do not match up with a normal run of /bin/ls from
gdb: the addresses can not be accessed.



It does appear that the code is in /lib/libc.so.7 in the
test context:

Breakpoint 2, reloc_non_plt (obj=0x41041600, 

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Nov-2, at 4:38 AM, Konstantin Belousov  wrote:

> On Fri, Nov 02, 2018 at 12:16:23AM -0700, Mark Millard wrote:
>> It stops when the dcbst in __syncicache runs into an address in
>> the p_align 65536 caused hole between the two PT_LOAD's with PF_X.
>> /bin/ls itself has such a hole, as do the .so libraries involved.
> 
> Try this.  I only compile-tested the change.
> 
> diff --git a/libexec/rtld-elf/powerpc/reloc.c 
> b/libexec/rtld-elf/powerpc/reloc.c
> index e921a4dc7d1..5f21e33bee3 100644
> --- a/libexec/rtld-elf/powerpc/reloc.c
> +++ b/libexec/rtld-elf/powerpc/reloc.c
> @@ -294,6 +294,7 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int 
> flags,
> {
>   const Elf_Rela *relalim;
>   const Elf_Rela *rela;
> + const Elf_Phdr *phdr;
>   SymCache *cache;
>   int r = -1;
> 
> @@ -327,8 +328,18 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int 
> flags,
>   if (cache != NULL)
>   free(cache);
> 
> - /* Synchronize icache for text seg in case we made any changes */
> - __syncicache(obj->mapbase, obj->textsize);
> + /*
> +  * Synchronize icache for executable segments in case we made
> +  * any changes.
> +  */
> + for (phdr = obj->phdr;
> + (const char *)phdr < (const char *)obj->phdr + obj->phsize;
> + phdr++) {
> + if (phdr->p_type == PT_LOAD && (phdr->p_flags & PF_X) != 0) {
> + __syncicache(obj->mapbase + phdr->p_vaddr,
> + phdr->p_memsz);
> + }
> + }
> 
>   return (r);
> }
> diff --git a/libexec/rtld-elf/powerpc64/reloc.c 
> b/libexec/rtld-elf/powerpc64/reloc.c
> index c2d6dac13b1..980b4933afe 100644
> --- a/libexec/rtld-elf/powerpc64/reloc.c
> +++ b/libexec/rtld-elf/powerpc64/reloc.c
> @@ -291,6 +291,7 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int 
> flags,
> {
>   const Elf_Rela *relalim;
>   const Elf_Rela *rela;
> + const Elf_Phdr *phdr;
>   SymCache *cache;
>   int bytes = obj->dynsymcount * sizeof(SymCache);
>   int r = -1;
> @@ -327,8 +328,18 @@ reloc_non_plt(Obj_Entry *obj, Obj_Entry *obj_rtld, int 
> flags,
>   if (cache)
>   munmap(cache, bytes);
> 
> - /* Synchronize icache for text seg in case we made any changes */
> - __syncicache(obj->mapbase, obj->textsize);
> + /*
> +  * Synchronize icache for executable segments in case we made
> +  * any changes.
> +  */
> + for (phdr = obj->phdr;
> + (const char *)phdr < (const char *)obj->phdr + obj->phsize;
> + phdr++) {
> + if (phdr->p_type == PT_LOAD && (phdr->p_flags & PF_X) != 0) {
> + __syncicache(obj->mapbase + phdr->p_vaddr,
> + phdr->p_memsz);
> + }
> + }
> 
>   return (r);
> }
> 

Unfortunately, that failed:

(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: 
/usr/obj/powerpcvtsc_gcc421/powerpc.powerpc/usr/src/powerpc.powerpc/libexec/rtld-elf/ld-elf.so.1.full
 /bin/ls

Breakpoint 4, reloc_non_plt (obj=0x41041000, obj_rtld=0x1801cc7, flags=4, 
lockstate=0x0) at /usr/src/libexec/rtld-elf/powerpc/reloc.c:338
338 __syncicache(obj->mapbase + phdr->p_vaddr,
1: x/i $pc
=> 0x1012b90 :   lwz r0,36(r29)
(gdb) print/x obj->mapbase+phdr->p_vaddr
$17 = 0x300
(gdb) print/x obj->mapbase  
$18 = 0x180
(gdb) print/x phdr->p_vaddr
$19 = 0x180
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
__syncicache (from=0x300, len=34112) at 
/usr/src/lib/libc/powerpc/gen/syncicache.c:94
94  __asm __volatile ("dcbst 0,%0" :: "r"(p));
1: x/i $pc
=> 0x10228b8 <__syncicache+96>: dcbst   0,r11



It looks to me like the 0x180 component of the
overall figure was double counted. ( phdr->p_vaddr
would vary but obj->mapbase would not. )

Omit "obj->mapbase + "?


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

[The backtrace confirms what I reported to Alexander Richardson
and Shawn Webb earlier.]

On 2018-Nov-1, at 6:40 PM, Mark Millard  wrote:

> On 2018-Nov-1, at 5:41 PM, Konstantin Belousov  wrote:
> 
>> On Tue, Oct 30, 2018 at 12:45:13PM -0700, Mark Millard wrote:
>>> Konstantin Belousov kostikbel at gmail.com wrote on
>>> Tue Oct 30 18:04:04 UTC 2018 :
>>> 
 On Tue, Oct 30, 2018 at 03:32:40PM +, Alexander Richardson wrote:
> On Tue, 30 Oct 2018 at 10:17, Michael Tuexen
>  wrote:
>> 
>>> On 29. Oct 2018, at 22:08, Alex Richardson  
>>> wrote:
>>> 
>>> Author: arichardson
>>> Date: Mon Oct 29 21:08:02 2018
>>> New Revision: 339876
>>> URL: https://svnweb.freebsd.org/changeset/base/339876
>>> 
>>> Log:
>>> rtld: set obj->textsize correctly
>>> 
>>> With lld-generated binaries the first PT_LOAD will usually be a 
>>> read-only
>>> segment unless you pass --no-rosegment. For those binaries the textsize 
>>> is
>>> determined by the next PT_LOAD. To allow both LLD and bfd 2.17 binaries 
>>> to
>>> be parsed correctly use the end of the last PT_LOAD that is marked as
>>> executable instead.
>>> 
>>> I noticed that the value was wrong while adding some debug prints for 
>>> some rtld
>>> changes for CHERI binaries. `obj->textsize` only seems to be used by 
>>> PPC so the
>>> effect is untested. However, the value before was definitely wrong and 
>>> the new
>>> result matches the phdrs.
>> I build kernel and world with a revision later than this on a PPC. 
>> Buildword
>> ends up with a world where almost all binaries are segfaulting 
>> Especially gdb
>> (but svn, ls or so all segfault).
>> 
>> Best regards
>> Michael
> 
> This is rather surprising since if anything the range of the icache
> flush should increase rather than decrease after this change.
> 
> I can only see this causing a behaviour change if we actually need to
> flush more than just the executable segments.
> Is it possible that some binary/library contains a non-executable
> segment as the first PT_LOAD?
> Or is there some linker script that adds custom PHDRS?
> 
 Could it be that there is a hole between start of the object mapping and
 the last PT_LOADable segment eligible for execution ?
>>> 
>>> [This note may be easier to deal with than the first
>>> note that I sent out.]
>>> 
>>> [My examples are from devel/powerpc64-xtoolchain-gcc used
>>> to buildworld buildkernel targeting a head -r339076 based
>>> powerpc64 environment. I do that on powerpc64 as well.]
>>> 
>>> powerpc64 loads the readonly data and the readonly code in one PT_LOAD,
>>> the first. The 2nd PT_LOAD is for sections without the readonly status,
>>> that includes .got and .plt being spanned. See below from
>>> objdump -ph for /bin/ls :
>>> 
>>> Program Header:
>>>   PHDR off0x0040 vaddr 0x1040 paddr 
>>> 0x1040 align 2**3
>>>filesz 0x0188 memsz 0x0188 flags r--
>>> INTERP off0x01c8 vaddr 0x11c8 paddr 
>>> 0x11c8 align 2**0
>>>filesz 0x0015 memsz 0x0015 flags r--
>>>   LOAD off0x vaddr 0x1000 paddr 
>>> 0x1000 align 2**16
>>>filesz 0x910c memsz 0x910c flags r-x
>>>   LOAD off0x9110 vaddr 0x10019110 paddr 
>>> 0x10019110 align 2**16
>>>filesz 0x0ee0 memsz 0x10e8 flags rw-
>>> DYNAMIC off0x9138 vaddr 0x10019138 paddr 
>>> 0x10019138 align 2**3
>>>filesz 0x01c0 memsz 0x01c0 flags rw-
>>>   NOTE off0x01e0 vaddr 0x11e0 paddr 
>>> 0x11e0 align 2**2
>>>filesz 0x0030 memsz 0x0030 flags r--
>>>  STACK off0x vaddr 0x paddr 
>>> 0x align 2**4
>>>filesz 0x memsz 0x flags rw-
>> 
>> We only need program headers, and we only need them from the object
>> which load causes the fault.  It might be not the binary but some library
>> that triggers the fault.  So the backtrace and some information from the
>> state of the image is needed.
>> 
>> You can build only rtld and use it as the standalone program to initiate
>> the image activation:
>>  /ld-elf.so.1 /bin/ls
>> or similar.
> 
> . . .

It stops when the dcbst in __syncicache runs into an address in
the p_align 65536 caused hole between the two PT_LOAD's with PF_X.
/bin/ls itself has such a hole, as do the .so libraries involved.

Details follow.

(gdb) run /bin/ls
Starting program: 
/usr/obj/powerpcvtsc_gcc421/powerpc.powerpc/usr/src/powerpc.powerpc/libexec/rtld-elf/ld-elf.so.1.full
 /bin/ls

Program received signal SIGSEGV, 

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Nov-1, at 5:41 PM, Konstantin Belousov  wrote:

> On Tue, Oct 30, 2018 at 12:45:13PM -0700, Mark Millard wrote:
>> Konstantin Belousov kostikbel at gmail.com wrote on
>> Tue Oct 30 18:04:04 UTC 2018 :
>> 
>>> On Tue, Oct 30, 2018 at 03:32:40PM +, Alexander Richardson wrote:
 On Tue, 30 Oct 2018 at 10:17, Michael Tuexen
  wrote:
> 
>> On 29. Oct 2018, at 22:08, Alex Richardson  
>> wrote:
>> 
>> Author: arichardson
>> Date: Mon Oct 29 21:08:02 2018
>> New Revision: 339876
>> URL: https://svnweb.freebsd.org/changeset/base/339876
>> 
>> Log:
>> rtld: set obj->textsize correctly
>> 
>> With lld-generated binaries the first PT_LOAD will usually be a read-only
>> segment unless you pass --no-rosegment. For those binaries the textsize 
>> is
>> determined by the next PT_LOAD. To allow both LLD and bfd 2.17 binaries 
>> to
>> be parsed correctly use the end of the last PT_LOAD that is marked as
>> executable instead.
>> 
>> I noticed that the value was wrong while adding some debug prints for 
>> some rtld
>> changes for CHERI binaries. `obj->textsize` only seems to be used by PPC 
>> so the
>> effect is untested. However, the value before was definitely wrong and 
>> the new
>> result matches the phdrs.
> I build kernel and world with a revision later than this on a PPC. 
> Buildword
> ends up with a world where almost all binaries are segfaulting 
> Especially gdb
> (but svn, ls or so all segfault).
> 
> Best regards
> Michael
 
 This is rather surprising since if anything the range of the icache
 flush should increase rather than decrease after this change.
 
 I can only see this causing a behaviour change if we actually need to
 flush more than just the executable segments.
 Is it possible that some binary/library contains a non-executable
 segment as the first PT_LOAD?
 Or is there some linker script that adds custom PHDRS?
 
>>> Could it be that there is a hole between start of the object mapping and
>>> the last PT_LOADable segment eligible for execution ?
>> 
>> [This note may be easier to deal with than the first
>> note that I sent out.]
>> 
>> [My examples are from devel/powerpc64-xtoolchain-gcc used
>> to buildworld buildkernel targeting a head -r339076 based
>> powerpc64 environment. I do that on powerpc64 as well.]
>> 
>> powerpc64 loads the readonly data and the readonly code in one PT_LOAD,
>> the first. The 2nd PT_LOAD is for sections without the readonly status,
>> that includes .got and .plt being spanned. See below from
>> objdump -ph for /bin/ls :
>> 
>> Program Header:
>>PHDR off0x0040 vaddr 0x1040 paddr 
>> 0x1040 align 2**3
>> filesz 0x0188 memsz 0x0188 flags r--
>>  INTERP off0x01c8 vaddr 0x11c8 paddr 
>> 0x11c8 align 2**0
>> filesz 0x0015 memsz 0x0015 flags r--
>>LOAD off0x vaddr 0x1000 paddr 
>> 0x1000 align 2**16
>> filesz 0x910c memsz 0x910c flags r-x
>>LOAD off0x9110 vaddr 0x10019110 paddr 
>> 0x10019110 align 2**16
>> filesz 0x0ee0 memsz 0x10e8 flags rw-
>> DYNAMIC off0x9138 vaddr 0x10019138 paddr 
>> 0x10019138 align 2**3
>> filesz 0x01c0 memsz 0x01c0 flags rw-
>>NOTE off0x01e0 vaddr 0x11e0 paddr 
>> 0x11e0 align 2**2
>> filesz 0x0030 memsz 0x0030 flags r--
>>   STACK off0x vaddr 0x paddr 
>> 0x align 2**4
>> filesz 0x memsz 0x flags rw-
> 
> We only need program headers, and we only need them from the object
> which load causes the fault.  It might be not the binary but some library
> that triggers the fault.  So the backtrace and some information from the
> state of the image is needed.
> 
> You can build only rtld and use it as the standalone program to initiate
> the image activation:
>   /ld-elf.so.1 /bin/ls
> or similar.

It will be a while before I can get an environment set
up which allows me to get a backtrace from a 32-bit
powerpc system. I have demonstrated that I get the crash
with what I tried to verify such.

I'll work on getting an environment in place that allows
what you are asking for.

Problem reproduction details (if you care) . . .

I've been trying to provide evidence but my 32-bit powerpc
environment is not as modern as has the problem. Also: the
buildworld it was installed from was built via clang, which
means that the libgcc_s.so that was built does not correctly
support C++ exceptions. (clang's generated output is the
source of that 

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Oct-31, at 11:53 AM, Alexander Richardson  
wrote:

> On Wed, 31 Oct 2018 at 18:38, Mark Millard  
> wrote:
>> 
>> On 2018-Oct-30, at 3:23 PM, Alexander Richardson > freebsd.org> wrote:
>> 
>>> . . .
>>> Before this change obj->textsize was always set as the end of
>>> PT_LOAD[0]. Now it will contain everything up to the end of the last
>>> PT_LOAD with execute permissions. In the binary you dumped this is
>>> PT_LOAD[0] both before and after the patch so there is no change in
>>> behaviour. The .got and .plt we not included in textsize before this
>>> patch either. Therefore the only way I can see this patch breaking
>>> anything is if the PHDRS of one of the loaded libraries causes a
>>> change in behaviour.
>> 
>> A fair night's sleep helps.
>> 
>> Not only did I read something into your description that was
>> not there, I looked at powerpc64 when it turns out that
>> only 32-bit powerpc shows the problem. (I asked.)
>> 
>> I'm also told reverting just those changes fixes 32-bit
>> powerpc context.
>> 
>> So I'm using an older 32-bit powerpc context to look at:
>> 
>> https://artifact.ci.freebsd.org/snapshot/head/r339870/powerpc/powerpc/base*.txz
>> 
>> materials now. -r339870 is the first 32-bit powerpc build
>> available there after the changes were made.
>> 
>> So in a /339870/ area from expanding the .txz's
>> inside it I get:
>> 
>> # elfdump -p ./bin/ls | less
>> 
>> program header:
>> 
>> entry: 0
>>p_type: PT_PHDR
>>p_offset: 52
>>p_vaddr: 0x1800034
>>p_paddr: 0x1800034
>>p_filesz: 224
>>p_memsz: 224
>>p_flags: PF_X|PF_R
>>p_align: 4
>> 
>> entry: 1
>>p_type: PT_INTERP
>>p_offset: 276
>>p_vaddr: 0x1800114
>>p_paddr: 0x1800114
>>p_filesz: 21
>>p_memsz: 21
>>p_flags: PF_R
>>p_align: 1
>> 
>> entry: 2
>>p_type: PT_LOAD
>>p_offset: 0
>>p_vaddr: 0x180
>>p_paddr: 0x180
>>p_filesz: 34112
>>p_memsz: 34112
>>p_flags: PF_X|PF_R
>>p_align: 65536
>> 
>> entry: 3
>>p_type: PT_LOAD
>>p_offset: 34112
>>p_vaddr: 0x1818540
>>p_paddr: 0x1818540
>>p_filesz: 316
>>p_memsz: 1752
>>p_flags: PF_X|PF_W|PF_R
>>p_align: 65536
>> 
>> entry: 4
>>p_type: PT_DYNAMIC
>>p_offset: 34132
>>p_vaddr: 0x1818554
>>p_paddr: 0x1818554
>>p_filesz: 216
>>p_memsz: 216
>>p_flags: PF_W|PF_R
>>p_align: 4
>> 
>> entry: 5
>>p_type: PT_NOTE
>>p_offset: 300
>>p_vaddr: 0x180012c
>>p_paddr: 0x180012c
>>p_filesz: 48
>>p_memsz: 48
>>p_flags: PF_R
>>p_align: 4
>> 
>> entry: 6
>>p_type: PT_LOAD
>>p_offset: 0
>>p_vaddr: 0
>>p_paddr: 0
>>p_filesz: 0
>>p_memsz: 0
>>p_flags: PF_W|PF_R
>>p_align: 4
>> 
>> I note some things unique to this compared
>> to powerpc64 and to what the old code would
>> do for what is unique:
>> 
>> There are 2 PT_LOADS with PF_X and there are a
>> bunch of pages between that are not covered by
>> any entry. This is from p_align being 65536 from
>> what I can tell.
>> 
>> entry 2:
>> 0x180+34112==0x1808540 for the ending of the read-only PF_X area.
>> entry 3:
>> 0x1818540 for the beginning of the writable PF_X area.
>> 0x001 differences: 65536 Bytes.
>> 
>> What is the handling of the page range that is
>> not described in any entry? Is
>> __syncicache(obj->mapbase, obj->textsize)
>> spanning the hole valid? Previously the
>> hole was not spanned as I understand.
>> 
>> 
>> Libraries also have the hole between the
>> PT_LOAD with PF_X ranges. Using /lib/libc.so
>> as an example:
>> 
>> # elfdump -p ./lib/libc.so.7 | less
>> 
>> program header:
>> 
>> entry: 0
>>p_type: PT_LOAD
>>p_offset: 0
>>p_vaddr: 0
>>p_paddr: 0
>>p_filesz: 1746472
>>p_memsz: 1746472
>>p_flags: PF_X|PF_R
>>p_align: 65536
>> 
>> entry: 1
>>p_type: PT_LOAD
>>p_offset: 1746480
>>p_vaddr: 0x1ba630
>>p_paddr: 0x1ba630
>>p_filesz: 44188
>>p_memsz: 201536
>>p_flags: PF_X|PF_W|PF_R
>>p_align: 65536
>> 
>> entry: 2
>>p_type: PT_DYNAMIC
>>p_offset: 1762600
>>p_vaddr: 0x1be528
>>p_paddr: 0x1be528
>>p_filesz: 192
>>p_memsz: 192
>>p_flags: PF_W|PF_R
>>p_align: 4
>> 
>> entry: 3
>>p_type: PT_TLS
>>p_offset: 1746480
>>p_vaddr: 0x1ba630
>>p_paddr: 0x1ba630
>>p_filesz: 2832
>>p_memsz: 2860
>>p_flags: PF_R
>>p_align: 16
>> 
>> entry: 4
>>p_type: PT_NULL
>>p_offset: 1745288
>>p_vaddr: 0x1aa188
>>p_paddr: 0x1aa188
>>p_filesz: 236
>>p_memsz: 236
>>p_flags: PF_R
>>p_align: 4
>> 
>> entry: 5
>>

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Oct-30, at 3:23 PM, Alexander Richardson  
wrote:

> . . .
> Before this change obj->textsize was always set as the end of
> PT_LOAD[0]. Now it will contain everything up to the end of the last
> PT_LOAD with execute permissions. In the binary you dumped this is
> PT_LOAD[0] both before and after the patch so there is no change in
> behaviour. The .got and .plt we not included in textsize before this
> patch either. Therefore the only way I can see this patch breaking
> anything is if the PHDRS of one of the loaded libraries causes a
> change in behaviour.

A fair night's sleep helps.

Not only did I read something into your description that was
not there, I looked at powerpc64 when it turns out that
only 32-bit powerpc shows the problem. (I asked.)

I'm also told reverting just those changes fixes 32-bit
powerpc context.

So I'm using an older 32-bit powerpc context to look at:

https://artifact.ci.freebsd.org/snapshot/head/r339870/powerpc/powerpc/base*.txz

materials now. -r339870 is the first 32-bit powerpc build
available there after the changes were made.

So in a /339870/ area from expanding the .txz's
inside it I get:

# elfdump -p ./bin/ls | less

program header:

entry: 0
p_type: PT_PHDR
p_offset: 52
p_vaddr: 0x1800034
p_paddr: 0x1800034
p_filesz: 224
p_memsz: 224
p_flags: PF_X|PF_R
p_align: 4

entry: 1
p_type: PT_INTERP
p_offset: 276
p_vaddr: 0x1800114
p_paddr: 0x1800114
p_filesz: 21
p_memsz: 21
p_flags: PF_R
p_align: 1

entry: 2
p_type: PT_LOAD
p_offset: 0
p_vaddr: 0x180
p_paddr: 0x180
p_filesz: 34112
p_memsz: 34112
p_flags: PF_X|PF_R
p_align: 65536

entry: 3
p_type: PT_LOAD
p_offset: 34112
p_vaddr: 0x1818540
p_paddr: 0x1818540
p_filesz: 316
p_memsz: 1752
p_flags: PF_X|PF_W|PF_R
p_align: 65536

entry: 4
p_type: PT_DYNAMIC
p_offset: 34132
p_vaddr: 0x1818554
p_paddr: 0x1818554
p_filesz: 216
p_memsz: 216
p_flags: PF_W|PF_R
p_align: 4

entry: 5
p_type: PT_NOTE
p_offset: 300
p_vaddr: 0x180012c
p_paddr: 0x180012c
p_filesz: 48
p_memsz: 48
p_flags: PF_R
p_align: 4

entry: 6
p_type: PT_LOAD
p_offset: 0
p_vaddr: 0
p_paddr: 0
p_filesz: 0
p_memsz: 0
p_flags: PF_W|PF_R
p_align: 4

I note some things unique to this compared
to powerpc64 and to what the old code would
do for what is unique:

There are 2 PT_LOADS with PF_X and there are a
bunch of pages between that are not covered by
any entry. This is from p_align being 65536 from
what I can tell.

entry 2:
0x180+34112==0x1808540 for the ending of the read-only PF_X area.
entry 3:
0x1818540 for the beginning of the writable PF_X area.
0x001 differences: 65536 Bytes.

What is the handling of the page range that is
not described in any entry? Is
__syncicache(obj->mapbase, obj->textsize)
spanning the hole valid? Previously the
hole was not spanned as I understand.


Libraries also have the hole between the
PT_LOAD with PF_X ranges. Using /lib/libc.so
as an example:

# elfdump -p ./lib/libc.so.7 | less

program header:

entry: 0
p_type: PT_LOAD
p_offset: 0
p_vaddr: 0
p_paddr: 0
p_filesz: 1746472
p_memsz: 1746472
p_flags: PF_X|PF_R
p_align: 65536

entry: 1
p_type: PT_LOAD
p_offset: 1746480
p_vaddr: 0x1ba630
p_paddr: 0x1ba630
p_filesz: 44188
p_memsz: 201536
p_flags: PF_X|PF_W|PF_R
p_align: 65536

entry: 2
p_type: PT_DYNAMIC
p_offset: 1762600
p_vaddr: 0x1be528
p_paddr: 0x1be528
p_filesz: 192
p_memsz: 192
p_flags: PF_W|PF_R
p_align: 4

entry: 3
p_type: PT_TLS
p_offset: 1746480
p_vaddr: 0x1ba630
p_paddr: 0x1ba630
p_filesz: 2832
p_memsz: 2860
p_flags: PF_R
p_align: 16

entry: 4
p_type: PT_NULL
p_offset: 1745288
p_vaddr: 0x1aa188
p_paddr: 0x1aa188
p_filesz: 236
p_memsz: 236
p_flags: PF_R
p_align: 4

entry: 5
p_type: PT_LOAD
p_offset: 0
p_vaddr: 0
p_paddr: 0
p_filesz: 0
p_memsz: 0
p_flags: PF_W|PF_R
p_align: 4

1746472==0x1AA628 for the ending of the readonly code.
0x1ba630 for the beginning of the writable code.
0x10008 difference: 655544 bytes.


I doubt the following would contribute but I
note them:

PT_PHDR for ./bin/ls has PF_X indicated.

Various other entries overlap with the
PT_LOAD's that have PF_X.


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Oct-30, at 2:40 PM, Alexander Richardson  
wrote:

> On Tue, 30 Oct 2018 at 21:32, Mark Millard  
> wrote:
>> 
>> 
>> 
>> On 2018-Oct-30, at 2:23 PM, Alexander Richardson > freebsd.org> wrote:
>> 
>>> On Tue, 30 Oct 2018 at 18:19, Mark Millard  
>>> wrote:
 
 Alexander Richardson arichardson at freebsd.org wrote on
 Tue Oct 30 15:33:00 UTC 2018 :
 
> On Tue, 30 Oct 2018 at 10:17, Michael Tuexen
>  wrote:
>> 
>>> On 29. Oct 2018, at 22:08, Alex Richardson  
>>> wrote:
>>> 
>>> Author: arichardson
>>> Date: Mon Oct 29 21:08:02 2018
>>> New Revision: 339876
>>> URL: https://svnweb.freebsd.org/changeset/base/339876
>>> 
>>> Log:
>>> rtld: set obj->textsize correctly
>>> 
>>> With lld-generated binaries the first PT_LOAD will usually be a 
>>> read-only
>>> segment unless you pass --no-rosegment. For those binaries the textsize 
>>> is
>>> determined by the next PT_LOAD. To allow both LLD and bfd 2.17 binaries 
>>> to
>>> be parsed correctly use the end of the last PT_LOAD that is marked as
>>> executable instead.
>>> 
>>> I noticed that the value was wrong while adding some debug prints for 
>>> some rtld
>>> changes for CHERI binaries. `obj->textsize` only seems to be used by 
>>> PPC so the
>>> effect is untested. However, the value before was definitely wrong and 
>>> the new
>>> result matches the phdrs.
>> I build kernel and world with a revision later than this on a PPC. 
>> Buildword
>> ends up with a world where almost all binaries are segfaulting 
>> Especially gdb
>> (but svn, ls or so all segfault).
>> 
>> Best regards
>> Michael
> 
> This is rather surprising since if anything the range of the icache
> flush should increase rather than decrease after this change.
> 
> I can only see this causing a behaviour change if we actually need to
> flush more than just the executable segments.
> Is it possible that some binary/library contains a non-executable
> segment as the first PT_LOAD?
> Or is there some linker script that adds custom PHDRS?
 
 The following is based on using devel/powerpc64-xtoolchain-gcc
 to buildworld buildkernel on/for powerpc64. (I experiment with
 using fairly modern tools to target powerpc64 and powerpc.)
 The build context is head -r339076 based, both for what
 did the build and for what it was building.
 
 I report from both elfdump and objdump output
 because each seems to have some oddities in what
 it outputs.
 
 I start with elfdump (which leaves sh_flags blank
 and shows a section header with sh_name empty
 that objdump does not list at all):
 
 # elfdump -pc /bin/ls | less
 
 . . .
 
 As for objdump on the same file (section
 one less than elfdump listed, no empty sh_name
 section listed):
 
 # objdump -ph /bin/ls | less
 
 /bin/ls: file format elf64-powerpc-freebsd
 
 Program Header:
   PHDR off0x0040 vaddr 0x1040 paddr 
 0x1040 align 2**3
filesz 0x0188 memsz 0x0188 flags r--
 INTERP off0x01c8 vaddr 0x11c8 paddr 
 0x11c8 align 2**0
filesz 0x0015 memsz 0x0015 flags r--
   LOAD off0x vaddr 0x1000 paddr 
 0x1000 align 2**16
filesz 0x910c memsz 0x910c flags r-x
   LOAD off0x9110 vaddr 0x10019110 paddr 
 0x10019110 align 2**16
filesz 0x0ee0 memsz 0x10e8 flags rw-
 DYNAMIC off0x9138 vaddr 0x10019138 paddr 
 0x10019138 align 2**3
filesz 0x01c0 memsz 0x01c0 flags rw-
   NOTE off0x01e0 vaddr 0x11e0 paddr 
 0x11e0 align 2**2
filesz 0x0030 memsz 0x0030 flags r--
  STACK off0x vaddr 0x paddr 
 0x align 2**4
filesz 0x memsz 0x flags rw-
 
 Dynamic Section:
 NEEDED   libutil.so.9
 NEEDED   libncursesw.so.8
 NEEDED   libc.so.7
 INIT 0x10019328
 FINI 0x10019340
 HASH 0x1210
 STRTAB   0x1d00
 SYMTAB   0x1490
 STRSZ0x035a
 SYMENT   0x0018
 DEBUG0x
 PLTGOT   0x10019898
 PLTRELSZ 0x06f0
 PLTREL   0x0007
 

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

On 2018-Oct-30, at 2:23 PM, Alexander Richardson  
wrote:

> On Tue, 30 Oct 2018 at 18:19, Mark Millard  
> wrote:
>> 
>> Alexander Richardson arichardson at freebsd.org wrote on
>> Tue Oct 30 15:33:00 UTC 2018 :
>> 
>>> On Tue, 30 Oct 2018 at 10:17, Michael Tuexen
>>>  wrote:
 
> On 29. Oct 2018, at 22:08, Alex Richardson  
> wrote:
> 
> Author: arichardson
> Date: Mon Oct 29 21:08:02 2018
> New Revision: 339876
> URL: https://svnweb.freebsd.org/changeset/base/339876
> 
> Log:
> rtld: set obj->textsize correctly
> 
> With lld-generated binaries the first PT_LOAD will usually be a read-only
> segment unless you pass --no-rosegment. For those binaries the textsize is
> determined by the next PT_LOAD. To allow both LLD and bfd 2.17 binaries to
> be parsed correctly use the end of the last PT_LOAD that is marked as
> executable instead.
> 
> I noticed that the value was wrong while adding some debug prints for 
> some rtld
> changes for CHERI binaries. `obj->textsize` only seems to be used by PPC 
> so the
> effect is untested. However, the value before was definitely wrong and 
> the new
> result matches the phdrs.
 I build kernel and world with a revision later than this on a PPC. 
 Buildword
 ends up with a world where almost all binaries are segfaulting 
 Especially gdb
 (but svn, ls or so all segfault).
 
 Best regards
 Michael
>>> 
>>> This is rather surprising since if anything the range of the icache
>>> flush should increase rather than decrease after this change.
>>> 
>>> I can only see this causing a behaviour change if we actually need to
>>> flush more than just the executable segments.
>>> Is it possible that some binary/library contains a non-executable
>>> segment as the first PT_LOAD?
>>> Or is there some linker script that adds custom PHDRS?
>> 
>> The following is based on using devel/powerpc64-xtoolchain-gcc
>> to buildworld buildkernel on/for powerpc64. (I experiment with
>> using fairly modern tools to target powerpc64 and powerpc.)
>> The build context is head -r339076 based, both for what
>> did the build and for what it was building.
>> 
>> I report from both elfdump and objdump output
>> because each seems to have some oddities in what
>> it outputs.
>> 
>> I start with elfdump (which leaves sh_flags blank
>> and shows a section header with sh_name empty
>> that objdump does not list at all):
>> 
>> # elfdump -pc /bin/ls | less
>> 
>> . . .
>> 
>> As for objdump on the same file (section
>> one less than elfdump listed, no empty sh_name
>> section listed):
>> 
>> # objdump -ph /bin/ls | less
>> 
>> /bin/ls: file format elf64-powerpc-freebsd
>> 
>> Program Header:
>>PHDR off0x0040 vaddr 0x1040 paddr 
>> 0x1040 align 2**3
>> filesz 0x0188 memsz 0x0188 flags r--
>>  INTERP off0x01c8 vaddr 0x11c8 paddr 
>> 0x11c8 align 2**0
>> filesz 0x0015 memsz 0x0015 flags r--
>>LOAD off0x vaddr 0x1000 paddr 
>> 0x1000 align 2**16
>> filesz 0x910c memsz 0x910c flags r-x
>>LOAD off0x9110 vaddr 0x10019110 paddr 
>> 0x10019110 align 2**16
>> filesz 0x0ee0 memsz 0x10e8 flags rw-
>> DYNAMIC off0x9138 vaddr 0x10019138 paddr 
>> 0x10019138 align 2**3
>> filesz 0x01c0 memsz 0x01c0 flags rw-
>>NOTE off0x01e0 vaddr 0x11e0 paddr 
>> 0x11e0 align 2**2
>> filesz 0x0030 memsz 0x0030 flags r--
>>   STACK off0x vaddr 0x paddr 
>> 0x align 2**4
>> filesz 0x memsz 0x flags rw-
>> 
>> Dynamic Section:
>>  NEEDED   libutil.so.9
>>  NEEDED   libncursesw.so.8
>>  NEEDED   libc.so.7
>>  INIT 0x10019328
>>  FINI 0x10019340
>>  HASH 0x1210
>>  STRTAB   0x1d00
>>  SYMTAB   0x1490
>>  STRSZ0x035a
>>  SYMENT   0x0018
>>  DEBUG0x
>>  PLTGOT   0x10019898
>>  PLTRELSZ 0x06f0
>>  PLTREL   0x0007
>>  JMPREL   0x100012f8
>>  0x7000   0x100089b4
>>  RELA 0x10001160
>>  RELASZ   0x0198
>>  RELAENT  0x0018
>>  VERNEED  0x10001110
>>  VERNEEDNUM   0x0001
>>  VERSYM   0x1000105a
>> 
>> Version References:

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

Konstantin Belousov kostikbel at gmail.com wrote on
Tue Oct 30 18:04:04 UTC 2018 :

> On Tue, Oct 30, 2018 at 03:32:40PM +, Alexander Richardson wrote:
> > On Tue, 30 Oct 2018 at 10:17, Michael Tuexen
> >  wrote:
> > >
> > > > On 29. Oct 2018, at 22:08, Alex Richardson  
> > > > wrote:
> > > >
> > > > Author: arichardson
> > > > Date: Mon Oct 29 21:08:02 2018
> > > > New Revision: 339876
> > > > URL: https://svnweb.freebsd.org/changeset/base/339876
> > > >
> > > > Log:
> > > >  rtld: set obj->textsize correctly
> > > >
> > > >  With lld-generated binaries the first PT_LOAD will usually be a 
> > > > read-only
> > > >  segment unless you pass --no-rosegment. For those binaries the 
> > > > textsize is
> > > >  determined by the next PT_LOAD. To allow both LLD and bfd 2.17 
> > > > binaries to
> > > >  be parsed correctly use the end of the last PT_LOAD that is marked as
> > > >  executable instead.
> > > >
> > > >  I noticed that the value was wrong while adding some debug prints for 
> > > > some rtld
> > > >  changes for CHERI binaries. `obj->textsize` only seems to be used by 
> > > > PPC so the
> > > >  effect is untested. However, the value before was definitely wrong and 
> > > > the new
> > > >  result matches the phdrs.
> > > I build kernel and world with a revision later than this on a PPC. 
> > > Buildword
> > > ends up with a world where almost all binaries are segfaulting 
> > > Especially gdb
> > > (but svn, ls or so all segfault).
> > >
> > > Best regards
> > > Michael
> > 
> > This is rather surprising since if anything the range of the icache
> > flush should increase rather than decrease after this change.
> > 
> > I can only see this causing a behaviour change if we actually need to
> > flush more than just the executable segments.
> > Is it possible that some binary/library contains a non-executable
> > segment as the first PT_LOAD?
> > Or is there some linker script that adds custom PHDRS?
> > 
> Could it be that there is a hole between start of the object mapping and
> the last PT_LOADable segment eligible for execution ?

[This note may be easier to deal with than the first
note that I sent out.]

[My examples are from devel/powerpc64-xtoolchain-gcc used
to buildworld buildkernel targeting a head -r339076 based
powerpc64 environment. I do that on powerpc64 as well.]

powerpc64 loads the readonly data and the readonly code in one PT_LOAD,
the first. The 2nd PT_LOAD is for sections without the readonly status,
that includes .got and .plt being spanned. See below from
objdump -ph for /bin/ls :

Program Header:
PHDR off0x0040 vaddr 0x1040 paddr 
0x1040 align 2**3
 filesz 0x0188 memsz 0x0188 flags r--
  INTERP off0x01c8 vaddr 0x11c8 paddr 
0x11c8 align 2**0
 filesz 0x0015 memsz 0x0015 flags r--
LOAD off0x vaddr 0x1000 paddr 
0x1000 align 2**16
 filesz 0x910c memsz 0x910c flags r-x
LOAD off0x9110 vaddr 0x10019110 paddr 
0x10019110 align 2**16
 filesz 0x0ee0 memsz 0x10e8 flags rw-
 DYNAMIC off0x9138 vaddr 0x10019138 paddr 
0x10019138 align 2**3
 filesz 0x01c0 memsz 0x01c0 flags rw-
NOTE off0x01e0 vaddr 0x11e0 paddr 
0x11e0 align 2**2
 filesz 0x0030 memsz 0x0030 flags r--
   STACK off0x vaddr 0x paddr 
0x align 2**4
 filesz 0x memsz 0x flags rw-

Sections:
Idx Name  Size  VMA   LMA   File off  Algn
  0 .interp   0015  11c8  11c8  01c8  2**0
  CONTENTS, ALLOC, LOAD, READONLY, DATA
  1 .note.tag 0030  11e0  11e0  01e0  2**2
  CONTENTS, ALLOC, LOAD, READONLY, DATA
  2 .hash 027c  1210  1210  0210  2**3
  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .dynsym   0870  1490  1490  0490  2**3
  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .dynstr   035a  1d00  1d00  0d00  2**0
  CONTENTS, ALLOC, LOAD, READONLY, DATA
  5 .gnu.version  00b4  1000105a  1000105a  105a  2**1
  CONTENTS, ALLOC, LOAD, READONLY, DATA
  6 .gnu.version_r 0050  10001110  10001110  1110  2**3
  CONTENTS, ALLOC, LOAD, READONLY, DATA
  7 .rela.dyn 0198  10001160  10001160  1160  2**3
  CONTENTS, ALLOC, LOAD, READONLY, DATA
  8 .rela.plt 06f0  100012f8  100012f8  12f8  

Re: svn commit: r339876 - head/libexec/rtld-elf

[Mark Millard via svn-src-head]

Alexander Richardson arichardson at freebsd.org wrote on
Tue Oct 30 15:33:00 UTC 2018 :

> On Tue, 30 Oct 2018 at 10:17, Michael Tuexen
>  wrote:
> >
> > > On 29. Oct 2018, at 22:08, Alex Richardson  
> > > wrote:
> > >
> > > Author: arichardson
> > > Date: Mon Oct 29 21:08:02 2018
> > > New Revision: 339876
> > > URL: https://svnweb.freebsd.org/changeset/base/339876
> > >
> > > Log:
> > >  rtld: set obj->textsize correctly
> > >
> > >  With lld-generated binaries the first PT_LOAD will usually be a read-only
> > >  segment unless you pass --no-rosegment. For those binaries the textsize 
> > > is
> > >  determined by the next PT_LOAD. To allow both LLD and bfd 2.17 binaries 
> > > to
> > >  be parsed correctly use the end of the last PT_LOAD that is marked as
> > >  executable instead.
> > >
> > >  I noticed that the value was wrong while adding some debug prints for 
> > > some rtld
> > >  changes for CHERI binaries. `obj->textsize` only seems to be used by PPC 
> > > so the
> > >  effect is untested. However, the value before was definitely wrong and 
> > > the new
> > >  result matches the phdrs.
> > I build kernel and world with a revision later than this on a PPC. Buildword
> > ends up with a world where almost all binaries are segfaulting 
> > Especially gdb
> > (but svn, ls or so all segfault).
> >
> > Best regards
> > Michael
> 
> This is rather surprising since if anything the range of the icache
> flush should increase rather than decrease after this change.
> 
> I can only see this causing a behaviour change if we actually need to
> flush more than just the executable segments.
> Is it possible that some binary/library contains a non-executable
> segment as the first PT_LOAD?
> Or is there some linker script that adds custom PHDRS?

The following is based on using devel/powerpc64-xtoolchain-gcc
to buildworld buildkernel on/for powerpc64. (I experiment with
using fairly modern tools to target powerpc64 and powerpc.)
The build context is head -r339076 based, both for what
did the build and for what it was building.

I report from both elfdump and objdump output
because each seems to have some oddities in what
it outputs.

I start with elfdump (which leaves sh_flags blank
and shows a section header with sh_name empty
that objdump does not list at all):

# elfdump -pc /bin/ls | less

program header:

entry: 0
p_type: PT_PHDR
p_offset: 64
p_vaddr: 0x1040
p_paddr: 0x1040
p_filesz: 392
p_memsz: 392
p_flags: PF_R
p_align: 8

entry: 1
p_type: PT_INTERP
p_offset: 456
p_vaddr: 0x11c8
p_paddr: 0x11c8
p_filesz: 21
p_memsz: 21
p_flags: PF_R
p_align: 1

entry: 2
p_type: PT_LOAD
p_offset: 0
p_vaddr: 0x1000
p_paddr: 0x1000
p_filesz: 37132
p_memsz: 37132
p_flags: PF_X|PF_R
p_align: 65536

entry: 3
p_type: PT_LOAD
p_offset: 37136
p_vaddr: 0x10019110
p_paddr: 0x10019110
p_filesz: 3808
p_memsz: 4328
p_flags: PF_W|PF_R
p_align: 65536

entry: 4
p_type: PT_DYNAMIC
p_offset: 37176
p_vaddr: 0x10019138
p_paddr: 0x10019138
p_filesz: 448
p_memsz: 448
p_flags: PF_W|PF_R
p_align: 8

entry: 5
p_type: PT_NOTE
p_offset: 480
p_vaddr: 0x11e0
p_paddr: 0x11e0
p_filesz: 48
p_memsz: 48
p_flags: PF_R
p_align: 4

entry: 6
p_type: PT_LOAD
p_offset: 0
p_vaddr: 0
p_paddr: 0
p_filesz: 0
p_memsz: 0
p_flags: PF_W|PF_R
p_align: 16

section header:

entry: 0
sh_name: 
sh_type: SHT_NULL
sh_flags: 
sh_addr: 0
sh_offset: 0
sh_size: 0
sh_link: 0
sh_info: 0
sh_addralign: 0
sh_entsize: 0

entry: 1
sh_name: .interp
sh_type: SHT_PROGBITS
sh_flags: 
sh_addr: 0x11c8
sh_offset: 456
sh_size: 21
sh_link: 0
sh_info: 0
sh_addralign: 1
sh_entsize: 0

entry: 2
sh_name: .note.tag
sh_type: SHT_NOTE
sh_flags: 
sh_addr: 0x11e0
sh_offset: 480
sh_size: 48
sh_link: 0
sh_info: 0
sh_addralign: 4
sh_entsize: 0

entry: 3
sh_name: .hash
sh_type: SHT_HASH
sh_flags: 
sh_addr: 0x1210
sh_offset: 528
sh_size: 636
sh_link: 4
sh_info: 0
sh_addralign: 8
sh_entsize: 4

entry: 4
sh_name: .dynsym
sh_type: SHT_DYNSYM
sh_flags: 
sh_addr: 0x1490
sh_offset: 1168
sh_size: 2160
sh_link: 5
sh_info: 1
sh_addralign: 8
sh_entsize: 24

entry: 5
sh_name: .dynstr
sh_type: SHT_STRTAB

Re: svn commit: r338949 - in head/sys/dev: cxgb ida iwn ixl mfi mpr mps mvs my oce pccbb pcn ral rl

[Mark Millard via svn-src-head]

[Similar to an old note about -r336099 .]

Both:

https://ci.freebsd.org/job/FreeBSD-head-amd64-build/10300/consoleText
https://ci.freebsd.org/job/FreeBSD-head-i386-build/9428/consoleText

show (from the amd64 example):

===> zlib (install)
install -N /usr/src/etc -T release -o root -g wheel -m 555   zlib.ko 
/usr/obj/usr/src/amd64.amd64/release/dist/kernel/boot/kernel/
install -N /usr/src/etc -T debug -o root -g wheel -m 555   zlib.ko.debug 
/usr/obj/usr/src/amd64.amd64/release/dist/kernel/usr/lib/debug/boot/kernel/
kldxref /usr/obj/usr/src/amd64.amd64/release/dist/kernel/boot/kernel
kldxref: Parse error of description string U16:vendor; U16:device
*** Error code 1

Stop.
make[4]: stopped in /usr/src/sys/modules
*** Error code 1

Stop.
make[3]: stopped in /usr/obj/usr/src/amd64.amd64/sys/GENERIC
*** Error code 1




The examples of the extra space in "; U16" appear to be:

Modified: head/sys/dev/pcn/if_pcn.c
==
--- head/sys/dev/pcn/if_pcn.c   Wed Sep 26 17:12:14 2018(r338948)
+++ head/sys/dev/pcn/if_pcn.c   Wed Sep 26 17:12:30 2018(r338949)
@@ -193,6 +193,8 @@ static driver_t pcn_driver = {
 static devclass_t pcn_devclass;
 
 DRIVER_MODULE(pcn, pci, pcn_driver, pcn_devclass, 0, 0);
+MODULE_PNP_INFO("U16:vendor; U16:device", pci, pcn, pcn_devs,
+nitems(pcn_devs) - 1);
 DRIVER_MODULE(miibus, pcn, miibus_driver, miibus_devclass, 0, 0);
 
 #define PCN_CSR_SETBIT(sc, reg, x) \

Modified: head/sys/dev/ral/if_ral_pci.c
==
--- head/sys/dev/ral/if_ral_pci.c   Wed Sep 26 17:12:14 2018
(r338948)
+++ head/sys/dev/ral/if_ral_pci.c   Wed Sep 26 17:12:30 2018
(r338949)
@@ -178,6 +178,8 @@ static driver_t ral_pci_driver = {
 static devclass_t ral_devclass;
 
 DRIVER_MODULE(ral, pci, ral_pci_driver, ral_devclass, NULL, NULL);
+MODULE_PNP_INFO("U16:vendor; U16:device; D:#", pci, ral, ral_pci_ids,
+nitems(ral_pci_ids) - 1);
 
 static int
 ral_pci_probe(device_t dev)

Modified: head/sys/dev/rl/if_rl.c
==
--- head/sys/dev/rl/if_rl.c Wed Sep 26 17:12:14 2018(r338948)
+++ head/sys/dev/rl/if_rl.c Wed Sep 26 17:12:30 2018(r338949)
@@ -259,6 +259,8 @@ static driver_t rl_driver = {
 static devclass_t rl_devclass;
 
 DRIVER_MODULE(rl, pci, rl_driver, rl_devclass, 0, 0);
+MODULE_PNP_INFO("U16:vendor; U16:device", pci, rl, rl_devs,
+nitems(rl_devs) - 1);
 DRIVER_MODULE(rl, cardbus, rl_driver, rl_devclass, 0, 0);
 DRIVER_MODULE(miibus, rl, miibus_driver, miibus_devclass, 0, 0);

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r336299 - in head: include lib/msun lib/msun/ld128 lib/msun/ld80 lib/msun/man lib/msun/src

[Mark Millard via svn-src-head]

[I've got a historical WERROR= used in my amd64-gcc based builds.]

On 2018-Sep-20, at 6:05 PM, Mark Millard  wrote:


> Li-Wen Hsu lwhsu at freebsd.org wrote on
> Thu Sep 20 22:10:19 UTC 2018 :
> 
>> On Thu, Sep 20, 2018 at 10:06 PM Mark Johnston  wrote:
>>> 
>>> On Thu, Sep 20, 2018 at 09:39:24AM -0700, John Baldwin wrote:
 On 9/20/18 8:54 AM, Mark Johnston wrote:
> On Sun, Jul 15, 2018 at 12:23:11AM +, Matt Macy wrote:
>> Author: mmacy
>> Date: Sun Jul 15 00:23:10 2018
>> New Revision: 336299
>> URL: https://svnweb.freebsd.org/changeset/base/336299
>> 
>> Log:
>>  msun: add ld80/ld128 powl, cpow, cpowf, cpowl from openbsd
>> 
>>  This corresponds to the latest status (hasn't changed in 9+
>>  years) from openbsd of ld80/ld128 powl, and source cpowf, cpow,
>>  cpowl (the complex power functions for float complex, double
>>  complex, and long double complex) which are required for C99
>>  compliance and were missing from FreeBSD. Also required for
>>  some numerical codes using complex numbered Hamiltonians.
>> 
>>  Thanks to jhb for tracking down the issue with making
>>  weak_reference compile on powerpc.
>> 
>>  When asked to review, bde said "I don't like it" - but
>>  provided no actionable feedback or superior implementations.
>> 
>>  Discussed with: jhb
>>  Submitted by: jmd
>>  Differential Revision: https://reviews.freebsd.org/D15919
> 
> This seems to have broken the gcc build:
> https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc/
> 
> /workspace/src/lib/msun/ld80/e_powl.c:275:1: error: floating constant 
> exceeds range of 'long double' [-Werror=overflow]
>  if( y >= LDBL_MAX )
>  ^~
 
 Which architecture?  i386 doesn't get build with i386-xtoolchain-gcc 
 pending
 some patches I haven't yet posted for review related to the weirdness we do
 with floating point on i386.
>>> 
>>> This is the -m32 build on amd64.  I haven't tested it myself, but Mark
>>> Millard noted that the issue might be fixed by a gcc update.
>>> 
>> 
>> I suspect this.  Each build is in a fresh created jail with the latest
>> branch of packages from pkg.freebsd.org.
>> 
>> At the beginning of (warning: 56MB file)
>> https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc/7262/consoleText
>> 
>> There is:
>> 
>> New packages to be INSTALLED:
>>amd64-xtoolchain-gcc: 0.4_1
>>amd64-gcc: 6.4.0_2
>>mpfr: 4.0.1
>>gmp: 6.1.2
>>mpc: 1.1.0_1
>>amd64-binutils: 2.30_5,1
>> 
>> Number of packages to be installed: 6
>> 
>> Or is there a newer version of devel/amd64-gcc I am not aware?
> 
> I wonder when ci.freebsd.org started using "amd64-gcc: 6.4.0_2".
> Looking: https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc/6665/consoleText
> from back on 2018-Jul-26. This suggests being based on
> -r475319 (2018-Jul-25) for devel/powerpc-gcc (the master
> port).
> 
> Looks like -r475290 (the float.h removal) is the first version
> with PORT_REVISION being 2, as are all later versions at
> this point. Technically -r476273 (2018-Aug-2) removed a typo from
> -r475290 but at the time the typo seemed to not make an operational
> difference. Still, ci.freebsd.org seems to not be
> using the version with the typo fixed.
> 
> Looks like system head being built was -r338831. (I'm only at
> -r338675 so far.)
> 
> I've started a buildworld buildkernel based -on head -r3338675
> and based on master port:
> 
> # svnlite diff /usr/ports/devel/powerpc64-gcc/ | more
> Index: /usr/ports/devel/powerpc64-gcc/Makefile
> ===
> --- /usr/ports/devel/powerpc64-gcc/Makefile (revision 480180)
> +++ /usr/ports/devel/powerpc64-gcc/Makefile (working copy)
> @@ -16,7 +16,8 @@
> LIB_DEPENDS=   libgmp.so:math/gmp \
>libmpfr.so:math/mpfr \
>libmpc.so:math/mpc
> -BUILD_DEPENDS= ${BU_PREFIX}-as:devel/${PKGNAMEPREFIX}binutils
> +BUILD_DEPENDS= ${BU_PREFIX}-as:devel/${PKGNAMEPREFIX}binutils \
> +   objdump:devel/binutils
> RUN_DEPENDS=   ${BU_PREFIX}-as:devel/${PKGNAMEPREFIX}binutils
> 
> USES=  gmake iconv libtool tar:xz makeinfo compiler
> 
> which does not have -r478209 involved ( un-breaking
> devel/aarch64-none-elf-gcc ).
> 
> # svnlite diff /usr/ports/devel/amd64-gcc/ | more
> #
> 
> So no differences there (relative to -r480180).
> 
> 
> (The objdump is from having built systems based on
> WITHOUT_BINUTILS and WITHOUT_BINUTILS_BOOTSTRAP at times
> but some devel/*-gcc builds trying to use objdump in the
> configuration part of gcc's build.)
> 
> My past builds using at or after -r475290 for various system
> head revisions have completed without being stopped by an
> error but it has been a while. I will see for this one, but
> it is only into building lib32 so far.
> 
> (Note: My port builds are via devel/poudriere-devel tied to
> my /usr/ports/ and to my
> 

Re: svn commit: r336299 - in head: include lib/msun lib/msun/ld128 lib/msun/ld80 lib/msun/man lib/msun/src

[Mark Millard via svn-src-head]

Li-Wen Hsu lwhsu at freebsd.org wrote on
Thu Sep 20 22:10:19 UTC 2018 :

> On Thu, Sep 20, 2018 at 10:06 PM Mark Johnston  wrote:
> >
> > On Thu, Sep 20, 2018 at 09:39:24AM -0700, John Baldwin wrote:
> > > On 9/20/18 8:54 AM, Mark Johnston wrote:
> > > > On Sun, Jul 15, 2018 at 12:23:11AM +, Matt Macy wrote:
> > > >> Author: mmacy
> > > >> Date: Sun Jul 15 00:23:10 2018
> > > >> New Revision: 336299
> > > >> URL: https://svnweb.freebsd.org/changeset/base/336299
> > > >>
> > > >> Log:
> > > >>   msun: add ld80/ld128 powl, cpow, cpowf, cpowl from openbsd
> > > >>
> > > >>   This corresponds to the latest status (hasn't changed in 9+
> > > >>   years) from openbsd of ld80/ld128 powl, and source cpowf, cpow,
> > > >>   cpowl (the complex power functions for float complex, double
> > > >>   complex, and long double complex) which are required for C99
> > > >>   compliance and were missing from FreeBSD. Also required for
> > > >>   some numerical codes using complex numbered Hamiltonians.
> > > >>
> > > >>   Thanks to jhb for tracking down the issue with making
> > > >>   weak_reference compile on powerpc.
> > > >>
> > > >>   When asked to review, bde said "I don't like it" - but
> > > >>   provided no actionable feedback or superior implementations.
> > > >>
> > > >>   Discussed with: jhb
> > > >>   Submitted by: jmd
> > > >>   Differential Revision: https://reviews.freebsd.org/D15919
> > > >
> > > > This seems to have broken the gcc build:
> > > > https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc/
> > > >
> > > > /workspace/src/lib/msun/ld80/e_powl.c:275:1: error: floating constant 
> > > > exceeds range of 'long double' [-Werror=overflow]
> > > >   if( y >= LDBL_MAX )
> > > >   ^~
> > >
> > > Which architecture?  i386 doesn't get build with i386-xtoolchain-gcc 
> > > pending
> > > some patches I haven't yet posted for review related to the weirdness we 
> > > do
> > > with floating point on i386.
> >
> > This is the -m32 build on amd64.  I haven't tested it myself, but Mark
> > Millard noted that the issue might be fixed by a gcc update.
> >
> 
> I suspect this.  Each build is in a fresh created jail with the latest
> branch of packages from pkg.freebsd.org.
> 
> At the beginning of (warning: 56MB file)
> https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc/7262/consoleText
> 
> There is:
> 
> New packages to be INSTALLED:
> amd64-xtoolchain-gcc: 0.4_1
> amd64-gcc: 6.4.0_2
> mpfr: 4.0.1
> gmp: 6.1.2
> mpc: 1.1.0_1
> amd64-binutils: 2.30_5,1
> 
> Number of packages to be installed: 6
> 
> Or is there a newer version of devel/amd64-gcc I am not aware?

I wonder when ci.freebsd.org started using "amd64-gcc: 6.4.0_2".
Looking: https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc/6665/consoleText
from back on 2018-Jul-26. This suggests being based on
-r475319 (2018-Jul-25) for devel/powerpc-gcc (the master
port).

Looks like -r475290 (the float.h removal) is the first version
with PORT_REVISION being 2, as are all later versions at
this point. Technically -r476273 (2018-Aug-2) removed a typo from
-r475290 but at the time the typo seemed to not make an operational
difference. Still, ci.freebsd.org seems to not be
using the version with the typo fixed.

Looks like system head being built was -r338831. (I'm only at
-r338675 so far.)

I've started a buildworld buildkernel based -on head -r3338675
and based on master port:

# svnlite diff /usr/ports/devel/powerpc64-gcc/ | more
Index: /usr/ports/devel/powerpc64-gcc/Makefile
===
--- /usr/ports/devel/powerpc64-gcc/Makefile (revision 480180)
+++ /usr/ports/devel/powerpc64-gcc/Makefile (working copy)
@@ -16,7 +16,8 @@
 LIB_DEPENDS=   libgmp.so:math/gmp \
libmpfr.so:math/mpfr \
libmpc.so:math/mpc
-BUILD_DEPENDS= ${BU_PREFIX}-as:devel/${PKGNAMEPREFIX}binutils
+BUILD_DEPENDS= ${BU_PREFIX}-as:devel/${PKGNAMEPREFIX}binutils \
+   objdump:devel/binutils
 RUN_DEPENDS=   ${BU_PREFIX}-as:devel/${PKGNAMEPREFIX}binutils
 
 USES=  gmake iconv libtool tar:xz makeinfo compiler

which does not have -r478209 involved ( un-breaking
devel/aarch64-none-elf-gcc ).

# svnlite diff /usr/ports/devel/amd64-gcc/ | more
#

So no differences there (relative to -r480180).


(The objdump is from having built systems based on
WITHOUT_BINUTILS and WITHOUT_BINUTILS_BOOTSTRAP at times
but some devel/*-gcc builds trying to use objdump in the
configuration part of gcc's build.)

My past builds using at or after -r475290 for various system
head revisions have completed without being stopped by an
error but it has been a while. I will see for this one, but
it is only into building lib32 so far.

(Note: My port builds are via devel/poudriere-devel tied to
my /usr/ports/ and to my
/usr/obj/DESTDIRs/clang-amd64-installworld-poud/ that is installed
from the same build that I install and boot from, just with
distrib-dirs distribution DB_FROM_SRC=1 also 

Re: svn commit: r336299 - in head: include lib/msun lib/msun/ld128 lib/msun/ld80 lib/msun/man lib/msun/src

[Mark Millard via svn-src-head]

John Baldwin jhb at FreeBSD.org wrote on
Thu Sep 20 16:39:27 UTC 2018 :

> On 9/20/18 8:54 AM, Mark Johnston wrote:
> > On Sun, Jul 15, 2018 at 12:23:11AM +, Matt Macy wrote:
> >> Author: mmacy
> >> Date: Sun Jul 15 00:23:10 2018
> >> New Revision: 336299
> >> URL: https://svnweb.freebsd.org/changeset/base/336299
> >>
> >> Log:
> >>   msun: add ld80/ld128 powl, cpow, cpowf, cpowl from openbsd
> >>   
> >>   This corresponds to the latest status (hasn't changed in 9+
> >>   years) from openbsd of ld80/ld128 powl, and source cpowf, cpow,
> >>   cpowl (the complex power functions for float complex, double
> >>   complex, and long double complex) which are required for C99
> >>   compliance and were missing from FreeBSD. Also required for
> >>   some numerical codes using complex numbered Hamiltonians.
> >>   
> >>   Thanks to jhb for tracking down the issue with making
> >>   weak_reference compile on powerpc.
> >>   
> >>   When asked to review, bde said "I don't like it" - but
> >>   provided no actionable feedback or superior implementations.
> >>   
> >>   Discussed with: jhb
> >>   Submitted by: jmd
> >>   Differential Revision: https://reviews.freebsd.org/D15919
> > 
> > This seems to have broken the gcc build:
> > https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc/
> > 
> > /workspace/src/lib/msun/ld80/e_powl.c:275:1: error: floating constant 
> > exceeds range of 'long double' [-Werror=overflow]
> >   if( y >= LDBL_MAX )
> >   ^~
> 
> Which architecture?  i386 doesn't get build with i386-xtoolchain-gcc pending
> some patches I haven't yet posted for review related to the weirdness we do
> with floating point on i386.


Looking at:

https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc/7260/consoleText
(the most recent completed build attempt when I looked)

It is the same as the 2018-Jun-28 list report about lib32 failing
to build in the amd64 build (that lead to your ports' head -r475290):

--- lib/msun__L ---
--- e_powl.o ---
/usr/local/bin/x86_64-unknown-freebsd11.1-gcc -DCOMPAT_32BIT -march=i686 -mmmx 
-msse -msse2 -m32  
-L/workspace/obj/workspace/src/amd64.amd64/obj-lib32/tmp/usr/lib32  
--sysroot=/workspace/obj/workspace/src/amd64.amd64/obj-lib32/tmp  
-B/usr/local/x86_64-unknown-freebsd11.1/bin/ 
-B/workspace/obj/workspace/src/amd64.amd64/obj-lib32/tmp/usr/lib32  -O2 -pipe 
-I/workspace/src/lib/msun/x86 -I/workspace/src/lib/msun/ld80 
-I/workspace/src/lib/msun/i387 -I/workspace/src/lib/msun/src 
-I/workspace/src/lib/libc/include  -I/workspace/src/lib/libc/i386  -g -MD  
-MF.depend.e_powl.o -MTe_powl.o -std=gnu99 -fstack-protector-strong 
-Wsystem-headers -Werror -Wno-pointer-sign -Wno-error=address 
-Wno-error=array-bounds -Wno-error=attributes -Wno-error=bool-compare 
-Wno-error=cast-align -Wno-error=clobbered -Wno-error=enum-compare 
-Wno-error=extra -Wno-error=inline -Wno-error=logical-not-parentheses 
-Wno-error=strict-aliasing -Wno-error=uninitialized 
-Wno-error=unused-but-set-variable -Wno-error=unused-func
 tion -Wno-error=unused-value -Wno-error=misleading-indentation 
-Wno-error=nonnull-compare -Wno-error=shift-negative-value 
-Wno-error=tautological-compare -Wno-error=unused-const-variable 
-Wno-unknown-pragmas -c /workspace/src/lib/msun/ld80/e_powl.c -o e_powl.o
--- secure/lib/libcrypto__L ---
. . .
--- lib/msun__L ---
/workspace/src/lib/msun/ld80/e_powl.c: In function 'powl':
/workspace/src/lib/msun/ld80/e_powl.c:275:1: error: floating constant exceeds 
range of 'long double' [-Werror=overflow]
 if( y >= LDBL_MAX )
 ^~
/workspace/src/lib/msun/ld80/e_powl.c:286:1: error: floating constant exceeds 
range of 'long double' [-Werror=overflow]
 if( y <= -LDBL_MAX )
 ^~
/workspace/src/lib/msun/ld80/e_powl.c:297:1: error: floating constant exceeds 
range of 'long double' [-Werror=overflow]
 if( x >= LDBL_MAX )
 ^~
/workspace/src/lib/msun/ld80/e_powl.c:321:1: error: floating constant exceeds 
range of 'long double' [-Werror=overflow]
 if( x <= -LDBL_MAX )
 ^~
/workspace/src/lib/msun/ld80/e_powl.c: In function 'powil':
/workspace/src/lib/msun/ld80/e_powl.c:577:3: error: floating constant exceeds 
range of 'long double' [-Werror=overflow]
   return( LDBL_MAX );
   ^~

Your ports' head -r475290 does avoid the problem and allow
such builds but ci.freebsd.org has not updated to some version
with the changes as I understand.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r336299 - in head: include lib/msun lib/msun/ld128 lib/msun/ld80 lib/msun/man lib/msun/src

[Mark Millard via svn-src-head]

Mark Johnston markj at freebsd.org wrote on
Thu Sep 20 15:54:08 UTC 2018 :

> On Sun, Jul 15, 2018 at 12:23:11AM +, Matt Macy wrote:
> > Author: mmacy
> > Date: Sun Jul 15 00:23:10 2018
> > New Revision: 336299
> > URL: https://svnweb.freebsd.org/changeset/base/336299
> > 
> > Log:
> >   msun: add ld80/ld128 powl, cpow, cpowf, cpowl from openbsd
> >   
> >   This corresponds to the latest status (hasn't changed in 9+
> >   years) from openbsd of ld80/ld128 powl, and source cpowf, cpow,
> >   cpowl (the complex power functions for float complex, double
> >   complex, and long double complex) which are required for C99
> >   compliance and were missing from FreeBSD. Also required for
> >   some numerical codes using complex numbered Hamiltonians.
> >   
> >   Thanks to jhb for tracking down the issue with making
> >   weak_reference compile on powerpc.
> >   
> >   When asked to review, bde said "I don't like it" - but
> >   provided no actionable feedback or superior implementations.
> >   
> >   Discussed with: jhb
> >   Submitted by: jmd
> >   Differential Revision: https://reviews.freebsd.org/D15919
> 
> This seems to have broken the gcc build:
> https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc/
> 
> /workspace/src/lib/msun/ld80/e_powl.c:275:1: error: floating constant exceeds 
> range of 'long double' [-Werror=overflow]
>   if( y >= LDBL_MAX )

Building with a more recent vintage of the devel/amd64-gcc port
does not do this (devel/powerpc64-=gcc master port).

I've built locally multiple-times since the below that
has by John Baldwin:

QUOTE
Revision 475290 - Directory Listing 
Modified Wed Jul 25 00:50:53 2018 UTC (8 weeks, 1 day ago) by jhb
Drop builtin float.h for amd64-gcc.

GCC's builtin  header is not compatible with
sys/x86/include/float.h.  Drop the builtin header for now.  If at
some point GCC's notion of floating point constants for i386 can
converge with sys/x86/include/float.h this can be restored.

Reviewed by:bapt
Differential Revision:  
https://reviews.freebsd.org/D16073
END QUOTE


Until ci.freebsd.org updates to be based on a version that
includes the change the issue will exists there.

There was activity on svn-src-head and freebsd-current starting
back on 2018-Jun-28 that reported the issue and lead to the
change, with subject:

head -r335782 (?) broke ci.freebsd.org's FreeBSD-head-amd64-gcc build (lib32 
part of build)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r336601 - head

[Mark Millard via svn-src-head]

[I was asked what the fix might be. So I suggest specifics.]

On 2018-Sep-16, at 10:06 AM, Mark Millard  wrote:

> Looks like this head/ObsoleteFiles.inc update has a typo
> in each thing added to OLD_FILES . . .
> 
> # ls -lTdt /usr/tests/usr.bin/indent/*
> -r--r--r--  1 root  wheel   121 Sep 13 22:53:30 2018 
> /usr/tests/usr.bin/indent/Kyuafile
> . . .
> -r--r--r--  1 root  wheel   295 Sep 13 22:53:29 2018 
> /usr/tests/usr.bin/indent/binary.0
> -r--r--r--  1 root  wheel92 May  1 19:35:24 2018 
> /usr/tests/usr.bin/indent/sac.0.pro
> -r--r--r--  1 root  wheel   130 May  1 19:35:24 2018 
> /usr/tests/usr.bin/indent/sac.0.stdout
> -r--r--r--  1 root  wheel   122 May  1 19:35:24 2018 
> /usr/tests/usr.bin/indent/sac.0
> -r--r--r--  1 root  wheel94 May  1 19:35:24 2018 
> /usr/tests/usr.bin/indent/nsac.0.pro
> -r--r--r--  1 root  wheel   130 May  1 19:35:24 2018 
> /usr/tests/usr.bin/indent/nsac.0.stdout
> -r--r--r--  1 root  wheel   123 May  1 19:35:24 2018 
> /usr/tests/usr.bin/indent/nsac.0
> 
> vs. ( note usr.bin vs. usr/bin ):
> 
> Modified: head/ObsoleteFiles.inc
> ==
> --- head/ObsoleteFiles.incSun Jul 22 12:04:21 2018(r336600)
> +++ head/ObsoleteFiles.incSun Jul 22 12:45:02 2018(r336601)
> @@ -38,6 +38,13 @@
> #   xargs -n1 | sort | uniq -d;
> # done
> 
> +# 20180722: indent(1) option renamed, test files follow
> +OLD_FILES+=usr/bin/indent/tests/nsac.0
> +OLD_FILES+=usr/bin/indent/tests/nsac.0.pro
> +OLD_FILES+=usr/bin/indent/tests/nsac.0.stdout
> +OLD_FILES+=usr/bin/indent/tests/sac.0
> +OLD_FILES+=usr/bin/indent/tests/sac.0.pro
> +OLD_FILES+=usr/bin/indent/tests/sac.0.stdout
> # 20180721: move of libmlx5.so.1 and libibverbs.so.1
> OLD_LIBS+=usr/lib/libmlx5.so.1
> OLD_LIBS+=usr/lib/libibverbs.so.1


As near as I can tell the fix would be something like
(leading whitespace seems to be disappearing):

# svnlite diff /usr/src/ObsoleteFiles.inc
Index: /usr/src/ObsoleteFiles.inc
===
--- /usr/src/ObsoleteFiles.inc  (revision 338675)
+++ /usr/src/ObsoleteFiles.inc  (working copy)
@@ -48,12 +48,12 @@
# 20180725: Cleanup old libcasper.so.0
OLD_LIBS+=lib/libcasper.so.0
# 20180722: indent(1) option renamed, test files follow
-OLD_FILES+=usr/bin/indent/tests/nsac.0
-OLD_FILES+=usr/bin/indent/tests/nsac.0.pro
-OLD_FILES+=usr/bin/indent/tests/nsac.0.stdout
-OLD_FILES+=usr/bin/indent/tests/sac.0
-OLD_FILES+=usr/bin/indent/tests/sac.0.pro
-OLD_FILES+=usr/bin/indent/tests/sac.0.stdout
+OLD_FILES+=usr/tests/usr.bin/indent/nsac.0
+OLD_FILES+=usr/tests/usr.bin/indent/nsac.0.pro
+OLD_FILES+=usr/tests/usr.bin/indent/nsac.0.stdout
+OLD_FILES+=usr/tests/usr.bin/indent/sac.0
+OLD_FILES+=usr/tests/usr.bin/indent/sac.0.pro
+OLD_FILES+=usr/tests/usr.bin/indent/sac.0.stdout
# 20180721: move of libmlx5.so.1 and libibverbs.so.1
OLD_LIBS+=usr/lib/libmlx5.so.1
OLD_LIBS+=usr/lib/libibverbs.so.1


This is sort of like the older part of the file:

# 20170322: rename  to _test to match the FreeBSD test suite name scheme
OLD_FILES+=usr/tests/usr.bin/col/col
OLD_FILES+=usr/tests/usr.bin/diff/diff
OLD_FILES+=usr/tests/usr.bin/ident/ident
OLD_FILES+=usr/tests/usr.bin/mkimg/mkimg
OLD_FILES+=usr/tests/usr.bin/sdiff/sdiff
OLD_FILES+=usr/tests/usr.bin/soelim/soelim
OLD_FILES+=usr/tests/usr.sbin/pw/pw_config
OLD_FILES+=usr/tests/usr.sbin/pw/pw_etcdir
OLD_FILES+=usr/tests/usr.sbin/pw/pw_groupadd
OLD_FILES+=usr/tests/usr.sbin/pw/pw_groupdel
OLD_FILES+=usr/tests/usr.sbin/pw/pw_groupmod
OLD_FILES+=usr/tests/usr.sbin/pw/pw_lock
OLD_FILES+=usr/tests/usr.sbin/pw/pw_useradd
OLD_FILES+=usr/tests/usr.sbin/pw/pw_userdel
OLD_FILES+=usr/tests/usr.sbin/pw/pw_usermod
OLD_FILES+=usr/tests/usr.sbin/pw/pw_usernext



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r336601 - head

[Mark Millard via svn-src-head]

Looks like this head/ObsoleteFiles.inc update has a typo
in each thing added to OLD_FILES . . .

# ls -lTdt /usr/tests/usr.bin/indent/*
-r--r--r--  1 root  wheel   121 Sep 13 22:53:30 2018 
/usr/tests/usr.bin/indent/Kyuafile
. . .
-r--r--r--  1 root  wheel   295 Sep 13 22:53:29 2018 
/usr/tests/usr.bin/indent/binary.0
-r--r--r--  1 root  wheel92 May  1 19:35:24 2018 
/usr/tests/usr.bin/indent/sac.0.pro
-r--r--r--  1 root  wheel   130 May  1 19:35:24 2018 
/usr/tests/usr.bin/indent/sac.0.stdout
-r--r--r--  1 root  wheel   122 May  1 19:35:24 2018 
/usr/tests/usr.bin/indent/sac.0
-r--r--r--  1 root  wheel94 May  1 19:35:24 2018 
/usr/tests/usr.bin/indent/nsac.0.pro
-r--r--r--  1 root  wheel   130 May  1 19:35:24 2018 
/usr/tests/usr.bin/indent/nsac.0.stdout
-r--r--r--  1 root  wheel   123 May  1 19:35:24 2018 
/usr/tests/usr.bin/indent/nsac.0

vs. ( note usr.bin vs. usr/bin ):

Modified: head/ObsoleteFiles.inc
==
--- head/ObsoleteFiles.inc  Sun Jul 22 12:04:21 2018(r336600)
+++ head/ObsoleteFiles.inc  Sun Jul 22 12:45:02 2018(r336601)
@@ -38,6 +38,13 @@
 #   xargs -n1 | sort | uniq -d;
 # done
 
+# 20180722: indent(1) option renamed, test files follow
+OLD_FILES+=usr/bin/indent/tests/nsac.0
+OLD_FILES+=usr/bin/indent/tests/nsac.0.pro
+OLD_FILES+=usr/bin/indent/tests/nsac.0.stdout
+OLD_FILES+=usr/bin/indent/tests/sac.0
+OLD_FILES+=usr/bin/indent/tests/sac.0.pro
+OLD_FILES+=usr/bin/indent/tests/sac.0.stdout
 # 20180721: move of libmlx5.so.1 and libibverbs.so.1
 OLD_LIBS+=usr/lib/libmlx5.so.1
 OLD_LIBS+=usr/lib/libibverbs.so.1


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r338478 - in head: contrib/elftoolchain/elfcopy contrib/elftoolchain/libelf sys/sys [breaks ci.freebsd.org builds for: FreeBSD-head-riscv64-build]

[Mark Millard via svn-src-head]

https://ci.freebsd.org/job/FreeBSD-head-riscv64-build/10428/consoleText
shows:


--- objcopy.full ---
cc -O2 -pipe -I/workspace/src/contrib/elftoolchain/libelftc 
-I/workspace/src/contrib/elftoolchain/libpe 
-I/workspace/src/contrib/elftoolchain/common -DWITH_PE=1 -I. -g -std=gnu99 
-Qunused-arguments 
-I/workspace/obj/workspace/src/riscv.riscv64/tmp/legacy/usr/include  -static  
-L/workspace/obj/workspace/src/riscv.riscv64/tmp/legacy/usr/lib -o objcopy.full 
archive.o ascii.o binary.o main.o pe.o sections.o segments.o symbols.o  
-L/workspace/obj/workspace/src/riscv.riscv64/tmp/obj-tools/lib/libarchive 
-larchive -L/workspace/obj/workspace/src/riscv.riscv64/tmp/obj-tools/lib/libz 
-lz -L/workspace/obj/workspace/src/riscv.riscv64/tmp/obj-tools/lib/libbz2 -lbz2 
-L/workspace/obj/workspace/src/riscv.riscv64/tmp/obj-tools/lib/liblzma -llzma 
-L/workspace/obj/workspace/src/riscv.riscv64/tmp/obj-tools/lib/libthr 
-L/workspace/obj/workspace/src/riscv.riscv64/tmp/obj-tools/lib/libexpat 
-lbsdxml 
-L/workspace/obj/workspace/src/riscv.riscv64/tmp/obj-tools/secure/lib/libcrypto 
-lcrypto -L/workspace/obj/wor
 kspace/src/riscv.riscv64/tmp/obj-tools/lib/libelftc -lelftc 
-L/workspace/obj/workspace/src/riscv.riscv64/tmp/obj-tools/lib/libelf -lelf 
-L/workspace/obj/workspace/src/riscv.riscv64/tmp/obj-tools/lib/libpe -lpe 
-lpthread -legacy
/workspace/obj/workspace/src/riscv.riscv64/tmp/legacy/usr/lib/libelf.a(gelf_rela.o):
 In function `gelf_getrela':
/workspace/src/contrib/elftoolchain/libelf/gelf_rela.c:95: undefined reference 
to `_libelf_is_mips64el'
/workspace/src/contrib/elftoolchain/libelf/gelf_rela.c:97: undefined reference 
to `_libelf_mips64el_r_info_tom'
/workspace/obj/workspace/src/riscv.riscv64/tmp/legacy/usr/lib/libelf.a(gelf_rela.o):
 In function `gelf_update_rela':
/workspace/src/contrib/elftoolchain/libelf/gelf_rela.c:167: undefined reference 
to `_libelf_is_mips64el'
/workspace/src/contrib/elftoolchain/libelf/gelf_rela.c:168: undefined reference 
to `_libelf_mips64el_r_info_tof'
/workspace/obj/workspace/src/riscv.riscv64/tmp/legacy/usr/lib/libelf.a(gelf_rel.o):
 In function `gelf_getrel':
/workspace/src/contrib/elftoolchain/libelf/gelf_rel.c:94: undefined reference 
to `_libelf_is_mips64el'
/workspace/src/contrib/elftoolchain/libelf/gelf_rel.c:95: undefined reference 
to `_libelf_mips64el_r_info_tom'
/workspace/obj/workspace/src/riscv.riscv64/tmp/legacy/usr/lib/libelf.a(gelf_rel.o):
 In function `gelf_update_rel':
/workspace/src/contrib/elftoolchain/libelf/gelf_rel.c:163: undefined reference 
to `_libelf_is_mips64el'
/workspace/src/contrib/elftoolchain/libelf/gelf_rel.c:164: undefined reference 
to `_libelf_mips64el_r_info_tof'
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** [objcopy.full] Error code 1

make[3]: stopped in /workspace/src/usr.bin/objcopy
1 error

make[3]: stopped in /workspace/src/usr.bin/objcopy
*** [cross-tools] Error code 2



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r338400 - head/release/tools

[Mark Millard via svn-src-head]

I'm not sure if the following is intended or not.

I tried using

beastie_disable="YES"
loader_color="NO"

in boot/loader.conf on head -r338341 .

The default colors are still changed during:
(some characters are filtered out)

QUOTE
Setting currdev to disk2p2:


/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|Loading
 /boot/defaults/loader.conf
END QUOTE

(Note the  in the long line: White FG code and Black BG code.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r338404 - head/sys/conf

[Mark Millard via svn-src-head]

On 2018-Aug-31, at 9:40 AM, Glen Barber  wrote:

> On Fri, Aug 31, 2018 at 09:34:00AM -0700, Mark Millard wrote:
>> On Fri, Aug 31, 2018 at 02:20:09PM +, Glen Barber wrote:
 On Fri, Aug 31, 2018 at 08:51:29AM -0400, Ed Maste wrote:
> On 30 August 2018 at 22:46, Glen Barber  wrote:
>> 
>> As I look closer at the log, I have a sneaking suspicion this may have
>> been a transient.  I'm redoing the i386 build against the original
>> r338401 for i386 alone as a test, if nothing else, to see if it fails in
>> the same place.
> 
> What's the failure?
 
>>> stage 3.1: building everything
 --
 In file included from /usr/src/sys/kern/genoffset.c:35:
 In file included from /usr/src/sys/sys/param.h:96:
 /usr/src/sys/sys/types.h:46:10: fatal error: 'machine/endian.h' file not 
 found
 #include 
  ^~
 1 error generated.
 --- genoffset.o ---
 *** [genoffset.o] Error code 1
 
 So far, 100% reproducible with an empty /usr/obj/.
 
> I see head/i386 is green in CI.
> 
 
 Does CI prune .OBJDIR for each build?
 
>>> 
>>> Make that 75% reproducible.  I suspect there is a build race occurring.
>> 
>> ci.freebsd.org occasionally shows examples of this race. For example:
>> 
>> https://ci.freebsd.org/job/FreeBSD-head-powerpc-build/7454/consoleText
>> 
>> shows:
>> 
>> --- genoffset.o ---
>> /usr/src/sys/sys/types.h:46:28: error: machine/endian.h: No such file or 
>> directory
>> *** [genoffset.o] Error code 1
>> 
>> for -r338319 but the prior build (for -r338318) and
>> following build (for -r338320) worked fine.
>> 
>> -r338319 was for something that should make no difference
>> to the issue:
>> 
>> QUOTE
>> Remove trailing slash in pathname so that valid METALOG is created in the
>> NO_ROOT case of make packages.
>> END QUOTE
>> 
>> I've seen other occasional examples.
>> 
> 
> Thanks for the link to this specific failure.  I see nothing in that
> range of commits that should have any impact here.  What caused me to
> back out the ALPHA rename is this is the first time I've seen this since
> the code freeze started, and following the first failure, restarted the
> builds with the same result.
> 
> I'll go poking around at the CI console output to try to find when the
> first occurrence happened.

Other than I'd seen the one on ci.free.org recently and thought
"oh, that again" and "although its been a long time", I think
the failures have been rare enough that without an archive of files
to search automatically, it would not be practical to find examples
via just looking around via the web interface for ci.freebsd.org .

I've seen rare failures in personal builds. But most of my
personal builds are incremental where a old file would be
around. So I expect more rare for this context. But I do
not remember the ci.freebsd.org vs. personal status for
the one I saw prior to the one that I reported --or when.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r338404 - head/sys/conf

[Mark Millard via svn-src-head]

On Fri, Aug 31, 2018 at 02:20:09PM +, Glen Barber wrote:
> > On Fri, Aug 31, 2018 at 08:51:29AM -0400, Ed Maste wrote:
> > > On 30 August 2018 at 22:46, Glen Barber  wrote:
> > > >
> > > > As I look closer at the log, I have a sneaking suspicion this may have
> > > > been a transient.  I'm redoing the i386 build against the original
> > > > r338401 for i386 alone as a test, if nothing else, to see if it fails in
> > > > the same place.
> > > 
> > > What's the failure?
> > 
> >  >>> stage 3.1: building everything
> >  --
> >  In file included from /usr/src/sys/kern/genoffset.c:35:
> >  In file included from /usr/src/sys/sys/param.h:96:
> >  /usr/src/sys/sys/types.h:46:10: fatal error: 'machine/endian.h' file not 
> > found
> >  #include 
> >   ^~
> >  1 error generated.
> >  --- genoffset.o ---
> >  *** [genoffset.o] Error code 1
> > 
> > So far, 100% reproducible with an empty /usr/obj/.
> > 
> > > I see head/i386 is green in CI.
> > > 
> > 
> > Does CI prune .OBJDIR for each build?
> > 
> 
> Make that 75% reproducible.  I suspect there is a build race occurring.

ci.freebsd.org occasionally shows examples of this race. For example:

https://ci.freebsd.org/job/FreeBSD-head-powerpc-build/7454/consoleText

shows:

--- genoffset.o ---
/usr/src/sys/sys/types.h:46:28: error: machine/endian.h: No such file or 
directory
*** [genoffset.o] Error code 1

for -r338319 but the prior build (for -r338318) and
following build (for -r338320) worked fine.

-r338319 was for something that should make no difference
to the issue:

QUOTE
Remove trailing slash in pathname so that valid METALOG is created in the
NO_ROOT case of make packages.
END QUOTE

I've seen other occasional examples.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"