Re: gitup issues [was Re: No update for a day on ports?]

[Ed Maste]

On Sat, 10 Apr 2021 at 21:34, Peter Jeremy via freebsd-ports
 wrote:
>
> gitup walks the destination tree (ie /usr/ports), hashing every file
> it finds (by reading the complete file into memory).  Whilst the
> default gitup configuration ignores /usr/ports/packages and
> /usr/ports/distfiles, it reads and hashes all the files down both
> paths before ignoring them.

This seems like something that should be fixed relatively easily. In
any case there's an enhancement request in gitup's GitHub project for
reducing gitup's memory use in general. I hope something comes of it.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Portsnap restoration after Git migration

[Ed Maste]

Colin (cperciva) and I are making good progress on the portsnap build
infrastructure Git migration. I'll follow up when it is back in
operation.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Proposed ports git transition schedule

[Ed Maste]

The doc and src tree git migrations are complete, and the git working
group is preparing the ports tree transition in advance of the 2021Q2
quarterly branch.

The proposed schedule is available at https://wiki.freebsd.org/git and
the status will be updated throughout the process, and important
milestones will be sent here as well.

The committer's guide has been updated with Git documentation, at
https://docs.freebsd.org/en/articles/committers-guide/#git-primer.
Some additional draft / in-progress notes are available in Warner's
repo at https://github.com/bsdimp/freebsd-git-docs/, and additional
documentation will be updated over the next few days.

When the conversion is complete, Subversion will become read-only with
no further updates to the ports tree. If you are fetching ports using
svn, you will need to switch to the new git repository at the
following URLS:

Web repository browser:
https://cgit.freebsd.org/ports/

Distributed mirrors for anonymous readonly checkout/clone
https://git.freebsd.org/ports.git
ssh://anon...@git.freebsd.org/ports.git

Read/write Repository for committers
ssh://g...@gitrepo.freebsd.org/ports.git

We will be monitoring #gitcvt on EFnet (irc.efnet.org) and #git on the
unofficial FreeBSD Discord server (https://discord.gg/WKEKSPtJGp)
during and after the cutover.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: the state of amd64 ports on -CURRENT as of 20200827

[Ed Maste]

On Thu, 27 Aug 2020 at 10:03, David Wolfskill  wrote:
>
> It's https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248859, in case you
> wanted to see for yourself.
>
> As I'm not a committer, it's not clear to me what (else) I can do to
> help fix this.

Hopefully your email will help raise the visibility of this issue and
a ports committer will pick it up.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

HEADS-UP: obsolete GNU as 2.17.50 retirement for FreeBSD 13, expected 2020-05-31

[Ed Maste]

All architectures supported by FreeBSD now using Clang and lld, and
tools from obsolete GNU binutils 2.17.50 have been retired one by one
- most recently objdump, in r360698. There is one binutil tool left:
GNU as. I plan to disable it at the end of the month, and then remove
all of binutils some weeks later if no additional issues are found.

We have two src.conf build knobs for as - WITH_BINUTILS, which
determines whether /usr/bin/as exists in the built system, and
WITH_BINUTILS_BOOTSTRAP, which controls whether as is built as a
bootstrap tool to build the rest of the system.

1. WITH_BINUTILS (default on i386 and amd64)

Turning this off means the installed system will not have /usr/bin/as.
A ports exp-run without as is in PR 205250. On i386 the most recent
run failed in:

comms/libfec
comms/syncterm
devel/plan9port
emulators/vmw
games/dxx-rebirth
graphics/vulkan-loader
lang/hla
lang/mit-scheme
lang/ocaml
lang/ocaml-nox11
math/ldouble
multimedia/mencoder
multimedia/mplayer

amd64 has similar failures in an earlier run.

There are generally two ways to address a ports need for an assembler:

a) Add a dependency on devel/binutils to the port's Makefile:
BUILD_DEPENDS+=as:devel/binutils

b) Use Clang's integrated assembler, via the compiler driver (cc).
This is what we do for all but one file in the base system.

Option (b) is the nicer choice, as it doesn't introduce a new
dependency, but takes some more work. Option (a) is straightforward,
and I have a proposed initial set of patches at
https://reviews.freebsd.org/D24739.

My hope is that individual maintainers of affected ports can prepare
their port for the upcoming as retirement in whichever way they find
most suitable; I'll have BUILD_DEPENDS patches ready for whichever
ones are not ready at the end of the month.

2. WITH_BINUTILS_BOOTSTRAP

GNU as is built only on amd64, and is used to assemble one file, the
optimized assembly version of the Skein cryptographic hash -
skein_block_asm.s. Unfortunately it makes extensive use of macro
features that are not yet supported by Clang's integrated assembler.
There are some proposed methods of addressing (for example, committing
an assembled .o, or translating the assembly into something that IAS
can handle) but none of them are particularly palatable. Right now I
expect the most likely path forward is that we revert to the somewhat
slower C-language skein implementation.

Please let me know if you have any questions or concerns; I'm happy to
help maintainers of affected ports find and test a solution.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

HEADS-UP: /usr/bin/objdump to be removed

[Ed Maste]

I plan to remove the obsolete GNU binutils 2.17.50 objdump from the
base system in the next few days.

If you currently use objdump from the base system you can give
llvm-objdump a try instead - it is mostly compatible, but has a few
missing options and the output format may be slightly different.
Otherwise, install binutils from ports or packages.

There are a few ports that need adjustment as well, identified in
exp-run PR 212319:

- PR 246229 databases/galera
Fixed by adding a ${LOCALBASE}/bin/objdump:devel/binutils BUILD_DEPENDS.

- PR 241159 devel/powerpc64-gcc (and other *-gcc ports)
The build finishes, but "make package" fails because certain files are missing.

- PR 241157: www/node (and node*)

- No PR yet, graphics/embree

Once this is committed the only remaining obsolete binutils tool will
be /usr/bin/as, on amd64 and i386 only. I expect it will also be
removed for FreeBSD 13.0.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Retiring GNU objdump 2.17.50

[Ed Maste]

On Sun, 12 Jan 2020 at 20:40, Slawa Olhovchenkov  wrote:
>
>  As long as pmcannotate relies on objdump(1) and pmcstat(8) to
>  work, it
>   will fail if one of them is not available.

Ah, we could have pmcannotate check for both llvm-objdump and objdump in $PATH.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Retiring GNU objdump 2.17.50

[Ed Maste]

We currently install and use at most three tools from GNU binutils
2.17.50, depending on target architecture:

1. as - assembler
2. ld - linker
3. objdump - diagnostic / information tool

I hope to retire all use of these obsolete binutils before FreeBSD 13.
Here I'd like to discuss objdump. It is a diagnostic tool that
provides information about object files, binaries and libraries. It's
not required as a bootstrap tool (i.e., not needed to build FreeBSD
world or kernel). It is required to build a limited number of ports,
and is used by some developers.

I have a tracking PR for GNU objdump's retirement open in PR 229046.
https://bugs.freebsd.org/229046.

There are two ways we can proceed with its retirement:

1. Remove it without replacement. Ports that need objdump to build
will have to depend on the binutils package/port, and users who wish
to use it will have to install it.

Related links for this path:
Ports exp-run: https://bugs.freebsd.org/212319
Patch review: https://reviews.freebsd.org/D7338

2. Install llvm-objdump in its place (perhaps via a symlink).
llvm-objdump is broadly compatible in both command-line argument
parsing and output format, but there are many small differences and
it's not a full drop-in replacement.

Related links for this path:
Patch review: https://reviews.freebsd.org/D18307

I am interested in feedback on the preferred approach. Installing
llvm's objdump has the advantage that for most use cases everything
will "just work", but may also introduce subtle failures.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: FreeCAD 0.17 && /lib//libgcc_s.so.1

[Ed Maste]

On Thu, 21 Feb 2019 at 16:47, Steve Kargl
 wrote:
>
> The missing symbols are
>
> % objdump -x lib/libgfortran.so | grep GCC_4.6.0 | awk '{print $5}' | sort

Thank you for collecting these.

> It looks like we may be able to grab some of these from libc/softfloat:
> getf2.c, gttf2.c, letf2.c, lttf2.c, netf2.c.
>
> It looks like we might be able to grab a few more from NetBSD:
> eqtf2.c and unordtf2.c

All seven of these are available in compiler-rt, I believe they just
need to be built and added to the version map.

That leaves: __addtf3 __divtf3 __floatditf __floatsitf __floatunditf
__multf3 __subtf3

compiler-rt also has these, but provided only in this case:
#if defined(CRT_HAS_128BIT) && defined(CRT_LDBL_128BIT)
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

HEADS-UP: switching i386 to lld as system linker

[Ed Maste]

As you may know I've been working on migrating to LLVM's lld as our
system linker, to replace the outdated GNU binutils ld 2.17.50. lld is
currently the system linker (i.e., is /usr/bin/ld) on amd64, arm64,
and armv7. i386 was not switched over at the same time as the other
architectures because many ports failed to build with lld, primarily
due to linking non-PIC code into shared objects.

Thanks to Antoine's sustained effort we've run through many
exp-runs[1], iterating on addressing ports that fail to build with
lld. The most recent i386 exp-run[2] had a little over 40 new failures
which resulted in about 70 skipped ports; a few of the failing ports
responsible for the most skipped have since been addressed. PRs are
open for the remaining failures and can be found via the dependency
tree[3] for the exp-run PR 214864.

I intend to enable lld as the system linker for i386 in -CURRENT in
the next week or so, and will continue helping address new failures
before and after the default is switched over.

The easiest way to configure an existing -CURRENT i386 system for
testing with lld as the system linker is to run as root:

# ln -fs ld.lld /usr/bin/ld

conversely, to switch (back) to GNU binutils ld:

# ln -fs ld.bfd /usr/bin/ld

There are a few ways to address failures with lld:

1. Add LLD_UNSAFE to the Makefile, which will cause the port to
(attempt to) depend on the binutils port/package and link with GNU ld.
This may not work if the port's build infrastructure does not honour
LD or LDFLAGS environment variables.

2. Add LDFLAGS_i386=-Wl,-z,notext to the Makefile. This will override
lld's default of disallowing .text relocations (and is a no-op with
GNU ld, just reaffirming GNU ld's default). As with #1 above, this may
not work if the port does not properly honour LDFLAGS.

3. Ensure that -fPIC is included in CFLAGS when compiling objects to
be linked into shared objects.

4. Rewrite assembly files to be linked into shared objects to be PIC.

Please let me know if you have any questions or want assistance with a
specific port.

[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=214864
[2] 
http://package18.nyi.freebsd.org/build.html?mastername=headi386PR214864-default=2019-01-05_21h00m02s
[3] 
https://bugs.freebsd.org/bugzilla/showdependencytree.cgi?id=214864_resolved=1
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: error: undefined symbol: main in poudriere jail

[Ed Maste]

On 25 September 2018 at 09:29, blubee blubeeme  wrote:
> On Tue, Sep 25, 2018 at 5:27 PM Lorenzo Salvadore via freebsd-ports <
> freebsd-ports@freebsd.org> wrote:
>
>> > > That's the problem, the same code works for earlier version of FreeBSD.
>> >
>> > You can try switching back to the old GNU ld via something like "ln
>> > -fs ld.bfd /usr/bin/ld" and building the port on 12. (Or, set
>> > WITHOUT_LLD_IS_LD in src.conf.) If that works I'll try to suggest some
>> > further steps.
>
> going into the jail on this failed poudiere build making the symlink and
> building; the build succeeds.

Ok - it is very strange that using BFD ld vs lld causes main() to
appear or not in one of the objects being linked; I suspect this is a
2nd-order effect of some build-time test that's failing for some
reason with lld.

Briefly, at a high-level there are two ways to approach a case like
this: get the port linking with GNU ld, or find and fix the root
cause. Ports has a knob LLD_UNSAFE that should cause base ld.bfd or
one from the binutils package to be used, but it does not work in some
cases (generally, ports with old embedded libtool or that otherwise do
not honour LDFLAGS or CFLAGS).

To take this to root cause there's probably a straightforward (if
long/tedious) path to be taken in comparing the results of building in
two jails, one with ld is ld.bfd and one with ld is ld.lld. The link
is failing with ld.lld, so compare the objects passed to the link
invocations between the two cases and find one that contains main() in
the bfd case and not in the lld case, and work backwards from there.

You can set LLD_REPRODUCE=_something_.tar in the environment and lld
will create a tarball of all inputs, command line, etc. for further
investigation.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: error: undefined symbol: main in poudriere jail

[Ed Maste]

> That's the problem, the same code works for earlier version of FreeBSD.

You can try switching back to the old GNU ld via something like "ln
-fs ld.bfd /usr/bin/ld" and building the port on 12. (Or, set
WITHOUT_LLD_IS_LD in src.conf.) If that works I'll try to suggest some
further steps.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Heads-up: linker (lld) changes for amd64 coming soon

[Ed Maste]

On 26 March 2018 at 22:14, Ed Maste <ema...@freebsd.org> wrote:
> Some changes related to the amd64 linker are nearly ready to be
> committed (within a week or three), so I'm sending this notice to
> request any final comments or concerns before these changes are made.

It took somewhat longer than a week or three, but these changes will
now happen quite soon.

> 1. Kostik (kib@) has a patch to start using kernel ifunc, with the
> first use being Supervisor Mode Access Prevention (SMAP) on amd64.
> This relies on linker support that is available in the in-tree lld and
> in contemporary binutils ld.bfd from ports, but not in the in-tree
> ld.bfd 2.17.50.

This is ready to be committed at any time.

> 2. WITH_LLD_IS_LD controls whether /usr/bin/ld is ld.bfd or ld.lld,
> and thus the linker used for linking ports. I plan to switch this to
> default on.

There was one significant remaining issue in the ports tree with lld
as /usr/bin/ld: lang/ghc. This was due (at least in part) to a bug in
lld's note handling. The bug is now fixed upstream and in FreeBSD in
r333401.

The latest version of ghc claims to have improved support for using
lld as the linker, and a lang/ghc update is currently in progress
(PR227968). Once this is committed I will request one more exp-run
with lld. As long as those results are acceptable, I'll then make the
switch to install lld as /usr/bin/ld on amd64.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Heads-up: linker (lld) changes for amd64 coming soon

[Ed Maste]

On 29 March 2018 at 13:14, Ed Maste <ema...@freebsd.org> wrote:
>
> There are now 14 PRs open for failures when lld is /usr/bin/ld.

Now down to 5:

PR 221800 - www/mod_jk
Appears to be an issue with libtool passing through linker flags. 0
dependent ports skipped.

PR 224673 - misc/seabios
Port maintainer has patch ready to be committed. 2 dependent ports skipped

PR 226872 - lang/ghc
Haskell compiler - the compiler builds fine by using ld.bfd via $LD,
but then the built compiler invokes ld. Report in the PR claims the
current version (8.4.1) includes some lld support so might "just work"
after the port's updated? 558 dependent ports skipped.

PR 226975 - x11/eaglemode
A common case due to disagreement about preemption of symbols with
protected visibility in a shared library. 0 dependent ports skipped.

PR 226980 - several ports using openal
Ports using openal failed due to the protected visibility symbol
preemption issue. LLD_UNSAFE has been added to many of them, with 14
more to go. As a group, 1 dependent port skipped.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Heads-up: linker (lld) changes for amd64 coming soon

[Ed Maste]

On 27 March 2018 at 18:21, Ed Maste <ema...@freebsd.org> wrote:
> (Moved from -current to -ports)
>
> On 27 March 2018 at 13:15, Ed Maste <ema...@freebsd.org> wrote:
>>
>> Fair enough - this was the reason I sent the email. I've now gone
>> through and submitted a PR for for each failure that did not already
>> have one. I've also added LLD_UNSAFE to a few ports where where it was
>> straightforward.
>
> Via tobik's commit to lang/myrddin (r465725) I discovered
> BINARY_ALIAS=ld=ld.bfd, which is a usable workaround for some ports
> which don't honour $LD or -fuse-ld=bfd in CFLAGS.

As of ports r465900 BINARY_ALIAS is now set automatically if LLD_UNSAFE is set.

There are now 14 PRs open for failures when lld is /usr/bin/ld. Thanks
to recent commits from krion@ all unmaintained ports that had issues
have been addressed, except for print/openprinting (PR221809) and some
openal-related failures (PR226980). I'll try to take a look at the
openal issues; it's likely that the interim solution will be just to
set LLD_UNSAFE in all of the dependent ports.

The remaining 12 PRs are all for maintained ports. I believe that now
the only port responsible for a significant number of skipped
dependent ports is lang/ghc (PR226872).
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Heads-up: linker (lld) changes for amd64 coming soon

[Ed Maste]

(Moved from -current to -ports)

On 27 March 2018 at 13:15, Ed Maste <ema...@freebsd.org> wrote:
>
> Fair enough - this was the reason I sent the email. I've now gone
> through and submitted a PR for for each failure that did not already
> have one. I've also added LLD_UNSAFE to a few ports where where it was
> straightforward.

Via tobik's commit to lang/myrddin (r465725) I discovered
BINARY_ALIAS=ld=ld.bfd, which is a usable workaround for some ports
which don't honour $LD or -fuse-ld=bfd in CFLAGS.

As you point out in reply to my r465755 BINARY_ALIAS alone is not
sufficient, because arm64 does not provide ld.bfd by default and
LLD_UNSAFE automatically brings in ports binutils if /usr/bin/ld.bfd
does not exist. So we need both LLD_UNSAFE and BINARY_ALIAS.

Should we just have LLD_UNSAFE also set BINARY_ALIAS=ld=ld.bfd?
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Ports and LLVM's lld linker

[Ed Maste]

On 18 December 2017 at 22:10, Ed Maste <ema...@freebsd.org> wrote:
>
> With a couple of recent changes in src head (r326831 and r326897) lld
> is now suitable for use as the base system /usr/bin/ld on amd64 and
> i386. We're working through ports failures, starting with those
> responsible for the largest number of skipped ports.
>
> The top four, on amd64:
>
> port# skipped
> devel/libunwind 7994
> databases/postgresql*-client230

These two are now addressed by setting LLD_UNSAFE=yes so that they'll
still link with ld.bfd once ld.lld is installed as /usr/bin/ld.

The issue with libunwind was already reported upstream (from someone
trying to use ld.gold, which fails in the same way), and libunwind's
developers will address it in a new version.

The postgresql*-client issues need more investigation, but now will
not prevent the migration to lld.

> lang/fpc76

This one's a little tricky, because it's a (Pascal) tool chain that
doesn't support usual environment variables like LD to specify a
linker explicitly.

> lang/mono   22

I'll mark this one LLD_UNSAFE after a libtool dependency is sorted out
(PR224514).

After setting LLD_UNSAFE=yes for libunwind some previously-skipped
ports attempt to build, and some of those now fail with lld. I'm
working through that list and may have a dozen or so more that will
become LLD_UNSAFE.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Ports and LLVM's lld linker

[Ed Maste]

On 27 November 2017 at 15:39, Ed Maste <ema...@freebsd.org> wrote:
> We're making good progress on using LLVM's lld linker as FreeBSD's
> /usr/bin/ld, so I'd like to raise awareness of the new linker within
> the ports community.

With a couple of recent changes in src head (r326831 and r326897) lld
is now suitable for use as the base system /usr/bin/ld on amd64 and
i386. We're working through ports failures, starting with those
responsible for the largest number of skipped ports.

The top four, on amd64:

port# skipped
devel/libunwind 7994
databases/postgresql*-client230
lang/fpc76
lang/mono   22

The remaining failures are responsible for no more than 2 skipped ports each.

devel/libunwind fails due to the shared object protected visibility
symbol preemption issue; Dimitry Andric and I are searching for an
appropriate fix.

The databases/postgresql*-client failures have been worked around by
r456635, adding LLD_UNSAFE=yes so that the port uses ld.bfd.

lang/fpc appears to suffer from stricter validation performed by lld:
/usr/bin/ld: error: x86_64/units/x86_64-freebsd/i_linux.o: invalid
alignment of section headers

lang/mono fails because lld defaults to -z text, disallowing
relocations in read-only segments (like .text). A workaround is to add
-z notext to the link command line, which turns off lld's error for
this case and results in the same behaviour as ld.bfd and ld.gold
provide by default.

Unfortunately usual workarounds (LLD_UNSAFE=yes or
LDFLAGS=Wl,-z,notext) fail for both lang/fpc and lang/mono, and it's
not immediately obvious to me how their respective builds handle the
options. I'll probably need help from acm@ and mono@ for these.

For reference the remaining ports failing with lld on amd64 are:
archivers/lua51-zlib
audio/alure
benchmarks/wrk
databases/postgres-xl
devel/libds
devel/libtecla
devel/pdcurses
devel/ztcl
emulators/gem5
ftp/rexx-curl
irc/eggdrop
irc/eggdrop-devel
irc/evangeline
lang/rexx-imc
lang/rexx-regutil
lang/siod
lang/smlnj
lang/tclX
mail/qmail-dk
math/rexx-regmath
misc/seabios
net-im/uTox
net/py-netif
net/py-netif
print/openprinting
print/pdftk
security/otpw
shells/bash-static
sysutils/dupd
sysutils/e2fsprogs
sysutils/installwatch
sysutils/unieject
www/cgihtml
www/dummyflash
www/mod_jk
www/mozplugger
www/tdom
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Ports and LLVM's lld linker

[Ed Maste]

We're making good progress on using LLVM's lld linker as FreeBSD's
/usr/bin/ld, so I'd like to raise awareness of the new linker within
the ports community.

Why do we need a new linker? Compared to GNU ld 2.17.50 currently in
the FreeBSD base system, lld brings:

* AArch64 (arm64) support
* Link Time Optimization (LTO)
* New ABI support
* Other linker optimizations
* Much faster link times
* Maintained code base

There are a pair of src.conf build knobs, WITH_LLD_BOOTSTRAP and
WITH_LLD_IS_LD, which control whether lld is used to link during
buildworld, and whether lld is installed as /usr/bin/ld. You can also
switch the linker by making /usr/bin/ld a symlink to either ld.bfd or
ld.lld.

These build knobs are enabled by default on arm64 as of FreeBSD 11.1.
I have been using lld as /usr/bin/ld on my amd64 laptop and desktop
for over a year. (Note that there are a few outstanding issues on
i386, and lld is not yet fully as the system linker there. I expect
that both i386 and 32-bit arm will be fully working in time for
FreeBSD 12.0.)

For the ports tree we've been iterating on exp-runs using lld, in PR
214864. A number of issues have been identified during this process,
and we've added a LLD_UNSAFE option that can be set in a port's
Makefile to fall back to GNU ld (either /usr/bin/ld.bfd or by
depending on the binutils port if /usr/bin/ld.bfd does not exist).

Note that there's currently a bug in pkg (PR223776) which causes
issues registering shared libraries linked by lld; once it's resolved
I'd like to encourage port maintainers who are willing to experiment
with FreeBSD-current test environment to enable WITH_LLD_IS_LD and
ensure that their ports work as expected. I'm happy to help work
through any issues that are found.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Linking ports with LLD: no default library search paths

[Ed Maste]

As mentioned recently on this list I've been investigating building
the ports tree with LLVM's lld installed as /usr/bin/ld (via the
exp-run in PR 214864).

One of the common failure causes is that LLD does not include default
library search paths (e.g., /usr/lib). Paths must be explicitly
specified via -L on the command line. The vast majority of software
uses the compiler driver (cc) for linking which passes the correct
paths to the linker, but a small number of ports invoke the linker
directly and need some change to work with LLD.

This issue affects (at least) these 18 FreeBSD ports:

arabic/libitl
archivers/lua51-zlib
devel/jsl
devel/libds
devel/libtecla
devel/pdcurses
devel/ztcl
ftp/rexx-curl
irc/eggdrop
irc/eggdrop-devel
irc/evangeline
java/java3d
lang/rexx-imc
lang/rexx-regutil
lang/siod
lang/tclX
print/gl2ps
security/otpw
sysutils/installwatch

The best fix for each of these is probably to have a change upstream
to use the compiler driver for linking instead of a direct ld
invocation. Otherwise, we can just add -L/usr/lib to the linker flags,
or add LLD_UNSAFE=yes to the port Makefile to forbid the port from
using lld. Note that -L/usr/lib can in general be added
unconditionally; it causes no harm to pass it to GNU BFD ld.
print/gl2ps was already changed this way (PR218399, r448652).

I'm working through the list of ports that fail to link with LLD,
starting with those responsible for the largest number of dependent
skipped ports. If maintainers of these ports are interested I'm happy
to help look at changes for any of these ports or help with testing
with LLD.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

linking ports with lld: some ports pass $LDFLAGS to $LD instead of the compiler driver

[Ed Maste]

I've been investigating building the ports tree with LLVM's lld
installed as /usr/bin/ld (via the exp-run in PR 214864). Through this
I've found a few common issues, some of which are due to limitations
of lld or choices made by its authors, and some of which are problems
in the ports themselves. One of the latter is described here.

Normally $LDFLAGS contains flags passed to the compiler driver when
invoked for linking. One issue that affects a small number of ports is
passing $LDFLAGS to direct linker invocation (i.e, $LD $LDFLAGS ...),
when $LDFLAGS contains flags that are appropriate only for the
compiler driver, not the linker.

In several instances the flag comes from bsd.ssp.mk, which adds
-fstack-protector to LDFLAGS. GNU BFD ld accepts the flag but produces
undesired output (a bogus DT_AUXILIARY "stack-protector" entry):

% readelf -d /usr/local/lib/libcfg+.so.0

Dynamic section at offset 0x91a8 contains 15 entries:
  TagType  Name/Value
 0x7ffd AUXILIARYAuxiliary library: [stack-protector]
...

while lld produces an error:

ld -fstack-protector -shared -o libcfg+.so.0 ../src/cfg+.o
../src/parse.o ../src/props.o ../src/cmdline.o ../src/cfgfile.o
../src/shared.o ../src/platon/str/strplus.o
../src/platon/str/strctype.o ../src/platon/str/strdyn.o
../src/platon/str/dynfgets.o
ld: error: unknown argument: -fstack-protector

This affects at least the following ports:
arabic/libitl
devel/libcfg
ftp/rexx-curl
www/dummyflash
www/mod_jk
www/tdom

I've submitted PRs for these ports and am happy to work with the
maintainers on a fix (both for our ports tree and to be submitted
upstream).
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: The future of portmaster [and of ports-mgmt/synth]

[Ed Maste]

On 30 May 2017 at 15:00, Mark Millard  wrote:
>
> ports-mgmt/synth depends on lang/gcc6-aux

For reference, I've created PR 219667 to track the lang/gcc6-aux
issue. The pre-built bootstrap compilers need to be recreated I
believe, with a trick similar to the one used for lang/ghc.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

ino64 status update and upgrade caution

[Ed Maste]

The ino64 project was committed to src last week[1][2] with
UPDATING notes added shortly thereafter [3][4][5].

As some people have been tripped up by the ino64 change I want to
emphasize again that the upgrade procedure described in UPDATING should
be followed exactly. The reboot after installing the new ino64 kernel,
and the addition of COMPAT_FREEBSD11 to custom kernel configurations,
are crucial steps.

Note that this upgrade is just the usual standard, documented procedure
and is required whenever new syscalls or other forwards-incompatible
changes are introduced. Prior to ino64 we have had a period without
any significant Application Binary Interface (ABI) or Kernel Binary
Interface (KBI) changes and it was possible to take shortcuts with the
upgrade procedure. The ino64 change is backwards-compatible but not
forwards-compatible and shortcuts must not be taken. (As a general
rule, it is safest to always use the full procedure unless you follow
the commit mailing list and are aware of which changes are not
foward-compatible.)

In addition, some users have created a libarchive.so.6 symlink to
libarchive.so.7, in an attempt to run older binaries on a fresh post-
ino64 install. Do not do this: libarchive.so.6 and libarchive.so.7
have an incompatible ABI and this will result in incorrect and
unpredictable behaviour.

The most recent -CURRENT package set was built prior to the ino64
commit, and thus will not work on a fresh ino64 install or snapshot
image. Until the next package set is available please build from ports,
or use Poudriere to build your own package set.

Most ports with a dependency on ino64 have now been patched. The
remaining ports which still need a change for ino64 are:

PortResponsible for # Skipped
--  
lang/gcc6-aux 54
sysutils/py-psutil13  (fix in review D10801)
devel/llvm38   6
sysutils/py-psutil121  3
devel/radare2  0
emulators/qemu-user-static 0
lang/ccl   0
lang/dmd2  0
lang/gcc5-aux  0
lang/modula3   0
lang/rust-nightly  0
lang/twelf 0
lang/urweb 0
net-mgmt/netdata   0

[1] https://svnweb.freebsd.org/changeset/base/318736
[2] https://svnweb.freebsd.org/changeset/base/318737
[3] https://svnweb.freebsd.org/changeset/base/318757
[4] https://svnweb.freebsd.org/changeset/base/318758
[5] https://svnweb.freebsd.org/changeset/base/318792
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: FYI: amd64 built with WITH_LLD_IS_LD= vs. devel/libunwind : cannot preempt symbol (for various symbols)

[Ed Maste]

On 16 April 2017 at 04:10, Mark Millard  wrote:
> Context: amd64 FreeBSD -r316952 as a VirtualBox guest
> that was built using WITH_LLD_IS_LD= . ports -r438577.
>
> x11/xorg-minimal indirectly gets to devel/libunwind and
> devel/libunwind fails to build from source:
>
>
> --- Lperf-simple ---
> libtool: link: cc -O2 -pipe -g -fstack-protector -fno-strict-aliasing 
> -fexceptions -Wall -Wsign-compare -fstack-protector -o .libs/Lperf-simple 
> Lperf-simple.o  ../src/.libs/libunwind.so -lgcc -llzma -Wl,-rpath 
> -Wl,/usr/local/lib
> /usr/bin/ld: error: ./Gperf-simple.c:195: cannot preempt symbol 
> '_ULx86_64_init_local' defined in ../src/.libs/libunwind.so

The LLD ports exp-run identified the "cannot preempt symbol" issue as
being responsible for the largest number of failed or skipped ports.
You can find a description of the issue in LLVM PR 30960
(https://bugs.llvm.org//show_bug.cgi?id=30960). This is a tricky
issue, and one for which there's not a clear right answer, but is
arguably a problem that needs to be addressed in the individual pieces
of software (libunwind, openal-soft, etc.)

As a temporary workaround you can add CFLAGS+= -fPIC to the port's
Makefile, as in
https://github.com/emaste/freebsd-ports/commit/4857444b31ca546e29e221dce2a41092765e6062
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: os of system fetching port source

[Ed Maste]

On 3 April 2017 at 17:04, qjail1 <qja...@a1poweruser.com> wrote:
> Ed Maste wrote:
>
> Is "Libfetch" whats used in the current port system to download the port
> source?

Libfetch is the library used by fetch(1) and pkg(8). The ports
infrastructure will invoke the fetch binary. It looks like
Mk/Scripts/do-fetch.sh is the file you'll have to investigate to find
the actual fetch invocation.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: os of system fetching port source

[Ed Maste]

On 19 February 2017 at 10:06, qjail1  wrote:
>
> Is there some fetch/ftp option to tag the ports system download request with
> the operating system the request is coming from?

Libfetch uses the HTTP_USER_AGENT environment variable if set, and it
can also be set via a fetch command-line argument. If not set it
defaults to just getprogname() with the libfetch version appended.

I'm not sure what would be the best approach with respect to having
this set by default in the ports infrastructure though.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Some reproducible builds notes

[Ed Maste]

On 17 June 2016 at 04:47, Maxim Sobolev  wrote:
> Hi Ed, I have not got time to look at diffoscope myself yet, but it's
> definitely in my short-term TODO list. Quick question for you though. Would
> it work given two tar.bz2 packages or does it need two directories?

It does in fact process two packages, and will recursively unpack
what's inside. It can also take two directories.

Note that it has specific support for .deb and .rpm packages, but
doesn't know about our pkg format right now. It works fine, but just
compares the +MANIFEST etc. as text files rather than doing a slightly
more "human-friendly" presentation.

To generate the diffoscope results I posted I basically did:

for pkg in $(cat nonrepro.txt); do
timeout $timeout diffoscope \
--jquery https://code.jquery.com/jquery-2.2.4.min.js \
--html-dir /mumble/diffoscope/${pkg%.txz} \
/mumble/build1/$pkg /mumble/build2/$pkg
done

> We
> have code to do just that (i.e. inspect mtree and extract files that differ,
> so if you don't, we can probably contribute that.

Yeah, I think something like this would be a good addition when
pkg-specific support gets added, although perhaps under some flag --
if we had a case where the content was changed from the hash in the
pkg metadata we'd still like to compare and report that. At least for
the way I've used it my use case

> P.S. For anyone interested in Ed's work, here is his BSDCan talk here live:
> https://youtu.be/z7pDnBO5wSM?t=5h37m6s

Thank you for recording it!
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Some reproducible builds notes

[Ed Maste]

I recently presented on "Reproducible Builds in FreeBSD" at BSDCan.
For anyone unfamiliar with the topic, from
https://reproducible-builds.org/ "Reproducible builds are a set of
software development practices which create a verifiable path from
human readable source code to the binary code used by computers." In
brief, the idea is that building the same binary, software package,
document or other binary artifact twice from the same source produces
identical output. There's good background information, documentation
on making builds reproducible, and links to test results on the
reproducible-builds.org site.

Many folks have contributed to the reproducible build effort in
FreeBSD src and ports over time -- at least a decade. There are many
practical benefits of reproducible builds (such as bandwidth and
storage savings). However, there's been a growing interest over the
last few years in the broad open source and free software community in
the topic, coming primarily from a software and toolchain integrity
perspective. Over the last few years some Debian folks have been
leading a comprehensive and structured reproducible builds effort.
bapt@ and I attended the first Reproducible Builds Summit in Athens
last year, and I had a talk accepted at BSDCan on it. The BSDCan
schedule page for my talk[1] has a link to the slides[2].

I'd like to continue discussing reproducible builds in the FreeBSD
context, but for now just want to capture some data from my talk so
that it's available for interested maintainers of individual ports
who'd like to take a look. I used src r300165 and ports r415464, with
a few patches as described in the talk.

I've put data from the ports build runs for my talk at [3]. In that
directory nonrepro.1.txt contains the set of packages that built
nonreproducibly (with a patch set the timestamps in pkg's output).
nonrepro.4.txt contains the set of packages that built nonreproducibly
with the patch above, SOURCE_DATE_EPOCH set in the build environment,
a Clang patch[4] to honour SOURCE_DATE_EPOCH, and a change to make GNU
ar default to deterministic archives, since committed as ports
r416639.

Diffoscope[5] is a tool that attempts to show the differences between
two binary artifacts in a concise and human-readable form. It's
available in ports as sysutils/py-diffoscope and in the
py34-diffoscope package. You can also try it out online[6]. In the
diffoscope/ subdirectory[7] I've put the output for most of the
nonreproducible packages. (Some packages[8] are excluded because of
excessive diffoscope runtime.)


[1] http://www.bsdcan.org/2016/schedule/events/714.en.html
[2] 
http://www.bsdcan.org/2016/schedule/attachments/375_2016-06-11-BSDCan-2016-Reproducible-Builds.pdf
[3] https://people.freebsd.org/~emaste/reproducible-builds/iteration-1/
[4] http://reviews.llvm.org/D20791
[5] https://diffoscope.org/
[6] https://try.diffoscope.org/
[7] 
https://people.freebsd.org/~emaste/reproducible-builds/iteration-1/diffoscope/
[8] 
https://people.freebsd.org/~emaste/reproducible-builds/iteration-1/excessive-diffoscope-runtime.txt
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Recording TIMESTAMPs in distinfo for reproducible builds work

[Ed Maste]

Baptiste and I have been looking at reproducible builds in the FreeBSD
ports tree, and one thing we'll need is a consistent timestamp that
doesn't change when a port is rebuilt without changes.

We considered a few different ideas, and have settled on experimenting
with the time 'make makesum' is run.

I have a bsd.port.mk change that I'll commit shortly to record the
TIMESTAMP when "make makesum" is run.  I want to do this now so that
this data is collected and stored "for free" along with regular
distfile updates.  This will allow experimentation and development of
reproducible package builds with real data.

For now ports that have no distinfo file, and distfile updates done
without using "make makesum," can just ignore the TIMESTAMP.

-Ed
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Removal of legacy X.Org (aka non-WITH_NEW_XORG)

[Ed Maste]

On 3 October 2014 10:00,  l...@lena.kiev.ua wrote:

 Does 10.1 with vt in text mode support _all_ the following in rc.conf?

 scrnmap=koi8-u2cp866u
 font8x16=cp866u-8x16
 allscreens_flags=80x30
 keymap=ru.koi8-r

No, in text mode vt currently has support for cp437 only.

vt(4) uses UTF-8 only right now, and the default font (for graphics
mode) includes a fairly wide character set.  Resolution switching is
not yet provided; vt(4) uses the resolution chosen by the underlying
driver.  That is, vt_vga will use 640x480, i915kms uses the native
panel resolution.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

Re: [FreeBSD-Announce] FreeBSD bug tracking moves from GNATS to Bugzilla

[Ed Maste]

On 4 June 2014 06:34, Darren Pilgrim list_free...@bluerosetech.com wrote:

 Requiring oauth will literally guarantee me and a whole bunch of other
 people will never have bugzilla accounts.

I don't think anyone's suggesting oauth would be required.  It would
just be available as an alternate method for those who don't want to
create a bugzilla-specific account.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org

sysutils/aaccli to be deprecated

[Ed Maste]

The sysutils/aaccli port is a precompiled binary for managing Adaptec
aac(4) RAID cards.  Unfortunately it's incompatible with some changes
in Adaptec's latest drivers, needed for supporting their recent cards.
It is also unmaintained and no longer being supported by Adaptec, and
also is a Linux binary.

The sysutils/arcconf port provides Adaptec's replacement tool (arcconf)
which is a native FreeBSD binary available in i386 and amd64 versions.

How do we go about marking sysutils/aaccli deprecated and have it point
people at arcconf instead?

-Ed
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]

databases/unixODBC installs empty config files

[Ed Maste]

The unixODBC port installs an etc/odbc.ini file which wipes out
an existing config on install of a new package.  (It's not even
a sample config from the distribution; it's just an empty file
created by virtue of existing in the plist it seems.)

Can it just be removed from the plist:

Index: pkg-plist
===
RCS file: /usr/cvs/ports/databases/unixODBC/pkg-plist,v
retrieving revision 1.26
diff -u -u -3 -r1.26 pkg-plist
--- pkg-plist   4 Nov 2006 23:14:18 -   1.26
+++ pkg-plist   18 Apr 2007 21:27:27 -
@@ -7,8 +7,6 @@
 bin/odbc_config
 bin/odbcinst
 %%GUI:%%bin/odbctest
-etc/odbc.ini
-etc/odbcinst.ini
 include/autotest.h
 include/odbcinst.h
 include/odbcinstext.h

___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: databases/unixODBC installs empty config files

[Ed Maste]

On Thu, Apr 19, 2007 at 03:12:53PM -0500, Scot Hetzel wrote:

 You can't simply remove them from the plist, as those files (odbc.ini,
 odbcinst.ini) are created by work/unixODBC-2.2.12/Makefile in the
 install-data-am target.

It's a shame that the target just touches the files to create
empty ones; I'm not sure if there are suitable sample files around
somewhere already.  But I guess having an empty odbc.ini-dist file
is better than an empty odbc.ini file.

 You'll also need to add a post-install target to the ports Makefile,
 so that it will create the odbc.ini and odbcinst.ini files.

Basically just mirroring the @exec lines from the plist, right?

- Ed
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to [EMAIL PROTECTED]