Re: CVS commit: src/doc

2020-02-29 Thread Kamil Rytarowski
On 29.02.2020 22:29, Kamil Rytarowski wrote:
> On 29.02.2020 21:58, Joerg Sonnenberger wrote:
>> On Sat, Feb 29, 2020 at 07:36:00PM +0100, Kamil Rytarowski wrote:
>>> On 29.02.2020 19:00, Taylor R Campbell wrote:
> Module Name:src
> Committed By:   kamil
> Date:   Sat Feb 29 04:27:01 UTC 2020
>
> Modified Files:
> src/doc: CHANGES
>
> Log Message:
> ld.elf_so(1): Implement DT_GNU_HASH

 Was this discussed anywhere?
>>>
>>> In the toolchain circles, for some time now (2+ years).
>>>
>>> It was one of the pending task to modernize our ELF loader on par with
>>> at least FreeBSD.
>>
>> Can we please stop with this "we must need XXX to be on par with YYY"

Speaking of what. We are still waiting for over a year now for
multisegment support in the ELF loader. We can barely build a functional
binary with a patched lld.

Can we get at least architectural design what is the proper direction
here (we received no feedback) so we can do it on our own?



signature.asc
Description: OpenPGP digital signature


Re: CVS commit: src/doc

2020-02-29 Thread Joerg Sonnenberger
On Sat, Feb 29, 2020 at 10:29:02PM +0100, Kamil Rytarowski wrote:
> On 29.02.2020 21:58, Joerg Sonnenberger wrote:
> > On Sat, Feb 29, 2020 at 07:36:00PM +0100, Kamil Rytarowski wrote:
> >> On 29.02.2020 19:00, Taylor R Campbell wrote:
>  Module Name:src
>  Committed By:   kamil
>  Date:   Sat Feb 29 04:27:01 UTC 2020
> 
>  Modified Files:
>  src/doc: CHANGES
> 
>  Log Message:
>  ld.elf_so(1): Implement DT_GNU_HASH
> >>>
> >>> Was this discussed anywhere?
> >>
> >> In the toolchain circles, for some time now (2+ years).
> >>
> >> It was one of the pending task to modernize our ELF loader on par with
> >> at least FreeBSD.
> > 
> > Can we please stop with this "we must need XXX to be on par with YYY"
> > nonsense without actually looking at the details first and discuss them?
> > Practically speaking, there was moderately little discussion about the
> > actual design choices of DT_GNU_HASH, especially some of its deficits.
> > Especially because we already have some important improvements in our
> > tree *without breaking compatibility*. Especially the size claims are
> > questionable at best as justification are weak at best. It also ignores
> > that by design DT_GNU_HASH conflicts with at least one platform ABI.
> > 
> > Joerg
> > 
> 
> Just keeping DT_GNU_HASH support around does not break compat. Full
> replacement of HASH algorithm would break compat but nobody wants to do
> it (at least in NetBSD).

Can you please stop speaking for the NetBSD community? Seriously, that
alone makes me want to just /dev/null all messages. "Keeping DT_GNU_HASH
around" is funny, given that it just got added without any discussion.

Joerg


Re: CVS commit: src/doc

2020-02-29 Thread Kamil Rytarowski
On 29.02.2020 21:58, Joerg Sonnenberger wrote:
> On Sat, Feb 29, 2020 at 07:36:00PM +0100, Kamil Rytarowski wrote:
>> On 29.02.2020 19:00, Taylor R Campbell wrote:
 Module Name:src
 Committed By:   kamil
 Date:   Sat Feb 29 04:27:01 UTC 2020

 Modified Files:
 src/doc: CHANGES

 Log Message:
 ld.elf_so(1): Implement DT_GNU_HASH
>>>
>>> Was this discussed anywhere?
>>
>> In the toolchain circles, for some time now (2+ years).
>>
>> It was one of the pending task to modernize our ELF loader on par with
>> at least FreeBSD.
> 
> Can we please stop with this "we must need XXX to be on par with YYY"
> nonsense without actually looking at the details first and discuss them?
> Practically speaking, there was moderately little discussion about the
> actual design choices of DT_GNU_HASH, especially some of its deficits.
> Especially because we already have some important improvements in our
> tree *without breaking compatibility*. Especially the size claims are
> questionable at best as justification are weak at best. It also ignores
> that by design DT_GNU_HASH conflicts with at least one platform ABI.
> 
> Joerg
> 

Just keeping DT_GNU_HASH support around does not break compat. Full
replacement of HASH algorithm would break compat but nobody wants to do
it (at least in NetBSD).

Improvement. I presume we are talking about enhancements such as
fast_divide32_prepare(3). This is reused also in DT_GNU_HASH.

DT_GNU_HASH is still not fully supported on MIPS ABI with our basesystem
toolchain (as it is not recent enough). Paper work stopped it for some
years and it is now in mainline GNU as DT_MIPS_XHASH.

https://github.com/bminor/binutils-gdb/commit/f16a9783c5f085443d806646074e9c06fdee9a88

So, I presume it is good to ifdef DT_GNU_HASH for __mips__ in the loader.



signature.asc
Description: OpenPGP digital signature


Re: CVS commit: src/doc

2020-02-29 Thread Joerg Sonnenberger
On Sat, Feb 29, 2020 at 07:36:00PM +0100, Kamil Rytarowski wrote:
> On 29.02.2020 19:00, Taylor R Campbell wrote:
> >> Module Name:src
> >> Committed By:   kamil
> >> Date:   Sat Feb 29 04:27:01 UTC 2020
> >>
> >> Modified Files:
> >> src/doc: CHANGES
> >>
> >> Log Message:
> >> ld.elf_so(1): Implement DT_GNU_HASH
> > 
> > Was this discussed anywhere?
> 
> In the toolchain circles, for some time now (2+ years).
> 
> It was one of the pending task to modernize our ELF loader on par with
> at least FreeBSD.

Can we please stop with this "we must need XXX to be on par with YYY"
nonsense without actually looking at the details first and discuss them?
Practically speaking, there was moderately little discussion about the
actual design choices of DT_GNU_HASH, especially some of its deficits.
Especially because we already have some important improvements in our
tree *without breaking compatibility*. Especially the size claims are
questionable at best as justification are weak at best. It also ignores
that by design DT_GNU_HASH conflicts with at least one platform ABI.

Joerg


Re: CVS commit: src/doc

2020-02-29 Thread Kamil Rytarowski
On 29.02.2020 19:29, Martin Husemann wrote:
> On Sat, Feb 29, 2020 at 06:00:29PM +, Taylor R Campbell wrote:
>>> Module Name:src
>>> Committed By:   kamil
>>> Date:   Sat Feb 29 04:27:01 UTC 2020
>>>
>>> Modified Files:
>>> src/doc: CHANGES
>>>
>>> Log Message:
>>> ld.elf_so(1): Implement DT_GNU_HASH
>>
>> Was this discussed anywhere?  What are the advantages and drawbacks of
>> this over what we were doing before?  What other toolchain changes are
>> involved in using it?  What maintenance burden does it put on us for
>> compatibility?  What's the impact on systems that prioritize size over
>> speed?
> 
> It also does not compile on most architectures.
> 
> Martin
> 

Looking!



signature.asc
Description: OpenPGP digital signature


Re: CVS commit: src/doc

2020-02-29 Thread Kamil Rytarowski
On 29.02.2020 19:00, Taylor R Campbell wrote:
>> Module Name:src
>> Committed By:   kamil
>> Date:   Sat Feb 29 04:27:01 UTC 2020
>>
>> Modified Files:
>> src/doc: CHANGES
>>
>> Log Message:
>> ld.elf_so(1): Implement DT_GNU_HASH
> 
> Was this discussed anywhere?

In the toolchain circles, for some time now (2+ years).

It was one of the pending task to modernize our ELF loader on par with
at least FreeBSD.

>  What are the advantages and drawbacks of
> this over what we were doing before?

Reading:

https://blogs.oracle.com/solaris/gnu-hash-elf-sections-v2

https://flapenguin.me/2017/05/10/elf-lookup-dt-gnu-hash/

1. Advantages.

We support now an alternative hash format that is ~50% faster, de facto
standard in ELF Operating Systems and thinner in size.

2. Disadvantages.

If we switch to GNU Hash only we are no longer producing compliant ELF
binaries.

Once we will deliver GNU Hash binaries, delivering sysv hash for
compatibility and conformance reason, binaries will be fatter for 0.5-5kb.

Some OSs like FreeBSD and most Linux distros decided to build both
hashes concurrently. Upstream LLD developers attempted to switch as many
OSs as possible to GNU Hash only, but it was resisted.

ps(1) sizes
---

54064 sysv hash
54216 both hashes
53520 gnu hash


>  What other toolchain changes are
> involved in using it?

1. On the ld level.

 - dynamically: make LDFLAGS="-Wl,--hash-style=both"
 - statically: possibly flipping

/* Define to 1 if you want to emit gnu hash in the ELF linker by default. */
#define DEFAULT_EMIT_GNU_HASH 0

in config.h

2. gcc/clang

pass - -Wl,--hash-style=both

>  What maintenance burden does it put on us for
> compatibility?

After designing in 10+ years ago, it works as is without changes in
other OSs.

Stable interface and if someone will invent new hash, we will likely get
a new type of: DT_*_HASH.

LLVM and GNU toolchains support it and recommend for ELF targets.

>  What's the impact on systems that prioritize size over
> speed?
> 

We win both size and speed with GNU Hash shipped exclusively.

We pay cost of extra around 1 kbyte per binary if we want both hashes.

If we want minimum size we can generate only one hash type.



signature.asc
Description: OpenPGP digital signature


Re: CVS commit: src/doc

2020-02-29 Thread Martin Husemann
On Sat, Feb 29, 2020 at 06:00:29PM +, Taylor R Campbell wrote:
> > Module Name:src
> > Committed By:   kamil
> > Date:   Sat Feb 29 04:27:01 UTC 2020
> > 
> > Modified Files:
> > src/doc: CHANGES
> > 
> > Log Message:
> > ld.elf_so(1): Implement DT_GNU_HASH
> 
> Was this discussed anywhere?  What are the advantages and drawbacks of
> this over what we were doing before?  What other toolchain changes are
> involved in using it?  What maintenance burden does it put on us for
> compatibility?  What's the impact on systems that prioritize size over
> speed?

It also does not compile on most architectures.

Martin


Re: CVS commit: src/doc

2020-02-29 Thread Taylor R Campbell
> Module Name:src
> Committed By:   kamil
> Date:   Sat Feb 29 04:27:01 UTC 2020
> 
> Modified Files:
> src/doc: CHANGES
> 
> Log Message:
> ld.elf_so(1): Implement DT_GNU_HASH

Was this discussed anywhere?  What are the advantages and drawbacks of
this over what we were doing before?  What other toolchain changes are
involved in using it?  What maintenance burden does it put on us for
compatibility?  What's the impact on systems that prioritize size over
speed?