Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Alberto Bertogli]

On Sun, Apr 28, 2024 at 06:50:30PM +0100, Alberto Bertogli wrote:

I'm still trying to find a reasonable workaround.


I found a workaround to fix the compilation issue, by detecting and not 
building the 64-bit variants when there is asm aliasing for the relevant 
functions.


However, it is now failing some of the tests, because of some weird 
API/ABI issue with the size of off_t.


The calls to the wrapped function are okay, but then the underlying call 
to glibc's functions have the wrong argument sizes :(


So I'm still trying to figure out to make this work.

Alberto

Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Alberto Bertogli]

On Sun, Apr 28, 2024 at 01:02:26PM +0100, Alberto Bertogli wrote:
The problem seems to be that some of the functions that have 64-bit 
variants (e.g. pread64, pwrite64) have an assembler name declared for 
the regular variant in the header; while other platforms don't do that 
and have the two functions declared separately.


https://gcc.gnu.org/onlinedocs/gcc/Asm-Labels.html

For example, this is the declaration of pread and pwrite in a 
post-preprocessed file, diff between x86_64 (without the bug, '-') and 
armel (with the bug, '+'):


```
@@ -1068,18 +,14 @@
extern ssize_t write (int __fd, const void *__buf, size_t __n)
__attribute__ ((__access__ (__read_only__, 2, 3)));
-# 389 "/usr/include/unistd.h" 3 4
-extern ssize_t pread (int __fd, void *__buf, size_t __nbytes,
-__off_t __offset)
-__attribute__ ((__access__ (__write_only__, 2, 3)));
-
-
+# 404 "/usr/include/unistd.h" 3 4
+extern ssize_t pread (int __fd, void *__buf, size_t __nbytes, __off64_t __offset) __asm__ 
("" "pread64")
+__attribute__ ((__access__ (__write_only__, 2, 3)));
+extern ssize_t pwrite (int __fd, const void *__buf, size_t __nbytes, __off64_t __offset) __asm__ 
("" "pwrite64")
-extern ssize_t pwrite (int __fd, const void *__buf, size_t __n,
- __off_t __offset)
__attribute__ ((__access__ (__read_only__, 2, 3)));
# 422 "/usr/include/unistd.h" 3 4
extern ssize_t pread64 (int __fd, void *__buf, size_t __nbytes,
```

That's why it's the assembler (and not linking) stage that's 
complaining, because that means both functions end up named as the 
64-bit variant. This can be seen in the assembly file. Continuing to 
use pread/pread64 as an example, there is no definition for pread(), 
only pread64() twice: once for pread and one for pread64.


It's tricky to support this in a generic way, because it's difficult 
to detect this is even happening, as the assembler name operates at a 
compiler level so we can't just undo it.


More things that make this interesting.

This program:

```
#include 
#include 
#include 

int main() {
printf("pread: %p\n", pread);
printf("pread64: %p\n", pread64);
printf("pread64 is pread: %b\n", pread == pread64);

void *lib = dlopen(NULL, RTLD_NOW);
void *l_pread = dlsym(lib, "pread");
void *l_pread64 = dlsym(lib, "pread64");

printf("l_pread: %p\n", l_pread);
printf("l_pread64: %p\n", l_pread64);
printf("l_pread64 is l_pread: %b\n", l_pread == l_pread64);
}
```

Built with:
  cc -save-temps=obj -D_XOPEN_SOURCE=600 -D_LARGEFILE64_SOURCE=1  -std=c99 
-Wall demo.c

Prints this on x86_64 Debian testing (which does not show this bug):

```
pread: 0x7fc1970f0c10
pread64: 0x7fc1970f0c10
pread64 is pread: 1
l_pread: 0x7fc1970f0c10
l_pread64: 0x7fc1970f0c10
l_pread64 is l_pread: 1
```

And prints this on the armel chroot:

```
pread: 0xf7da6a90
pread64: 0xf7da6a90
pread64 is pread: 1
l_pread: 0xf7da68f8
l_pread64: 0xf7da6a90
l_pread64 is l_pread: 0
```

So on x86_64 both pread() and pread64() are the same function, yet the 
headers allow us to declare them individually.


But on armel, even though there is a separate implementation of pread() 
on libc, the headers prevent us from declaring them separately (because 
of the asm name declaration in unistd.h).


Just putting this here in case it helps someone else debug a similar 
problem in the future.


I'm still trying to find a reasonable workaround.

Thanks,
Alberto

Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Alberto Bertogli]

On Sat, Apr 20, 2024 at 10:30:20AM +0100, Alberto Bertogli wrote:

On Sat, Apr 20, 2024 at 12:00:00AM +0200, Santiago Vila wrote:

I can't offer ssh access either (for now), but I've checked and
this error may be reproduced easily on an arm64 machine using an
armel chroot.


Oohhh this is good to know, I didn't know that was a viable option.  
Thank you for the suggestion!


I will try to reproduce it this way, I'll let you know how it goes.


I managed to reproduce this the way you suggested, on a Hetzner VM and 
an armel chroot.


The problem seems to be that some of the functions that have 64-bit 
variants (e.g. pread64, pwrite64) have an assembler name declared for 
the regular variant in the header; while other platforms don't do that 
and have the two functions declared separately.


https://gcc.gnu.org/onlinedocs/gcc/Asm-Labels.html

For example, this is the declaration of pread and pwrite in a 
post-preprocessed file, diff between x86_64 (without the bug, '-') and 
armel (with the bug, '+'):


```
@@ -1068,18 +,14 @@
 
 extern ssize_t write (int __fd, const void *__buf, size_t __n)

 __attribute__ ((__access__ (__read_only__, 2, 3)));
-# 389 "/usr/include/unistd.h" 3 4
-extern ssize_t pread (int __fd, void *__buf, size_t __nbytes,
-__off_t __offset)
-__attribute__ ((__access__ (__write_only__, 2, 3)));
-
-
+# 404 "/usr/include/unistd.h" 3 4
+extern ssize_t pread (int __fd, void *__buf, size_t __nbytes, __off64_t __offset) __asm__ 
("" "pread64")
 
 
+__attribute__ ((__access__ (__write_only__, 2, 3)));

+extern ssize_t pwrite (int __fd, const void *__buf, size_t __nbytes, __off64_t __offset) __asm__ 
("" "pwrite64")
 
 
-extern ssize_t pwrite (int __fd, const void *__buf, size_t __n,

- __off_t __offset)
 __attribute__ ((__access__ (__read_only__, 2, 3)));
 # 422 "/usr/include/unistd.h" 3 4
 extern ssize_t pread64 (int __fd, void *__buf, size_t __nbytes,
```

That's why it's the assembler (and not linking) stage that's 
complaining, because that means both functions end up named as the 
64-bit variant. This can be seen in the assembly file. Continuing to use 
pread/pread64 as an example, there is no definition for pread(), only 
pread64() twice: once for pread and one for pread64.


It's tricky to support this in a generic way, because it's difficult to 
detect this is even happening, as the assembler name operates at a 
compiler level so we can't just undo it.


I'll keep trying to find a viable workaround for this.

Thanks,
Alberto

Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Alberto Bertogli]

On Sat, Apr 20, 2024 at 12:00:00AM +0200, Santiago Vila wrote:

El 25/3/24 a las 20:12, Chris Lamb escribió:

Alberto Bertogli wrote:


If you know of a functional official image that I can use to try to
reproduce the problem, or recently-tested steps I can follow to get
a working qemu install, please let me know.


Alas, I can't actually be helpful here. There are no official images
as far as I know… and somewhat annoyingly I (ie. a Debian Developer)
actually have access to some machines set aside for this purpose. I
call this "annoying" because I naturally can't then give you direct
SSH access transitively — but I can proxy ideas, of course.


Hi.

I can't offer ssh access either (for now), but I've checked and
this error may be reproduced easily on an arm64 machine using an
armel chroot.


Oohhh this is good to know, I didn't know that was a viable option.  
Thank you for the suggestion!


I will try to reproduce it this way, I'll let you know how it goes.

Thank you!
Alberto

Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Santiago Vila]

El 25/3/24 a las 20:12, Chris Lamb escribió:

Alberto Bertogli wrote:


If you know of a functional official image that I can use to try to
reproduce the problem, or recently-tested steps I can follow to get
a working qemu install, please let me know.


Alas, I can't actually be helpful here. There are no official images
as far as I know… and somewhat annoyingly I (ie. a Debian Developer)
actually have access to some machines set aside for this purpose. I
call this "annoying" because I naturally can't then give you direct
SSH access transitively — but I can proxy ideas, of course.


Hi.

I can't offer ssh access either (for now), but I've checked and
this error may be reproduced easily on an arm64 machine using an
armel chroot.

Several cloud vendors (like GCP, AWS or Hetzner) offer arm64 machines,
billed hourly and relatively cheap.

Also (but only if your stomach allows it :-), Oracle Cloud has arm64 machines in
their Free Tier. In this case I think they only have Ubuntu and not Debian,
but this would be not matter if you are using a Debian chroot.

Thanks.

Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Alberto Bertogli]

On Mon, Mar 25, 2024 at 07:12:24PM +, Chris Lamb wrote:

Alberto Bertogli wrote:


If you know of a functional official image that I can use to try to
reproduce the problem, or recently-tested steps I can follow to get
a working qemu install, please let me know.


Alas, I can't actually be helpful here. There are no official images
as far as I know… and somewhat annoyingly I (ie. a Debian Developer)
actually have access to some machines set aside for this purpose. I
call this "annoying" because I naturally can't then give you direct
SSH access transitively — but I can proxy ideas, of course.


I totally understand the access part, that's very reasonable on Debian's 
part.


But unfortunately, if I can't even run a local VM to try to reproduce 
the problem, it's too limiting for me. Especially considering the kind 
of issues libfiu often runs into, which tend to be a bit on the esoteric 
side :)




Hm, googling the actual error message a little, I think this might be
a bigger issue... or perhaps more accurately, at least one that has
potentially been also solved elsewhere:

 * Same think in lightdm: 

 * Some kind of "_FILE_OFFSET_BITS"-related patch for v4l-utils
   


Thank you for looking at this!

I think they could be similar; in particular the second one.

Maybe there's something like `#define pread pread64` in the 
architecture's headers that is triggering these errors?




Does this spark anything worth trying? :-)


Maybe seeing the preprocessor output and the actual (temporary) file 
getting the complaints could be useful in figuring out what's going on.


That said, even if we find what the problem is, we may keep finding 
other issues in the future. If I'm not able to have a VM for this
platform where I can try to reproduce problems, I'm not sure it's viable 
to support the package on it :(


Thanks!
Alberto

Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Chris Lamb]

Alberto Bertogli wrote:

> If you know of a functional official image that I can use to try to
> reproduce the problem, or recently-tested steps I can follow to get
> a working qemu install, please let me know.

Alas, I can't actually be helpful here. There are no official images
as far as I know… and somewhat annoyingly I (ie. a Debian Developer)
actually have access to some machines set aside for this purpose. I
call this "annoying" because I naturally can't then give you direct
SSH access transitively — but I can proxy ideas, of course.

Hm, googling the actual error message a little, I think this might be
a bigger issue... or perhaps more accurately, at least one that has
potentially been also solved elsewhere:

  * Same think in lightdm: 

  * Some kind of "_FILE_OFFSET_BITS"-related patch for v4l-utils


etc.

Does this spark anything worth trying? :-)


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org 🍥 chris-lamb.co.uk
   `-

Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Alberto Bertogli]

On Sun, Mar 24, 2024 at 06:00:51PM +, Chris Lamb wrote:

Alberto Bertogli wrote:


I'll try to get a debian install to boot for armhf, but it'll take me a
bit because it's not straightforward (to put it mildly :).


Oh, yeah. :/ Perhaps qemu might be better option here. There might
even be pre-built disk images flying around.


Sorry, yes that's what I meant! To install it under qemu!

Unfortunately, I haven't been able to get it to work.

I managed to get bookworm installed, and then extracted the 
kernel+initrd from inside the virtual disk (as apparently that's 
needed), but the kernel won't boot due to: `Unhandled fault: synchronous 
abort (translation table walk)`.


At this point I'm not really keen on debugging whatever that armhf 
kernel issue is :(


If you know of a functional official image that I can use to try to 
reproduce the problem, or recently-tested steps I can follow to get a 
working qemu install, please let me know.


Thanks,
Alberto

Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Chris Lamb]

Alberto Bertogli wrote:

> I'll try to get a debian install to boot for armhf, but it'll take me a 
> bit because it's not straightforward (to put it mildly :).

Oh, yeah. :/ Perhaps qemu might be better option here. There might
even be pre-built disk images flying around.

> How urgent/important is this issue?

Medium? As it is technically a regression (as in, armhf used to build
before), this was filed with a severity of "serious". What this means
in practical terms is that newer versions of libfiu we upload will not
migrate to the staging area for the next release of Debian.


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org 🍥 chris-lamb.co.uk
   `-

Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Alberto Bertogli]

On Mon, Mar 18, 2024 at 12:29:40PM +, Chris Lamb wrote:

Dear Alberto,


Hi!



Hope this finds you well. Any quick/immediate ideas on what might be
behind this build failure? Note that this is on ARM architectures
rather than amd64 — I often misread and conflate them at speed. :) Oh,
and I can't reproduce this on amd64 locally, at least, so I don't think
it is, say, due to some *general* update to the GLIBC build toolchain.


Nothing obvious.

It's a bit strange because the "symbol X already defined" errors most 
likely come from the assembler when building the individual files, not 
from linking stage.


While it's impossible to be 100% sure because of the parallel build and 
the lack of command-output correlation in the logs, everything points to 
that being the case.


I'll try to get a debian install to boot for armhf, but it'll take me a 
bit because it's not straightforward (to put it mildly :).


How urgent/important is this issue?

Thanks,
Alberto




- Original message -
From: Sebastian Ramacher 
To: Debian Bug Tracking System 
Subject: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: 
symbol `open64' is already defined
Date: Friday, 15 March 2024 6:50 PM

Source: libfiu
Version: 1.2-2
Severity: serious
Tags: ftbfs
Justification: fails to build from source (but built successfully in the past)
X-Debbugs-Cc: sramac...@debian.org

https://buildd.debian.org/status/fetch.php?pkg=libfiu&arch=armhf&ver=1.2-2&stamp=1710292712&raw=0

cc -D_XOPEN_SOURCE=600 -fPIC -DFIU_ENABLE=1 -D_LARGEFILE64_SOURCE=1 -I. -I../../libfiu/ -g 
-O2 -Werror=implicit-function-declaration -ffile-prefix-map=/<>=. 
-fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64 -Wdate-time -D_FORTIFY_SOURCE=2 
-Wl,-z,relro -Wl,-z,now -D_GNU_SOURCE -c codegen.c -o codegen.o
/tmp/cc54dEva.s: Assembler messages:
/tmp/cc54dEva.s:726: Error: symbol `open64' is already defined
/tmp/cchEoHpC.s: Assembler messages:
/tmp/cchEoHpC.s:474: Error: symbol `mmap64' is already defined
make[4]: *** [Makefile:67: modules/posix.mm.mod.o] Error 1
make[4]: *** Waiting for unfinished jobs
make[4]: *** [Makefile:67: modules/posix.custom.o] Error 1
/tmp/cct4HXD3.s: Assembler messages:
/tmp/cct4HXD3.s:1810: Error: symbol `pread64' is already defined
/tmp/cct4HXD3.s:3995: Error: symbol `pwrite64' is already defined
/tmp/cct4HXD3.s:5803: Error: symbol `truncate64' is already defined
/tmp/cct4HXD3.s:6480: Error: symbol `ftruncate64' is already defined
/tmp/ccInAMjZ.s: Assembler messages:
/tmp/ccInAMjZ.s:437: Error: symbol `fopen64' is already defined
make[4]: *** [Makefile:67: modules/posix.io.mod.o] Error 1
/tmp/ccInAMjZ.s:1099: Error: symbol `freopen64' is already defined
/tmp/ccInAMjZ.s:3393: Error: symbol `tmpfile64' is already defined
/tmp/ccInAMjZ.s:5973: Error: symbol `ftello64' is already defined
/tmp/ccInAMjZ.s:7007: Error: symbol `fseeko64' is already defined
/tmp/ccInAMjZ.s:7699: Error: symbol `fsetpos64' is already defined
make[4]: *** [Makefile:67: modules/posix.stdio.mod.o] Error 1





Alberto

Bug#1066938: Fwd: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Chris Lamb]

Dear Alberto,

Hope this finds you well. Any quick/immediate ideas on what might be
behind this build failure? Note that this is on ARM architectures
rather than amd64 — I often misread and conflate them at speed. :) Oh,
and I can't reproduce this on amd64 locally, at least, so I don't think
it is, say, due to some *general* update to the GLIBC build toolchain.


Best wishes,

Chris


- Original message -
From: Sebastian Ramacher 
To: Debian Bug Tracking System 
Subject: Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: 
symbol `open64' is already defined
Date: Friday, 15 March 2024 6:50 PM

Source: libfiu
Version: 1.2-2
Severity: serious
Tags: ftbfs
Justification: fails to build from source (but built successfully in the past)
X-Debbugs-Cc: sramac...@debian.org

https://buildd.debian.org/status/fetch.php?pkg=libfiu&arch=armhf&ver=1.2-2&stamp=1710292712&raw=0

cc -D_XOPEN_SOURCE=600 -fPIC -DFIU_ENABLE=1 -D_LARGEFILE64_SOURCE=1 -I. 
-I../../libfiu/ -g -O2 -Werror=implicit-function-declaration 
-ffile-prefix-map=/<>=. -fstack-protector-strong 
-fstack-clash-protection -Wformat -Werror=format-security -D_LARGEFILE_SOURCE 
-D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64 -Wdate-time -D_FORTIFY_SOURCE=2 
-Wl,-z,relro -Wl,-z,now -D_GNU_SOURCE -c codegen.c -o codegen.o
/tmp/cc54dEva.s: Assembler messages:
/tmp/cc54dEva.s:726: Error: symbol `open64' is already defined
/tmp/cchEoHpC.s: Assembler messages:
/tmp/cchEoHpC.s:474: Error: symbol `mmap64' is already defined
make[4]: *** [Makefile:67: modules/posix.mm.mod.o] Error 1
make[4]: *** Waiting for unfinished jobs
make[4]: *** [Makefile:67: modules/posix.custom.o] Error 1
/tmp/cct4HXD3.s: Assembler messages:
/tmp/cct4HXD3.s:1810: Error: symbol `pread64' is already defined
/tmp/cct4HXD3.s:3995: Error: symbol `pwrite64' is already defined
/tmp/cct4HXD3.s:5803: Error: symbol `truncate64' is already defined
/tmp/cct4HXD3.s:6480: Error: symbol `ftruncate64' is already defined
/tmp/ccInAMjZ.s: Assembler messages:
/tmp/ccInAMjZ.s:437: Error: symbol `fopen64' is already defined
make[4]: *** [Makefile:67: modules/posix.io.mod.o] Error 1
/tmp/ccInAMjZ.s:1099: Error: symbol `freopen64' is already defined
/tmp/ccInAMjZ.s:3393: Error: symbol `tmpfile64' is already defined
/tmp/ccInAMjZ.s:5973: Error: symbol `ftello64' is already defined
/tmp/ccInAMjZ.s:7007: Error: symbol `fseeko64' is already defined
/tmp/ccInAMjZ.s:7699: Error: symbol `fsetpos64' is already defined
make[4]: *** [Makefile:67: modules/posix.stdio.mod.o] Error 1

Bug#1066938: libfiu: FTBFS on arm{el,hf}: /tmp/cc54dEva.s:726: Error: symbol `open64' is already defined

[Sebastian Ramacher]

Source: libfiu
Version: 1.2-2
Severity: serious
Tags: ftbfs
Justification: fails to build from source (but built successfully in the past)
X-Debbugs-Cc: sramac...@debian.org

https://buildd.debian.org/status/fetch.php?pkg=libfiu&arch=armhf&ver=1.2-2&stamp=1710292712&raw=0

cc -D_XOPEN_SOURCE=600 -fPIC -DFIU_ENABLE=1 -D_LARGEFILE64_SOURCE=1 -I. 
-I../../libfiu/ -g -O2 -Werror=implicit-function-declaration 
-ffile-prefix-map=/<>=. -fstack-protector-strong 
-fstack-clash-protection -Wformat -Werror=format-security -D_LARGEFILE_SOURCE 
-D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64 -Wdate-time -D_FORTIFY_SOURCE=2 
-Wl,-z,relro -Wl,-z,now -D_GNU_SOURCE -c codegen.c -o codegen.o
/tmp/cc54dEva.s: Assembler messages:
/tmp/cc54dEva.s:726: Error: symbol `open64' is already defined
/tmp/cchEoHpC.s: Assembler messages:
/tmp/cchEoHpC.s:474: Error: symbol `mmap64' is already defined
make[4]: *** [Makefile:67: modules/posix.mm.mod.o] Error 1
make[4]: *** Waiting for unfinished jobs
make[4]: *** [Makefile:67: modules/posix.custom.o] Error 1
/tmp/cct4HXD3.s: Assembler messages:
/tmp/cct4HXD3.s:1810: Error: symbol `pread64' is already defined
/tmp/cct4HXD3.s:3995: Error: symbol `pwrite64' is already defined
/tmp/cct4HXD3.s:5803: Error: symbol `truncate64' is already defined
/tmp/cct4HXD3.s:6480: Error: symbol `ftruncate64' is already defined
/tmp/ccInAMjZ.s: Assembler messages:
/tmp/ccInAMjZ.s:437: Error: symbol `fopen64' is already defined
make[4]: *** [Makefile:67: modules/posix.io.mod.o] Error 1
/tmp/ccInAMjZ.s:1099: Error: symbol `freopen64' is already defined
/tmp/ccInAMjZ.s:3393: Error: symbol `tmpfile64' is already defined
/tmp/ccInAMjZ.s:5973: Error: symbol `ftello64' is already defined
/tmp/ccInAMjZ.s:7007: Error: symbol `fseeko64' is already defined
/tmp/ccInAMjZ.s:7699: Error: symbol `fsetpos64' is already defined
make[4]: *** [Makefile:67: modules/posix.stdio.mod.o] Error 1

Cheers
-- 
Sebastian Ramacher