Aw: Re: Aw: Re: Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-09-14 Thread Steffen Möller
Hi Alexandre,

> Gesendet: Samstag, 13. September 2025 um 10:53
> Von: "Alexandre Detiste" 
> An: "Steffen Möller" 
> CC: [email protected]
> Betreff: Re: Aw: Re: Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - 
> where are we?
> By only looking at the log and not the repository,
> it seems  libarrow-python-flight2100,libarrow-python2100 and python3-pyarrow
> are tagged "Architecture: all" and should instead be tagged "Architecture: 
> any".

Thank you for your feedback. I just checked, it is any. And with another look at
the lintian error's description


arch-dependent-file-not-in-arch-specific-directory
This package is Multi-Arch "same", but it installs an ELF binary in the 
directory that is not architecture-specific.


and no immediate idea surfacing after looking at d/rules, as a quick fix I went 
for having those packages as
MultiArch: no.


I is now building locally (and on Salsa, I presume) for a while. Will update 
the remaining lintian
bits then later today.

Best,
Steffen



Re: Aw: Re: Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-09-13 Thread Alexandre Detiste
Hi,

By only looking at the log and not the repository,
it seems  libarrow-python-flight2100,libarrow-python2100 and python3-pyarrow
are tagged "Architecture: all" and should instead be tagged "Architecture: any".

I'll have a look again later.

Thank you for working on this

Greetings

Le jeu. 11 sept. 2025 à 17:27, Steffen Möller  a écrit :
>
>
> Hello again,
>
> cowbuilder builds the arrow packages just fine, we are darn close with 
> debian/copyright. Lintian complaints, including some errors.
> What do you think is missing to rightously upload to experimental?
>
> Best,
> Steffen
>
>
> E: libarrow-python-flight2100: 
> arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0]
> E: libarrow-python-flight2100: 
> arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100]
> E: libarrow-python2100: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0]
> E: libarrow-python2100: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so]
> E: r-cran-arrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/R/site-library/arrow/libs/arrow.so]
> E: python3-pyarrow: missing-dependency-on-libc needed by 
> usr/lib/python3.13/dist-packages/pyarrow/_acero.cpython-313-x86_64-linux-gnu.so
>  and 19 others
> E: r-cran-arrow: missing-dependency-on-libc needed by 
> usr/lib/R/site-library/arrow/libs/arrow.so
> E: libarrow-python-flight2100: python-package-missing-depends-on-python
> E: python3-pyarrow: python-package-missing-depends-on-python
> E: apache-arrow source: python3-depends-but-no-python3-helper 
> libarrow-python-flight2100 pyarrow-dev python3-pyarrow
> E: r-cran-arrow: requires-r-api
> W: apache-arrow source: binaries-have-file-conflict libarrow-dev pyarrow-dev 
> usr/share/arrow/gdb/gdb_arrow.py
> W: apache-arrow source: binaries-have-file-conflict 
> libarrow-python-flight2100 python3-pyarrow 
> usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100
> W: apache-arrow source: binaries-have-file-conflict 
> libarrow-python-flight2100 python3-pyarrow 
> usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0
> W: apache-arrow source: binaries-have-file-conflict 
> libarrow-python-flight2100-dbgsym python3-pyarrow-dbgsym 
> usr/lib/debug/.build-id/bb/128659158a15063e2be34818a4b9ebab8f8ec3.debug
> W: apache-arrow source: binaries-have-file-conflict libarrow-python2100 
> python3-pyarrow 
> usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100
> W: apache-arrow source: binaries-have-file-conflict libarrow-python2100 
> python3-pyarrow 
> usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0
> W: apache-arrow source: binaries-have-file-conflict 
> libarrow-python2100-dbgsym python3-pyarrow-dbgsym 
> usr/lib/debug/.build-id/91/2272dc914dfe99c9c2a26d68f042b904bef25a.debug
> W: libarrow-python2100: distant-prerequisite-in-shlibs libarrow-python2100 
> (>= ${binary:Version}) [shlibs]
> W: gir1.2-arrow-1.0: empty-binary-package
> W: gir1.2-arrow-dataset-1.0: empty-binary-package
> W: gir1.2-parquet-1.0: empty-binary-package
> W: apache-arrow source: obsolete-url-in-packaging 
> https://code.google.com/archive/p/mman-win32 [debian/copyright]
> W: libarrow-python-flight2100: python-module-in-wrong-location 
> usr/lib/python3.13/dist-packages/pyarrow -> 
> usr/lib/python3/dist-packages/pyarrow
> W: libarrow-python2100: python-module-in-wrong-location 
> usr/lib/python3.13/dist-packages/pyarrow -> 
> usr/lib/python3/dist-packages/pyarrow
> W: python3-pyarrow: python-module-in-wrong-location 
> usr/lib/python3.13/dist-packages/pyarrow -> 
> usr/lib/python3/dist-packages/pyarrow
> W: python3-pyarrow: python-module-in-wrong-location 
> usr/lib/python3.13/dist-packages/pyarrow-21.0.0.egg-info -> 
> usr/lib/python3/dist-packages/pyarrow-21.0.0.egg-info
>



Aw: Aw: Re: Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-09-11 Thread Steffen Möller


Hello again,

cowbuilder builds the arrow packages just fine, we are darn close with 
debian/copyright. Lintian complaints, including some errors.
What do you think is missing to rightously upload to experimental?

Best,
Steffen


E: libarrow-python-flight2100: 
arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0]
E: libarrow-python-flight2100: 
arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100]
E: libarrow-python2100: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0]
E: libarrow-python2100: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so]
E: r-cran-arrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/R/site-library/arrow/libs/arrow.so]
E: python3-pyarrow: missing-dependency-on-libc needed by 
usr/lib/python3.13/dist-packages/pyarrow/_acero.cpython-313-x86_64-linux-gnu.so 
and 19 others
E: r-cran-arrow: missing-dependency-on-libc needed by 
usr/lib/R/site-library/arrow/libs/arrow.so
E: libarrow-python-flight2100: python-package-missing-depends-on-python
E: python3-pyarrow: python-package-missing-depends-on-python
E: apache-arrow source: python3-depends-but-no-python3-helper 
libarrow-python-flight2100 pyarrow-dev python3-pyarrow
E: r-cran-arrow: requires-r-api
W: apache-arrow source: binaries-have-file-conflict libarrow-dev pyarrow-dev 
usr/share/arrow/gdb/gdb_arrow.py
W: apache-arrow source: binaries-have-file-conflict libarrow-python-flight2100 
python3-pyarrow 
usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100
W: apache-arrow source: binaries-have-file-conflict libarrow-python-flight2100 
python3-pyarrow 
usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0
W: apache-arrow source: binaries-have-file-conflict 
libarrow-python-flight2100-dbgsym python3-pyarrow-dbgsym 
usr/lib/debug/.build-id/bb/128659158a15063e2be34818a4b9ebab8f8ec3.debug
W: apache-arrow source: binaries-have-file-conflict libarrow-python2100 
python3-pyarrow usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100
W: apache-arrow source: binaries-have-file-conflict libarrow-python2100 
python3-pyarrow 
usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0
W: apache-arrow source: binaries-have-file-conflict libarrow-python2100-dbgsym 
python3-pyarrow-dbgsym 
usr/lib/debug/.build-id/91/2272dc914dfe99c9c2a26d68f042b904bef25a.debug
W: libarrow-python2100: distant-prerequisite-in-shlibs libarrow-python2100 (>= 
${binary:Version}) [shlibs]
W: gir1.2-arrow-1.0: empty-binary-package
W: gir1.2-arrow-dataset-1.0: empty-binary-package
W: gir1.2-parquet-1.0: empty-binary-package
W: apache-arrow source: obsolete-url-in-packaging 
https://code.google.com/archive/p/mman-win32 [debian/copyright]
W: libarrow-python-flight2100: python-module-in-wrong-location 
usr/lib/python3.13/dist-packages/pyarrow -> 
usr/lib/python3/dist-packages/pyarrow
W: libarrow-python2100: python-module-in-wrong-location 
usr/lib/python3.13/dist-packages/pyarrow -> 
usr/lib/python3/dist-packages/pyarrow
W: python3-pyarrow: python-module-in-wrong-location 
usr/lib/python3.13/dist-packages/pyarrow -> 
usr/lib/python3/dist-packages/pyarrow
W: python3-pyarrow: python-module-in-wrong-location 
usr/lib/python3.13/dist-packages/pyarrow-21.0.0.egg-info -> 
usr/lib/python3/dist-packages/pyarrow-21.0.0.egg-info



Aw: Aw: Aw: Re: Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-09-11 Thread Steffen Möller
The package builds fine in cowbuilder but salsa triggers

-- Build files have been written to: 
/builds/science-team/arrow/debian/output/source_dir/cpp_build
ninja: error: manifest 'build.ninja' still dirty after 100 tries, perhaps 
system time is not set
dh_auto_build: error: cd cpp_build && LC_ALL=C.UTF-8 ninja -j2 -v returned exit 
code 1
make[1]: *** [debian/rules:102: override_dh_auto_build] Error 25
make[1]: Leaving directory '/builds/science-team/arrow/debian/output/source_dir'
make: *** [debian/rules:37: binary] Error 2

I presume this is some feature for reproducible builds, anyway. Ideas how to 
address this are appreciated.

Cheers,
Steffen



> Gesendet: Donnerstag, 11. September 2025 um 17:26
> Von: "Steffen Möller" 
> An: [email protected]
> Betreff: Aw: Aw: Re: Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - 
> where are we?
>
> 
> Hello again,
> 
> cowbuilder builds the arrow packages just fine, we are darn close with 
> debian/copyright. Lintian complaints, including some errors.
> What do you think is missing to rightously upload to experimental?
> 
> Best,
> Steffen
> 
> 
> E: libarrow-python-flight2100: 
> arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0]
> E: libarrow-python-flight2100: 
> arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100]
> E: libarrow-python2100: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0]
> E: libarrow-python2100: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100]
> E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so]
> E: r-cran-arrow: arch-dependent-file-not-in-arch-specific-directory 
> [usr/lib/R/site-library/arrow/libs/arrow.so]
> E: python3-pyarrow: missing-dependency-on-libc needed by 
> usr/lib/python3.13/dist-packages/pyarrow/_acero.cpython-313-x86_64-linux-gnu.so
>  and 19 others
> E: r-cran-arrow: missing-dependency-on-libc needed by 
> usr/lib/R/site-library/arrow/libs/arrow.so
> E: libarrow-python-flight2100: python-package-missing-depends-on-python
> E: python3-pyarrow: python-package-missing-depends-on-python
> E: apache-arrow source: python3-depends-but-no-python3-helper 
> libarrow-python-flight2100 pyarrow-dev python3-pyarrow
> E: r-cran-arrow: requires-r-api
> W: apache-arrow source: binaries-have-file-conflict libarrow-dev pyarrow-dev 
> usr/share/arrow/gdb/gdb_arrow.py
> W: apache-arrow source: binaries-have-file-conflict 
> libarrow-python-flight2100 python3-pyarrow 
> usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100
> W: apache-arrow source: binaries-have-file-conflict 
> libarrow-python-flight2100 python3-pyarrow 
> usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0
> W: apache-arrow source: binaries-have-file-conflict 
> libarrow-python-flight2100-dbgsym python3-pyarrow-dbgsym 
> usr/lib/debug/.build-id/bb/128659158a15063e2be34818a4b9ebab8f8ec3.debug
> W: apache-arrow source: binaries-have-file-conflict libarrow-python2100 
> python3-pyarrow 
> usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100
> W: apache-arrow source: binaries-have-file-conflict libarrow-python2100 
> python3-pyarrow 
> usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0
> W: apache-arrow source: binaries-have-file-conflict 
> libarrow-python2100-dbgsym python3-pyarrow-dbgsym 
> usr/lib/debug/.build-id/91/2272dc914dfe99c9c2a26d68f042b904bef25a.debug
> W: libarrow-python2100: distant-prerequisite-in-shlibs libarrow-python2100 
> (>= ${binary:Version}) [shlibs]
> W: gir1.2-arrow-1.0: empty-binary-package
> W: gir1.2-arrow-dataset-1.0: empty-binary-package
> W: gir1.2-parquet-1.0: empty-binary-pa

Lintian errors for Apache Arrow (was: Aw: Aw: Re: Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?)

2025-08-28 Thread Steffen Möller
Hello,

https://salsa.debian.org/science-team/arrow/-/jobs/8135311 kindly presents the 
following

E: libarrow-python-flight2100: 
arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0]
E: libarrow-python-flight2100: 
arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100]
E: libarrow-python2100: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0]
E: libarrow-python2100: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100]
E: python3-pyarrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so]
E: r-cran-arrow: arch-dependent-file-not-in-arch-specific-directory 
[usr/lib/R/site-library/arrow/libs/arrow.so]

Other packages like python3-numpy also have their .so underneath 
usr/lib/python3/dist-packages?!


E: apache-arrow source: build-depends-indep-without-arch-indep
E: libgandiva-dev: description-synopsis-is-duplicated line 1
E: libgandiva2100: description-synopsis-is-duplicated line 1
E: libgandiva-dev: description-too-short gandiva-dev
E: libgandiva2100: description-too-short gandiva
E: r-cran-arrow: description-too-short R

Fixed locally.


E: python3-pyarrow: missing-dependency-on-libc needed by 
usr/lib/python3.13/dist-packages/pyarrow/_acero.cpython-313-x86_64-linux-gnu.so 
and 19 others
E: r-cran-arrow: missing-dependency-on-libc needed by 
usr/lib/R/site-library/arrow/libs/arrow.so

Hm. It is not what I thought it is. Ideas?


E: python3-pyarrow: package-installs-python-pycache-dir 
[usr/lib/python3.13/dist-packages/pyarrow/interchange/__pycache__/]
E: python3-pyarrow: package-installs-python-pycache-dir 
[usr/lib/python3.13/dist-packages/pyarrow/parquet/__pycache__/]
E: python3-pyarrow: package-installs-python-pycache-dir 
[usr/lib/python3.13/dist-packages/pyarrow/tests/__pycache__/]
E: python3-pyarrow: package-installs-python-pycache-dir 
[usr/lib/python3.13/dist-packages/pyarrow/tests/interchange/__pycache__/]
E: python3-pyarrow: package-installs-python-pycache-dir 
[usr/lib/python3.13/dist-packages/pyarrow/tests/parquet/__pycache__/]
E: python3-pyarrow: package-installs-python-pycache-dir 
[usr/lib/python3.13/dist-packages/pyarrow/vendored/__pycache__/]

Fixed locally.

E: libarrow-python-flight2100: python-package-missing-depends-on-python
E: python3-pyarrow: python-package-missing-depends-on-python

${python3:Depends} is placed but presumed empty. Ideas?

help?


E: apache-arrow source: python3-depends-but-no-python3-helper pyarrow-dev 
python3-pyarrow


E: r-cran-arrow: requires-r-api
? Will figure that one out.

E: apache-arrow source: version-substvar-for-external-package Depends 
${binary:Version} libarrow-acero-dev -> libarrow-acero [debian/control:286]
W: apache-arrow source: binaries-have-file-conflict libarrow-dev pyarrow-dev 
usr/share/arrow/gdb/gdb_arrow.py
W: apache-arrow source: binaries-have-file-conflict libarrow-python-flight2100 
python3-pyarrow 
usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100
W: apache-arrow source: binaries-have-file-conflict libarrow-python-flight2100 
python3-pyarrow 
usr/lib/python3.13/dist-packages/pyarrow/libarrow_python_flight.so.2100.0.0
W: apache-arrow source: binaries-have-file-conflict libarrow-python2100 
python3-pyarrow usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100
W: apache-arrow source: binaries-have-file-conflict libarrow-python2100 
python3-pyarrow 
usr/lib/python3.13/dist-packages/pyarrow/libarrow_python.so.2100.0.0
W: libarrow-python2100: distant-prerequisite-in-shlibs libarrow-python2100 (>= 
${binary:Version}) [shlibs]

help!?

W: gir1.2-arrow-1.0: empty-binary-package
W: gir1.2-arrow-dataset-1.0: empty-binary-package
W: gir1.2-parquet-1.0: empty-binary-package

help!?

W: libarrow-python-flight2100: python-module-in-wrong-location 
usr/lib/python3.13/dist-packages/pyarrow -> 
usr/lib/python3/dist-packages/pyarrow
W: libarrow-python2100: python-module-in-wrong-location 
usr/lib/python3.13/dist-packages/pyarrow -> 
usr/lib/python3/dist-packages/pyarrow

Aw: Re: Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-08-27 Thread Steffen Möller
Hello,

> Gesendet: Dienstag, 26. August 2025 um 19:24
> Von: "Diane Trout" 
> An: "Steffen Möller" , [email protected]
> Betreff: Re: Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where 
> are we?
>
> On Tue, 2025-08-26 at 16:22 +, Steffen Möller wrote:
> > 
> > 
> > On a sidenote, when linking the R package (from CRAN with the tools
> > subdir
> > removed) against the arrow C++ library then I get a link error wrt
> > some
> > Hive objects. Are there some files omitted from building the library?
> 
> I haven't had a chance to investigate why but I did notice lintian said
> several library packages were empty.
> 
> When I tried to do an update on the package earlier it seems like
> upstream was fond of moving files around and adding and removing
> libraries to the overall archive.
> 
> The other common lintian warninings were about needing symbols files
> and the python libraries were missing the hardening build flags.
> 

We now have an R package that can be loaded from within an R shell,
have not tried it, though.

To me it feels like the package is closing in for an upload.
What are you all's feelings about d/copyright?

Best,
Steffen



Re: Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-08-26 Thread Diane Trout
On Tue, 2025-08-26 at 16:22 +, Steffen Möller wrote:
> 
> 
> On a sidenote, when linking the R package (from CRAN with the tools
> subdir
> removed) against the arrow C++ library then I get a link error wrt
> some
> Hive objects. Are there some files omitted from building the library?

I haven't had a chance to investigate why but I did notice lintian said
several library packages were empty.

When I tried to do an update on the package earlier it seems like
upstream was fond of moving files around and adding and removing
libraries to the overall archive.

The other common lintian warninings were about needing symbols files
and the python libraries were missing the hardening build flags.

Diane



Aw: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-08-26 Thread Steffen Möller





> Gesendet: Samstag, 23. August 2025 um 02:13
> Von: "Diane Trout" 
> An: "Steffen Möller" , [email protected]
> Betreff: Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?
>
> > 
> > Yes, please have a look. I suggest we start with a very core of the
> > software (C++ and Python) and then see who wants to join with
> > additional
> > languages.
> 
> 
> I figured out the reason it didn't build on i386 was the multiarch
> paths were hard coded in d/rules. I fixed that and tested it locally.
> I've pushed the fix to salsa, though I haven't seen the results yet.
> 
> (There was some missing dependencies in the python autopkgtest that I
> also fixed in an earlier commit)
> 

Excellent!

I am experimenting with building the r-cran-arrow package also from the
same source tree. We break the "r-cran-"-semantics just slightly that way,
but I guess it is the most practical to keep the versions in sync.

On a sidenote, when linking the R package (from CRAN with the tools subdir
removed) against the arrow C++ library then I get a link error wrt some
Hive objects. Are there some files omitted from building the library?

Cheers,
Steffen



Re: Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-08-22 Thread Diane Trout
> 
> Yes, please have a look. I suggest we start with a very core of the
> software (C++ and Python) and then see who wants to join with
> additional
> languages.


I figured out the reason it didn't build on i386 was the multiarch
paths were hard coded in d/rules. I fixed that and tested it locally.
I've pushed the fix to salsa, though I haven't seen the results yet.

(There was some missing dependencies in the python autopkgtest that I
also fixed in an earlier commit)



Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-08-19 Thread Dirk Eddelbuettel


On 19 August 2025 at 08:15, Steffen Möller wrote:
| Yes, please have a look. I suggest we start with a very core of the
| software (C++ and Python) and then see who wants to join with additional
| languages.

There is also a feature-matrix of what is enabled. Here is what the (current)
R package shows (built outside of the distro as usual) on my box:


| > arrow::arrow_info()
Arrow package version: 21.0.0.1

Capabilities:
   
acero  TRUE
datasetTRUE
substrait FALSE
parquetTRUE
json   TRUE
s3 TRUE
gcsTRUE
utf8proc   TRUE
re2TRUE
snappy TRUE
gzip   TRUE
brotli TRUE
zstd   TRUE
lz4TRUE
lz4_frame  TRUE
lzo   FALSE
bz2TRUE
jemalloc   TRUE
mimalloc   TRUE

Memory:
  
Allocator mimalloc
Current0 bytes
Max0 bytes

Runtime:
  
SIMD Level  avx512
Detected SIMD Level avx512

Build:
   
C++ Library Version  21.0.0
C++ CompilerGNU
C++ Compiler Version 11.4.0

> 

My C++ compiler is newer, but this was a 'narrower and faster' build that
pulled pre-made shared libraries in. Which we can't do.

Thanks for working on this. It is both important and difficult.

Cheers, Dirk
 

-- 
dirk.eddelbuettel.com | @eddelbuettel | [email protected]



Aw: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-08-19 Thread Steffen Möller





> Gesendet: Montag, 18. August 2025 um 21:02
> Von: "Diane Trout" 
> An: "Steffen Möller" , [email protected]
> Betreff: Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?
>
> On Mon, 2025-08-18 at 18:31 +, Steffen Möller wrote:
> > > > 
> > 
> > (with python3-pyarrow - near miss)
> > https://salsa.debian.org/science-team/arrow/-/pipelines/912189 
> > 
> 
> I'll take a look at the python part. In my attempts to build it it
> seemed like they regularly added and removed shared libraries, and the
> build error looks like something is missing or not found.
> 
> I kind of feel like arrow will need a team to maintain it. I doubt one
> person is going to know the details of the build systems for every
> language they build bindings for.

Just before leaving, I yesterday had a build success with 21.0.0
https://salsa.debian.org/science-team/arrow/-/pipelines/912266
which is still running.

Yes, please have a look. I suggest we start with a very core of the
software (C++ and Python) and then see who wants to join with additional
languages.

Best,
Steffen



Re: Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-08-18 Thread Diane Trout
On Mon, 2025-08-18 at 18:31 +, Steffen Möller wrote:
> > > 
> 
> (with python3-pyarrow - near miss)
> https://salsa.debian.org/science-team/arrow/-/pipelines/912189 
> 

I'll take a look at the python part. In my attempts to build it it
seemed like they regularly added and removed shared libraries, and the
build error looks like something is missing or not found.

I kind of feel like arrow will need a team to maintain it. I doubt one
person is going to know the details of the build systems for every
language they build bindings for.

Diane



Aw: Aw: Re: Packaging of Apache Arrow - where are we?

2025-08-18 Thread Steffen Möller
> > Gesendet: Montag, 30. Juni 2025 um 12:30
> > Von: "Alexandre Detiste" 
> > An: "Steffen Möller" 
> > CC: [email protected]
> > Betreff: Re: Packaging of Apache Arrow - where are we?
> >
> > Le lun. 30 juin 2025 à 12:03, Steffen Möller  a 
> > écrit :
> > > I have updated to version 20.0.0. The situation got a bit worse as the 
> > > Python interface now demands Apache Thrift
> > > (https://thrift.apache.org/, https://github.com/apache/thrift) to be 
> > > available, which it is not.
> > 
> > https://tracker.debian.org/pkg/thrift is here
> 
> And it is an amazingly popular package
> https://qa.debian.org/popcon.php?package=thrift
> No idea why I missed it.
> 
> > It's also needed to update python-feather-format and remove the
> > dependency on cython-legacy.
> 
> I'll have a look. 

I got Apache Arrow to build,

(without any Python)
https://salsa.debian.org/science-team/arrow/-/pipelines/912000

(with python3-pyarrow - near miss)
https://salsa.debian.org/science-team/arrow/-/pipelines/912189 

and am now upgrading to 21.0.0.

Many thanks go to upstream's Raúl Cumplido his overlooked merge request for 
19.0.0 and
for offering help whenever we get stuck with anything. 

I fail to specify d/watch in a way that it does not prefer the Rust 
implementation and would appreciate some help on that.

Next will (eventually) be r-cran-arrow.

Best,
Steffen








Aw: Re: Packaging of Apache Arrow - where are we?

2025-06-30 Thread Steffen Möller





> Gesendet: Montag, 30. Juni 2025 um 12:30
> Von: "Alexandre Detiste" 
> An: "Steffen Möller" 
> CC: [email protected]
> Betreff: Re: Packaging of Apache Arrow - where are we?
>
> Le lun. 30 juin 2025 à 12:03, Steffen Möller  a écrit 
> :
> > I have updated to version 20.0.0. The situation got a bit worse as the 
> > Python interface now demands Apache Thrift
> > (https://thrift.apache.org/, https://github.com/apache/thrift) to be 
> > available, which it is not.
> 
> https://tracker.debian.org/pkg/thrift is here

And it is an amazingly popular package
https://qa.debian.org/popcon.php?package=thrift
No idea why I missed it.

> It's also needed to update python-feather-format and remove the
> dependency on cython-legacy.

I'll have a look. 

Thank you tons!

Steffen