On Fri, Aug 23, 2019 at 2:28 AM Jakub Piotr Cłapa <jpc...@zenburn.net>
wrote:
> Hi,
>
> I spent some time trying to cross-compile NumPy for Linux ARM64 on a
> macOS host as part of an OpenWRT compilation [1]. It was not so bad but
> certainly not trivial and I was wondering if we could use my experience
> to help streamline the process.
>
Thanks, that would be great!
We have quite some issues with cross-compilation, and usually they're hard
to resolve (plus it's not something we officially support - but there's a
clear need for improvement). Note that conda is also a cross-compiler (to
the same system), sometimes worth looking into how it deals with
(numpy.)distutils issues.
> I encountered two challenges:
>
> 1. Python distutils did not allow me to provide a proper cross-ranlib
> program path. I opened an issue for this on the Python bug tracker [2].
>
That seems like a straightforward enough patch. distutils isn't
well-maintained but hopefully that gets merged soon. If it doesn't get
responses in a couple of weeks, please feel free to ping me and I can
review/test (I'm not a core Python dev, but it may help push it forwards).
>
> 2. numpy distutils scripts check sys.platform in many places.
> sys.platform returns the platform on which compilation occurs (darwin in
> my case) which in case of cross-compilation is not the one on which the
> code will be run (linux). This of course results in invalid compiler and
> linker flags.
>
> I made a quick'n'dirty fix [3] and got numpy to compile and run
> correctly (but I have yet to run the full test suite). Of course this is
> not a proper way to solve this so I was hoping to get some feedback on
> the preferred/best way. I did not find any info about an official way to
> do this with distutils.
>
> One idea (from @commodo on GitHub) would be to just change my hardcoded
> string ("Linux") to something like this:
>
> target_platform = os.getenviron(“TARGET_PLATFORM”, sys.platform)
>
That seems reasonable enough. Prefixing that name with NPY_ would be good,
to not get accidental clashes.
I'm not sure if there's a more standard/preferred way, I haven't come
across it in distutils. There's some issues with reasonably detailed
discussion for SciPy - this one may be useful:
https://github.com/scipy/scipy/issues/8571
> It could also be further refactored by moving it to a single place but
> while certainly more elegant and "proper" it may ultimately make it less
> readable and harder to follow. I am also unsure where would be the best
> place to put it and how to pass it around to other modules.
>
Keeping all platform comparisons where they are now seems right. Just need
one central place to parse NPY_TARGET_PLATFORM - perhaps in
numpy/distutils/misc_util.py
Cheers,
Ralf
> [1]: https://github.com/openwrt/packages/pull/9797
> [2]: https://bugs.python.org/issue37916
> [3]:
>
> https://github.com/openwrt/packages/pull/9797/commits/e701e122fb9c78d5cecffb88c1d750178a239083#diff-1ea8d839590bcdc406854d3998a96704
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion
