[
https://issues.apache.org/jira/browse/HADOOP-18401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17579716#comment-17579716
]
Veli-Jussi Raitila edited comment on HADOOP-18401 at 8/15/22 1:50 PM:
----------------------------------------------------------------------
It is indeed unfortunate transitive Python dependencies can break the build -
of the development container in this case. Hadoop itself does build just fine
on an M1 Mac.
Earlier version of the lazy-object-proxy library does not have this problem,
and neither does builds for Intel since a prebuilt wheel from PyPI is
appropriately picked. The dependency chain for pylint always picks the latest
versions and after the packaging mishap the build started breaking from
external causes.
So the solutions are:
# Do not install pylint for the ARM development container at all. Not sure
where this is actually used as I was able to run a minimal build, skipping
tests + docs but with native libs without it
# Pin a specific version of lazy-object-proxy which still has the appropriate
wheels available
# Wait for a packaging fix upstream for the lazy-object-proxy library
# Get a newer Python installed in the development container
I also noticed some modularization effort being started for Dockerfiles in
trunk. I wonder if there is a valid reason to treat ARM and Intel differently:
For the 3.3.4 release, Hugo has an ARM build available, as does the Java SDK,
so those are obvious. But the packages sources from Ubuntu repos also differ
slightly, and either hadolint nor the ISA-L library are installed in the ARM
container. ARM container also sets the QT_QPA_PLATFORM environment variable.
The rationale for these choices are not clear to me.
was (Author: JIRAUSER294345):
It is indeed unfortunate transitive Python dependencies can break the build -
of the development container in this case. Hadoop itself does build just fine
on an M1 Mac.
Earlier version of the lazy-object-proxy library does not have this problem,
and neither does builds for Intel since a prebuilt wheel from PyPI is
appropriately picked. The dependency chain for pylint always picks the latest
versions and after the packaging mishap the build started breaking from
external causes.
So the solutions are:
# Do not install pylint for the ARM development container at all. Not sure
where this is actually used as I was able to run a minimal build, skipping
tests + docs but with native libs without it
# Pin a specific version of lazy-object-proxy which still has the appropriate
wheels available
# Wait for a packaging fix upstream for the lazy-object-proxy library
I also noticed some modularization effort being started for Dockerfiles in
trunk. I wonder if there is a valid reason to treat ARM and Intel differently:
Hugo has an ARM build as does the Java SDK, so those are obvious. But the
packages installed from Ubuntu repos also differ slightly, and hadolint nor the
ISA-L library are not installed in the ARM container - and rationale for these
choices are not clear to me.
> release files miss hadoop-3.3.4-aarch64.tar.gz
> ----------------------------------------------
>
> Key: HADOOP-18401
> URL: https://issues.apache.org/jira/browse/HADOOP-18401
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.3.2, 3.3.3, 3.3.4
> Reporter: Ling Xu
> Priority: Minor
> Attachments: image-2022-08-11-14-54-15-490.png
>
>
> release files miss hadoop-3.3.4-aarch64.tar.gz
> !image-2022-08-11-14-54-15-490.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
