[
https://issues.apache.org/jira/browse/IMPALA-14396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Gaal resolved IMPALA-14396.
----------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
https://github.com/cloudera/native-toolchain/commit/4c18e2672ede490b161e83739de6d848d3e05751
IMPALA-14396: Update build containers for Python 3 and public repo changes
Public infrastructure elements (package repos, individual packages) have
changed since the build container definitions have last been update.
This patch compensates such drift to keep the containers buildable.
Details:
1. Pull EPEL packages from the EPEL archives for CentOS 7
When CentOS 7 was retired, the corresponding EPEL repos were also
moved from the active download points on fedoraprojects.org to the
archives.
Since we still want the toolchain to build on CentOS 7, point the EPEL
repo descriptors to the archives for EPEL v7 on fedoraproject.org.
This allows the toolchain build containers to be rebuilt as well as keep
the toolchain itself building correctly on CentOS 7.
Tested by building new build containers, then using those containers for
building the toolchain itself.
2. Reset default Java to Java 8 on SLES 15
On SLES 15 the default JRE needs to be reset to Java 8 JRE.
Recent package updates caused a Java 17 JRE to be installed and
configured as the default JRE as an implicit dependecy, which breaks the
assert-dependencies() check at the end of the container build, so the
default JRE is reset to Java 8 manually via update-alternatives.
3. Update container build Python scripts to Python3
This patch prepares the toolchain container build process to run on
recent operating systems, where Python2 is not readily available. It
replaces Python2 idioms with constructs or modules available across both
Python major versions:
Old code used distutils.sysconfig for locating Python.h during dependency
installation checking. This module is not supported for recent versions
of Python3, so switch to the sysconfig module, which is supported
everywhere.
The filter/iterator logic in in-docker.py used to take len() of an
iterator directly, which cannot be done in Python 3. Change the filter
to a list comprehension, so the result list can be checked the same way.
4. Update the supported OS list to contain all currently supported
platforms.
Change-Id: Ie491cfcac6efdecbe9f3c18a48fc142ca6269d1c
Reviewed-on: http://gerrit.cloudera.org:8080/23128
Reviewed-by: Michael Smith <michael.sm...@cloudera.com>
Tested-by: Laszlo Gaal <laszlo.g...@cloudera.com>
> Toolchain build container images fail to build because of platform changes
> and deprecations
> -------------------------------------------------------------------------------------------
>
> Key: IMPALA-14396
> URL: https://issues.apache.org/jira/browse/IMPALA-14396
> Project: IMPALA
> Issue Type: Bug
> Components: Toolchain
> Affects Versions: Impala 5.0.0
> Reporter: Laszlo Gaal
> Assignee: Laszlo Gaal
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> Recent toolchain work for adding Ubuntu 24.04 support (tracked as
> IMPALA-14139) exposed failures in the process of generating the build
> container images for concurrent/Docker-based toolchain builds.
> * Red Hat ended support for RHEL 7, which also caused CentOS 7 to end
> support. This led to the removal of almost all CentOS 7 package repos,
> blocking package installations in the current CentOS 7 Dockerfile for the
> build container image
> * Some SLES 15 packages were updated to become dependent on OpenJDK 17. This
> interferes with Impala's preference for OpenJDK 8 during toolchain builds, as
> the JDK 17 dependence was installed earlier, getting also installed as the
> preferred Java version via the {{alternatives}} mechanism, overriding the
> Impala preference.
> * The Python script driving container construction is specific to Python2, so
> it failed to run on Ubuntu 22.04 and 24.04, where Python2 is not available
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
