This is an automated email from the ASF dual-hosted git repository.
ash pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v1-10-test by this push:
new 4b40eaf Hive/Hadoop minicluster needs JDK8 and JAVA_HOME to work
(#8938)
4b40eaf is described below
commit 4b40eaf02c55a2d510387d79d1fa145cc107ba9e
Author: Ash Berlin-Taylor <ash_git...@firemirror.com>
AuthorDate: Thu May 21 06:19:49 2020 +0100
Hive/Hadoop minicluster needs JDK8 and JAVA_HOME to work (#8938)
Debian Buster only ships with a JDK11, and Hive/Hadoop fails in odd,
hard to debug ways (complains about metastore not being initalized,
possibly related to the class loader issues.)
Until we rip Hive out from the CI (replacing it with Hadoop in a seprate
integration, only on for some builds) we'll have to stick with JRE8
Our previous approach of installing openjdk-8 from Sid/Unstable started
failing as Debian Sid has a new (and conflicting) version of GCC/libc.
The adoptopenjdk package archive is designed for Buster so should be
more resilient
(cherry picked from commit 8476c1e387bb98db3feb9f26247bcef6e92948ec)
---
Dockerfile.ci | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/Dockerfile.ci b/Dockerfile.ci
index 1666f13..ffa9a28 100644
--- a/Dockerfile.ci
+++ b/Dockerfile.ci
@@ -119,11 +119,14 @@ RUN adduser airflow \
# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863199
RUN mkdir -pv /usr/share/man/man1 \
&& mkdir -pv /usr/share/man/man7 \
+ && curl -fsSL
https://adoptopenjdk.jfrog.io/adoptopenjdk/api/gpg/key/public | apt-key add - \
+ && echo 'deb https://adoptopenjdk.jfrog.io/adoptopenjdk/deb/ buster main'
> \
+ /etc/apt/sources.list.d/adoptopenjdk.list \
&& apt-get update \
&& apt-get install --no-install-recommends -y \
gnupg \
libgcc-8-dev \
- default-jre-headless \
+ adoptopenjdk-8-hotspot-jre \
apt-transport-https \
bash-completion \
ca-certificates \
@@ -147,6 +150,8 @@ RUN mkdir -pv /usr/share/man/man1 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
+ENV JAVA_HOME=/usr/lib/jvm/adoptopenjdk-8-hotspot-jre-amd64
+
# Install Hadoop and Hive
# It is done in one step to share variables.
ENV HADOOP_HOME="/opt/hadoop-cdh" HIVE_HOME="/opt/hive"
