Quanlong Huang created IMPALA-11359:
---------------------------------------
Summary: bootstrap_toolchain.py failed to download Apache Hive3
Key: IMPALA-11359
URL: https://issues.apache.org/jira/browse/IMPALA-11359
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Reporter: Quanlong Huang
I try building with Apache Hive 3 locally and hit this error:
{code:bash}
$ export USE_APACHE_HIVE=true
$ ./buildall.sh -noclean -notests -ninja
IMPALA_VERSION = 4.2.0-SNAPSHOT
IMPALA_HOME = /home/quanlong/workspace/Impala
HADOOP_HOME =
/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33/
HADOOP_CONF_DIR = /home/quanlong/workspace/Impala/fe/src/test/resources
HADOOP_INCLUDE_DIR =
/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//include
HADOOP_LIB_DIR =
/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib
IMPALA_CLUSTER_NODES_DIR= /home/quanlong/workspace/Impala/testdata/cluster/cdh7
HIVE_HOME =
/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin
HIVE_CONF_DIR =
/home/quanlong/workspace/Impala/fe/./src/test/resources
HIVE_SRC_DIR =
/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src
HBASE_HOME =
/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hbase-2.4.6.7.2.16.0-33/
HBASE_CONF_DIR = /home/quanlong/workspace/Impala/fe/src/test/resources
RANGER_HOME =
/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/ranger-2.1.0.7.2.16.0-33-admin
RANGER_CONF_DIR = /home/quanlong/workspace/Impala/fe/src/test/resources
THRIFT_HOME =
/home/quanlong/workspace/Impala/toolchain/toolchain-packages-gcc7.5.0/thrift-0.11.0-p5
CLASSPATH =
/home/quanlong/workspace/Impala/fe/src/test/resources:/home/quanlong/workspace/Impala/fe/target/classes:/home/quanlong/workspace/Impala/fe/target/dependency::/home/quanlong/workspace/Impala/fe/src/test/resources:/home/quanlong/workspace/Impala/fe/target/classes:/home/quanlong/workspace/Impala/fe/target/dependency::/home/quanlong/workspace/Impala/fe/src/test/resources:/home/quanlong/workspace/Impala/fe/target/classes:/home/quanlong/workspace/Impala/fe/target/dependency:
LIBHDFS_OPTS =
-Djava.library.path=/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib/native/
-XX:ErrorFile=/home/quanlong/workspace/Impala/logs/hs_err_pid%p.log
-XX:MaxPermSize=128m
-Djava.library.path=/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib/native/
-XX:ErrorFile=/home/quanlong/workspace/Impala/logs/hs_err_pid%p.log
-XX:MaxPermSize=128m
-Djava.library.path=/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553/hadoop-3.1.1.7.2.16.0-33//lib/native/
-XX:ErrorFile=/home/quanlong/workspace/Impala/logs/hs_err_pid%p.log
-XX:MaxPermSize=128m
JAVA_HOME = /usr/lib/jvm/java-8-openjdk-amd64
POSTGRES_JDBC_DRIVER =
/home/quanlong/workspace/Impala/fe/target/dependency/postgresql-42.3.3.jar
IMPALA_TOOLCHAIN = /home/quanlong/workspace/Impala/toolchain
IMPALA_TOOLCHAIN_PACKAGES_HOME =
/home/quanlong/workspace/Impala/toolchain/toolchain-packages-gcc7.5.0
METASTORE_DB = HMS_home_quanlong_workspace_Impala_cdp
DOWNLOAD_CDH_COMPONENTS = true
IMPALA_MAVEN_OPTIONS =
IMPALA_TOOLCHAIN_HOST = native-toolchain.s3.amazonaws.com
CDP_BUILD_NUMBER = 26065553
CDP_COMPONENTS_HOME =
/home/quanlong/workspace/Impala/toolchain/cdp_components-26065553
APACHE_MIRROR = https://www.apache.org/dyn/closer.cgi
APACHE_COMPONENTS_HOME =
/home/quanlong/workspace/Impala/toolchain/apache_components
IMPALA_HADOOP_VERSION = 3.1.1.7.2.16.0-33
IMPALA_AVRO_JAVA_VERSION= 1.8.2.7.2.16.0-33
IMPALA_PARQUET_VERSION = 1.10.99.7.2.16.0-33
IMPALA_HIVE_VERSION = 3.1.2
IMPALA_HBASE_VERSION = 2.4.6.7.2.16.0-33
IMPALA_HUDI_VERSION = 0.5.0-incubating
IMPALA_KUDU_VERSION = 09a6c8833e
IMPALA_RANGER_VERSION = 2.1.0.7.2.16.0-33
IMPALA_ICEBERG_VERSION = 0.13.1.7.2.16.0-33
IMPALA_GCS_VERSION = 2.1.2.7.2.16.0-33
IMPALA_COS_VERSION = 3.1.0-5.9.3
SKIP_PYTHON_DOWNLOAD is true, skipping python dependencies download.
SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping toolchain bootstrap.
>>> Downloading and extracting cdh components.
2022-06-16 08:12:30,701 Thread-2 INFO: Downloading
https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download
to
/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz
(attempt 1)
2022-06-16 08:12:30,702 Thread-1 INFO: Downloading
https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
to
/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz
(attempt 1)
2022-06-16 08:12:31,534 Thread-2 ERROR: Download failed; retrying after sleep:
Command with args '['wget',
'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download',
'--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz']'
failed with exit code 5:
--2022-06-16 08:12:30--
https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download
Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's
Encrypt,C=US’:
Issued certificate has expired.
To connect to www.apache.org insecurely, use `--no-check-certificate'.
2022-06-16 08:12:31,535 Thread-1 ERROR: Download failed; retrying after sleep:
Command with args '['wget',
'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download',
'--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz']'
failed with exit code 5:
--2022-06-16 08:12:30--
https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's
Encrypt,C=US’:
Issued certificate has expired.
To connect to www.apache.org insecurely, use `--no-check-certificate'.
2022-06-16 08:12:43,924 Thread-1 INFO: Downloading
https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
to
/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz
(attempt 2)
2022-06-16 08:12:44,749 Thread-1 ERROR: Download failed; retrying after sleep:
Command with args '['wget',
'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download',
'--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz']'
failed with exit code 5:
--2022-06-16 08:12:43--
https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's
Encrypt,C=US’:
Issued certificate has expired.
To connect to www.apache.org insecurely, use `--no-check-certificate'.
2022-06-16 08:12:45,670 Thread-2 INFO: Downloading
https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download
to
/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz
(attempt 2)
2022-06-16 08:12:46,503 Thread-2 ERROR: Download failed; retrying after sleep:
Command with args '['wget',
'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download',
'--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-bin.tar.gz']'
failed with exit code 5:
--2022-06-16 08:12:45--
https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz?action=download
Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's
Encrypt,C=US’:
Issued certificate has expired.
To connect to www.apache.org insecurely, use `--no-check-certificate'.
2022-06-16 08:12:55,981 Thread-1 INFO: Downloading
https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
to
/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz
(attempt 3)
Traceback (most recent call last):
File "/home/quanlong/workspace/Impala/bin/bootstrap_toolchain.py", line 568,
in <module>
if __name__ == "__main__": main()
File "/home/quanlong/workspace/Impala/bin/bootstrap_toolchain.py", line 565,
in main
execute_many(download, components_needing_download)
File "/home/quanlong/workspace/Impala/bin/bootstrap_toolchain.py", line 441,
in execute_many
return pool.map(f, args, 1)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
raise self._value
Exception: Command with args '['wget',
'https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download',
'--output-document=/home/quanlong/workspace/Impala/toolchain/apache_components/apache-hive-3.1.2-src.tar.gz']'
failed with exit code 5:
--2022-06-16 08:12:55--
https://www.apache.org/dyn/closer.cgi/hive/hive-3.1.2/apache-hive-3.1.2-src.tar.gz?action=download
Resolving www.apache.org (www.apache.org)... 151.101.2.132, 2a04:4e42::644
Connecting to www.apache.org (www.apache.org)|151.101.2.132|:443... connected.
ERROR: cannot verify www.apache.org's certificate, issued by ‘CN=R3,O=Let's
Encrypt,C=US’:
Issued certificate has expired.
To connect to www.apache.org insecurely, use `--no-check-certificate'.{code}
CC [~chufucun]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
