Laszlo Gaal has posted comments on this change. ( http://gerrit.cloudera.org:8080/20737 )
Change subject: IMPALA-11157: Use native-toolchain hadoop for aarch64 ...................................................................... Patch Set 2: (2 comments) I've tried running a core-mode test build with this change, but dataload seems to be quite flaky: - https://jenkins.impala.io/view/Ubuntu%2020/job/ubuntu-20.04-from-scratch-ARM/62/ crashed with symptoms indicating an Impala crash (Transport endpoint is not connected): 00:52:56.956 ERROR: INSERT OVERWRITE TABLE functional_parquet.alltypessmall partition (year, month) 00:52:56.956 SELECT id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col, year, month 00:52:56.956 FROM functional.alltypessmall 00:52:56.956 Traceback (most recent call last): 00:52:56.956 File "/home/ubuntu/Impala/bin/load-data.py", line 189, in exec_impala_query_from_file 00:52:56.956 result = impala_client.execute(query) 00:52:56.956 File "/home/ubuntu/Impala/tests/beeswax/impala_beeswax.py", line 191, in execute 00:52:56.956 handle = self.__execute_query(query_string.strip(), user=user) 00:52:56.956 File "/home/ubuntu/Impala/tests/beeswax/impala_beeswax.py", line 369, in __execute_query 00:52:56.956 self.wait_for_finished(handle) 00:52:56.956 File "/home/ubuntu/Impala/tests/beeswax/impala_beeswax.py", line 390, in wait_for_finished 00:52:56.956 raise ImpalaBeeswaxException("Query aborted:" + error_log, None) 00:52:56.956 ImpalaBeeswaxException: ImpalaBeeswaxException: 00:52:56.956 Query aborted:Exec() rpc failed: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) Another build on a private instance (similar size with Ubuntu 20.04) crashed in a different location during dataload, leaving a number of JVM crash reports in /var/crash, and several hs_err_pidNNN.logs in the Impala working directory. (These are pretty large; I'm happy to send them to you directly if you think they would be helpful). My primary suspicion is the version mismatch between CDP_HADOOP_VERSION and IMPALA_HADOOP_CLIENT_BINARY_VERSION, see the comments. http://gerrit.cloudera.org:8080/#/c/20737/2/bin/impala-config.sh File bin/impala-config.sh: http://gerrit.cloudera.org:8080/#/c/20737/2/bin/impala-config.sh@207 PS2, Line 207: 3.3.6 I'm afraid this version is too new, and much too different from the regular Hadoop version, see below http://gerrit.cloudera.org:8080/#/c/20737/2/bin/impala-config.sh@253 PS2, Line 253: 3.1.1.7.2.18.0-369 I'm afraid this version of Hadoop is not really compatible with the significantly newer 3.3.6 that's specified for the ARM native libraries above. 3.1.1 was released 5 years ago; 3.3.6 is quite fresh, and it has recently received a full upgrade to the AWS Java SDK v2, among other changes. -- To view, visit http://gerrit.cloudera.org:8080/20737 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ide5ad327d6ce7c2a6b7d0ec4cf1dd53fef987720 Gerrit-Change-Number: 20737 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Laszlo Gaal <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Comment-Date: Fri, 01 Dec 2023 17:58:07 +0000 Gerrit-HasComments: Yes
