Kengo Seki created BIGTOP-3572:
----------------------------------
Summary: Oozie smoke test fails on the ARM platform
Key: BIGTOP-3572
URL: https://issues.apache.org/jira/browse/BIGTOP-3572
Project: Bigtop
Issue Type: Bug
Components: tests
Reporter: Kengo Seki
Fix For: 3.0.0
https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=debian-10-arm64-deploy/console
https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=debian-9-arm64-deploy/console
https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=ubuntu-18.04-arm64-deploy/console
https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=ubuntu-20.04-arm64-deploy/console
It looks like to be caused by the version mismatch of Guava, but I'm not sure
why it happens only on arm64 (and maybe ppc64le?) platform.
{code}
+ oozie-setup sharelib create -fs hdfs://28c610858918.bigtop.apache.org:8020/
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/lib/oozie/embedded-oozie-server/webapp/WEB-INF/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/lib/oozie/lib-client/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
Exception in thread "main" java.lang.NoSuchMethodError:
com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at
org.apache.oozie.service.XLogService.extractInfoForLogWebService(XLogService.java:218)
at org.apache.oozie.service.XLogService.init(XLogService.java:203)
at
org.apache.oozie.service.Services.setServiceInternal(Services.java:387)
at org.apache.oozie.service.Services.<init>(Services.java:111)
at
org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:198)
at
org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:103)
{code}
I also tried to reproduce it, but I couldn't on the m6g EC2 instance.
{code}
ubuntu@ip-172-30-1-254:~/bigtop/provisioner/docker$ cat /etc/issue
Ubuntu 18.04.5 LTS \n \l
ubuntu@ip-172-30-1-254:~/bigtop/provisioner/docker$ arch
aarch64
ubuntu@ip-172-30-1-254:~/bigtop/provisioner/docker$ ./docker-hadoop.sh -d -C
config_ubuntu-18.04.yaml -G -L -k hdfs,yarn,mapreduce,hive,oozie -s oozie -c 1
...
+ oozie-setup sharelib create -fs hdfs://3349cb325787.bigtop.apache.org:8020/
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/usr/lib/oozie/lib-client/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/lib/oozie/embedded-oozie-server/webapp/WEB-INF/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
8324 [main] INFO org.apache.oozie.service.XLogService -
*******************************************************************************
STARTUP MSG: Oozie BUILD_VERSION [5.2.1] compiled by [ubuntu] on
[2021.07.12-00:33:17GMT -Dvc.revision=unavailable -Dvc.url=unavailable
-DgenerateDocs]
STARTUP MSG: revision
[branch-5.2@8f0e5ee09]@[[email protected]:apache/oozie.git]
*******************************************************************************
8325 [main] INFO oozieops -
*******************************************************************************
STARTUP MSG: Oozie BUILD_VERSION [5.2.1] compiled by [ubuntu] on
[2021.07.12-00:33:17GMT -Dvc.revision=unavailable -Dvc.url=unavailable
-DgenerateDocs]
STARTUP MSG: revision
[branch-5.2@8f0e5ee09]@[[email protected]:apache/oozie.git]
*******************************************************************************
8325 [main] INFO org.apache.oozie.service.XLogService - Log4j configuration
file [oozie-log4j.properties]
8325 [main] INFO org.apache.oozie.service.XLogService - Log4j configuration
file loaded from [/etc/oozie/conf]
8326 [main] INFO org.apache.oozie.service.XLogService - Log4j reload interval
[10 sec]
8333 [main] INFO org.apache.oozie.service.ConfigurationService - Oozie home dir
[/usr/lib/oozie]
8334 [main] INFO org.apache.oozie.service.ConfigurationService - Oozie conf dir
[/etc/oozie/conf]
8334 [main] INFO org.apache.oozie.service.ConfigurationService - Oozie conf
file [oozie-site.xml]
8613 [main] WARN org.apache.oozie.service.ConfigurationService - Invalid
configuration defined, [use.system.libpath.for.mapreduce.and.pig.jobs]
8625 [main] WARN org.apache.oozie.service.ConfigurationService - Invalid
configuration defined, [oozie.authentication.signature.secret]
8625 [main] WARN org.apache.oozie.service.ConfigurationService - Invalid
configuration defined, [oozie.service.AuthorizationService.security.enabled]
8626 [main] WARN org.apache.oozie.util.DateUtils - GMT, UTC or Region/City
Timezone formats are preferred instead of UTC
8627 [main] WARN org.apache.oozie.service.Services - System ID [oozie-oozi]
exceeds maximum length [10], trimming
8629 [main] INFO org.apache.oozie.service.Services - Exiting null Entering
NORMAL
8629 [main] INFO oozieops - Exiting null Entering NORMAL
8630 [main] INFO org.apache.oozie.service.Services - Initialized runtime
directory [/tmp/oozie-oozi2045196064457886659.dir]
8630 [main] INFO org.apache.oozie.service.ConfigurationService - Programmatic
configuration change,
property[oozie.services]=[org.apache.oozie.service.LiteWorkflowAppService,
org.apache.oozie.service.HadoopAccessorService]
8630 [main] INFO oozieops - Programmatic configuration change,
property[oozie.services]=[org.apache.oozie.service.LiteWorkflowAppService,
org.apache.oozie.service.HadoopAccessorService]
8631 [main] INFO org.apache.oozie.service.ConfigurationService - Programmatic
configuration change, property[oozie.services.ext]=[]
8631 [main] INFO oozieops - Programmatic configuration change,
property[oozie.services.ext]=[]
8636 [main] INFO org.apache.oozie.service.HadoopAccessorService -
JOB_TRACKER_WHITELIST :[], Total entries :0
8636 [main] INFO org.apache.oozie.service.HadoopAccessorService -
NAME_NODE_WHITELIST :[], Total entries :0
8636 [main] INFO org.apache.oozie.service.HadoopAccessorService - Oozie
Kerberos Authentication [disabled]
8872 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
9041 [main] INFO org.apache.hadoop.security.SecurityUtil - Updating
Configuration
9041 [main] INFO org.apache.oozie.service.Services - Initialized
9044 [main] INFO org.apache.oozie.service.Services - Running with JARs for
Hadoop version [3.2.2]
9048 [main] INFO org.apache.oozie.service.Services - Oozie System ID
[oozie-oozi] started!
Found Hadoop that supports Erasure Coding. Trying to disable Erasure Coding for
path: /user/oozie/share/lib
Done
the destination path for sharelib is: /user/oozie/share/lib/lib_20210713230159
44357 [main] INFO org.apache.oozie.service.Services - Shutdown
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
