Quanlong Huang created IMPALA-12999:
---------------------------------------
Summary: Missing log4j.properties in the package
Key: IMPALA-12999
URL: https://issues.apache.org/jira/browse/IMPALA-12999
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Reporter: Quanlong Huang
I got a NoClassDefFoundError when launching catalogd deployed by the RPM package
{noformat}
I0411 13:38:14.642594 113789 init.cc:602] Default AES cipher mode for
spill-to-disk: AES-GCM
I0411 13:38:14.746702 113789 init.cc:440] Using Java weigher sizeof
I0411 13:38:15.480756 113789 webserver.cc:395] Starting webserver on
0.0.0.0:25020
I0411 13:38:15.480811 113789 webserver.cc:411] Document root: /opt/impala
I0411 13:38:15.481134 113789 webserver.cc:548] Webserver started
I0411 13:38:15.609732 113789 jni-util.cc:302] java.lang.NoClassDefFoundError:
org.apache.hadoop.fs.FileSystem
at
org.apache.hadoop.hive.conf.valcoersion.JavaIOTmpdirVariableCoercion.<clinit>(JavaIOTmpdirVariableCoercion.java:37)
at
org.apache.hadoop.hive.conf.SystemVariables.<clinit>(SystemVariables.java:37)
at
org.apache.hadoop.hive.conf.HiveConf$ConfVars.<init>(HiveConf.java:5736)
at
org.apache.hadoop.hive.conf.HiveConf$ConfVars.<init>(HiveConf.java:5717)
at
org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:536)
at org.apache.hadoop.hive.conf.HiveConf.<clinit>(HiveConf.java:256)
at org.apache.impala.service.JniCatalog.<clinit>(JniCatalog.java:135)
I0411 13:38:15.609787 113789 status.cc:129] NoClassDefFoundError:
org.apache.hadoop.fs.FileSystem
@ 0x10a02f4
@ 0x1c21104
@ 0x1036151
@ 0xfd433d
@ 0xf5a43e
@ 0xe5b774
@ 0x7f81d2bf66a3
@ 0xf05c2e
F0411 13:38:15.611681 113789 catalog.cc:86] NoClassDefFoundError:
org.apache.hadoop.fs.FileSystem
. Impalad exiting.
Minidump with no thread info available.
Wrote minidump to
/var/log/impala-minidumps/catalogd/86355b1b-4d2c-4195-61ac0d9e-68059041.dmp
{noformat}
No more info in catalogd.ERROR:
{noformat}
Log file created at: 2024/04/11 13:38:14
Running on machine: ccycloud-3.quanlong.root.comops.site
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0411 13:38:14.627285 113789 logging.cc:256] stderr will be logged to this file.
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.conf.Configuration).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
F0411 13:38:15.611681 113789 catalog.cc:86] NoClassDefFoundError:
org.apache.hadoop.fs.FileSystem
. Impalad exiting.
*** Check failure stack trace: ***
@ 0x391054d
@ 0x3912484
@ 0x390ff2c
@ 0x39129a9
@ 0x103625b
@ 0xfd433d
@ 0xf5a43e
@ 0xe5b774
@ 0x7f81d2bf66a3
@ 0xf05c2e
Minidump with no thread info available.
Wrote minidump to
/var/log/impala-minidumps/catalogd/86355b1b-4d2c-4195-61ac0d9e-68059041.dmp{noformat}
It seems an issue of the CLASSPATH that the jar of hadoop-common is missing.
But actually in my case, it's due to the process doesn't have permission to
read the core-site.xml file. Due to missing log4j.properties, the exception is
not shown in the log, which makes it pretty hard to debug.
Note that the exception happens before GlogAppender is installed. So logs from
log4j are not redirected to glog yet.
After adding the missing log4j.properties file, the exception is shown,
explaning the NoClassDefFoundError. It's due to a failure in initializing a
static field of FileSystem.
{noformat}
Log file created at: 2024/04/12 02:27:41
Running on machine: ccycloud-3.quanlong.root.comops.site
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0412 02:27:41.413901 190384 logging.cc:256] stderr will be logged to this file.
24/04/12 02:27:42 ERROR conf.Configuration: error parsing conf core-site.xml
java.io.FileNotFoundException: /opt/impala/conf/core-site.xml (Permission
denied)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at
sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
at
sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
at org.apache.hadoop.conf.Configuration.parse(Configuration.java:3023)
at
org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3119)
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3077)
at
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:3050)
at
org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2928)
at
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2910)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1264)
at
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1318)
at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1543)
at org.apache.hadoop.fs.FileSystem$Cache.<init>(FileSystem.java:3495)
at org.apache.hadoop.fs.FileSystem.<clinit>(FileSystem.java:194)
24/04/12 02:27:42 INFO util.JvmPauseMonitor: Starting JVM pause monitor
24/04/12 02:27:42 INFO conf.HiveConf: Found configuration file
file:/opt/impala/conf/hive-site.xml
F0412 02:27:42.422082 190384 catalog.cc:86] NoClassDefFoundError:
org.apache.hadoop.fs.FileSystem
. Impalad exiting.
*** Check failure stack trace: ***
@ 0x391054d
@ 0x3912484
@ 0x390ff2c
@ 0x39129a9
@ 0x103625b
@ 0xfd433d
@ 0xf5a43e
@ 0xe5b774
@ 0x7fb2a83cd6a3
@ 0xf05c2e
Minidump with no thread info available.
Wrote minidump to
/var/log/impala-minidumps/catalogd/1c092b19-c5db-442a-dbbfff82-47299b9e.dmp
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
