stepanovD opened a new issue #10698:
URL: https://github.com/apache/druid/issues/10698
Hi! I try execute ingestion by hadoop mapreduce. My tutning config:
`
"tuningConfig": {
"type": "hadoop",
"jobProperties": {
"mapreduce.job.classloader": "true",
"mapreduce.job.classloader.system.classes":
"-javax.validation.,java.,javax.,org.apache.commons.logging.,org.apache.log4j.,org.apache.hadoop.,com.google.,org.apache.druid.",
"mapreduce.job.counters.max": "4"
}
}
`
But I got problem:
`
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.druid.indexer.DetermineHashedPartitionsJob$DetermineCardinalityMapper
not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2638)
at
org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:187)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: java.lang.ClassNotFoundException: Class
org.apache.druid.indexer.DetermineHashedPartitionsJob$DetermineCardinalityMapper
not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2542)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2636)
... 8 more
`
I tried change "mapreduce.job.classloader": "true" to
"mapreduce.job.user.classpath.first": "true", but then I got other problem:
`
2020-12-21T14:02:03,347 INFO [task-runner-0-priority-0]
org.apache.hadoop.mapreduce.Job - Job job_1608555203540_0005 failed with state
FAILED due to: Application application_1608555203540_0005 failed 2 times due to
AM Container for appattempt_1608555203540_0005_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2020-12-21 17:02:02.529]Exception from
container-launch.
Container id: container_1608555203540_0005_02_000001
Exit code: 1
[2020-12-21 17:02:02.536]Container exited with a non-zero exit code 1. Error
file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/tmp/hadoop-hadoop/nm-local-dir/filecache/179/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/abc/app/hadoop-3.2.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type
[org.apache.logging.slf4j.Log4jLoggerFactory]
ERROR StatusLogger No log4j2 configuration file found. Using default
configuration: logging only errors to the console. Set system property
'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show Log4j2
internal initialization logging.
`
Please help me solve this problem someone.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
