[
https://issues.apache.org/jira/browse/SPARK-17787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Taukir updated SPARK-17787:
---------------------------
Description:
While trying to push spark app logs to kafka , I am trying to use
KafkaAppender. Please find the command below.It throws the following error
spark-submit --class com.SampleApp --conf spark.ui.port=8081 --master
yarn-cluster
--files
files/home/hos/KafkaApp/log4j.properties#log4j.properties,/home/hos/KafkaApp/kafka_2.10-0.8.0.jar
--conf
spark.driver.extraJavaOptions='-Dlog4j.configuration=file:/home/hos/KafkaApp/log4j.properties'
--conf "spark.driver.extraClassPath=/home/hos/KafkaApp/kafka_2.10-0.8.0.jar"
--conf
spark.executor.extraJavaOptions='-Dlog4j.configuration=file:/home/hos/KafkaApp/log4j.properties'
--conf "spark.executor.extraClassPath=/home/hos/KafkaApp/kafka_2.10-0.8.0.jar"
Kafka-App-assembly-1.0.jar
it it throws java.lang.ClassNotFoundException:
kafka.producer.KafkaLog4jAppender exception
was:
While trying to push spark app logs to kafka , I am trying to use
KafkaAppender. Please find the command below.It throws the following error
spark-submit --class com.SampleApp --conf spark.ui.port=8081 --master
yarn-cluster
--files
files/home/hos/KafkaApp/log4j.properties#log4j.properties,/home/hos/KafkaApp/kafka_2.10-0.8.0.jar
--conf
spark.driver.extraJavaOptions='-Dlog4j.configuration=file:/home/hos/KafkaApp/log4j.properties'
--conf "spark.driver.extraClassPath=/home/hos/KafkaApp/kafka_2.10-0.8.0.jar"
--conf
spark.executor.extraJavaOptions='-Dlog4j.configuration=file:/home/hos/KafkaApp/log4j.properties'
--conf "spark.executor.extraClassPath=/home/hos/KafkaApp/kafka_2.10-0.8.0.jar"
Kafka-App-assembly-1.0.jar
it it throws java.lang.ClassNotFoundException:
kafka.producer.KafkaLog4jAppender exception
> spark submit throws error while using kafka Appender log4j:ERROR Could not
> instantiate class [kafka.producer.KafkaLog4jAppender].
> java.lang.ClassNotFoundException: kafka.producer.KafkaLog4jAppender
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-17787
> URL: https://issues.apache.org/jira/browse/SPARK-17787
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.6.2
> Reporter: Taukir
>
> While trying to push spark app logs to kafka , I am trying to use
> KafkaAppender. Please find the command below.It throws the following error
> spark-submit --class com.SampleApp --conf spark.ui.port=8081 --master
> yarn-cluster
> --files
> files/home/hos/KafkaApp/log4j.properties#log4j.properties,/home/hos/KafkaApp/kafka_2.10-0.8.0.jar
> --conf
> spark.driver.extraJavaOptions='-Dlog4j.configuration=file:/home/hos/KafkaApp/log4j.properties'
> --conf "spark.driver.extraClassPath=/home/hos/KafkaApp/kafka_2.10-0.8.0.jar"
> --conf
> spark.executor.extraJavaOptions='-Dlog4j.configuration=file:/home/hos/KafkaApp/log4j.properties'
> --conf
> "spark.executor.extraClassPath=/home/hos/KafkaApp/kafka_2.10-0.8.0.jar"
> Kafka-App-assembly-1.0.jar
> it it throws java.lang.ClassNotFoundException:
> kafka.producer.KafkaLog4jAppender exception
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
