Re: Configuring Log4J (Spark 1.5 on EMR 4.1)
try to assemble log4j.xml or log4j.properties in your jar...probably you'll get what you want, however pay attention that when you'll move to multinode cluster - there will be difference On 20 November 2015 at 05:10, Afshartous, Nick wrote: > > < log4j.properties file only exists on the master and not the slave nodes, > so you are probably running into > https://issues.apache.org/jira/browse/SPARK-11105, which has already been > fixed in the not-yet-released Spark 1.6.0. EMR will upgrade to Spark 1.6.0 > once it is released. > > Thanks for the info, though this is a single-node cluster so that can't be > the cause of the error (which is in the driver log). > -- > Nick > > From: Jonathan Kelly [jonathaka...@gmail.com] > Sent: Thursday, November 19, 2015 6:45 PM > To: Afshartous, Nick > Cc: user@spark.apache.org > Subject: Re: Configuring Log4J (Spark 1.5 on EMR 4.1) > > This file only exists on the master and not the slave nodes, so you are > probably running into https://issues.apache.org/jira/browse/SPARK-11105, > which has already been fixed in the not-yet-released Spark 1.6.0. EMR will > upgrade to Spark 1.6.0 once it is released. > > ~ Jonathan > > On Thu, Nov 19, 2015 at 1:30 PM, Afshartous, Nick <mailto:nafshart...@turbine.com>> wrote: > > Hi, > > On Spark 1.5 on EMR 4.1 the message below appears in stderr in the Yarn UI. > > ERROR StatusLogger No log4j2 configuration file found. Using default > configuration: logging only errors to the console. > > I do see that there is > >/usr/lib/spark/conf/log4j.properties > > Can someone please advise on how to setup log4j properly. > > Thanks, > -- > Nick > > Notice: This communication is for the intended recipient(s) only and may > contain confidential, proprietary, legally protected or privileged > information of Turbine, Inc. If you are not the intended recipient(s), > please notify the sender at once and delete this communication. > Unauthorized use of the information in this communication is strictly > prohibited and may be unlawful. For those recipients under contract with > Turbine, Inc., the information in this communication is subject to the > terms and conditions of any applicable contracts or agreements. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org user-unsubscr...@spark.apache.org> > For additional commands, e-mail: user-h...@spark.apache.org user-h...@spark.apache.org> > > > > Notice: This communication is for the intended recipient(s) only and may > contain confidential, proprietary, legally protected or privileged > information of Turbine, Inc. If you are not the intended recipient(s), > please notify the sender at once and delete this communication. > Unauthorized use of the information in this communication is strictly > prohibited and may be unlawful. For those recipients under contract with > Turbine, Inc., the information in this communication is subject to the > terms and conditions of any applicable contracts or agreements. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
RE: Configuring Log4J (Spark 1.5 on EMR 4.1)
< log4j.properties file only exists on the master and not the slave nodes, so you are probably running into https://issues.apache.org/jira/browse/SPARK-11105, which has already been fixed in the not-yet-released Spark 1.6.0. EMR will upgrade to Spark 1.6.0 once it is released. Thanks for the info, though this is a single-node cluster so that can't be the cause of the error (which is in the driver log). -- Nick From: Jonathan Kelly [jonathaka...@gmail.com] Sent: Thursday, November 19, 2015 6:45 PM To: Afshartous, Nick Cc: user@spark.apache.org Subject: Re: Configuring Log4J (Spark 1.5 on EMR 4.1) This file only exists on the master and not the slave nodes, so you are probably running into https://issues.apache.org/jira/browse/SPARK-11105, which has already been fixed in the not-yet-released Spark 1.6.0. EMR will upgrade to Spark 1.6.0 once it is released. ~ Jonathan On Thu, Nov 19, 2015 at 1:30 PM, Afshartous, Nick mailto:nafshart...@turbine.com>> wrote: Hi, On Spark 1.5 on EMR 4.1 the message below appears in stderr in the Yarn UI. ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. I do see that there is /usr/lib/spark/conf/log4j.properties Can someone please advise on how to setup log4j properly. Thanks, -- Nick Notice: This communication is for the intended recipient(s) only and may contain confidential, proprietary, legally protected or privileged information of Turbine, Inc. If you are not the intended recipient(s), please notify the sender at once and delete this communication. Unauthorized use of the information in this communication is strictly prohibited and may be unlawful. For those recipients under contract with Turbine, Inc., the information in this communication is subject to the terms and conditions of any applicable contracts or agreements. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org<mailto:user-unsubscr...@spark.apache.org> For additional commands, e-mail: user-h...@spark.apache.org<mailto:user-h...@spark.apache.org> Notice: This communication is for the intended recipient(s) only and may contain confidential, proprietary, legally protected or privileged information of Turbine, Inc. If you are not the intended recipient(s), please notify the sender at once and delete this communication. Unauthorized use of the information in this communication is strictly prohibited and may be unlawful. For those recipients under contract with Turbine, Inc., the information in this communication is subject to the terms and conditions of any applicable contracts or agreements. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Configuring Log4J (Spark 1.5 on EMR 4.1)
This file only exists on the master and not the slave nodes, so you are probably running into https://issues.apache.org/jira/browse/SPARK-11105, which has already been fixed in the not-yet-released Spark 1.6.0. EMR will upgrade to Spark 1.6.0 once it is released. ~ Jonathan On Thu, Nov 19, 2015 at 1:30 PM, Afshartous, Nick wrote: > > Hi, > > On Spark 1.5 on EMR 4.1 the message below appears in stderr in the Yarn UI. > > ERROR StatusLogger No log4j2 configuration file found. Using default > configuration: logging only errors to the console. > > I do see that there is > >/usr/lib/spark/conf/log4j.properties > > Can someone please advise on how to setup log4j properly. > > Thanks, > -- > Nick > > Notice: This communication is for the intended recipient(s) only and may > contain confidential, proprietary, legally protected or privileged > information of Turbine, Inc. If you are not the intended recipient(s), > please notify the sender at once and delete this communication. > Unauthorized use of the information in this communication is strictly > prohibited and may be unlawful. For those recipients under contract with > Turbine, Inc., the information in this communication is subject to the > terms and conditions of any applicable contracts or agreements. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
Configuring Log4J (Spark 1.5 on EMR 4.1)
Hi, On Spark 1.5 on EMR 4.1 the message below appears in stderr in the Yarn UI. ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. I do see that there is /usr/lib/spark/conf/log4j.properties Can someone please advise on how to setup log4j properly. Thanks, -- Nick Notice: This communication is for the intended recipient(s) only and may contain confidential, proprietary, legally protected or privileged information of Turbine, Inc. If you are not the intended recipient(s), please notify the sender at once and delete this communication. Unauthorized use of the information in this communication is strictly prohibited and may be unlawful. For those recipients under contract with Turbine, Inc., the information in this communication is subject to the terms and conditions of any applicable contracts or agreements. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
