I am new to spark and we are developing a data science pipeline based on spark on ec2. So far we have been using python on spark standalone cluster. However, being a newbie I would like to know more about how can I do debugging (program level) from spark logs (is it stderr ?). I find it a bit difficult to debug since, spark itself has many messages there. Any ideas or suggestion regarding configuration change to facilitate this would be highly appreciated !!
-- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Log-analysis-tp6168.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
