[ https://issues.apache.org/jira/browse/SPARK-4989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260667#comment-14260667 ]
Zhang, Liye commented on SPARK-4989: ------------------------------------ Assume that we set *spark.eventLog.dir=hdfs://host:port/user/username/eventLog0, hdfs://host:port/user/username/eventLog1*, when starting the application, application will fail due to the exception, like following: {code} Exception in thread "main" java.net.URISyntaxException: Illegal character in path at index 41: hdfs//host:port/user/username/eventLog0, hdfs://host:port/user/username/eventLog1 at java.net.URI$Parser.fail(URI.java:2829) at java.net.URI$Parser.checkChars(URI.java:3002) at java.net.URI$Parser.parseHierarchical(URI.java:3086) at java.net.URI$Parser.parse(URI.java:3034) at java.net.URI.<init>(URI.java:595) at org.apache.spark.scheduler.EventLoggingListener.<init>(EventLoggingListener.scala:64) at org.apache.spark.SparkContext.<init>(SparkContext.scala:357) at org.apache.spark.SparkContext.<init>(SparkContext.scala:136) at com.intel.benchmark.wordcount.Driver$.main(Driver.scala:44) at com.intel.benchmark.wordcount.Driver.main(Driver.scala) {code} This will cause the *Master* to remove the application and rebuild the sparkUI. In which will also throw the exception to *Master Actor*, which will lead to *Master*'s restart. The exception will be like the following: {code} 14/12/30 09:48:18 ERROR OneForOneStrategy: Illegal character in path at index 41: hdfs://sr148:9000/user/liyezhan/eventLog, hdfs://sr148:9000/user/liyezhan/eventLog1 java.net.URISyntaxException: Illegal character in path at index 41: hdfs//host:port/user/username/eventLog0, hdfs://host:port/user/username/eventLog1 at java.net.URI$Parser.fail(URI.java:2829) at java.net.URI$Parser.checkChars(URI.java:3002) at java.net.URI$Parser.parseHierarchical(URI.java:3086) at java.net.URI$Parser.parse(URI.java:3034) at java.net.URI.<init>(URI.java:595) at org.apache.spark.util.Utils$.resolveURI(Utils.scala:1561) at org.apache.spark.scheduler.EventLoggingListener$.getLogPath(EventLoggingListener.scala:262) at org.apache.spark.deploy.master.Master$$anonfun$11.apply(Master.scala:724) at org.apache.spark.deploy.master.Master$$anonfun$11.apply(Master.scala:724) at scala.Option.map(Option.scala:145) at org.apache.spark.deploy.master.Master.rebuildSparkUI(Master.scala:724) at org.apache.spark.deploy.master.Master.removeApplication(Master.scala:695) at org.apache.spark.deploy.master.Master.finishApplication(Master.scala:673) at org.apache.spark.deploy.master.Master$$anonfun$receiveWithLogging$1$$anonfun$applyOrElse$29.apply(Master.scala:419) at org.apache.spark.deploy.master.Master$$anonfun$receiveWithLogging$1$$anonfun$applyOrElse$29.apply(Master.scala:419) at scala.Option.foreach(Option.scala:236) at org.apache.spark.deploy.master.Master$$anonfun$receiveWithLogging$1.applyOrElse(Master.scala:419) at scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33) at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33) at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25) at org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:53) at org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:42) at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118) at org.apache.spark.util.ActorLogReceive$$anon$1.applyOrElse(ActorLogReceive.scala:42) at akka.actor.Actor$class.aroundReceive(Actor.scala:465) at org.apache.spark.deploy.master.Master.aroundReceive(Master.scala:51) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) at akka.actor.ActorCell.invoke(ActorCell.scala:487) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238) at akka.dispatch.Mailbox.run(Mailbox.scala:220) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) {code} > wrong application configuration cause cluster down in standalone mode > --------------------------------------------------------------------- > > Key: SPARK-4989 > URL: https://issues.apache.org/jira/browse/SPARK-4989 > Project: Spark > Issue Type: Bug > Components: Deploy, Spark Core > Affects Versions: 1.0.0, 1.1.0, 1.2.0 > Reporter: Zhang, Liye > Priority: Critical > > when enabling eventlog in standalone mode, if give the wrong configuration, > the standalone cluster will down (cause master restart, lose connection with > workers). > How to reproduce: just give an invalid value to "spark.eventLog.dir", for > example: *spark.eventLog.dir=hdfs://tmp/logdir1, hdfs://tmp/logdir2*. This > will throw illegalArgumentException, which will cause the *Master* restart. > And the whole cluster is not available. > This is not acceptable that cluster is crashed by one application's wrong > setting. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org