[jira] [Commented] (SPARK-5966) Spark-submit deploy-mode incorrectly affecting submission when master = local[4]

2015-10-26 Thread Rick Hillegas (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974509#comment-14974509
 ] 

Rick Hillegas commented on SPARK-5966:
--

Should this issue be assigned to Kevin now, since he submitted the pull 
request? Thanks.

> Spark-submit deploy-mode incorrectly affecting submission when master = 
> local[4] 
> -
>
> Key: SPARK-5966
> URL: https://issues.apache.org/jira/browse/SPARK-5966
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Submit
>Affects Versions: 1.3.0
>Reporter: Tathagata Das
>Assignee: Andrew Or
>Priority: Critical
> Fix For: 1.5.3, 1.6.0
>
>
> {code}
> [tdas @ Zion spark] bin/spark-submit --master local[10]  --class 
> test.MemoryTest --driver-memory 1G 
> /Users/tdas/Projects/Others/simple-project/target/scala-2.10/simple-app-assembly-0.1-SNAPSHOT.jar
> JVM arguments: [-XX:MaxPermSize=128m, -Xms1G, -Xmx1G]
> App arguments:
> Usage: MemoryTest  
> [tdas @ Zion spark] bin/spark-submit --master local[10] --deploy-mode cluster 
>  --class test.MemoryTest --driver-memory 1G 
> /Users/tdas/Projects/Others/simple-project/target/scala-2.10/simple-app-assembly-0.1-SNAPSHOT.jar
> java.lang.ClassNotFoundException:
>   at java.lang.Class.forName0(Native Method)
>   at java.lang.Class.forName(Class.java:270)
>   at 
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:538)
>   at 
> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
>   at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)
>   at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
>   at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-5966) Spark-submit deploy-mode incorrectly affecting submission when master = local[4]

2015-10-22 Thread kevin yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968711#comment-14968711
 ] 

kevin yu commented on SPARK-5966:
-

Hello Tathagata & Andrew:
I have code a possible fix, and the error message will be like this.
$ ./bin/spark-submit --master local[10] --deploy-mode cluster 
examples/src/main/python/pi.py
Error: Cluster deploy mode is not compatible with master "local"
Run with --help for usage help or --verbose for debug output

Let me know if you have any comments. Otherwise, I am going to submit a PR 
shortly. Thanks.

Kevin


> Spark-submit deploy-mode incorrectly affecting submission when master = 
> local[4] 
> -
>
> Key: SPARK-5966
> URL: https://issues.apache.org/jira/browse/SPARK-5966
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Submit
>Affects Versions: 1.3.0
>Reporter: Tathagata Das
>Assignee: Andrew Or
>Priority: Critical
>
> {code}
> [tdas @ Zion spark] bin/spark-submit --master local[10]  --class 
> test.MemoryTest --driver-memory 1G 
> /Users/tdas/Projects/Others/simple-project/target/scala-2.10/simple-app-assembly-0.1-SNAPSHOT.jar
> JVM arguments: [-XX:MaxPermSize=128m, -Xms1G, -Xmx1G]
> App arguments:
> Usage: MemoryTest  
> [tdas @ Zion spark] bin/spark-submit --master local[10] --deploy-mode cluster 
>  --class test.MemoryTest --driver-memory 1G 
> /Users/tdas/Projects/Others/simple-project/target/scala-2.10/simple-app-assembly-0.1-SNAPSHOT.jar
> java.lang.ClassNotFoundException:
>   at java.lang.Class.forName0(Native Method)
>   at java.lang.Class.forName(Class.java:270)
>   at 
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:538)
>   at 
> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
>   at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)
>   at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
>   at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-5966) Spark-submit deploy-mode incorrectly affecting submission when master = local[4]

2015-10-22 Thread Apache Spark (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968739#comment-14968739
 ] 

Apache Spark commented on SPARK-5966:
-

User 'kevinyu98' has created a pull request for this issue:
https://github.com/apache/spark/pull/9220

> Spark-submit deploy-mode incorrectly affecting submission when master = 
> local[4] 
> -
>
> Key: SPARK-5966
> URL: https://issues.apache.org/jira/browse/SPARK-5966
> Project: Spark
>  Issue Type: Bug
>  Components: Spark Submit
>Affects Versions: 1.3.0
>Reporter: Tathagata Das
>Assignee: Andrew Or
>Priority: Critical
>
> {code}
> [tdas @ Zion spark] bin/spark-submit --master local[10]  --class 
> test.MemoryTest --driver-memory 1G 
> /Users/tdas/Projects/Others/simple-project/target/scala-2.10/simple-app-assembly-0.1-SNAPSHOT.jar
> JVM arguments: [-XX:MaxPermSize=128m, -Xms1G, -Xmx1G]
> App arguments:
> Usage: MemoryTest  
> [tdas @ Zion spark] bin/spark-submit --master local[10] --deploy-mode cluster 
>  --class test.MemoryTest --driver-memory 1G 
> /Users/tdas/Projects/Others/simple-project/target/scala-2.10/simple-app-assembly-0.1-SNAPSHOT.jar
> java.lang.ClassNotFoundException:
>   at java.lang.Class.forName0(Native Method)
>   at java.lang.Class.forName(Class.java:270)
>   at 
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:538)
>   at 
> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
>   at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)
>   at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
>   at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-5966) Spark-submit deploy-mode incorrectly affecting submission when master = local[4]

2015-05-20 Thread Andrew Or (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552923#comment-14552923
 ] 

Andrew Or commented on SPARK-5966:
--

You're not supposed to run with cluster deploy mode using a local master

 Spark-submit deploy-mode incorrectly affecting submission when master = 
 local[4] 
 -

 Key: SPARK-5966
 URL: https://issues.apache.org/jira/browse/SPARK-5966
 Project: Spark
  Issue Type: Bug
  Components: Spark Submit
Affects Versions: 1.3.0
Reporter: Tathagata Das
Assignee: Andrew Or
Priority: Critical

 {code}
 [tdas @ Zion spark] bin/spark-submit --master local[10]  --class 
 test.MemoryTest --driver-memory 1G 
 /Users/tdas/Projects/Others/simple-project/target/scala-2.10/simple-app-assembly-0.1-SNAPSHOT.jar
 JVM arguments: [-XX:MaxPermSize=128m, -Xms1G, -Xmx1G]
 App arguments:
 Usage: MemoryTest allocation size in MB interval in seconds
 [tdas @ Zion spark] bin/spark-submit --master local[10] --deploy-mode cluster 
  --class test.MemoryTest --driver-memory 1G 
 /Users/tdas/Projects/Others/simple-project/target/scala-2.10/simple-app-assembly-0.1-SNAPSHOT.jar
 java.lang.ClassNotFoundException:
   at java.lang.Class.forName0(Native Method)
   at java.lang.Class.forName(Class.java:270)
   at 
 org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:538)
   at 
 org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
   at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)
   at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
   at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-5966) Spark-submit deploy-mode incorrectly affecting submission when master = local[4]

2015-05-20 Thread Tathagata Das (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552988#comment-14552988
 ] 

Tathagata Das commented on SPARK-5966:
--

Yes, but its super non-intuitive error message when you accidentally do so. 
There should something like a IllegalArgumentException rather than 
ClassNotFoundException.

 Spark-submit deploy-mode incorrectly affecting submission when master = 
 local[4] 
 -

 Key: SPARK-5966
 URL: https://issues.apache.org/jira/browse/SPARK-5966
 Project: Spark
  Issue Type: Bug
  Components: Spark Submit
Affects Versions: 1.3.0
Reporter: Tathagata Das
Assignee: Andrew Or
Priority: Critical

 {code}
 [tdas @ Zion spark] bin/spark-submit --master local[10]  --class 
 test.MemoryTest --driver-memory 1G 
 /Users/tdas/Projects/Others/simple-project/target/scala-2.10/simple-app-assembly-0.1-SNAPSHOT.jar
 JVM arguments: [-XX:MaxPermSize=128m, -Xms1G, -Xmx1G]
 App arguments:
 Usage: MemoryTest allocation size in MB interval in seconds
 [tdas @ Zion spark] bin/spark-submit --master local[10] --deploy-mode cluster 
  --class test.MemoryTest --driver-memory 1G 
 /Users/tdas/Projects/Others/simple-project/target/scala-2.10/simple-app-assembly-0.1-SNAPSHOT.jar
 java.lang.ClassNotFoundException:
   at java.lang.Class.forName0(Native Method)
   at java.lang.Class.forName(Class.java:270)
   at 
 org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:538)
   at 
 org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
   at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)
   at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
   at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org