[jira] [Created] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

Thomas Graves created SPARK-11701: - Summary: YARN - dynamic allocation and speculation active task accounting wrong Key: SPARK-11701 URL: https://issues.apache.org/jira/browse/SPARK-11701 Project: Spa

[jira] [Commented] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15002505#comment-15002505 ] Thomas Graves commented on SPARK-11701: --- I'm not exactly sure if this is same issue

[jira] [Commented] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003977#comment-15003977 ] Thomas Graves commented on SPARK-11701: --- [~jerryshao] ARe you referring to my last

[jira] [Commented] (SPARK-4117) Spark on Yarn handle AM being told command from RM

[ https://issues.apache.org/jira/browse/SPARK-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024603#comment-15024603 ] Thomas Graves commented on SPARK-4117: -- [~devaraj.k] Where is Spark handling the Ap

[jira] [Updated] (SPARK-10911) Executors should System.exit on clean shutdown

[ https://issues.apache.org/jira/browse/SPARK-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-10911: -- Assignee: Zhuo Liu > Executors should System.exit on clean shutdown > -

[jira] [Commented] (SPARK-4117) Spark on Yarn handle AM being told command from RM

[ https://issues.apache.org/jira/browse/SPARK-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15031843#comment-15031843 ] Thomas Graves commented on SPARK-4117: -- [~devaraj.k] thanks for explaining. Sounds

[jira] [Commented] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034042#comment-15034042 ] Thomas Graves commented on SPARK-11701: --- tested on latest 1.6 branch and I am no lo

[jira] [Commented] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034625#comment-15034625 ] Thomas Graves commented on SPARK-11701: --- So it looks like this is a race condition.

[jira] [Assigned] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-11701: - Assignee: Thomas Graves > YARN - dynamic allocation and speculation active task accounti

[jira] [Commented] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034749#comment-15034749 ] Thomas Graves commented on SPARK-11701: --- The same issue existing with dynamic alloc

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036341#comment-15036341 ] Thomas Graves commented on SPARK-1239: -- I have another user hitting this also. The a

[jira] [Commented] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036366#comment-15036366 ] Thomas Graves commented on SPARK-11701: --- this looks like a dup of SPARK-9038 > YAR

[jira] [Commented] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15036395#comment-15036395 ] Thomas Graves commented on SPARK-11701: --- Also seems related to https://github.com/a

[jira] [Updated] (SPARK-10873) can't sort columns on history page

[ https://issues.apache.org/jira/browse/SPARK-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-10873: -- Assignee: Zhuo Liu > can't sort columns on history page > -- >

[jira] [Commented] (SPARK-10911) Executors should System.exit on clean shutdown

[ https://issues.apache.org/jira/browse/SPARK-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15038418#comment-15038418 ] Thomas Graves commented on SPARK-10911: --- there are other cases this can happen. We'

[jira] [Commented] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15038603#comment-15038603 ] Thomas Graves commented on SPARK-11701: --- Note after investigating this some more p

[jira] [Created] (SPARK-12384) Allow -Xms to be set differently then -Xmx

Thomas Graves created SPARK-12384: - Summary: Allow -Xms to be set differently then -Xmx Key: SPARK-12384 URL: https://issues.apache.org/jira/browse/SPARK-12384 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-12384) Allow -Xms to be set differently then -Xmx

[ https://issues.apache.org/jira/browse/SPARK-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062068#comment-15062068 ] Thomas Graves commented on SPARK-12384: --- Yes and that is why this change it meant f

[jira] [Commented] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15064200#comment-15064200 ] Thomas Graves commented on SPARK-11701: --- I ran into another instance of this and it

[jira] [Assigned] (SPARK-12654) sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop

[ https://issues.apache.org/jira/browse/SPARK-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-12654: - Assignee: Thomas Graves > sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on se

[jira] [Created] (SPARK-12654) sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop

Thomas Graves created SPARK-12654: - Summary: sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop Key: SPARK-12654 URL: https://issues.apache.org/jira/browse/SPARK-12654 Project:

[jira] [Commented] (SPARK-12654) sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop

[ https://issues.apache.org/jira/browse/SPARK-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083643#comment-15083643 ] Thomas Graves commented on SPARK-12654: --- So the bug here is that WholeTextFileRDD.g

[jira] [Commented] (SPARK-12654) sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop

[ https://issues.apache.org/jira/browse/SPARK-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083663#comment-15083663 ] Thomas Graves commented on SPARK-12654: --- It looks like the version of getConf in Ha

[jira] [Commented] (SPARK-1291) Link the spark UI to RM ui in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025244#comment-14025244 ] Thomas Graves commented on SPARK-1291: -- https://github.com/apache/spark/pull/1002 >

[jira] [Comment Edited] (SPARK-1291) Link the spark UI to RM ui in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025244#comment-14025244 ] Thomas Graves edited comment on SPARK-1291 at 6/9/14 2:57 PM: --

[jira] [Updated] (SPARK-1720) use LD_LIBRARY_PATH instead of -Djava.library.path

[ https://issues.apache.org/jira/browse/SPARK-1720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1720: - Description: I think it would be better to use LD_LIBRARY_PATH rather then -Djava.library.path.

[jira] [Resolved] (SPARK-1522) YARN ClientBase will throw a NPE if there is no YARN application specific classpath.

[ https://issues.apache.org/jira/browse/SPARK-1522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1522. -- Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Bernardo Gomez Palacio > YARN

[jira] [Resolved] (SPARK-1978) In some cases, spark-yarn does not automatically restart the failed container

[ https://issues.apache.org/jira/browse/SPARK-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1978. -- Resolution: Fixed > In some cases, spark-yarn does not automatically restart the failed contain

[jira] [Resolved] (SPARK-1639) Some tidying of Spark on YARN code

[ https://issues.apache.org/jira/browse/SPARK-1639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1639. -- Resolution: Fixed Fix Version/s: 1.1.0 > Some tidying of Spark on YARN code > --

[jira] [Created] (SPARK-2111) pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1

Thomas Graves created SPARK-2111: Summary: pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1 Key: SPARK-2111 URL: https://issues.apache.org/jira/browse/SPARK-2111 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-1740) Pyspark cancellation kills unrelated pyspark workers

[ https://issues.apache.org/jira/browse/SPARK-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029416#comment-14029416 ] Thomas Graves commented on SPARK-1740: -- On yarn when I run pyspark it kills the execu

[jira] [Updated] (SPARK-1740) Pyspark cancellation kills unrelated pyspark workers

[ https://issues.apache.org/jira/browse/SPARK-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1740: - Priority: Critical (was: Major) > Pyspark cancellation kills unrelated pyspark workers > ---

[jira] [Commented] (SPARK-1718) pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat

[ https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029422#comment-14029422 ] Thomas Graves commented on SPARK-1718: -- So this actually appears to be an issue becau

[jira] [Commented] (SPARK-2080) Yarn: history UI link missing, wrong reported user

[ https://issues.apache.org/jira/browse/SPARK-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14029828#comment-14029828 ] Thomas Graves commented on SPARK-2080: -- Note that this is only fixing it for running

[jira] [Resolved] (SPARK-2080) Yarn: history UI link missing, wrong reported user

[ https://issues.apache.org/jira/browse/SPARK-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2080. -- Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 > Yarn: history UI link

[jira] [Created] (SPARK-2139) spark.yarn.dist.* configs are not documented

Thomas Graves created SPARK-2139: Summary: spark.yarn.dist.* configs are not documented Key: SPARK-2139 URL: https://issues.apache.org/jira/browse/SPARK-2139 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2051) spark.yarn.dist.* configs are not supported in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2051: - Issue Type: Improvement (was: Bug) > spark.yarn.dist.* configs are not supported in yarn-cluster

[jira] [Updated] (SPARK-2051) spark.yarn.dist.* configs are not supported in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2051: - Summary: spark.yarn.dist.* configs are not supported in yarn-cluster mode (was: In yarn.ClientBa

[jira] [Resolved] (SPARK-2139) spark.yarn.dist.* configs are not documented

[ https://issues.apache.org/jira/browse/SPARK-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2139. -- Resolution: Duplicate > spark.yarn.dist.* configs are not documented >

[jira] [Commented] (SPARK-2051) spark.yarn.dist.* configs are not supported in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14030612#comment-14030612 ] Thomas Graves commented on SPARK-2051: -- Note that the configs weren't originally inte

[jira] [Updated] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

[ https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2140: - Fix Version/s: 1.1.0 1.0.1 > yarn stable client doesn't properly handle MEMORY

[jira] [Created] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

Thomas Graves created SPARK-2140: Summary: yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM Key: SPARK-2140 URL: https://issues.apache.org/jira/browse/SPARK-2140 Project: Spark

[jira] [Resolved] (SPARK-1951) spark on yarn can't start

[ https://issues.apache.org/jira/browse/SPARK-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1951. -- Resolution: Invalid As pointed by Sean out spark-submit changed the default scheme to be file:/

[jira] [Comment Edited] (SPARK-1951) spark on yarn can't start

[ https://issues.apache.org/jira/browse/SPARK-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14030900#comment-14030900 ] Thomas Graves edited comment on SPARK-1951 at 6/13/14 5:55 PM: -

[jira] [Resolved] (SPARK-1930) The Container is running beyond physical memory limits, so as to be killed.

[ https://issues.apache.org/jira/browse/SPARK-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1930. -- Resolution: Fixed Fix Version/s: 1.1.0 > The Container is running beyond physical memory

[jira] [Created] (SPARK-2165) spark on yarn: add support for setting maxAppAttempts in the ApplicationSubmissionContext

Thomas Graves created SPARK-2165: Summary: spark on yarn: add support for setting maxAppAttempts in the ApplicationSubmissionContext Key: SPARK-2165 URL: https://issues.apache.org/jira/browse/SPARK-2165

[jira] [Updated] (SPARK-1907) spark-submit: add exec at the end of the script

[ https://issues.apache.org/jira/browse/SPARK-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1907: - Assignee: Colin Patrick McCabe > spark-submit: add exec at the end of the script > --

[jira] [Created] (SPARK-2167) spark-submit should return exit code based on failure/success

Thomas Graves created SPARK-2167: Summary: spark-submit should return exit code based on failure/success Key: SPARK-2167 URL: https://issues.apache.org/jira/browse/SPARK-2167 Project: Spark

[jira] [Resolved] (SPARK-2051) spark.yarn.dist.* configs are not supported in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2051. -- Resolution: Fixed Fix Version/s: 1.1.0 > spark.yarn.dist.* configs are not supported in

[jira] [Created] (SPARK-2223) Building and running tests with maven is extremely slow

Thomas Graves created SPARK-2223: Summary: Building and running tests with maven is extremely slow Key: SPARK-2223 URL: https://issues.apache.org/jira/browse/SPARK-2223 Project: Spark Issue T

[jira] [Created] (SPARK-2224) allow running tests for one sub module

Thomas Graves created SPARK-2224: Summary: allow running tests for one sub module Key: SPARK-2224 URL: https://issues.apache.org/jira/browse/SPARK-2224 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038971#comment-14038971 ] Thomas Graves commented on SPARK-2224: -- Thanks, wasn't away of -pl.. doesn't seem to

[jira] [Commented] (SPARK-2223) Building and running tests with maven is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038976#comment-14038976 ] Thomas Graves commented on SPARK-2223: -- Some of our nightly builds are taking 50 minu

[jira] [Commented] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038981#comment-14038981 ] Thomas Graves commented on SPARK-2224: -- Oh, I bet only ran with package and not insta

[jira] [Commented] (SPARK-2223) Building and running tests with maven is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039076#comment-14039076 ] Thomas Graves commented on SPARK-2223: -- [~srowen] do you know how long it takes you t

[jira] [Resolved] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2224. -- Resolution: Invalid > allow running tests for one sub module >

[jira] [Commented] (SPARK-2224) allow running tests for one sub module

[ https://issues.apache.org/jira/browse/SPARK-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039081#comment-14039081 ] Thomas Graves commented on SPARK-2224: -- well that worked as in it actually ran in the

[jira] [Assigned] (SPARK-1528) Spark on Yarn: Add option for user to specify additional namenodes to get tokens from

[ https://issues.apache.org/jira/browse/SPARK-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1528: Assignee: Thomas Graves > Spark on Yarn: Add option for user to specify additional namenode

[jira] [Commented] (SPARK-1528) Spark on Yarn: Add option for user to specify additional namenodes to get tokens from

[ https://issues.apache.org/jira/browse/SPARK-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039186#comment-14039186 ] Thomas Graves commented on SPARK-1528: -- https://github.com/apache/spark/pull/1159 >

[jira] [Commented] (SPARK-1395) Cannot launch jobs on Yarn cluster with "local:" scheme in SPARK_JAR

[ https://issues.apache.org/jira/browse/SPARK-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040782#comment-14040782 ] Thomas Graves commented on SPARK-1395: -- https://github.com/apache/spark/pull/560 > C

[jira] [Resolved] (SPARK-1395) Cannot launch jobs on Yarn cluster with "local:" scheme in SPARK_JAR

[ https://issues.apache.org/jira/browse/SPARK-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1395. -- Resolution: Fixed Fix Version/s: (was: 1.0.0) 1.1.0 > Cannot laun

[jira] [Assigned] (SPARK-1890) add modify acls to the web UI for the "kill" button

[ https://issues.apache.org/jira/browse/SPARK-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1890: Assignee: Thomas Graves > add modify acls to the web UI for the "kill" button > ---

[jira] [Assigned] (SPARK-1891) Add admin acls to the Web UI

[ https://issues.apache.org/jira/browse/SPARK-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1891: Assignee: Thomas Graves > Add admin acls to the Web UI > > >

[jira] [Commented] (SPARK-1891) Add admin acls to the Web UI

[ https://issues.apache.org/jira/browse/SPARK-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042516#comment-14042516 ] Thomas Graves commented on SPARK-1891: -- https://github.com/apache/spark/pull/1196 com

[jira] [Commented] (SPARK-1890) add modify acls to the web UI for the "kill" button

[ https://issues.apache.org/jira/browse/SPARK-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042517#comment-14042517 ] Thomas Graves commented on SPARK-1890: -- https://github.com/apache/spark/pull/1196 >

[jira] [Resolved] (SPARK-1718) pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat

[ https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1718. -- Resolution: Duplicate > pyspark doesn't work with assembly jar containing over 65536 files/dirs

[jira] [Resolved] (SPARK-2289) Remove use of spark.worker.instances

[ https://issues.apache.org/jira/browse/SPARK-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2289. -- Resolution: Fixed > Remove use of spark.worker.instances >

[jira] [Commented] (SPARK-2289) Remove use of spark.worker.instances

[ https://issues.apache.org/jira/browse/SPARK-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044643#comment-14044643 ] Thomas Graves commented on SPARK-2289: -- https://github.com/apache/spark/pull/1214 >

[jira] [Created] (SPARK-2289) Remove use of spark.worker.instances

Thomas Graves created SPARK-2289: Summary: Remove use of spark.worker.instances Key: SPARK-2289 URL: https://issues.apache.org/jira/browse/SPARK-2289 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2167) spark-submit should return exit code based on failure/success

[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048890#comment-14048890 ] Thomas Graves commented on SPARK-2167: -- There might be multiple issues. I know there

[jira] [Resolved] (SPARK-2400) config spark.yarn.max.executor.failures is not explained accurately

[ https://issues.apache.org/jira/browse/SPARK-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2400. -- Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Chen Chao Target V

[jira] [Commented] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058881#comment-14058881 ] Thomas Graves commented on SPARK-2444: -- Can you be more specific, what are you lookin

[jira] [Commented] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058930#comment-14058930 ] Thomas Graves commented on SPARK-2444: -- Maybe I'm missing something in what you are a

[jira] [Resolved] (SPARK-1946) Submit stage after executors have been registered

[ https://issues.apache.org/jira/browse/SPARK-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1946. -- Resolution: Fixed Fix Version/s: 1.1.0 > Submit stage after executors have been register

[jira] [Resolved] (SPARK-1453) Improve the way Spark on Yarn waits for executors before starting

[ https://issues.apache.org/jira/browse/SPARK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1453. -- Resolution: Duplicate duplicate of https://issues.apache.org/jira/browse/SPARK-1946 > Improve

[jira] [Updated] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1576: - Fix Version/s: (was: 1.0.0) > Passing of JAVA_OPTS to YARN on command line >

[jira] [Updated] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1576: - Affects Version/s: (was: 1.0.0) > Passing of JAVA_OPTS to YARN on command line >

[jira] [Resolved] (SPARK-1291) Link the spark UI to RM ui in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1291. -- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 > Link the spar

[jira] [Resolved] (SPARK-2111) pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1

[ https://issues.apache.org/jira/browse/SPARK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2111. -- Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 > pyspark errors when SP

[jira] [Updated] (SPARK-2111) pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1

[ https://issues.apache.org/jira/browse/SPARK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2111: - Assignee: Prashant Sharma > pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1 > --

[jira] [Created] (SPARK-2541) Standalone mode can't access secure HDFS anymore

Thomas Graves created SPARK-2541: Summary: Standalone mode can't access secure HDFS anymore Key: SPARK-2541 URL: https://issues.apache.org/jira/browse/SPARK-2541 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14066872#comment-14066872 ] Thomas Graves commented on SPARK-1680: -- so what do we want to do here. Do we want to

[jira] [Commented] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068528#comment-14068528 ] Thomas Graves commented on SPARK-2604: -- Just to clarify, are you referring to the max

[jira] [Assigned] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-2604: Assignee: Thomas Graves > Spark Application hangs on yarn in edge case scenario of executor

[jira] [Updated] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2604: - Assignee: (was: Thomas Graves) > Spark Application hangs on yarn in edge case scenario of exe

[jira] [Commented] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068531#comment-14068531 ] Thomas Graves commented on SPARK-2604: -- Also note that it shouldn't hang, it should f

[jira] [Resolved] (SPARK-1831) add the security guide to the "More" drop down menu

[ https://issues.apache.org/jira/browse/SPARK-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1831. -- Resolution: Invalid looks like this is already got fixed. > add the security guide to the "Mor

[jira] [Commented] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068581#comment-14068581 ] Thomas Graves commented on SPARK-1680: -- https://github.com/apache/spark/pull/1512 >

[jira] [Assigned] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1680: Assignee: Thomas Graves > Clean up use of setExecutorEnvs in SparkConf > -

[jira] [Resolved] (SPARK-1707) Remove 3 second sleep before starting app on YARN

[ https://issues.apache.org/jira/browse/SPARK-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1707. -- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 > Remove 3 seco

[jira] [Resolved] (SPARK-2606) In some cases, pages display incorrect in spark UI

[ https://issues.apache.org/jira/browse/SPARK-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2606. -- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 > In some cases

[jira] [Commented] (SPARK-2037) yarn client mode doesn't support spark.yarn.max.executor.failures

[ https://issues.apache.org/jira/browse/SPARK-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14071261#comment-14071261 ] Thomas Graves commented on SPARK-2037: -- https://github.com/apache/spark/pull/1180 >

[jira] [Resolved] (SPARK-2577) File upload to viewfs is broken due to mount point resolution

[ https://issues.apache.org/jira/browse/SPARK-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2577. -- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 > File upload t

[jira] [Commented] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14071786#comment-14071786 ] Thomas Graves commented on SPARK-2604: -- Yes we should be adding the overhead in at t

[jira] [Updated] (SPARK-2150) Provide direct link to finished application UI in yarn resource manager UI

[ https://issues.apache.org/jira/browse/SPARK-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2150: - Assignee: Rahul Singhal > Provide direct link to finished application UI in yarn resource manager

[jira] [Resolved] (SPARK-2150) Provide direct link to finished application UI in yarn resource manager UI

[ https://issues.apache.org/jira/browse/SPARK-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2150. -- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 > Provide direc

[jira] [Commented] (SPARK-2668) Support log4j log to yarn container log directory

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073246#comment-14073246 ] Thomas Graves commented on SPARK-2668: -- Sorry I don't follow what you are saying here

[jira] [Commented] (SPARK-2668) Support log4j log to yarn container log directory

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073310#comment-14073310 ] Thomas Graves commented on SPARK-2668: -- Oh, I see you just want a variable to referen

[jira] [Resolved] (SPARK-2037) yarn client mode doesn't support spark.yarn.max.executor.failures

[ https://issues.apache.org/jira/browse/SPARK-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2037. -- Resolution: Fixed Fix Version/s: 1.1.0 > yarn client mode doesn't support spark.yarn.max

[jira] [Resolved] (SPARK-1528) Spark on Yarn: Add option for user to specify additional namenodes to get tokens from

[ https://issues.apache.org/jira/browse/SPARK-1528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1528. -- Resolution: Fixed Fix Version/s: 1.1.0 > Spark on Yarn: Add option for user to specify a

<    3   4   5   6   7   8   9   10   11   12   >