[jira] [Resolved] (SPARK-2289) Remove use of spark.worker.instances

[ https://issues.apache.org/jira/browse/SPARK-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2289. -- Resolution: Fixed Remove use of spark.worker.instances

[jira] [Commented] (SPARK-2289) Remove use of spark.worker.instances

[ https://issues.apache.org/jira/browse/SPARK-2289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044643#comment-14044643 ] Thomas Graves commented on SPARK-2289: -- https://github.com/apache/spark/pull/1214

[jira] [Created] (SPARK-2289) Remove use of spark.worker.instances

Thomas Graves created SPARK-2289: Summary: Remove use of spark.worker.instances Key: SPARK-2289 URL: https://issues.apache.org/jira/browse/SPARK-2289 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2167) spark-submit should return exit code based on failure/success

[ https://issues.apache.org/jira/browse/SPARK-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048890#comment-14048890 ] Thomas Graves commented on SPARK-2167: -- There might be multiple issues. I know there

[jira] [Resolved] (SPARK-2400) config spark.yarn.max.executor.failures is not explained accurately

[ https://issues.apache.org/jira/browse/SPARK-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2400. -- Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Chen Chao Target

[jira] [Commented] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14058930#comment-14058930 ] Thomas Graves commented on SPARK-2444: -- Maybe I'm missing something in what you are

[jira] [Resolved] (SPARK-1946) Submit stage after executors have been registered

[ https://issues.apache.org/jira/browse/SPARK-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1946. -- Resolution: Fixed Fix Version/s: 1.1.0 Submit stage after executors have been

[jira] [Resolved] (SPARK-1453) Improve the way Spark on Yarn waits for executors before starting

[ https://issues.apache.org/jira/browse/SPARK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1453. -- Resolution: Duplicate duplicate of https://issues.apache.org/jira/browse/SPARK-1946 Improve

[jira] [Updated] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1576: - Fix Version/s: (was: 1.0.0) Passing of JAVA_OPTS to YARN on command line

[jira] [Updated] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1576: - Affects Version/s: (was: 1.0.0) Passing of JAVA_OPTS to YARN on command line

[jira] [Resolved] (SPARK-1291) Link the spark UI to RM ui in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1291. -- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 Link the

[jira] [Resolved] (SPARK-2111) pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1

[ https://issues.apache.org/jira/browse/SPARK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2111. -- Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 pyspark errors when

[jira] [Updated] (SPARK-2111) pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1

[ https://issues.apache.org/jira/browse/SPARK-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2111: - Assignee: Prashant Sharma pyspark errors when SPARK_PRINT_LAUNCH_COMMAND=1

[jira] [Created] (SPARK-2541) Standalone mode can't access secure HDFS anymore

Thomas Graves created SPARK-2541: Summary: Standalone mode can't access secure HDFS anymore Key: SPARK-2541 URL: https://issues.apache.org/jira/browse/SPARK-2541 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14066872#comment-14066872 ] Thomas Graves commented on SPARK-1680: -- so what do we want to do here. Do we want to

[jira] [Commented] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068528#comment-14068528 ] Thomas Graves commented on SPARK-2604: -- Just to clarify, are you referring to the max

[jira] [Assigned] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-2604: Assignee: Thomas Graves Spark Application hangs on yarn in edge case scenario of executor

[jira] [Updated] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2604: - Assignee: (was: Thomas Graves) Spark Application hangs on yarn in edge case scenario of

[jira] [Commented] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068531#comment-14068531 ] Thomas Graves commented on SPARK-2604: -- Also note that it shouldn't hang, it should

[jira] [Resolved] (SPARK-1831) add the security guide to the More drop down menu

[ https://issues.apache.org/jira/browse/SPARK-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1831. -- Resolution: Invalid looks like this is already got fixed. add the security guide to the More

[jira] [Commented] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068581#comment-14068581 ] Thomas Graves commented on SPARK-1680: -- https://github.com/apache/spark/pull/1512

[jira] [Assigned] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1680: Assignee: Thomas Graves Clean up use of setExecutorEnvs in SparkConf

[jira] [Resolved] (SPARK-2606) In some cases, pages display incorrect in spark UI

[ https://issues.apache.org/jira/browse/SPARK-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2606. -- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 In some

[jira] [Commented] (SPARK-2037) yarn client mode doesn't support spark.yarn.max.executor.failures

[ https://issues.apache.org/jira/browse/SPARK-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14071261#comment-14071261 ] Thomas Graves commented on SPARK-2037: -- https://github.com/apache/spark/pull/1180

[jira] [Resolved] (SPARK-2577) File upload to viewfs is broken due to mount point resolution

[ https://issues.apache.org/jira/browse/SPARK-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2577. -- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 File upload

[jira] [Commented] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14071786#comment-14071786 ] Thomas Graves commented on SPARK-2604: -- Yes we should be adding the overhead in at

[jira] [Updated] (SPARK-2150) Provide direct link to finished application UI in yarn resource manager UI

[ https://issues.apache.org/jira/browse/SPARK-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2150: - Assignee: Rahul Singhal Provide direct link to finished application UI in yarn resource manager

[jira] [Commented] (SPARK-2668) Support log4j log to yarn container log directory

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073246#comment-14073246 ] Thomas Graves commented on SPARK-2668: -- Sorry I don't follow what you are saying

[jira] [Commented] (SPARK-2668) Support log4j log to yarn container log directory

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073310#comment-14073310 ] Thomas Graves commented on SPARK-2668: -- Oh, I see you just want a variable to

[jira] [Resolved] (SPARK-2037) yarn client mode doesn't support spark.yarn.max.executor.failures

[ https://issues.apache.org/jira/browse/SPARK-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2037. -- Resolution: Fixed Fix Version/s: 1.1.0 yarn client mode doesn't support

[jira] [Resolved] (SPARK-1890) add modify acls to the web UI for the kill button

[ https://issues.apache.org/jira/browse/SPARK-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1890. -- Resolution: Fixed add modify acls to the web UI for the kill button

[jira] [Resolved] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1680. -- Resolution: Fixed Clean up use of setExecutorEnvs in SparkConf

[jira] [Created] (SPARK-2930) clarify docs on using webhdfs with spark.yarn.access.namenodes

Thomas Graves created SPARK-2930: Summary: clarify docs on using webhdfs with spark.yarn.access.namenodes Key: SPARK-2930 URL: https://issues.apache.org/jira/browse/SPARK-2930 Project: Spark

[jira] [Commented] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092881#comment-14092881 ] Thomas Graves commented on SPARK-2089: -- Sandy, just wondering if you have any ETA on

[jira] [Commented] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

[ https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14094211#comment-14094211 ] Thomas Graves commented on SPARK-2140: -- The commented out code was sufficient at the

[jira] [Commented] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

[ https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095721#comment-14095721 ] Thomas Graves commented on SPARK-2140: -- ah it seems things have changed. Its now

[jira] [Created] (SPARK-3072) Yarn AM not always properly exiting after unregistering from RM

Thomas Graves created SPARK-3072: Summary: Yarn AM not always properly exiting after unregistering from RM Key: SPARK-3072 URL: https://issues.apache.org/jira/browse/SPARK-3072 Project: Spark

[jira] [Commented] (SPARK-3072) Yarn AM not always properly exiting after unregistering from RM

[ https://issues.apache.org/jira/browse/SPARK-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098705#comment-14098705 ] Thomas Graves commented on SPARK-3072: -- Note that in yarn-cluster mode the client

[jira] [Commented] (SPARK-3072) Yarn AM not always properly exiting after unregistering from RM

[ https://issues.apache.org/jira/browse/SPARK-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14101134#comment-14101134 ] Thomas Graves commented on SPARK-3072: -- Note that this also isn't an issue on hadoop

[jira] [Commented] (SPARK-3120) Local Dirs is not useful in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14102199#comment-14102199 ] Thomas Graves commented on SPARK-3120: -- Can you please clarify this? You are trying

[jira] [Resolved] (SPARK-3072) Yarn AM not always properly exiting after unregistering from RM

[ https://issues.apache.org/jira/browse/SPARK-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3072. -- Resolution: Fixed Fix Version/s: 1.1.0 Yarn AM not always properly exiting after

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14102504#comment-14102504 ] Thomas Graves commented on SPARK-3129: -- A couple of random thoughts on this for yarn.

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104077#comment-14104077 ] Thomas Graves commented on SPARK-3129: -- Yes that probably means using reflection. I

[jira] [Created] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

Thomas Graves created SPARK-3152: Summary: Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly Key: SPARK-3152 URL: https://issues.apache.org/jira/browse/SPARK-3152 Project:

[jira] [Commented] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

[ https://issues.apache.org/jira/browse/SPARK-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104430#comment-14104430 ] Thomas Graves commented on SPARK-3152: -- I'm hoping we can fix this in SPARK-2933 for

[jira] [Resolved] (SPARK-2742) The variable inputFormatInfo and inputFormatMap never used

[ https://issues.apache.org/jira/browse/SPARK-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2742. -- Resolution: Fixed Fix Version/s: 1.2.0 The variable inputFormatInfo and inputFormatMap

[jira] [Commented] (SPARK-2742) The variable inputFormatInfo and inputFormatMap never used

[ https://issues.apache.org/jira/browse/SPARK-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14106997#comment-14106997 ] Thomas Graves commented on SPARK-2742: -- https://github.com/apache/spark/pull/1614

[jira] [Commented] (SPARK-2541) Standalone mode can't access secure HDFS anymore

[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110730#comment-14110730 ] Thomas Graves commented on SPARK-2541: -- So I haven't had time to full investigate

[jira] [Updated] (SPARK-2541) Standalone mode can't access secure HDFS anymore

[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2541: - Attachment: SPARK-2541-partial.patch Standalone mode can't access secure HDFS anymore

[jira] [Comment Edited] (SPARK-2541) Standalone mode can't access secure HDFS anymore

[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110730#comment-14110730 ] Thomas Graves edited comment on SPARK-2541 at 8/26/14 2:11 PM:

[jira] [Resolved] (SPARK-2933) Cleanup unnecessary and duplicated code in Yarn module

[ https://issues.apache.org/jira/browse/SPARK-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2933. -- Resolution: Fixed Fix Version/s: 1.2.0 Cleanup unnecessary and duplicated code in Yarn

[jira] [Resolved] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

[ https://issues.apache.org/jira/browse/SPARK-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3152. -- Resolution: Duplicate Yarn AM cluster mode doesn't cleanup staging directory when it exits

[jira] [Commented] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

[ https://issues.apache.org/jira/browse/SPARK-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112377#comment-14112377 ] Thomas Graves commented on SPARK-3152: -- this was fixed by SPARK-2933 in master. I

[jira] [Updated] (SPARK-3260) Yarn - pass acls along with executor launch

[ https://issues.apache.org/jira/browse/SPARK-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3260: - Target Version/s: 1.1.1, 1.2.0 (was: 1.2.0) Yarn - pass acls along with executor launch

[jira] [Created] (SPARK-3260) Yarn - pass acls along with executor launch

Thomas Graves created SPARK-3260: Summary: Yarn - pass acls along with executor launch Key: SPARK-3260 URL: https://issues.apache.org/jira/browse/SPARK-3260 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3168) The ServletContextHandler of webui lacks a SessionManager

[ https://issues.apache.org/jira/browse/SPARK-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14118173#comment-14118173 ] Thomas Graves commented on SPARK-3168: -- Spark is using the ServletContextHandler.

[jira] [Created] (SPARK-3347) Spark alpha doesn't compile due to SPARK-2889

Thomas Graves created SPARK-3347: Summary: Spark alpha doesn't compile due to SPARK-2889 Key: SPARK-3347 URL: https://issues.apache.org/jira/browse/SPARK-3347 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-3347) Spark on yarn alpha doesn't compile due to SPARK-2889

[ https://issues.apache.org/jira/browse/SPARK-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3347. -- Resolution: Fixed Fix Version/s: 1.2.0 Spark on yarn alpha doesn't compile due to

[jira] [Created] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

Thomas Graves created SPARK-3351: Summary: Yarn YarnRMClientImpl.shutdown can be called before register - NPE Key: SPARK-3351 URL: https://issues.apache.org/jira/browse/SPARK-3351 Project: Spark

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14118574#comment-14118574 ] Thomas Graves commented on SPARK-3351: -- Note I was running on yarn alpha but I think

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14118616#comment-14118616 ] Thomas Graves commented on SPARK-3351: -- Note that this failure scenario also can

[jira] [Commented] (SPARK-2718) YARN does not handle spark configs with quotes or backslashes

[ https://issues.apache.org/jira/browse/SPARK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119854#comment-14119854 ] Thomas Graves commented on SPARK-2718: -- [~andrewor] the pr for this is closed, can

[jira] [Comment Edited] (SPARK-2718) YARN does not handle spark configs with quotes or backslashes

[ https://issues.apache.org/jira/browse/SPARK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119854#comment-14119854 ] Thomas Graves edited comment on SPARK-2718 at 9/3/14 1:37 PM: --

[jira] [Resolved] (SPARK-3187) Refactor and cleanup Yarn allocator code

[ https://issues.apache.org/jira/browse/SPARK-3187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3187. -- Resolution: Fixed Fix Version/s: 1.2.0 Refactor and cleanup Yarn allocator code

[jira] [Created] (SPARK-3375) spark on yarn alpha container allocation issues

Thomas Graves created SPARK-3375: Summary: spark on yarn alpha container allocation issues Key: SPARK-3375 URL: https://issues.apache.org/jira/browse/SPARK-3375 Project: Spark Issue Type:

[jira] [Updated] (SPARK-3375) spark on yarn container allocation issues

[ https://issues.apache.org/jira/browse/SPARK-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3375: - Summary: spark on yarn container allocation issues (was: spark on yarn alpha container

[jira] [Commented] (SPARK-3375) spark on yarn alpha container allocation issues

[ https://issues.apache.org/jira/browse/SPARK-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120361#comment-14120361 ] Thomas Graves commented on SPARK-3375: -- Ok so the real issue here is that we are

[jira] [Assigned] (SPARK-3375) spark on yarn container allocation issues

[ https://issues.apache.org/jira/browse/SPARK-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-3375: Assignee: Thomas Graves spark on yarn container allocation issues

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122064#comment-14122064 ] Thomas Graves commented on SPARK-3129: -- On yarn, it generates the secret

[jira] [Resolved] (SPARK-3260) Yarn - pass acls along with executor launch

[ https://issues.apache.org/jira/browse/SPARK-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3260. -- Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 (was: 1.1.1,

[jira] [Resolved] (SPARK-3375) spark on yarn container allocation issues

[ https://issues.apache.org/jira/browse/SPARK-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3375. -- Resolution: Fixed Fix Version/s: 1.2.0 spark on yarn container allocation issues

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14125981#comment-14125981 ] Thomas Graves commented on SPARK-3129: -- yes that should be enough. Prevent data

[jira] [Created] (SPARK-3456) YarnAllocator can lose container requests to RM

Thomas Graves created SPARK-3456: Summary: YarnAllocator can lose container requests to RM Key: SPARK-3456 URL: https://issues.apache.org/jira/browse/SPARK-3456 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3456) YarnAllocator can lose container requests to RM

[ https://issues.apache.org/jira/browse/SPARK-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127088#comment-14127088 ] Thomas Graves commented on SPARK-3456: -- Note this is only a problem on yarn alpha

[jira] [Commented] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127188#comment-14127188 ] Thomas Graves commented on SPARK-3174: -- Since you mention the graceful decommission

[jira] [Comment Edited] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127188#comment-14127188 ] Thomas Graves edited comment on SPARK-3174 at 9/9/14 4:51 PM: --

[jira] [Commented] (SPARK-3445) Deprecate and later remove YARN alpha support

[ https://issues.apache.org/jira/browse/SPARK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127318#comment-14127318 ] Thomas Graves commented on SPARK-3445: -- We are in progress of moving and the timeline

[jira] [Commented] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127446#comment-14127446 ] Thomas Graves commented on SPARK-3174: -- {quote} Yeah so how about we create a

[jira] [Resolved] (SPARK-3286) Cannot view ApplicationMaster UI when Yarn’s url scheme is https

[ https://issues.apache.org/jira/browse/SPARK-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3286. -- Resolution: Fixed Fix Version/s: 1.2.0 Cannot view ApplicationMaster UI when Yarn’s url

[jira] [Created] (SPARK-3475) dev/merge_spark_pr.py fails on mac

Thomas Graves created SPARK-3475: Summary: dev/merge_spark_pr.py fails on mac Key: SPARK-3475 URL: https://issues.apache.org/jira/browse/SPARK-3475 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-3477) Clean up code in Yarn Client / ClientBase

[ https://issues.apache.org/jira/browse/SPARK-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3477: - Description: With the addition of new features and supporting multiple versions of yarn the code

[jira] [Resolved] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

[ https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2140. -- Resolution: Fixed Fix Version/s: (was: 1.0.1) (was:

[jira] [Updated] (SPARK-3374) Spark on Yarn config cleanup

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3374: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-3492 Spark on Yarn config cleanup

[jira] [Resolved] (SPARK-1891) Add admin acls to the Web UI

[ https://issues.apache.org/jira/browse/SPARK-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1891. -- Resolution: Fixed Add admin acls to the Web UI

[jira] [Resolved] (SPARK-2558) Mention --queue argument in YARN documentation

[ https://issues.apache.org/jira/browse/SPARK-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2558. -- Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 (was: 1.1.0,

[jira] [Created] (SPARK-3508) annotate the Spark configs to indicate which ones are meant for the end user

Thomas Graves created SPARK-3508: Summary: annotate the Spark configs to indicate which ones are meant for the end user Key: SPARK-3508 URL: https://issues.apache.org/jira/browse/SPARK-3508 Project:

[jira] [Resolved] (SPARK-3014) Log a more informative messages in a couple failure scenarios

[ https://issues.apache.org/jira/browse/SPARK-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3014. -- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Sandy Ryza Log a more

[jira] [Resolved] (SPARK-3456) YarnAllocator can lose container requests to RM

[ https://issues.apache.org/jira/browse/SPARK-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3456. -- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Thomas Graves YarnAllocator

[jira] [Resolved] (SPARK-3410) The priority of shutdownhook for ApplicationMaster should not be integer literal

[ https://issues.apache.org/jira/browse/SPARK-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3410. -- Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Kousuke Saruta The priority

[jira] [Resolved] (SPARK-3177) Yarn-alpha ClientBaseSuite Unit test failed

[ https://issues.apache.org/jira/browse/SPARK-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3177. -- Resolution: Fixed Fix Version/s: (was: 1.1.1) 1.2.0 Yarn-alpha

[jira] [Resolved] (SPARK-3557) Yarn client config prioritization is backwards

[ https://issues.apache.org/jira/browse/SPARK-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3557. -- Resolution: Duplicate Yarn client config prioritization is backwards

[jira] [Commented] (SPARK-3557) Yarn client config prioritization is backwards

[ https://issues.apache.org/jira/browse/SPARK-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138993#comment-14138993 ] Thomas Graves commented on SPARK-3557: -- This is a dup of SPARK-2872, although this

[jira] [Commented] (SPARK-2872) Fix conflict between code and doc in YarnClientSchedulerBackend

[ https://issues.apache.org/jira/browse/SPARK-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138996#comment-14138996 ] Thomas Graves commented on SPARK-2872: -- adding description from spark-3557 as it

[jira] [Created] (SPARK-3596) Support changing the yarn client monitor interval

Thomas Graves created SPARK-3596: Summary: Support changing the yarn client monitor interval Key: SPARK-3596 URL: https://issues.apache.org/jira/browse/SPARK-3596 Project: Spark Issue Type:

[jira] [Created] (SPARK-3607) ConnectionManager threads.max configs on the thread pools don't work

Thomas Graves created SPARK-3607: Summary: ConnectionManager threads.max configs on the thread pools don't work Key: SPARK-3607 URL: https://issues.apache.org/jira/browse/SPARK-3607 Project: Spark

[jira] [Created] (SPARK-3627) spark on yarn reports success even though job fails

Thomas Graves created SPARK-3627: Summary: spark on yarn reports success even though job fails Key: SPARK-3627 URL: https://issues.apache.org/jira/browse/SPARK-3627 Project: Spark Issue

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143438#comment-14143438 ] Thomas Graves commented on SPARK-3627: -- We could make this a separate issue, but I've

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143523#comment-14143523 ] Thomas Graves commented on SPARK-3627: -- this might be the same as SPARK-3293 spark

[jira] [Commented] (SPARK-2848) Shade Guava in Spark deliverables

[ https://issues.apache.org/jira/browse/SPARK-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143877#comment-14143877 ] Thomas Graves commented on SPARK-2848: -- [~vanzin] [~pwendell] I think the fix version

[jira] [Resolved] (SPARK-2668) Add variable of yarn log directory for reference from the log4j configuration

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2668. -- Resolution: Fixed Fix Version/s: 1.2.0 Add variable of yarn log directory for reference

<    1   2   3   4   5   6   7   8   9   10   >