[jira] [Commented] (SPARK-13775) history server sort by completed time by default

[ https://issues.apache.org/jira/browse/SPARK-13775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187707#comment-15187707 ] Thomas Graves commented on SPARK-13775: --- As I stated in the PR, I merged that in because it was

[jira] [Commented] (SPARK-13775) history server sort by completed time by default

[ https://issues.apache.org/jira/browse/SPARK-13775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187820#comment-15187820 ] Thomas Graves commented on SPARK-13775: --- Note those were basically rhetorical questions. You are

[jira] [Updated] (SPARK-13642) Properly handle signal kill of ApplicationMaster

[ https://issues.apache.org/jira/browse/SPARK-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13642: -- Assignee: Saisai Shao > Properly handle signal kill of ApplicationMaster >

[jira] [Commented] (SPARK-13775) history server sort by completed time by default

[ https://issues.apache.org/jira/browse/SPARK-13775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187804#comment-15187804 ] Thomas Graves commented on SPARK-13775: --- why does it really matter? Did the version I merged harm

[jira] [Updated] (SPARK-13577) Allow YARN to handle multiple jars, archive when uploading Spark dependencies

[ https://issues.apache.org/jira/browse/SPARK-13577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13577: -- Assignee: Marcelo Vanzin > Allow YARN to handle multiple jars, archive when uploading Spark

[jira] [Resolved] (SPARK-13577) Allow YARN to handle multiple jars, archive when uploading Spark dependencies

[ https://issues.apache.org/jira/browse/SPARK-13577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13577. --- Resolution: Fixed > Allow YARN to handle multiple jars, archive when uploading Spark

[jira] [Updated] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3374: - Assignee: Boyang Jerry Peng > Spark on Yarn remove deprecated configs for 2.0 >

[jira] [Commented] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187141#comment-15187141 ] Thomas Graves commented on SPARK-3374: -- It seems this work is also being done under

[jira] [Commented] (SPARK-12800) Subtle bug on Spark Yarn Client under Kerberos Security Mode

[ https://issues.apache.org/jira/browse/SPARK-12800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15216243#comment-15216243 ] Thomas Graves commented on SPARK-12800: --- You are talking about launching a job using

[jira] [Commented] (SPARK-13723) YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true

[ https://issues.apache.org/jira/browse/SPARK-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218792#comment-15218792 ] Thomas Graves commented on SPARK-13723: --- I'm saying if either the --num-executors or

[jira] [Commented] (SPARK-13723) YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true

[ https://issues.apache.org/jira/browse/SPARK-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218463#comment-15218463 ] Thomas Graves commented on SPARK-13723: --- Any other opinions on this? [~andrewor14] [~vanzin]

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file instead of package.scala

[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Assignee: Sanket Reddy (was: Thomas Graves) > Improve the spark build to pick the version

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file instead of package.scala

[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Assignee: Thomas Graves > Improve the spark build to pick the version information from the pom

[jira] [Created] (SPARK-14405) broadcast variables can cause lots of warning messages on shutdown

Thomas Graves created SPARK-14405: - Summary: broadcast variables can cause lots of warning messages on shutdown Key: SPARK-14405 URL: https://issues.apache.org/jira/browse/SPARK-14405 Project: Spark

[jira] [Resolved] (SPARK-14245) webUI should display the user

[ https://issues.apache.org/jira/browse/SPARK-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-14245. --- Resolution: Fixed Fix Version/s: 2.0.0 > webUI should display the user >

[jira] [Resolved] (SPARK-12384) Allow -Xms to be set differently then -Xmx

[ https://issues.apache.org/jira/browse/SPARK-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-12384. --- Resolution: Fixed Fix Version/s: 2.0.0 > Allow -Xms to be set differently then -Xmx >

[jira] [Resolved] (SPARK-13063) Make the SPARK YARN STAGING DIR as configurable

[ https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13063. --- Resolution: Fixed Fix Version/s: 2.0.0 > Make the SPARK YARN STAGING DIR as

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file and add git commit information

[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Summary: Improve the spark build to pick the version information from the pom file and add git

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file and add git commit information

[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Description: Right now the spark-submit --version and other parts of the code pick up version

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file instead of package.scala

[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Description: Right now the spark-submit --version and other parts of the code pick up

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205279#comment-15205279 ] Thomas Graves commented on SPARK-1239: -- I do like the idea of broadcast and originally when I had

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204900#comment-15204900 ] Thomas Graves commented on SPARK-1239: -- So I have been looking at this and testing a few changes out.

[jira] [Updated] (SPARK-13642) Properly handle signal kill of ApplicationMaster

[ https://issues.apache.org/jira/browse/SPARK-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13642: -- Fix Version/s: 1.6.2 > Properly handle signal kill of ApplicationMaster >

[jira] [Resolved] (SPARK-13675) The url link in historypage is not correct for application running in yarn cluster mode

[ https://issues.apache.org/jira/browse/SPARK-13675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13675. --- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.0.0 > The url link

[jira] [Commented] (SPARK-2666) Always try to cancel running tasks when a stage is marked as zombie

[ https://issues.apache.org/jira/browse/SPARK-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176149#comment-15176149 ] Thomas Graves commented on SPARK-2666: -- [~lianhuiwang] were you going to work on this? I'm running

[jira] [Commented] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15180027#comment-15180027 ] Thomas Graves commented on SPARK-3374: -- [~srowen] can you add [~jerrypeng] as a contributor so he

[jira] [Commented] (SPARK-13642) Inconsistent finishing state between driver and AM

[ https://issues.apache.org/jira/browse/SPARK-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177905#comment-15177905 ] Thomas Graves commented on SPARK-13642: --- The problem is that you really don't want the opposite to

[jira] [Created] (SPARK-13723) YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true

Thomas Graves created SPARK-13723: - Summary: YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true Key: SPARK-13723 URL: https://issues.apache.org/jira/browse/SPARK-13723

[jira] [Commented] (SPARK-13723) YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true

[ https://issues.apache.org/jira/browse/SPARK-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15183272#comment-15183272 ] Thomas Graves commented on SPARK-13723: --- see some discussion on

[jira] [Updated] (SPARK-13459) Separate Alive and Dead Executors in Executor Totals Table

[ https://issues.apache.org/jira/browse/SPARK-13459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13459: -- Assignee: Alex Bozarth > Separate Alive and Dead Executors in Executor Totals Table >

[jira] [Resolved] (SPARK-13459) Separate Alive and Dead Executors in Executor Totals Table

[ https://issues.apache.org/jira/browse/SPARK-13459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13459. --- Resolution: Fixed Fix Version/s: 2.0.0 > Separate Alive and Dead Executors in

[jira] [Updated] (SPARK-1239) Improve fetching of map output statuses

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1239: - Summary: Improve fetching of map output statuses (was: Don't fetch all map output statuses at

[jira] [Created] (SPARK-14331) Exceptions saving to parquetFile after join from dataframes in master

Thomas Graves created SPARK-14331: - Summary: Exceptions saving to parquetFile after join from dataframes in master Key: SPARK-14331 URL: https://issues.apache.org/jira/browse/SPARK-14331 Project:

[jira] [Resolved] (SPARK-12864) Fetch failure from AM restart

[ https://issues.apache.org/jira/browse/SPARK-12864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-12864. --- Resolution: Fixed Fix Version/s: 2.0.0 > Fetch failure from AM restart >

[jira] [Updated] (SPARK-12384) Allow -Xms to be set differently then -Xmx

[ https://issues.apache.org/jira/browse/SPARK-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-12384: -- Assignee: Dhruve Ashar > Allow -Xms to be set differently then -Xmx >

[jira] [Created] (SPARK-14245) webUI should display the user

Thomas Graves created SPARK-14245: - Summary: webUI should display the user Key: SPARK-14245 URL: https://issues.apache.org/jira/browse/SPARK-14245 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-13669) Job will always fail in the external shuffle service unavailable situation

[ https://issues.apache.org/jira/browse/SPARK-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214398#comment-15214398 ] Thomas Graves commented on SPARK-13669: --- I think there is a bigger blacklisting change required for

[jira] [Commented] (SPARK-12956) add spark.yarn.hdfs.home.directory property

[ https://issues.apache.org/jira/browse/SPARK-12956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214406#comment-15214406 ] Thomas Graves commented on SPARK-12956: --- Are you talking it uses the home directory for the spark

[jira] [Commented] (SPARK-13829) Spark submit with keytab can't submit to a non-HDFS yarn cluster

[ https://issues.apache.org/jira/browse/SPARK-13829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214394#comment-15214394 ] Thomas Graves commented on SPARK-13829: --- Can you clarify this, is this the case you are using

[jira] [Commented] (SPARK-13063) Make the SPARK YARN STAGING DIR as configurable

[ https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214419#comment-15214419 ] Thomas Graves commented on SPARK-13063: --- it seems harmless to allow this to be configurable if

[jira] [Commented] (SPARK-14245) webUI should display the user

[ https://issues.apache.org/jira/browse/SPARK-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15224589#comment-15224589 ] Thomas Graves commented on SPARK-14245: --- ok, personally I would prefer people to assign jira to

[jira] [Commented] (SPARK-14245) webUI should display the user

[ https://issues.apache.org/jira/browse/SPARK-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15224288#comment-15224288 ] Thomas Graves commented on SPARK-14245: --- [~ajbozarth] Please go ahead and assign yourself the jira

[jira] [Updated] (SPARK-14245) webUI should display the user

[ https://issues.apache.org/jira/browse/SPARK-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14245: -- Assignee: Alex Bozarth > webUI should display the user > - > >

[jira] [Updated] (SPARK-13063) Make the SPARK YARN STAGING DIR as configurable

[ https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13063: -- Assignee: Devaraj K > Make the SPARK YARN STAGING DIR as configurable >

[jira] [Comment Edited] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204900#comment-15204900 ] Thomas Graves edited comment on SPARK-1239 at 3/21/16 7:27 PM: --- So I have

[jira] [Commented] (SPARK-14963) YarnShuffleService should use YARN getRecoveryPath() for leveldb location

[ https://issues.apache.org/jira/browse/SPARK-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264010#comment-15264010 ] Thomas Graves commented on SPARK-14963: --- I'm definitely fine with it but someone else might also be

[jira] [Commented] (SPARK-11316) coalesce doesn't handle UnionRDD with partial locality properly

[ https://issues.apache.org/jira/browse/SPARK-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264205#comment-15264205 ] Thomas Graves commented on SPARK-11316: --- Simple steps to reproduce an RDD with partial preferred

[jira] [Commented] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260492#comment-15260492 ] Thomas Graves commented on SPARK-3374: -- I think these were all removed under SPARK-12343 > Spark on

[jira] [Closed] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-3374. Resolution: Duplicate > Spark on Yarn remove deprecated configs for 2.0 >

[jira] [Created] (SPARK-14963) YarnShuffleService should use YARN getRecoveryPath() for leveldb location

Thomas Graves created SPARK-14963: - Summary: YarnShuffleService should use YARN getRecoveryPath() for leveldb location Key: SPARK-14963 URL: https://issues.apache.org/jira/browse/SPARK-14963 Project:

[jira] [Updated] (SPARK-14963) YarnShuffleService should use YARN getRecoveryPath() for leveldb location

[ https://issues.apache.org/jira/browse/SPARK-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14963: -- Issue Type: Improvement (was: Bug) > YarnShuffleService should use YARN getRecoveryPath() for

[jira] [Resolved] (SPARK-6735) Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.

[ https://issues.apache.org/jira/browse/SPARK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-6735. -- Resolution: Fixed Fix Version/s: 2.0.0 > Provide options to make maximum executor

[jira] [Updated] (SPARK-6735) Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.

[ https://issues.apache.org/jira/browse/SPARK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-6735: - Assignee: Saisai Shao > Provide options to make maximum executor failure count ( which kills the

[jira] [Commented] (SPARK-14331) Exceptions saving to parquetFile after join from dataframes in master

[ https://issues.apache.org/jira/browse/SPARK-14331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296440#comment-15296440 ] Thomas Graves commented on SPARK-14331: --- Note I was running spark on yarn. > Exceptions saving to

[jira] [Commented] (SPARK-14331) Exceptions saving to parquetFile after join from dataframes in master

[ https://issues.apache.org/jira/browse/SPARK-14331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296416#comment-15296416 ] Thomas Graves commented on SPARK-14331: --- I was trying to reproduce this to get you the rest but now

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file and add git commit information

[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Assignee: (was: Sanket Reddy) > Improve the spark build to pick the version information

[jira] [Commented] (SPARK-14331) Exceptions saving to parquetFile after join from dataframes in master

[ https://issues.apache.org/jira/browse/SPARK-14331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296705#comment-15296705 ] Thomas Graves commented on SPARK-14331: --- that might be https://github.com/apache/spark/pull/13175.

[jira] [Commented] (SPARK-15405) YARN uploading the same __spark_conf__.zip twice

[ https://issues.apache.org/jira/browse/SPARK-15405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291180#comment-15291180 ] Thomas Graves commented on SPARK-15405: --- [~vanzin] I missed this when I did the review, do we need

[jira] [Created] (SPARK-15405) YARN uploading the same __spark_conf__.zip twice

Thomas Graves created SPARK-15405: - Summary: YARN uploading the same __spark_conf__.zip twice Key: SPARK-15405 URL: https://issues.apache.org/jira/browse/SPARK-15405 Project: Spark Issue

[jira] [Created] (SPARK-15410) spark-submit --help throws exception

Thomas Graves created SPARK-15410: - Summary: spark-submit --help throws exception Key: SPARK-15410 URL: https://issues.apache.org/jira/browse/SPARK-15410 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4924) Factor out code to launch Spark applications into a separate library

[ https://issues.apache.org/jira/browse/SPARK-4924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284483#comment-15284483 ] Thomas Graves commented on SPARK-4924: -- [~javadba] If you have ideas on improving the documentation

[jira] [Updated] (SPARK-4224) Support group acls

[ https://issues.apache.org/jira/browse/SPARK-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-4224: - Target Version/s: 2.0.0 > Support group acls > -- > > Key:

[jira] [Resolved] (SPARK-4224) Support group acls

[ https://issues.apache.org/jira/browse/SPARK-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-4224. -- Resolution: Fixed Fix Version/s: 2.0.0 > Support group acls > -- > >

[jira] [Updated] (SPARK-4224) Support group acls

[ https://issues.apache.org/jira/browse/SPARK-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-4224: - Assignee: Dhruve Ashar > Support group acls > -- > > Key:

[jira] [Created] (SPARK-15121) Improve logging of external shuffle handler

Thomas Graves created SPARK-15121: - Summary: Improve logging of external shuffle handler Key: SPARK-15121 URL: https://issues.apache.org/jira/browse/SPARK-15121 Project: Spark Issue Type:

[jira] [Updated] (SPARK-15121) Improve logging of external shuffle handler

[ https://issues.apache.org/jira/browse/SPARK-15121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-15121: -- Assignee: (was: Thomas Graves) > Improve logging of external shuffle handler >

[jira] [Commented] (SPARK-14963) YarnShuffleService should use YARN getRecoveryPath() for leveldb location

[ https://issues.apache.org/jira/browse/SPARK-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272466#comment-15272466 ] Thomas Graves commented on SPARK-14963: --- [~jerryshao] I think the other pr is being closed so if

[jira] [Created] (SPARK-15178) Remove LazyFileRegion

Thomas Graves created SPARK-15178: - Summary: Remove LazyFileRegion Key: SPARK-15178 URL: https://issues.apache.org/jira/browse/SPARK-15178 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-14963) YarnShuffleService should use YARN getRecoveryPath() for leveldb location

[ https://issues.apache.org/jira/browse/SPARK-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14963: -- Assignee: Saisai Shao > YarnShuffleService should use YARN getRecoveryPath() for leveldb

[jira] [Resolved] (SPARK-14963) YarnShuffleService should use YARN getRecoveryPath() for leveldb location

[ https://issues.apache.org/jira/browse/SPARK-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-14963. --- Resolution: Fixed Fix Version/s: 2.1.0 > YarnShuffleService should use YARN

[jira] [Resolved] (SPARK-13988) Large history files block new applications from showing up in History UI.

[ https://issues.apache.org/jira/browse/SPARK-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13988. --- Resolution: Fixed Fix Version/s: 2.0.0 > Large history files block new applications

[jira] [Updated] (SPARK-13988) Large history files block new applications from showing up in History UI.

[ https://issues.apache.org/jira/browse/SPARK-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13988: -- Assignee: Parth Brahmbhatt > Large history files block new applications from showing up in

[jira] [Resolved] (SPARK-14572) Update Config Doc to specify -Xms in extraJavaOptions

[ https://issues.apache.org/jira/browse/SPARK-14572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-14572. --- Resolution: Fixed Fix Version/s: 2.0.0 > Update Config Doc to specify -Xms in

[jira] [Updated] (SPARK-14572) Update Config Doc to specify -Xms in extraJavaOptions

[ https://issues.apache.org/jira/browse/SPARK-14572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14572: -- Assignee: Dhruve Ashar > Update Config Doc to specify -Xms in extraJavaOptions >

[jira] [Commented] (SPARK-1989) Exit executors faster if they get into a cycle of heavy GC

[ https://issues.apache.org/jira/browse/SPARK-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15262761#comment-15262761 ] Thomas Graves commented on SPARK-1989: -- Personally I don't agree with this and think we should close

[jira] [Updated] (SPARK-15703) Make ListenerBus event queue size configurable

[ https://issues.apache.org/jira/browse/SPARK-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-15703: -- Issue Type: Improvement (was: Bug) > Make ListenerBus event queue size configurable >

[jira] [Commented] (SPARK-15703) Make ListenerBus event queue size configurable

[ https://issues.apache.org/jira/browse/SPARK-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15394271#comment-15394271 ] Thomas Graves commented on SPARK-15703: --- Using this jira to make lsitenerbus configurable and we

[jira] [Resolved] (SPARK-15703) Make ListenerBus event queue size configurable

[ https://issues.apache.org/jira/browse/SPARK-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-15703. --- Resolution: Fixed Fix Version/s: 2.1.0 > Make ListenerBus event queue size

[jira] [Updated] (SPARK-15703) Make ListenerBus event queue size configurable

[ https://issues.apache.org/jira/browse/SPARK-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-15703: -- Priority: Minor (was: Critical) > Make ListenerBus event queue size configurable >

[jira] [Updated] (SPARK-15703) Make ListenerBus event queue size configurable

[ https://issues.apache.org/jira/browse/SPARK-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-15703: -- Assignee: Dhruve Ashar > Make ListenerBus event queue size configurable >

[jira] [Updated] (SPARK-15703) Make ListenerBus event queue size configurable

[ https://issues.apache.org/jira/browse/SPARK-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-15703: -- Summary: Make ListenerBus event queue size configurable (was: Spark UI doesn't show all tasks

[jira] [Resolved] (SPARK-16505) YARN shuffle service should throw errors when it fails to start

[ https://issues.apache.org/jira/browse/SPARK-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-16505. --- Resolution: Fixed Fix Version/s: 2.1.0 > YARN shuffle service should throw errors

[jira] [Commented] (SPARK-14963) YarnShuffleService should use YARN getRecoveryPath() for leveldb location

[ https://issues.apache.org/jira/browse/SPARK-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376913#comment-15376913 ] Thomas Graves commented on SPARK-14963: --- committed fix in

[jira] [Updated] (SPARK-16505) YARN shuffle service should throw errors when it fails to start

[ https://issues.apache.org/jira/browse/SPARK-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-16505: -- Assignee: Marcelo Vanzin > YARN shuffle service should throw errors when it fails to start >

[jira] [Commented] (SPARK-16711) YarnShuffleService doesn't re-init properly on YARN rolling upgrade

[ https://issues.apache.org/jira/browse/SPARK-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392145#comment-15392145 ] Thomas Graves commented on SPARK-16711: --- Note this happens when security is on. The

[jira] [Commented] (SPARK-16711) YarnShuffleService doesn't re-init properly on YARN rolling upgrade

[ https://issues.apache.org/jira/browse/SPARK-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392135#comment-15392135 ] Thomas Graves commented on SPARK-16711: --- Note that what happens here is the yarnshuffleservice

[jira] [Created] (SPARK-16711) YarnShuffleService doesn't re-init properly on YARN rolling upgrade

Thomas Graves created SPARK-16711: - Summary: YarnShuffleService doesn't re-init properly on YARN rolling upgrade Key: SPARK-16711 URL: https://issues.apache.org/jira/browse/SPARK-16711 Project: Spark

[jira] [Commented] (SPARK-16664) Spark 1.6.2 - Persist call on Data frames with more than 200 columns is wiping out the data.

[ https://issues.apache.org/jira/browse/SPARK-16664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15399168#comment-15399168 ] Thomas Graves commented on SPARK-16664: --- I am out of the office until 8/8. Please contact my

[jira] [Commented] (SPARK-16744) spark.yarn.appMasterEnv handling assumes values should be appended

[ https://issues.apache.org/jira/browse/SPARK-16744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395611#comment-15395611 ] Thomas Graves commented on SPARK-16744: --- Along with this we may consider thinking about an admin

[jira] [Resolved] (SPARK-16435) Behavior changes if initialExecutor is less than minExecutor for dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-16435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-16435. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Behavior changes if

[jira] [Updated] (SPARK-16435) Behavior changes if initialExecutor is less than minExecutor for dynamic allocation

[ https://issues.apache.org/jira/browse/SPARK-16435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-16435: -- Assignee: Saisai Shao > Behavior changes if initialExecutor is less than minExecutor for

[jira] [Commented] (SPARK-2666) Always try to cancel running tasks when a stage is marked as zombie

[ https://issues.apache.org/jira/browse/SPARK-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386561#comment-15386561 ] Thomas Graves commented on SPARK-2666: -- thanks for the explanation. I guess we would have to look

[jira] [Commented] (SPARK-12661) Drop Python 2.6 support in PySpark

[ https://issues.apache.org/jira/browse/SPARK-12661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384392#comment-15384392 ] Thomas Graves commented on SPARK-12661: --- I think [~nchammas] is talking about dropping it, not

[jira] [Commented] (SPARK-15703) Spark UI doesn't show all tasks as completed when it should

[ https://issues.apache.org/jira/browse/SPARK-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384571#comment-15384571 ] Thomas Graves commented on SPARK-15703: --- I think we should do both. event queue size configurable

[jira] [Commented] (SPARK-16650) Improve documentation of spark.task.maxFailures

[ https://issues.apache.org/jira/browse/SPARK-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385868#comment-15385868 ] Thomas Graves commented on SPARK-16650: --- I'll try to put up a patch for this later today. >

[jira] [Created] (SPARK-16650) Improve documentation of spark.task.maxFailures

Thomas Graves created SPARK-16650: - Summary: Improve documentation of spark.task.maxFailures Key: SPARK-16650 URL: https://issues.apache.org/jira/browse/SPARK-16650 Project: Spark Issue

[jira] [Created] (SPARK-16630) Blacklist a node if executors won't launch on it.

Thomas Graves created SPARK-16630: - Summary: Blacklist a node if executors won't launch on it. Key: SPARK-16630 URL: https://issues.apache.org/jira/browse/SPARK-16630 Project: Spark Issue

[jira] [Commented] (SPARK-16630) Blacklist a node if executors won't launch on it.

[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384744#comment-15384744 ] Thomas Graves commented on SPARK-16630: --- Note this is more at the resource manager level where in

[jira] [Updated] (SPARK-16630) Blacklist a node if executors won't launch on it.

[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-16630: -- Description: On YARN, its possible that a node is messed or misconfigured such that a

[jira] [Resolved] (SPARK-15951) Change Executors Page to use datatables to support sorting columns and searching

[ https://issues.apache.org/jira/browse/SPARK-15951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-15951. --- Resolution: Fixed Fix Version/s: 2.1.0 > Change Executors Page to use datatables to

<    2   3   4   5   6   7   8   9   10   11   >