[jira] [Assigned] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

2015-09-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-10640: - Assignee: Thomas Graves > Spark history server fails to parse taskEndReasonFromJson Task

[jira] [Commented] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

2015-09-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14791438#comment-14791438 ] Thomas Graves commented on SPARK-10640: --- yes 1.5 history server reading 1.5.0 logs.

[jira] [Updated] (SPARK-13642) Properly handle signal kill of ApplicationMaster

2016-03-23 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13642: -- Fix Version/s: 1.6.2 > Properly handle signal kill of ApplicationMaster > -

[jira] [Commented] (SPARK-13829) Spark submit with keytab can't submit to a non-HDFS yarn cluster

2016-03-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214394#comment-15214394 ] Thomas Graves commented on SPARK-13829: --- Can you clarify this, is this the case you

[jira] [Commented] (SPARK-13669) Job will always fail in the external shuffle service unavailable situation

2016-03-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214398#comment-15214398 ] Thomas Graves commented on SPARK-13669: --- I think there is a bigger blacklisting cha

[jira] [Commented] (SPARK-12956) add spark.yarn.hdfs.home.directory property

2016-03-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214406#comment-15214406 ] Thomas Graves commented on SPARK-12956: --- Are you talking it uses the home directory

[jira] [Commented] (SPARK-13063) Make the SPARK YARN STAGING DIR as configurable

2016-03-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214419#comment-15214419 ] Thomas Graves commented on SPARK-13063: --- it seems harmless to allow this to be conf

[jira] [Updated] (SPARK-13063) Make the SPARK YARN STAGING DIR as configurable

2016-03-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13063: -- Assignee: Devaraj K > Make the SPARK YARN STAGING DIR as configurable > ---

[jira] [Commented] (SPARK-12800) Subtle bug on Spark Yarn Client under Kerberos Security Mode

2016-03-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15216243#comment-15216243 ] Thomas Graves commented on SPARK-12800: --- You are talking about launching a job usin

[jira] [Updated] (SPARK-12384) Allow -Xms to be set differently then -Xmx

2016-03-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-12384: -- Assignee: Dhruve Ashar > Allow -Xms to be set differently then -Xmx > -

[jira] [Created] (SPARK-14245) webUI should display the user

2016-03-29 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-14245: - Summary: webUI should display the user Key: SPARK-14245 URL: https://issues.apache.org/jira/browse/SPARK-14245 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-13723) YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true

2016-03-30 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218463#comment-15218463 ] Thomas Graves commented on SPARK-13723: --- Any other opinions on this? [~andrewor14]

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file instead of package.scala

2016-03-30 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Assignee: Thomas Graves > Improve the spark build to pick the version information from the pom

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file instead of package.scala

2016-03-30 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Assignee: Sanket Reddy (was: Thomas Graves) > Improve the spark build to pick the version info

[jira] [Commented] (SPARK-13723) YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true

2016-03-30 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218792#comment-15218792 ] Thomas Graves commented on SPARK-13723: --- I'm saying if either the --num-executors o

[jira] [Created] (SPARK-14331) Exceptions saving to parquetFile after join from dataframes in master

2016-04-01 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-14331: - Summary: Exceptions saving to parquetFile after join from dataframes in master Key: SPARK-14331 URL: https://issues.apache.org/jira/browse/SPARK-14331 Project: Spar

[jira] [Updated] (SPARK-1239) Improve fetching of map output statuses

2016-04-01 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1239: - Summary: Improve fetching of map output statuses (was: Don't fetch all map output statuses at eac

[jira] [Resolved] (SPARK-12864) Fetch failure from AM restart

2016-04-01 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-12864. --- Resolution: Fixed Fix Version/s: 2.0.0 > Fetch failure from AM restart > -

[jira] [Updated] (SPARK-14245) webUI should display the user

2016-04-04 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14245: -- Assignee: Alex Bozarth > webUI should display the user > - > >

[jira] [Commented] (SPARK-14245) webUI should display the user

2016-04-04 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224288#comment-15224288 ] Thomas Graves commented on SPARK-14245: --- [~ajbozarth] Please go ahead and assign yo

[jira] [Commented] (SPARK-14245) webUI should display the user

2016-04-04 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224589#comment-15224589 ] Thomas Graves commented on SPARK-14245: --- ok, personally I would prefer people to as

[jira] [Created] (SPARK-14405) broadcast variables can cause lots of warning messages on shutdown

2016-04-05 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-14405: - Summary: broadcast variables can cause lots of warning messages on shutdown Key: SPARK-14405 URL: https://issues.apache.org/jira/browse/SPARK-14405 Project: Spark

[jira] [Resolved] (SPARK-13063) Make the SPARK YARN STAGING DIR as configurable

2016-04-05 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13063. --- Resolution: Fixed Fix Version/s: 2.0.0 > Make the SPARK YARN STAGING DIR as configurab

[jira] [Resolved] (SPARK-14245) webUI should display the user

2016-04-07 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-14245. --- Resolution: Fixed Fix Version/s: 2.0.0 > webUI should display the user > -

[jira] [Resolved] (SPARK-12384) Allow -Xms to be set differently then -Xmx

2016-04-07 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-12384. --- Resolution: Fixed Fix Version/s: 2.0.0 > Allow -Xms to be set differently then -Xmx >

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file instead of package.scala

2016-04-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Description: Right now the spark-submit --version and other parts of the code pick up version

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file and add git commit information

2016-04-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Summary: Improve the spark build to pick the version information from the pom file and add git

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file and add git commit information

2016-04-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14279: -- Description: Right now the spark-submit --version and other parts of the code pick up version

[jira] [Updated] (SPARK-14572) Update Config Doc to specify -Xms in extraJavaOptions

2016-04-14 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-14572: -- Assignee: Dhruve Ashar > Update Config Doc to specify -Xms in extraJavaOptions > --

[jira] [Resolved] (SPARK-14572) Update Config Doc to specify -Xms in extraJavaOptions

2016-04-14 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-14572. --- Resolution: Fixed Fix Version/s: 2.0.0 > Update Config Doc to specify -Xms in extraJav

[jira] [Updated] (SPARK-13988) Large history files block new applications from showing up in History UI.

2016-04-21 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13988: -- Assignee: Parth Brahmbhatt > Large history files block new applications from showing up in Hist

[jira] [Resolved] (SPARK-13988) Large history files block new applications from showing up in History UI.

2016-04-21 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13988. --- Resolution: Fixed Fix Version/s: 2.0.0 > Large history files block new applications fr

[jira] [Assigned] (SPARK-27371) Standalone master receives resource info from worker and allocate driver/executor properly

2019-08-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-27371: - Assignee: wuyi > Standalone master receives resource info from worker and allocate > d

[jira] [Resolved] (SPARK-27368) Design: Standalone supports GPU scheduling

2019-08-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-27368. --- Resolution: Fixed resolving since code committed > Design: Standalone supports GPU scheduli

[jira] [Resolved] (SPARK-27371) Standalone master receives resource info from worker and allocate driver/executor properly

2019-08-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-27371. --- Resolution: Fixed Fix Version/s: 3.0.0 > Standalone master receives resource info fro

[jira] [Created] (SPARK-28679) Spark Yarn ResourceRequestHelper shouldn't lookup setResourceInformation is no resources specified

2019-08-09 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-28679: - Summary: Spark Yarn ResourceRequestHelper shouldn't lookup setResourceInformation is no resources specified Key: SPARK-28679 URL: https://issues.apache.org/jira/browse/SPARK-286

[jira] [Commented] (SPARK-23151) Provide a distribution of Spark with Hadoop 3.0

2019-08-13 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906218#comment-16906218 ] Thomas Graves commented on SPARK-23151: --- Hey so for the hadoop-3.2 profile, the de

[jira] [Commented] (SPARK-27361) YARN support for GPU-aware scheduling

2019-08-14 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907431#comment-16907431 ] Thomas Graves commented on SPARK-27361: --- Since that was done prior to this feature

[jira] [Updated] (SPARK-28577) Ensure executorMemoryHead requested value not less than MEMORY_OFFHEAP_SIZE when MEMORY_OFFHEAP_ENABLED is true

2019-08-22 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-28577: -- Labels: release-notes (was: ) > Ensure executorMemoryHead requested value not less than MEMOR

[jira] [Updated] (SPARK-28577) Ensure executorMemoryHead requested value not less than MEMORY_OFFHEAP_SIZE when MEMORY_OFFHEAP_ENABLED is true

2019-08-22 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-28577: -- Docs Text: On YARN, The off heap memory size is now separately included for the user in the co

[jira] [Resolved] (SPARK-28414) Standalone worker/master UI updates for Resource scheduling

2019-08-27 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-28414. --- Fix Version/s: 3.0.0 Assignee: wuyi Resolution: Fixed > Standalone worker/ma

[jira] [Resolved] (SPARK-27360) Standalone cluster mode support for GPU-aware scheduling

2019-08-27 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-27360. --- Fix Version/s: 3.0.0 Resolution: Fixed all subtasks are finished so resolving > Stan

[jira] [Commented] (SPARK-28917) Jobs can hang because of race of RDD.dependencies

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921424#comment-16921424 ] Thomas Graves commented on SPARK-28917: --- So for this to happen, you essentially ha

[jira] [Created] (SPARK-28960) maven 3.6.1 gone from apache maven repo

2019-09-03 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-28960: - Summary: maven 3.6.1 gone from apache maven repo Key: SPARK-28960 URL: https://issues.apache.org/jira/browse/SPARK-28960 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27495: -- Description: *Q1.* What are you trying to do? Articulate your objectives using absolutely no

[jira] [Commented] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921608#comment-16921608 ] Thomas Graves commented on SPARK-27495: --- Thanks for the feedback [~felixcheung] 1

[jira] [Commented] (SPARK-28960) maven 3.6.1 gone from apache maven repo

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921611#comment-16921611 ] Thomas Graves commented on SPARK-28960: --- its a bit odd I don't see 3.6.2 on their

[jira] [Updated] (SPARK-28961) Upgrade Maven to 3.6.2

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-28961: -- Description: It looks like maven 3.6.1 is missing from the apache maven repo: [http://apache.

[jira] [Resolved] (SPARK-28960) maven 3.6.1 gone from apache maven repo

2019-09-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-28960. --- Resolution: Duplicate > maven 3.6.1 gone from apache maven repo > --

[jira] [Resolved] (SPARK-28577) Ensure executorMemoryHead requested value not less than MEMORY_OFFHEAP_SIZE when MEMORY_OFFHEAP_ENABLED is true

2019-09-04 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-28577. --- Fix Version/s: 3.0.0 Assignee: Yang Jie Resolution: Fixed > Ensure executorM

[jira] [Updated] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

2019-09-04 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27495: -- Description: *Q1.* What are you trying to do? Articulate your objectives using absolutely no

[jira] [Assigned] (SPARK-27492) GPU scheduling - High level user documentation

2019-09-04 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-27492: - Assignee: Thomas Graves > GPU scheduling - High level user documentation >

[jira] [Commented] (SPARK-27492) GPU scheduling - High level user documentation

2019-09-05 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923528#comment-16923528 ] Thomas Graves commented on SPARK-27492: --- I think we talked about this a big in the

[jira] [Commented] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

2019-09-11 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16927558#comment-16927558 ] Thomas Graves commented on SPARK-27495: --- [~felixcheung]  [~jiangxb1987]  I put thi

[jira] [Resolved] (SPARK-27492) GPU scheduling - High level user documentation

2019-09-11 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-27492. --- Fix Version/s: 3.0.0 Resolution: Fixed > GPU scheduling - High level user documentati

[jira] [Commented] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

2019-09-16 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930600#comment-16930600 ] Thomas Graves commented on SPARK-27495: --- The SPIP vote passed -  https://mail-arch

[jira] [Created] (SPARK-29148) Modify dynamic allocation manager for stage level scheduling

2019-09-18 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-29148: - Summary: Modify dynamic allocation manager for stage level scheduling Key: SPARK-29148 URL: https://issues.apache.org/jira/browse/SPARK-29148 Project: Spark

[jira] [Created] (SPARK-29149) Update YARN cluster manager For Stage Level Scheduling

2019-09-18 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-29149: - Summary: Update YARN cluster manager For Stage Level Scheduling Key: SPARK-29149 URL: https://issues.apache.org/jira/browse/SPARK-29149 Project: Spark Issu

[jira] [Created] (SPARK-29150) Update RDD API for Stage level scheduling

2019-09-18 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-29150: - Summary: Update RDD API for Stage level scheduling Key: SPARK-29150 URL: https://issues.apache.org/jira/browse/SPARK-29150 Project: Spark Issue Type: Impro

[jira] [Updated] (SPARK-29148) Modify dynamic allocation manager for stage level scheduling

2019-09-18 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-29148: -- Issue Type: Improvement (was: Bug) > Modify dynamic allocation manager for stage level schedu

[jira] [Updated] (SPARK-29150) Update RDD API for Stage level scheduling

2019-09-18 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-29150: -- Issue Type: Story (was: Improvement) > Update RDD API for Stage level scheduling > --

[jira] [Updated] (SPARK-29148) Modify dynamic allocation manager for stage level scheduling

2019-09-18 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-29148: -- Issue Type: Story (was: Improvement) > Modify dynamic allocation manager for stage level sche

[jira] [Updated] (SPARK-29149) Update YARN cluster manager For Stage Level Scheduling

2019-09-18 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-29149: -- Issue Type: Story (was: Bug) > Update YARN cluster manager For Stage Level Scheduling > -

[jira] [Created] (SPARK-29151) Support fraction resources for resource scheduling

2019-09-18 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-29151: - Summary: Support fraction resources for resource scheduling Key: SPARK-29151 URL: https://issues.apache.org/jira/browse/SPARK-29151 Project: Spark Issue Ty

[jira] [Commented] (SPARK-29151) Support fraction resources for resource scheduling

2019-09-18 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932422#comment-16932422 ] Thomas Graves commented on SPARK-29151: --- Depending on when things go in we could d

[jira] [Created] (SPARK-29153) ResourceProfile conflict resolution stage level scheduling

2019-09-18 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-29153: - Summary: ResourceProfile conflict resolution stage level scheduling Key: SPARK-29153 URL: https://issues.apache.org/jira/browse/SPARK-29153 Project: Spark

[jira] [Updated] (SPARK-29150) Update RDD API for Stage level scheduling

2019-09-18 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-29150: -- Description: See the SPIP and design doc attached to SPARK-27495.  Note this is meant to be t

[jira] [Created] (SPARK-29154) Update Spark scheduler for stage level scheduling

2019-09-18 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-29154: - Summary: Update Spark scheduler for stage level scheduling Key: SPARK-29154 URL: https://issues.apache.org/jira/browse/SPARK-29154 Project: Spark Issue Typ

[jira] [Updated] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

2019-09-18 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27495: -- Labels: SPIP (was: ) > SPIP: Support Stage level resource configuration and scheduling >

[jira] [Updated] (SPARK-27495) SPIP: Support Stage level resource configuration and scheduling

2019-09-18 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27495: -- Epic Name: Stage Level Scheduling > SPIP: Support Stage level resource configuration and sched

[jira] [Commented] (SPARK-29151) Support fraction resources for resource scheduling

2019-09-20 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934419#comment-16934419 ] Thomas Graves commented on SPARK-29151: --- To keep this simple for a design, I think

[jira] [Updated] (SPARK-29151) Support fraction resources for task resource scheduling

2019-09-20 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-29151: -- Summary: Support fraction resources for task resource scheduling (was: Support fraction resou

[jira] [Updated] (SPARK-29151) Support fraction resources for task resource scheduling

2019-09-20 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-29151: -- Description: The current resource scheduling code for GPU/FPGA, etc only supports amounts as

[jira] [Commented] (SPARK-29206) Number of shuffle Netty server threads should be a multiple of number of chunk fetch handler threads

2019-09-23 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16935968#comment-16935968 ] Thomas Graves commented on SPARK-29206: --- that is definitely not the behavior I was

[jira] [Created] (SPARK-29303) UI updates for stage level scheduling

2019-09-30 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-29303: - Summary: UI updates for stage level scheduling Key: SPARK-29303 URL: https://issues.apache.org/jira/browse/SPARK-29303 Project: Spark Issue Type: Story

[jira] [Commented] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-09-30 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940974#comment-16940974 ] Thomas Graves commented on SPARK-27396: --- The main objectives have actually already

[jira] [Resolved] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

2019-09-30 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-27396. --- Assignee: Robert Joseph Evans Resolution: Fixed > SPIP: Public APIs for extended Colum

[jira] [Created] (SPARK-29306) Executors need to track what ResourceProfile they are created with

2019-09-30 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-29306: - Summary: Executors need to track what ResourceProfile they are created with Key: SPARK-29306 URL: https://issues.apache.org/jira/browse/SPARK-29306 Project: Spark

[jira] [Comment Edited] (SPARK-28759) Upgrade scala-maven-plugin to 4.2.0

2019-10-01 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942308#comment-16942308 ] Thomas Graves edited comment on SPARK-28759 at 10/1/19 9:05 PM: --

[jira] [Commented] (SPARK-28759) Upgrade scala-maven-plugin to 4.2.0

2019-10-01 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942308#comment-16942308 ] Thomas Graves commented on SPARK-28759: --- [~dongjoon] [~hyukjin.kwon]  Did either o

[jira] [Created] (SPARK-29329) maven incremental builds not working

2019-10-02 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-29329: - Summary: maven incremental builds not working Key: SPARK-29329 URL: https://issues.apache.org/jira/browse/SPARK-29329 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-28759) Upgrade scala-maven-plugin to 4.2.0

2019-10-02 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942787#comment-16942787 ] Thomas Graves commented on SPARK-28759: --- I rolled back this commit and the increme

[jira] [Updated] (SPARK-29329) maven incremental builds not working

2019-10-02 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-29329: -- Description: It looks like since we Upgraded scala-maven-plugin to 4.2.0 https://issues.apach

[jira] [Commented] (SPARK-29329) maven incremental builds not working

2019-10-02 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942790#comment-16942790 ] Thomas Graves commented on SPARK-29329: --- there are few comments on SPARK-28759 in

[jira] [Commented] (SPARK-29329) maven incremental builds not working

2019-10-03 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943839#comment-16943839 ] Thomas Graves commented on SPARK-29329: --- filed issue with scala-maven-plugin, we w

[jira] [Resolved] (SPARK-31029) Occasional class not found error in user's Future code using global ExecutionContext

2020-06-19 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-31029. --- Fix Version/s: 3.1.0 Assignee: shanyu zhao Resolution: Fixed > Occasional cl

[jira] [Commented] (SPARK-32037) Rename blacklisting feature to avoid language with racist connotation

2020-06-23 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17143151#comment-17143151 ] Thomas Graves commented on SPARK-32037: --- I agree healthy/unhealty could mean other

[jira] [Commented] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

2020-06-24 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17144194#comment-17144194 ] Thomas Graves commented on SPARK-24615: --- [~mengxr]  [~jiangxb1987]  it would be ni

[jira] [Resolved] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

2020-06-29 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-24615. --- Fix Version/s: 3.0.0 Resolution: Fixed > SPIP: Accelerator-aware task scheduling for

[jira] [Assigned] (SPARK-32068) Spark 3 UI task launch time show in error time zone

2020-06-30 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-32068: - Assignee: JinxinTang > Spark 3 UI task launch time show in error time zone > --

[jira] [Resolved] (SPARK-32068) Spark 3 UI task launch time show in error time zone

2020-06-30 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-32068. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed > Spark 3 UI task lau

[jira] [Commented] (SPARK-32135) Show Spark Driver name on Spark history web page

2020-06-30 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148742#comment-17148742 ] Thomas Graves commented on SPARK-32135: --- [~gaurangi]can you please clarify what yo

[jira] [Commented] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster

2020-06-30 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148746#comment-17148746 ] Thomas Graves commented on SPARK-32119: --- You can specify the jars in extraClassPat

[jira] [Commented] (SPARK-32120) Single GPU is allocated multiple times

2020-07-06 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152118#comment-17152118 ] Thomas Graves commented on SPARK-32120: --- Yes this was a known limitation when we d

[jira] [Resolved] (SPARK-32036) Remove references to "blacklist"/"whitelist" language (outside of blacklisting feature)

2020-07-15 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-32036. --- Fix Version/s: 3.1.0 Resolution: Fixed > Remove references to "blacklist"/"whitelist"

[jira] [Assigned] (SPARK-32036) Remove references to "blacklist"/"whitelist" language (outside of blacklisting feature)

2020-07-15 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-32036: - Assignee: Erik Krogen > Remove references to "blacklist"/"whitelist" language (outside

[jira] [Commented] (SPARK-32287) Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

2020-07-15 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17158521#comment-17158521 ] Thomas Graves commented on SPARK-32287: --- I'll try to reproduce and investigate loc

[jira] [Commented] (SPARK-32037) Rename blacklisting feature to avoid language with racist connotation

2020-07-15 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17158739#comment-17158739 ] Thomas Graves commented on SPARK-32037: --- Any other opinions on what we should go w

[jira] [Created] (SPARK-32332) AQE doesn't adequately allow for Columnar Processing extension

2020-07-16 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-32332: - Summary: AQE doesn't adequately allow for Columnar Processing extension Key: SPARK-32332 URL: https://issues.apache.org/jira/browse/SPARK-32332 Project: Spark

[jira] [Created] (SPARK-32333) Drop references to Master

2020-07-16 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-32333: - Summary: Drop references to Master Key: SPARK-32333 URL: https://issues.apache.org/jira/browse/SPARK-32333 Project: Spark Issue Type: Improvement

<    5   6   7   8   9   10   11   12   13   14   >