[jira] [Updated] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-23304: -- Description: The query below seems to ignore the coalesce. This is running spark 2.2 or spark

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349270#comment-16349270 ] Thomas Graves commented on SPARK-23304: --- so with the new ORC code is there anyway t

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349367#comment-16349367 ] Thomas Graves commented on SPARK-23304: --- ok I've attached 2 files one with spark 2.

[jira] [Updated] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-23304: -- Attachment: spark23_oldorc_explain.txt spark22_oldorc_explain.txt > Spark SQL c

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349369#comment-16349369 ] Thomas Graves commented on SPARK-23304: --- Note I've removed some of the columns from

[jira] [Created] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

Thomas Graves created SPARK-23309: - Summary: Spark 2.3 cached query performance 20-30% worse then spark 2.2 Key: SPARK-23309 URL: https://issues.apache.org/jira/browse/SPARK-23309 Project: Spark

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349430#comment-16349430 ] Thomas Graves commented on SPARK-23304: ---   I filed Jira https://issues.apache.org/

[jira] [Commented] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349431#comment-16349431 ] Thomas Graves commented on SPARK-23309: --- I'm curious if anyone else is seeing the s

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349444#comment-16349444 ] Thomas Graves commented on SPARK-23304: --- [~smilegator] just to make sure you saw my

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349446#comment-16349446 ] Thomas Graves commented on SPARK-23304: --- It still seems like a bug to me since the

[jira] [Updated] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-23304: -- Attachment: spark23_oldorc_explain_convermetastoreorcfalse.txt > Spark SQL coalesce() against h

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349480#comment-16349480 ] Thomas Graves commented on SPARK-23304: --- I just ran the query (show()) and saw the

[jira] [Commented] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349483#comment-16349483 ] Thomas Graves commented on SPARK-23309: --- sure, I can also run with the  --conf spar

[jira] [Commented] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349551#comment-16349551 ] Thomas Graves commented on SPARK-23309: --- seeing the same time difference after addi

[jira] [Commented] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349553#comment-16349553 ] Thomas Graves commented on SPARK-23309: --- [~dongjoon] is there any native way with t

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349555#comment-16349555 ] Thomas Graves commented on SPARK-23304: --- I don't have any hive tables backed by par

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350428#comment-16350428 ] Thomas Graves commented on SPARK-23304: --- well I guess that give you end # of partit

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350423#comment-16350423 ] Thomas Graves commented on SPARK-23304: --- it doesn't look like sql("xyz").rdd.partit

[jira] [Commented] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350440#comment-16350440 ] Thomas Graves commented on SPARK-23304: --- ok so I guess by that logic then the coale

[jira] [Resolved] (SPARK-23304) Spark SQL coalesce() against hive not working

[ https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-23304. --- Resolution: Invalid > Spark SQL coalesce() against hive not working > ---

[jira] [Commented] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350533#comment-16350533 ] Thomas Graves commented on SPARK-23309: --- Note the schema of "something" here is a "

[jira] [Commented] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350813#comment-16350813 ] Thomas Graves commented on SPARK-23309: --- I'm still seeing spark 2.3 slower by about

[jira] [Comment Edited] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350813#comment-16350813 ] Thomas Graves edited comment on SPARK-23309 at 2/2/18 7:04 PM:

[jira] [Commented] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350900#comment-16350900 ] Thomas Graves commented on SPARK-23309: --- So the last test I did was spark 2.3 with

[jira] [Commented] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350901#comment-16350901 ] Thomas Graves commented on SPARK-23309: --- I should ask is there a log statement or q

[jira] [Comment Edited] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350813#comment-16350813 ] Thomas Graves edited comment on SPARK-23309 at 2/2/18 8:15 PM:

[jira] [Comment Edited] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350901#comment-16350901 ] Thomas Graves edited comment on SPARK-23309 at 2/2/18 8:29 PM:

[jira] [Comment Edited] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350901#comment-16350901 ] Thomas Graves edited comment on SPARK-23309 at 2/2/18 8:29 PM:

[jira] [Created] (SPARK-25995) sparkR should ensure user args are after the argument used for the port

Thomas Graves created SPARK-25995: - Summary: sparkR should ensure user args are after the argument used for the port Key: SPARK-25995 URL: https://issues.apache.org/jira/browse/SPARK-25995 Project: Sp

[jira] [Commented] (SPARK-25995) sparkR should ensure user args are after the argument used for the port

[ https://issues.apache.org/jira/browse/SPARK-25995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16683876#comment-16683876 ] Thomas Graves commented on SPARK-25995: --- I haven't looked at the details but I wou

[jira] [Resolved] (SPARK-21809) Change Stage Page to use datatables to support sorting columns and searching

[ https://issues.apache.org/jira/browse/SPARK-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-21809. --- Resolution: Fixed Assignee: Parth Gandhi Fix Version/s: 3.0.0 > Change Stage

[jira] [Commented] (SPARK-26089) Handle large corrupt shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16699639#comment-16699639 ] Thomas Graves commented on SPARK-26089: --- it would definitely be nice to improve bl

[jira] [Commented] (SPARK-26089) Handle large corrupt shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700493#comment-16700493 ] Thomas Graves commented on SPARK-26089: --- Yeah that seems to make sense and I would

[jira] [Created] (SPARK-26201) python broadcast.value on driver fails with disk encryption enabled

Thomas Graves created SPARK-26201: - Summary: python broadcast.value on driver fails with disk encryption enabled Key: SPARK-26201 URL: https://issues.apache.org/jira/browse/SPARK-26201 Project: Spark

[jira] [Commented] (SPARK-26201) python broadcast.value on driver fails with disk encryption enabled

[ https://issues.apache.org/jira/browse/SPARK-26201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702028#comment-16702028 ] Thomas Graves commented on SPARK-26201: --- the issue here seems to be that it isn't

[jira] [Resolved] (SPARK-26201) python broadcast.value on driver fails with disk encryption enabled

[ https://issues.apache.org/jira/browse/SPARK-26201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-26201. --- Resolution: Fixed Assignee: Sanket Chintapalli Fix Version/s: 3.0.0

[jira] [Commented] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150583#comment-15150583 ] Thomas Graves commented on SPARK-12316: --- Ah ok, thanks for the clarification. I'll

[jira] [Commented] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150616#comment-15150616 ] Thomas Graves commented on SPARK-12316: --- [~hshreedharan] I think what you are sayin

[jira] [Created] (SPARK-13364) history server application column not sorting properly

Thomas Graves created SPARK-13364: - Summary: history server application column not sorting properly Key: SPARK-13364 URL: https://issues.apache.org/jira/browse/SPARK-13364 Project: Spark Issu

[jira] [Created] (SPARK-13365) should coalesce do anything if coalescing to same number of partitions without shuffle

Thomas Graves created SPARK-13365: - Summary: should coalesce do anything if coalescing to same number of partitions without shuffle Key: SPARK-13365 URL: https://issues.apache.org/jira/browse/SPARK-13365

[jira] [Reopened] (SPARK-2541) Standalone mode can't access secure HDFS anymore

[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reopened SPARK-2541: -- > Standalone mode can't access secure HDFS anymore > ---

[jira] [Commented] (SPARK-2541) Standalone mode can't access secure HDFS anymore

[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152357#comment-15152357 ] Thomas Graves commented on SPARK-2541: -- I'm fine with reopening this. Ideally we woul

[jira] [Updated] (SPARK-11316) coalesce setupGroups doesn't handle UnionRDD with partial localtiy properly

[ https://issues.apache.org/jira/browse/SPARK-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-11316: -- Summary: coalesce setupGroups doesn't handle UnionRDD with partial localtiy properly (was: isE

[jira] [Assigned] (SPARK-11316) coalesce setupGroups doesn't handle UnionRDD with partial localtiy properly

[ https://issues.apache.org/jira/browse/SPARK-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-11316: - Assignee: Thomas Graves > coalesce setupGroups doesn't handle UnionRDD with partial loca

[jira] [Resolved] (SPARK-13364) history server application column Id not sorting as number

[ https://issues.apache.org/jira/browse/SPARK-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13364. --- Resolution: Fixed Fix Version/s: 2.0.0 > history server application column Id not sort

[jira] [Updated] (SPARK-13364) history server application column Id not sorting as number

[ https://issues.apache.org/jira/browse/SPARK-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13364: -- Assignee: Zhuo Liu > history server application column Id not sorting as number > -

[jira] [Updated] (SPARK-11316) coalesce doesn't handle UnionRDD with partial locality properly

[ https://issues.apache.org/jira/browse/SPARK-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-11316: -- Summary: coalesce doesn't handle UnionRDD with partial locality properly (was: coalesce setupG

[jira] [Resolved] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-12316. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > Stack overflow with

[jira] [Resolved] (SPARK-12523) Support long-running of the Spark On HBase and hive meta store.

[ https://issues.apache.org/jira/browse/SPARK-12523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-12523. --- Resolution: Fixed Assignee: SaintBacchus Fix Version/s: 2.0.0 > Support long-

[jira] [Commented] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169068#comment-15169068 ] Thomas Graves commented on SPARK-3374: -- Just adding some more detail: Note we should

[jira] [Resolved] (SPARK-13481) History server page with a default sorting as "desc" time.

[ https://issues.apache.org/jira/browse/SPARK-13481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13481. --- Resolution: Fixed Fix Version/s: 2.0.0 > History server page with a default sorting as

[jira] [Updated] (SPARK-13481) History server page with a default sorting as "desc" time.

[ https://issues.apache.org/jira/browse/SPARK-13481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13481: -- Assignee: Zhuo Liu > History server page with a default sorting as "desc" time. > -

[jira] [Commented] (SPARK-2666) Always try to cancel running tasks when a stage is marked as zombie

[ https://issues.apache.org/jira/browse/SPARK-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176149#comment-15176149 ] Thomas Graves commented on SPARK-2666: -- [~lianhuiwang] were you going to work on this

[jira] [Commented] (SPARK-13642) Inconsistent finishing state between driver and AM

[ https://issues.apache.org/jira/browse/SPARK-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177905#comment-15177905 ] Thomas Graves commented on SPARK-13642: --- The problem is that you really don't want

[jira] [Commented] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180027#comment-15180027 ] Thomas Graves commented on SPARK-3374: -- [~srowen] can you add [~jerrypeng] as a cont

[jira] [Updated] (SPARK-13459) Separate Alive and Dead Executors in Executor Totals Table

[ https://issues.apache.org/jira/browse/SPARK-13459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13459: -- Assignee: Alex Bozarth > Separate Alive and Dead Executors in Executor Totals Table > -

[jira] [Resolved] (SPARK-13459) Separate Alive and Dead Executors in Executor Totals Table

[ https://issues.apache.org/jira/browse/SPARK-13459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13459. --- Resolution: Fixed Fix Version/s: 2.0.0 > Separate Alive and Dead Executors in Executor

[jira] [Commented] (SPARK-13723) YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true

[ https://issues.apache.org/jira/browse/SPARK-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15183272#comment-15183272 ] Thomas Graves commented on SPARK-13723: --- see some discussion on https://github.com/

[jira] [Created] (SPARK-13723) YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true

Thomas Graves created SPARK-13723: - Summary: YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true Key: SPARK-13723 URL: https://issues.apache.org/jira/browse/SPARK-13723

[jira] [Resolved] (SPARK-13675) The url link in historypage is not correct for application running in yarn cluster mode

[ https://issues.apache.org/jira/browse/SPARK-13675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13675. --- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.0.0 > The url link i

[jira] [Updated] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3374: - Assignee: Boyang Jerry Peng > Spark on Yarn remove deprecated configs for 2.0 > --

[jira] [Commented] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187141#comment-15187141 ] Thomas Graves commented on SPARK-3374: -- It seems this work is also being done under

[jira] [Commented] (SPARK-13723) YARN - Change behavior of --num-executors when spark.dynamicAllocation.enabled true

[ https://issues.apache.org/jira/browse/SPARK-13723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187424#comment-15187424 ] Thomas Graves commented on SPARK-13723: --- Warnings going by when spark submit is sta

[jira] [Created] (SPARK-13775) history server sort by completed time by default

Thomas Graves created SPARK-13775: - Summary: history server sort by completed time by default Key: SPARK-13775 URL: https://issues.apache.org/jira/browse/SPARK-13775 Project: Spark Issue Type

[jira] [Commented] (SPARK-13775) history server sort by completed time by default

[ https://issues.apache.org/jira/browse/SPARK-13775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187707#comment-15187707 ] Thomas Graves commented on SPARK-13775: --- As I stated in the PR, I merged that in be

[jira] [Updated] (SPARK-13642) Properly handle signal kill of ApplicationMaster

[ https://issues.apache.org/jira/browse/SPARK-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13642: -- Assignee: Saisai Shao > Properly handle signal kill of ApplicationMaster >

[jira] [Commented] (SPARK-13775) history server sort by completed time by default

[ https://issues.apache.org/jira/browse/SPARK-13775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187804#comment-15187804 ] Thomas Graves commented on SPARK-13775: --- why does it really matter? Did the versio

[jira] [Commented] (SPARK-13775) history server sort by completed time by default

[ https://issues.apache.org/jira/browse/SPARK-13775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187820#comment-15187820 ] Thomas Graves commented on SPARK-13775: --- Note those were basically rhetorical quest

[jira] [Updated] (SPARK-13577) Allow YARN to handle multiple jars, archive when uploading Spark dependencies

[ https://issues.apache.org/jira/browse/SPARK-13577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13577: -- Assignee: Marcelo Vanzin > Allow YARN to handle multiple jars, archive when uploading Spark dep

[jira] [Resolved] (SPARK-13577) Allow YARN to handle multiple jars, archive when uploading Spark dependencies

[ https://issues.apache.org/jira/browse/SPARK-13577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13577. --- Resolution: Fixed > Allow YARN to handle multiple jars, archive when uploading Spark dependen

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204900#comment-15204900 ] Thomas Graves commented on SPARK-1239: -- So I have been looking at this and testing a

[jira] [Comment Edited] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204900#comment-15204900 ] Thomas Graves edited comment on SPARK-1239 at 3/21/16 7:27 PM: -

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205279#comment-15205279 ] Thomas Graves commented on SPARK-1239: -- I do like the idea of broadcast and originall

[jira] [Commented] (SPARK-3168) The ServletContextHandler of webui lacks a SessionManager

[ https://issues.apache.org/jira/browse/SPARK-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118173#comment-14118173 ] Thomas Graves commented on SPARK-3168: -- Spark is using the ServletContextHandler. Wh

[jira] [Created] (SPARK-3347) Spark alpha doesn't compile due to SPARK-2889

Thomas Graves created SPARK-3347: Summary: Spark alpha doesn't compile due to SPARK-2889 Key: SPARK-3347 URL: https://issues.apache.org/jira/browse/SPARK-3347 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-3347) Spark on yarn alpha doesn't compile due to SPARK-2889

[ https://issues.apache.org/jira/browse/SPARK-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3347: - Summary: Spark on yarn alpha doesn't compile due to SPARK-2889 (was: Spark alpha doesn't compile

[jira] [Resolved] (SPARK-3347) Spark on yarn alpha doesn't compile due to SPARK-2889

[ https://issues.apache.org/jira/browse/SPARK-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3347. -- Resolution: Fixed Fix Version/s: 1.2.0 > Spark on yarn alpha doesn't compile due to SPARK

[jira] [Created] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

Thomas Graves created SPARK-3351: Summary: Yarn YarnRMClientImpl.shutdown can be called before register - NPE Key: SPARK-3351 URL: https://issues.apache.org/jira/browse/SPARK-3351 Project: Spark

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118574#comment-14118574 ] Thomas Graves commented on SPARK-3351: -- Note I was running on yarn alpha but I think

[jira] [Comment Edited] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118574#comment-14118574 ] Thomas Graves edited comment on SPARK-3351 at 9/2/14 7:34 PM: --

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118616#comment-14118616 ] Thomas Graves commented on SPARK-3351: -- Note that this failure scenario also can caus

[jira] [Commented] (SPARK-2718) YARN does not handle spark configs with quotes or backslashes

[ https://issues.apache.org/jira/browse/SPARK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119854#comment-14119854 ] Thomas Graves commented on SPARK-2718: -- [~andrewor] the pr for this is closed, can w

[jira] [Comment Edited] (SPARK-2718) YARN does not handle spark configs with quotes or backslashes

[ https://issues.apache.org/jira/browse/SPARK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14119854#comment-14119854 ] Thomas Graves edited comment on SPARK-2718 at 9/3/14 1:37 PM: --

[jira] [Created] (SPARK-3374) Spark on Yarn config cleanup

Thomas Graves created SPARK-3374: Summary: Spark on Yarn config cleanup Key: SPARK-3374 URL: https://issues.apache.org/jira/browse/SPARK-3374 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-3187) Refactor and cleanup Yarn allocator code

[ https://issues.apache.org/jira/browse/SPARK-3187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3187. -- Resolution: Fixed Fix Version/s: 1.2.0 > Refactor and cleanup Yarn allocator code > -

[jira] [Created] (SPARK-3375) spark on yarn alpha container allocation issues

Thomas Graves created SPARK-3375: Summary: spark on yarn alpha container allocation issues Key: SPARK-3375 URL: https://issues.apache.org/jira/browse/SPARK-3375 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3375) spark on yarn alpha container allocation issues

[ https://issues.apache.org/jira/browse/SPARK-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120281#comment-14120281 ] Thomas Graves commented on SPARK-3375: -- It appears there is a race condition in which

[jira] [Updated] (SPARK-3375) spark on yarn container allocation issues

[ https://issues.apache.org/jira/browse/SPARK-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3375: - Summary: spark on yarn container allocation issues (was: spark on yarn alpha container allocation

[jira] [Commented] (SPARK-3375) spark on yarn alpha container allocation issues

[ https://issues.apache.org/jira/browse/SPARK-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120361#comment-14120361 ] Thomas Graves commented on SPARK-3375: -- Ok so the real issue here is that we are send

[jira] [Commented] (SPARK-3168) The ServletContextHandler of webui lacks a SessionManager

[ https://issues.apache.org/jira/browse/SPARK-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120900#comment-14120900 ] Thomas Graves commented on SPARK-3168: -- posted to the pr. > The ServletContextHandle

[jira] [Assigned] (SPARK-3375) spark on yarn container allocation issues

[ https://issues.apache.org/jira/browse/SPARK-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-3375: Assignee: Thomas Graves > spark on yarn container allocation issues > -

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122064#comment-14122064 ] Thomas Graves commented on SPARK-3129: -- On yarn, it generates the secret automaticall

[jira] [Resolved] (SPARK-3260) Yarn - pass acls along with executor launch

[ https://issues.apache.org/jira/browse/SPARK-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3260. -- Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 (was: 1.1.1, 1.2

[jira] [Resolved] (SPARK-3375) spark on yarn container allocation issues

[ https://issues.apache.org/jira/browse/SPARK-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3375. -- Resolution: Fixed Fix Version/s: 1.2.0 > spark on yarn container allocation issues >

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125981#comment-14125981 ] Thomas Graves commented on SPARK-3129: -- yes that should be enough. > Prevent data lo

[jira] [Created] (SPARK-3456) YarnAllocator can lose container requests to RM

Thomas Graves created SPARK-3456: Summary: YarnAllocator can lose container requests to RM Key: SPARK-3456 URL: https://issues.apache.org/jira/browse/SPARK-3456 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3456) YarnAllocator can lose container requests to RM

[ https://issues.apache.org/jira/browse/SPARK-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127088#comment-14127088 ] Thomas Graves commented on SPARK-3456: -- Note this is only a problem on yarn alpha bec

[jira] [Commented] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127188#comment-14127188 ] Thomas Graves commented on SPARK-3174: -- Since you mention the graceful decommission a

[jira] [Commented] (SPARK-3445) Deprecate and later remove YARN alpha support

[ https://issues.apache.org/jira/browse/SPARK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127204#comment-14127204 ] Thomas Graves commented on SPARK-3445: -- Yahoo moving to 2.x and I'm +1 for this propo

[jira] [Comment Edited] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127188#comment-14127188 ] Thomas Graves edited comment on SPARK-3174 at 9/9/14 4:51 PM: --

<    1   2   3   4   5   6   7   8   9   10   >