[jira] [Created] (SPARK-1499) Workers continuously produce failing executors

Aaron Davidson created SPARK-1499: - Summary: Workers continuously produce failing executors Key: SPARK-1499 URL: https://issues.apache.org/jira/browse/SPARK-1499 Project: Spark Issue Type: Bu

[jira] [Assigned] (SPARK-1456) Clean up use of Ordered/Ordering in OrderedRDDFunctions

[ https://issues.apache.org/jira/browse/SPARK-1456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-1456: --- Assignee: Michael Armbrust > Clean up use of Ordered/Ordering in OrderedRDDFunctions

[jira] [Updated] (SPARK-1487) Support record filtering via predicate pushdown in Parquet

[ https://issues.apache.org/jira/browse/SPARK-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-1487: Fix Version/s: 1.1.0 > Support record filtering via predicate pushdown in Parquet > ---

[jira] [Commented] (SPARK-1498) Spark can hang if pyspark tasks fail

[ https://issues.apache.org/jira/browse/SPARK-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13969237#comment-13969237 ] Shivaram Venkataraman commented on SPARK-1498: -- I see this problem on the mas

[jira] [Updated] (SPARK-1495) support leftsemijoin for sparkSQL

[ https://issues.apache.org/jira/browse/SPARK-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-1495: Affects Version/s: (was: 1.0.0) > support leftsemijoin for sparkSQL > -

[jira] [Updated] (SPARK-1495) support leftsemijoin for sparkSQL

[ https://issues.apache.org/jira/browse/SPARK-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-1495: Fix Version/s: (was: 1.0.0) 1.1.0 > support leftsemijoin for sparkSQ

[jira] [Created] (SPARK-1498) Spark can hang if pyspark tasks fail

Kay Ousterhout created SPARK-1498: - Summary: Spark can hang if pyspark tasks fail Key: SPARK-1498 URL: https://issues.apache.org/jira/browse/SPARK-1498 Project: Spark Issue Type: Bug Affe

[jira] [Updated] (SPARK-1498) Spark can hang if pyspark tasks fail

[ https://issues.apache.org/jira/browse/SPARK-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-1498: -- Description: In pyspark, when some kinds of jobs fail, Spark hangs rather than returning an er

[jira] [Created] (SPARK-1497) Spark YARN code isn't checked with Scalastyle and has many style violations

Patrick Wendell created SPARK-1497: -- Summary: Spark YARN code isn't checked with Scalastyle and has many style violations Key: SPARK-1497 URL: https://issues.apache.org/jira/browse/SPARK-1497 Project

[jira] [Resolved] (SPARK-1137) ZK Persistence Engine crashes if stored data has wrong serialVersionUID

[ https://issues.apache.org/jira/browse/SPARK-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-1137. --- Resolution: Fixed > ZK Persistence Engine crashes if stored data has wrong serialVersionUID >

[jira] [Resolved] (SPARK-1136) Fix FaultToleranceTest for Docker 0.8.1

[ https://issues.apache.org/jira/browse/SPARK-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-1136. --- Resolution: Fixed > Fix FaultToleranceTest for Docker 0.8.1 > ---

[jira] [Resolved] (SPARK-1488) Resolve scalac feature warnings during build

[ https://issues.apache.org/jira/browse/SPARK-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1488. Resolution: Fixed Fix Version/s: 1.0.0 Assignee: Sean Owen > Resolve scalac

[jira] [Created] (SPARK-1496) SparkContext.jarOfClass should return Option instead of a sequence

Patrick Wendell created SPARK-1496: -- Summary: SparkContext.jarOfClass should return Option instead of a sequence Key: SPARK-1496 URL: https://issues.apache.org/jira/browse/SPARK-1496 Project: Spark

[jira] [Created] (SPARK-1495) support leftsemijoin for sparkSQL

Adrian Wang created SPARK-1495: -- Summary: support leftsemijoin for sparkSQL Key: SPARK-1495 URL: https://issues.apache.org/jira/browse/SPARK-1495 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-1494) Hive Dependencies being checked by MIMA

[ https://issues.apache.org/jira/browse/SPARK-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13969113#comment-13969113 ] Michael Armbrust commented on SPARK-1494: - We could change the object to contain a

[jira] [Created] (SPARK-1494) Hive Dependencies being checked by MIMA

Ahir Reddy created SPARK-1494: - Summary: Hive Dependencies being checked by MIMA Key: SPARK-1494 URL: https://issues.apache.org/jira/browse/SPARK-1494 Project: Spark Issue Type: Bug Com

[jira] [Comment Edited] (SPARK-1476) 2GB limit in spark for blocks

[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968974#comment-13968974 ] Patrick Wendell edited comment on SPARK-1476 at 4/14/14 11:05 PM: --

[jira] [Commented] (SPARK-1476) 2GB limit in spark for blocks

[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968974#comment-13968974 ] Patrick Wendell commented on SPARK-1476: Okay sounds good - a POC like that would

[jira] [Updated] (SPARK-1493) Apache RAT excludes don't work with file path (instead of file name)

[ https://issues.apache.org/jira/browse/SPARK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1493: --- Labels: starter (was: ) > Apache RAT excludes don't work with file path (instead of file nam

[jira] [Created] (SPARK-1493) Apache RAT excludes don't work with file path (instead of file name)

Patrick Wendell created SPARK-1493: -- Summary: Apache RAT excludes don't work with file path (instead of file name) Key: SPARK-1493 URL: https://issues.apache.org/jira/browse/SPARK-1493 Project: Spark

[jira] [Resolved] (SPARK-1436) Compression code broke in-memory store

[ https://issues.apache.org/jira/browse/SPARK-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1436. Resolution: Fixed > Compression code broke in-memory store > --

[jira] [Created] (SPARK-1492) running-on-yarn doc should use spark-submit script for examples

Thomas Graves created SPARK-1492: Summary: running-on-yarn doc should use spark-submit script for examples Key: SPARK-1492 URL: https://issues.apache.org/jira/browse/SPARK-1492 Project: Spark

[jira] [Commented] (SPARK-1474) Spark on yarn assembly doesn't include org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968884#comment-13968884 ] Thomas Graves commented on SPARK-1474: -- https://github.com/apache/spark/pull/406 > S

[jira] [Commented] (SPARK-1478) Upgrade FlumeInputDStream's FlumeReceiver to support FLUME-1915

[ https://issues.apache.org/jira/browse/SPARK-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968880#comment-13968880 ] Ted Malaska commented on SPARK-1478: Chated with tdas. He reviewed the code, but he h

[jira] [Commented] (SPARK-1474) Spark on yarn assembly doesn't include org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968577#comment-13968577 ] Xuan Gong commented on SPARK-1474: -- Thanks, [~tgraves]. I messed up the configurations..

[jira] [Created] (SPARK-1491) maven hadoop-provided profile fails to build

Thomas Graves created SPARK-1491: Summary: maven hadoop-provided profile fails to build Key: SPARK-1491 URL: https://issues.apache.org/jira/browse/SPARK-1491 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-1474) Spark on yarn assembly doesn't include org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968566#comment-13968566 ] Thomas Graves commented on SPARK-1474: -- do you have a /* at the end to pick up the ja

[jira] [Commented] (SPARK-1478) Upgrade FlumeInputDStream's FlumeReceiver to support FLUME-1915

[ https://issues.apache.org/jira/browse/SPARK-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968561#comment-13968561 ] Ted Malaska commented on SPARK-1478: I sent the initial patch. Here is the link. htt

[jira] [Commented] (SPARK-1474) Spark on yarn assembly doesn't include org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968550#comment-13968550 ] Xuan Gong commented on SPARK-1474: -- [~tgraves] I think so, I did add YARN_HOME into class

[jira] [Updated] (SPARK-1474) Spark on yarn assembly doesn't include org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1474: - Summary: Spark on yarn assembly doesn't include org.apache.hadoop.yarn.server.webproxy.amfilter.A

[jira] [Updated] (SPARK-1474) Spark on yarn assembly doesn't include

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1474: - Summary: Spark on yarn assembly doesn't include (was: Can not Launch spark application by YARN

[jira] [Assigned] (SPARK-1474) Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark code from master branch with hadoop-2.2.0

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1474: Assignee: Thomas Graves > Can not Launch spark application by YARN Client with yarn-standal

[jira] [Updated] (SPARK-1465) Spark compilation is broken with the latest hadoop-2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1465: --- Fix Version/s: 1.0.0 > Spark compilation is broken with the latest hadoop-2.4.0 release > ---

[jira] [Updated] (SPARK-1465) Spark compilation is broken with the latest hadoop-2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1465: --- Priority: Blocker (was: Major) > Spark compilation is broken with the latest hadoop-2.4.0 re

[jira] [Resolved] (SPARK-1480) Choose classloader consistently inside of Spark codebase

[ https://issues.apache.org/jira/browse/SPARK-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1480. Resolution: Fixed > Choose classloader consistently inside of Spark codebase >

[jira] [Created] (SPARK-1490) Add kerberos support to the HistoryServer

Thomas Graves created SPARK-1490: Summary: Add kerberos support to the HistoryServer Key: SPARK-1490 URL: https://issues.apache.org/jira/browse/SPARK-1490 Project: Spark Issue Type: Improveme

[jira] [Comment Edited] (SPARK-1474) Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark code from master branch with hadoop-2.2.0

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968465#comment-13968465 ] Thomas Graves edited comment on SPARK-1474 at 4/14/14 4:06 PM: -

[jira] [Commented] (SPARK-1474) Can not Launch spark application by YARN Client with yarn-standalone mode when compiling spark code from master branch with hadoop-2.2.0

[ https://issues.apache.org/jira/browse/SPARK-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968465#comment-13968465 ] Thomas Graves commented on SPARK-1474: -- are you including the yarn jars in the classp

[jira] [Created] (SPARK-1489) Fix the HistoryServer view acls

Thomas Graves created SPARK-1489: Summary: Fix the HistoryServer view acls Key: SPARK-1489 URL: https://issues.apache.org/jira/browse/SPARK-1489 Project: Spark Issue Type: Bug Compo

[jira] [Commented] (SPARK-1453) Improve the way Spark on Yarn waits for executors before starting

[ https://issues.apache.org/jira/browse/SPARK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968390#comment-13968390 ] Mridul Muralidharan commented on SPARK-1453: (d) becomes relevant in case of

[jira] [Commented] (SPARK-1453) Improve the way Spark on Yarn waits for executors before starting

[ https://issues.apache.org/jira/browse/SPARK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968374#comment-13968374 ] Thomas Graves commented on SPARK-1453: -- Actually there are 2 timeouts. The one you me

[jira] [Created] (SPARK-1487) Support record filtering via predicate pushdown in Parquet

Andre Schumacher created SPARK-1487: --- Summary: Support record filtering via predicate pushdown in Parquet Key: SPARK-1487 URL: https://issues.apache.org/jira/browse/SPARK-1487 Project: Spark

[jira] [Created] (SPARK-1488) Resolve scalac feature warnings during build

Sean Owen created SPARK-1488: Summary: Resolve scalac feature warnings during build Key: SPARK-1488 URL: https://issues.apache.org/jira/browse/SPARK-1488 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-1293) Support for reading/writing complex types in Parquet

[ https://issues.apache.org/jira/browse/SPARK-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968332#comment-13968332 ] Andre Schumacher commented on SPARK-1293: - There is a preliminary pull request now

[jira] [Commented] (SPARK-1476) 2GB limit in spark for blocks

[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968147#comment-13968147 ] Mridul Muralidharan commented on SPARK-1476: [~pwendell] IMO both are limitati

[jira] [Assigned] (SPARK-1476) 2GB limit in spark for blocks

[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-1476: -- Assignee: Mridul Muralidharan > 2GB limit in spark for blocks > ---

[jira] [Created] (SPARK-1486) Support multi-model training in MLlib

Xiangrui Meng created SPARK-1486: Summary: Support multi-model training in MLlib Key: SPARK-1486 URL: https://issues.apache.org/jira/browse/SPARK-1486 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-1485) Implement AllReduce

Xiangrui Meng created SPARK-1485: Summary: Implement AllReduce Key: SPARK-1485 URL: https://issues.apache.org/jira/browse/SPARK-1485 Project: Spark Issue Type: Improvement Component