[jira] [Updated] (SPARK-2456) Scheduler refactoring

[ https://issues.apache.org/jira/browse/SPARK-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2456: --- Description: This is an umbrella ticket to track scheduler refactoring. We want to clearly define

[jira] [Commented] (SPARK-2456) Scheduler refactoring

[ https://issues.apache.org/jira/browse/SPARK-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072860#comment-14072860 ] Reynold Xin commented on SPARK-2456: One related PR:

[jira] [Updated] (SPARK-2310) Support arbitrary options on the command line with spark-submit

[ https://issues.apache.org/jira/browse/SPARK-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2310: --- Assignee: Sandy Ryza Support arbitrary options on the command line with spark-submit

[jira] [Resolved] (SPARK-2310) Support arbitrary options on the command line with spark-submit

[ https://issues.apache.org/jira/browse/SPARK-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2310. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1253

[jira] [Created] (SPARK-2664) Deal with `--conf` options in spark-submit that relate to flags

Patrick Wendell created SPARK-2664: -- Summary: Deal with `--conf` options in spark-submit that relate to flags Key: SPARK-2664 URL: https://issues.apache.org/jira/browse/SPARK-2664 Project: Spark

[jira] [Updated] (SPARK-2664) Deal with `--conf` options in spark-submit that relate to flags

[ https://issues.apache.org/jira/browse/SPARK-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2664: --- Description: If someone sets a spark conf that relates to an existing flag `--master`, we

[jira] [Commented] (SPARK-2652) Turning default configurations for PySpark

[ https://issues.apache.org/jira/browse/SPARK-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072909#comment-14072909 ] Apache Spark commented on SPARK-2652: - User 'davies' has created a pull request for

[jira] [Resolved] (SPARK-2661) Unpersist last RDD in bagel iteration

[ https://issues.apache.org/jira/browse/SPARK-2661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-2661. -- Resolution: Fixed Unpersist last RDD in bagel iteration

[jira] [Commented] (SPARK-2664) Deal with `--conf` options in spark-submit that relate to flags

[ https://issues.apache.org/jira/browse/SPARK-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072925#comment-14072925 ] Sandy Ryza commented on SPARK-2664: --- I think the right behavior here is worth a little

[jira] [Comment Edited] (SPARK-2664) Deal with `--conf` options in spark-submit that relate to flags

[ https://issues.apache.org/jira/browse/SPARK-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072925#comment-14072925 ] Sandy Ryza edited comment on SPARK-2664 at 7/24/14 7:18 AM: I

[jira] [Created] (SPARK-2665) Add EqualNS support for HiveQL

Cheng Hao created SPARK-2665: Summary: Add EqualNS support for HiveQL Key: SPARK-2665 URL: https://issues.apache.org/jira/browse/SPARK-2665 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-2414) Remove jquery

[ https://issues.apache.org/jira/browse/SPARK-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2414: --- Assignee: (was: Reynold Xin) Remove jquery - Key: SPARK-2414

[jira] [Comment Edited] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073013#comment-14073013 ] lukovnikov edited comment on SPARK-1405 at 7/24/14 9:10 AM:

[jira] [Commented] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073020#comment-14073020 ] lukovnikov commented on SPARK-1405: --- btw, could this please be merged with the main?

[jira] [Commented] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073085#comment-14073085 ] Apache Spark commented on SPARK-2604: - User 'twinkle-sachdeva' has created a pull

[jira] [Commented] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073086#comment-14073086 ] Twinkle Sachdeva commented on SPARK-2604: - Please review the pull request :

[jira] [Commented] (SPARK-2575) SVMWithSGD throwing Input Validation failed

[ https://issues.apache.org/jira/browse/SPARK-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073111#comment-14073111 ] navanee commented on SPARK-2575: spark SVM supports multinomial or binomial

[jira] [Created] (SPARK-2666) when task is FetchFailed cancel running tasks of failedStage

Lianhui Wang created SPARK-2666: --- Summary: when task is FetchFailed cancel running tasks of failedStage Key: SPARK-2666 URL: https://issues.apache.org/jira/browse/SPARK-2666 Project: Spark

[jira] [Commented] (SPARK-2576) slave node throws NoClassDefFoundError $line11.$read$ when executing a Spark QL query on HDFS CSV file

[ https://issues.apache.org/jira/browse/SPARK-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073114#comment-14073114 ] Prashant Sharma commented on SPARK-2576: Looking at it. slave node throws

[jira] [Commented] (SPARK-2666) when task is FetchFailed cancel running tasks of failedStage

[ https://issues.apache.org/jira/browse/SPARK-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073116#comment-14073116 ] Apache Spark commented on SPARK-2666: - User 'lianhuiwang' has created a pull request

[jira] [Commented] (SPARK-2456) Scheduler refactoring

[ https://issues.apache.org/jira/browse/SPARK-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073122#comment-14073122 ] Nan Zhu commented on SPARK-2456: maybe it's also related:

[jira] [Updated] (SPARK-2667) getCallSiteInfo doesn't take into account that graphx is part of spark.

[ https://issues.apache.org/jira/browse/SPARK-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Budau updated SPARK-2667: Description: getCallSiteInfo from org.apache.spark.util.Utils uses a regex pattern to match when a

[jira] [Created] (SPARK-2667) getCallSiteInfo doesn't take into account that graphx is part of spark.

Adrian Budau created SPARK-2667: --- Summary: getCallSiteInfo doesn't take into account that graphx is part of spark. Key: SPARK-2667 URL: https://issues.apache.org/jira/browse/SPARK-2667 Project: Spark

[jira] [Created] (SPARK-2668) Support log4j log to yarn container log directory

Peng Zhang created SPARK-2668: - Summary: Support log4j log to yarn container log directory Key: SPARK-2668 URL: https://issues.apache.org/jira/browse/SPARK-2668 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2668) Support log4j log to yarn container log directory

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073199#comment-14073199 ] Apache Spark commented on SPARK-2668: - User 'renozhang' has created a pull request for

[jira] [Updated] (SPARK-2150) Provide direct link to finished application UI in yarn resource manager UI

[ https://issues.apache.org/jira/browse/SPARK-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2150: - Assignee: Rahul Singhal Provide direct link to finished application UI in yarn resource manager

[jira] [Commented] (SPARK-1112) When spark.akka.frameSize 10, task results bigger than 10MiB block execution

[ https://issues.apache.org/jira/browse/SPARK-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073244#comment-14073244 ] DjvuLee commented on SPARK-1112: Does anyone test in version0.9.2，I found it also failed ,

[jira] [Commented] (SPARK-2668) Support log4j log to yarn container log directory

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073246#comment-14073246 ] Thomas Graves commented on SPARK-2668: -- Sorry I don't follow what you are saying

[jira] [Commented] (SPARK-2668) Support log4j log to yarn container log directory

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073278#comment-14073278 ] Peng Zhang commented on SPARK-2668: --- [~tgraves] Original log works fine, and log will be

[jira] [Comment Edited] (SPARK-2668) Support log4j log to yarn container log directory

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073278#comment-14073278 ] Peng Zhang edited comment on SPARK-2668 at 7/24/14 3:12 PM:

[jira] [Commented] (SPARK-2668) Support log4j log to yarn container log directory

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073310#comment-14073310 ] Thomas Graves commented on SPARK-2668: -- Oh, I see you just want a variable to

[jira] [Commented] (SPARK-2575) SVMWithSGD throwing Input Validation failed

[ https://issues.apache.org/jira/browse/SPARK-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073329#comment-14073329 ] Xiangrui Meng commented on SPARK-2575: -- [~dbtsai] sent a PR for multinomial logistic

[jira] [Created] (SPARK-2669) Hadoop configuration is not localised when submitting job in yarn-cluster mode

Maxim Ivanov created SPARK-2669: --- Summary: Hadoop configuration is not localised when submitting job in yarn-cluster mode Key: SPARK-2669 URL: https://issues.apache.org/jira/browse/SPARK-2669 Project:

[jira] [Commented] (SPARK-2669) Hadoop configuration is not localised when submitting job in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073338#comment-14073338 ] Apache Spark commented on SPARK-2669: - User 'redbaron' has created a pull request for

[jira] [Updated] (SPARK-1264) Documentation for setting heap sizes across all configurations

[ https://issues.apache.org/jira/browse/SPARK-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson updated SPARK-1264: -- Assignee: (was: Aaron Davidson) Documentation for setting heap sizes across all

[jira] [Commented] (SPARK-2583) ConnectionManager cannot distinguish whether error occurred or not

[ https://issues.apache.org/jira/browse/SPARK-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073419#comment-14073419 ] Kousuke Saruta commented on SPARK-2583: --- I have added some test cases to my PR for

[jira] [Commented] (SPARK-2479) Comparing floating-point numbers using relative error in UnitTests

[ https://issues.apache.org/jira/browse/SPARK-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073430#comment-14073430 ] Apache Spark commented on SPARK-2479: - User 'mengxr' has created a pull request for

[jira] [Updated] (SPARK-2538) External aggregation in Python

[ https://issues.apache.org/jira/browse/SPARK-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2538: - Priority: Critical (was: Major) External aggregation in Python --

[jira] [Created] (SPARK-2670) FetchFailedException should be thrown when local fetch has failed

Kousuke Saruta created SPARK-2670: - Summary: FetchFailedException should be thrown when local fetch has failed Key: SPARK-2670 URL: https://issues.apache.org/jira/browse/SPARK-2670 Project: Spark

[jira] [Updated] (SPARK-2619) Configurable file-mode for spark/bin folder in the .deb package.

[ https://issues.apache.org/jira/browse/SPARK-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2619: --- Assignee: Christian Tzolov Configurable file-mode for spark/bin folder in the .deb package.

[jira] [Resolved] (SPARK-2619) Configurable file-mode for spark/bin folder in the .deb package.

[ https://issues.apache.org/jira/browse/SPARK-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2619. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1531

[jira] [Created] (SPARK-2671) BlockObjectWriter should create parent directory when the directory doesn't exist

Kousuke Saruta created SPARK-2671: - Summary: BlockObjectWriter should create parent directory when the directory doesn't exist Key: SPARK-2671 URL: https://issues.apache.org/jira/browse/SPARK-2671

[jira] [Updated] (SPARK-2603) Remove unnecessary toMap and toList in converting Java collections to Scala collections JsonRDD.scala

[ https://issues.apache.org/jira/browse/SPARK-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2603: Fix Version/s: 1.0.2 1.1.0 Remove unnecessary toMap and toList in

[jira] [Resolved] (SPARK-2603) Remove unnecessary toMap and toList in converting Java collections to Scala collections JsonRDD.scala

[ https://issues.apache.org/jira/browse/SPARK-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2603. - Resolution: Fixed Remove unnecessary toMap and toList in converting Java collections to

[jira] [Created] (SPARK-2672) support compressed file in wholeFile()

Davies Liu created SPARK-2672: - Summary: support compressed file in wholeFile() Key: SPARK-2672 URL: https://issues.apache.org/jira/browse/SPARK-2672 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-2673) Improve Spark so that we can attach Debugger to Executors easily

Kousuke Saruta created SPARK-2673: - Summary: Improve Spark so that we can attach Debugger to Executors easily Key: SPARK-2673 URL: https://issues.apache.org/jira/browse/SPARK-2673 Project: Spark

[jira] [Commented] (SPARK-2464) Twitter Receiver does not stop correctly when streamingContext.stop is called

[ https://issues.apache.org/jira/browse/SPARK-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073518#comment-14073518 ] Apache Spark commented on SPARK-2464: - User 'tdas' has created a pull request for this

[jira] [Commented] (SPARK-1154) Spark fills up disk with app-* folders

[ https://issues.apache.org/jira/browse/SPARK-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073521#comment-14073521 ] Andrew Ash commented on SPARK-1154: --- For the record, this is Evan's PR that closed this

[jira] [Commented] (SPARK-2670) FetchFailedException should be thrown when local fetch has failed

[ https://issues.apache.org/jira/browse/SPARK-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073539#comment-14073539 ] Apache Spark commented on SPARK-2670: - User 'sarutak' has created a pull request for

[jira] [Created] (SPARK-2674) Add date and time types to inferSchema

Hossein Falaki created SPARK-2674: - Summary: Add date and time types to inferSchema Key: SPARK-2674 URL: https://issues.apache.org/jira/browse/SPARK-2674 Project: Spark Issue Type: New

[jira] [Closed] (SPARK-2676) CLONE - LiveListenerBus should set higher capacity for its event queue

[ https://issues.apache.org/jira/browse/SPARK-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zongheng Yang closed SPARK-2676. Resolution: Duplicate CLONE - LiveListenerBus should set higher capacity for its event queue

[jira] [Commented] (SPARK-2675) LiveListenerBus should set higher capacity for its event queue

[ https://issues.apache.org/jira/browse/SPARK-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073546#comment-14073546 ] Apache Spark commented on SPARK-2675: - User 'concretevitamin' has created a pull

[jira] [Commented] (SPARK-2671) BlockObjectWriter should create parent directory when the directory doesn't exist

[ https://issues.apache.org/jira/browse/SPARK-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073548#comment-14073548 ] Apache Spark commented on SPARK-2671: - User 'sarutak' has created a pull request for

[jira] [Resolved] (SPARK-2037) yarn client mode doesn't support spark.yarn.max.executor.failures

[ https://issues.apache.org/jira/browse/SPARK-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2037. -- Resolution: Fixed Fix Version/s: 1.1.0 yarn client mode doesn't support

[jira] [Updated] (SPARK-2674) Add date and time types to inferSchema

[ https://issues.apache.org/jira/browse/SPARK-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2674: Assignee: Davies Liu (was: Michael Armbrust) Add date and time types to inferSchema

[jira] [Assigned] (SPARK-2674) Add date and time types to inferSchema

[ https://issues.apache.org/jira/browse/SPARK-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-2674: --- Assignee: Michael Armbrust Add date and time types to inferSchema

[jira] [Updated] (SPARK-2674) Add date and time types to inferSchema

[ https://issues.apache.org/jira/browse/SPARK-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2674: Target Version/s: 1.1.0 Add date and time types to inferSchema

[jira] [Commented] (SPARK-2387) Remove the stage barrier for better resource utilization

[ https://issues.apache.org/jira/browse/SPARK-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073597#comment-14073597 ] Kay Ousterhout commented on SPARK-2387: --- Have you done experiments to understand how

[jira] [Comment Edited] (SPARK-2387) Remove the stage barrier for better resource utilization

[ https://issues.apache.org/jira/browse/SPARK-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073597#comment-14073597 ] Kay Ousterhout edited comment on SPARK-2387 at 7/24/14 8:23 PM:

[jira] [Updated] (SPARK-2250) show stage RDDs in UI

[ https://issues.apache.org/jira/browse/SPARK-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2250: --- Assignee: Neville Li show stage RDDs in UI - Key:

[jira] [Resolved] (SPARK-2250) show stage RDDs in UI

[ https://issues.apache.org/jira/browse/SPARK-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2250. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1188

[jira] [Created] (SPARK-2677) BasicBlockFetchIterator#next can be wait forever

Kousuke Saruta created SPARK-2677: - Summary: BasicBlockFetchIterator#next can be wait forever Key: SPARK-2677 URL: https://issues.apache.org/jira/browse/SPARK-2677 Project: Spark Issue Type:

[jira] [Updated] (SPARK-2677) BasicBlockFetchIterator#next can wait forever

[ https://issues.apache.org/jira/browse/SPARK-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2677: -- Summary: BasicBlockFetchIterator#next can wait forever (was: BasicBlockFetchIterator#next can

[jira] [Commented] (SPARK-1855) Provide memory-and-local-disk RDD checkpointing

[ https://issues.apache.org/jira/browse/SPARK-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073784#comment-14073784 ] koert kuipers commented on SPARK-1855: -- i think this makes sense. we have iterative

[jira] [Updated] (SPARK-2678) `Spark-submit` overrides user application options

[ https://issues.apache.org/jira/browse/SPARK-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-2678: -- Priority: Major (was: Minor) `Spark-submit` overrides user application options

[jira] [Created] (SPARK-2679) Ser/De for Double to enable calling Java API from python in MLlib

Doris Xin created SPARK-2679: Summary: Ser/De for Double to enable calling Java API from python in MLlib Key: SPARK-2679 URL: https://issues.apache.org/jira/browse/SPARK-2679 Project: Spark

[jira] [Commented] (SPARK-2679) Ser/De for Double to enable calling Java API from python in MLlib

[ https://issues.apache.org/jira/browse/SPARK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073833#comment-14073833 ] Apache Spark commented on SPARK-2679: - User 'dorx' has created a pull request for this

[jira] [Updated] (SPARK-2298) Show stage attempt in UI

[ https://issues.apache.org/jira/browse/SPARK-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2298: --- Priority: Critical (was: Major) Show stage attempt in UI

[jira] [Commented] (SPARK-2515) Hypothesis testing

[ https://issues.apache.org/jira/browse/SPARK-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073879#comment-14073879 ] Doris Xin commented on SPARK-2515: -- Here's the proposed API for chi-squared tests (lives

[jira] [Resolved] (SPARK-2464) Twitter Receiver does not stop correctly when streamingContext.stop is called

[ https://issues.apache.org/jira/browse/SPARK-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-2464. -- Resolution: Fixed Twitter Receiver does not stop correctly when streamingContext.stop is

[jira] [Resolved] (SPARK-2014) Make PySpark store RDDs in MEMORY_ONLY_SER with compression by default

[ https://issues.apache.org/jira/browse/SPARK-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-2014. -- Resolution: Fixed Fix Version/s: 1.1.0 Make PySpark store RDDs in MEMORY_ONLY_SER with

[jira] [Commented] (SPARK-1044) Default spark logs location in EC2 AMI leads to out-of-disk space pretty soon

[ https://issues.apache.org/jira/browse/SPARK-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073930#comment-14073930 ] Andrew Ash commented on SPARK-1044: --- Filling up the work dir could be alleviated by

[jira] [Commented] (SPARK-786) Clean up old work directories in standalone worker

[ https://issues.apache.org/jira/browse/SPARK-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073932#comment-14073932 ] Andrew Ash commented on SPARK-786: -- Agreed. With SPARK-1860 we could re-enable that the

[jira] [Commented] (SPARK-1044) Default spark logs location in EC2 AMI leads to out-of-disk space pretty soon

[ https://issues.apache.org/jira/browse/SPARK-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073938#comment-14073938 ] Allan Douglas R. de Oliveira commented on SPARK-1044: - I think it is

[jira] [Resolved] (SPARK-1030) unneeded file required when running pyspark program using yarn-client

[ https://issues.apache.org/jira/browse/SPARK-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-1030. --- Resolution: Fixed Fix Version/s: 1.0.0 Closing this now, since it was addressed as part of

[jira] [Created] (SPARK-2680) Lower spark.shuffle.memoryFraction to 0.2 by default

Matei Zaharia created SPARK-2680: Summary: Lower spark.shuffle.memoryFraction to 0.2 by default Key: SPARK-2680 URL: https://issues.apache.org/jira/browse/SPARK-2680 Project: Spark Issue

[jira] [Updated] (SPARK-2529) Clean the closure in foreach and foreachPartition

[ https://issues.apache.org/jira/browse/SPARK-2529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2529: - Target Version/s: 1.1.0, 1.0.3 (was: 1.1.0, 1.0.2) Clean the closure in foreach and

[jira] [Updated] (SPARK-2531) Make BroadcastNestedLoopJoin take into account a BuildSide

[ https://issues.apache.org/jira/browse/SPARK-2531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2531: - Target Version/s: 1.1.0, 1.0.3 (was: 1.1.0, 1.0.2) Make BroadcastNestedLoopJoin take into

[jira] [Updated] (SPARK-2548) JavaRecoverableWordCount is missing

[ https://issues.apache.org/jira/browse/SPARK-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2548: - Target Version/s: 1.1.0, 0.9.3, 1.0.3 (was: 1.1.0, 1.0.2, 0.9.3) JavaRecoverableWordCount is

[jira] [Updated] (SPARK-2506) In yarn-cluster mode, ApplicationMaster does not clean up correctly at the end of the job if users call sc.stop manually

[ https://issues.apache.org/jira/browse/SPARK-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2506: - Target Version/s: 1.0.3 (was: 1.0.2) In yarn-cluster mode, ApplicationMaster does not clean up

[jira] [Updated] (SPARK-1667) Jobs never finish successfully once bucket file missing occurred

[ https://issues.apache.org/jira/browse/SPARK-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-1667: - Target Version/s: 1.1.0, 1.0.3 (was: 1.1.0, 1.0.2) Jobs never finish successfully once bucket

[jira] [Updated] (SPARK-2558) Mention --queue argument in YARN documentation

[ https://issues.apache.org/jira/browse/SPARK-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2558: - Target Version/s: 1.1.0, 1.0.3 (was: 1.1.0, 1.0.2) Mention --queue argument in YARN

[jira] [Updated] (SPARK-2576) slave node throws NoClassDefFoundError $line11.$read$ when executing a Spark QL query on HDFS CSV file

[ https://issues.apache.org/jira/browse/SPARK-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2576: - Target Version/s: 1.1.0, 1.0.3 (was: 1.1.0, 1.0.2) slave node throws NoClassDefFoundError

[jira] [Updated] (SPARK-2425) Standalone Master is too aggressive in removing Applications

[ https://issues.apache.org/jira/browse/SPARK-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2425: - Target Version/s: 1.0.3 (was: 1.0.2) Standalone Master is too aggressive in removing

[jira] [Updated] (SPARK-2541) Standalone mode can't access secure HDFS anymore

[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2541: - Target Version/s: 1.1.0, 1.0.3 (was: 1.1.0, 1.0.2) Standalone mode can't access secure HDFS

[jira] [Commented] (SPARK-2529) Clean the closure in foreach and foreachPartition

[ https://issues.apache.org/jira/browse/SPARK-2529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073964#comment-14073964 ] Apache Spark commented on SPARK-2529: - User 'rxin' has created a pull request for this

[jira] [Updated] (SPARK-2668) Support log4j log to yarn container log directory

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peng Zhang updated SPARK-2668: -- Affects Version/s: 1.0.0 Support log4j log to yarn container log directory

[jira] [Updated] (SPARK-2668) Add variable of yarn log directory for reference from the log4j configuration

[ https://issues.apache.org/jira/browse/SPARK-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peng Zhang updated SPARK-2668: -- Description: Assign value of yarn container log directory to java opts spark.yarn.log.dir, So user

[jira] [Commented] (SPARK-2681) With low probability, the Spark inexplicable hang

[ https://issues.apache.org/jira/browse/SPARK-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074031#comment-14074031 ] Patrick Wendell commented on SPARK-2681: Can you do a jstack of the executor when

[jira] [Commented] (SPARK-2681) Spark can hang when fetching shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074044#comment-14074044 ] Guoqiang Li commented on SPARK-2681: OK, but have some time. Spark can hang when

[jira] [Commented] (SPARK-2618) use config spark.scheduler.priority for specifying TaskSet's priority on DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074045#comment-14074045 ] Patrick Wendell commented on SPARK-2618: We shouldn't should expose these types of

[jira] [Comment Edited] (SPARK-2681) Spark can hang when fetching shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074044#comment-14074044 ] Guoqiang Li edited comment on SPARK-2681 at 7/25/14 4:39 AM: -

[jira] [Updated] (SPARK-2670) FetchFailedException should be thrown when local fetch has failed

[ https://issues.apache.org/jira/browse/SPARK-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2670: --- Priority: Critical (was: Major) FetchFailedException should be thrown when local fetch has

[jira] [Updated] (SPARK-2670) FetchFailedException should be thrown when local fetch has failed

[ https://issues.apache.org/jira/browse/SPARK-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2670: --- Component/s: Spark Core FetchFailedException should be thrown when local fetch has failed

[jira] [Updated] (SPARK-2670) FetchFailedException should be thrown when local fetch has failed

[ https://issues.apache.org/jira/browse/SPARK-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2670: --- Target Version/s: 1.1.0 FetchFailedException should be thrown when local fetch has failed

[jira] [Created] (SPARK-2682) Javadoc generated from Scala source code is not in javadoc's index

Yin Huai created SPARK-2682: --- Summary: Javadoc generated from Scala source code is not in javadoc's index Key: SPARK-2682 URL: https://issues.apache.org/jira/browse/SPARK-2682 Project: Spark

[jira] [Updated] (SPARK-2682) Javadoc generated from Scala source code is not in javadoc's index

[ https://issues.apache.org/jira/browse/SPARK-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-2682: Component/s: Documentation Javadoc generated from Scala source code is not in javadoc's index

[jira] [Commented] (SPARK-2682) Javadoc generated from Scala source code is not in javadoc's index

[ https://issues.apache.org/jira/browse/SPARK-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074109#comment-14074109 ] Apache Spark commented on SPARK-2682: - User 'yhuai' has created a pull request for

[jira] [Commented] (SPARK-2664) Deal with `--conf` options in spark-submit that relate to flags

[ https://issues.apache.org/jira/browse/SPARK-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074108#comment-14074108 ] Patrick Wendell commented on SPARK-2664: Hey Sandy, The reason why we originally

[jira] [Resolved] (SPARK-2538) External aggregation in Python

[ https://issues.apache.org/jira/browse/SPARK-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-2538. -- Resolution: Fixed Fix Version/s: (was: 1.0.1) (was: 1.0.0)

  1   2   >