[jira] [Resolved] (SPARK-3361) Expand PEP 8 checks to include EC2 script and Python examples

[ https://issues.apache.org/jira/browse/SPARK-3361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3361. Resolution: Fixed Fix Version/s: (was: 1.1.0) 1.2.0 > Expand PEP 8 che

[jira] [Updated] (SPARK-3349) Incorrect partitioning after LIMIT operator

[ https://issues.apache.org/jira/browse/SPARK-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3349: --- Assignee: Eric Liang > Incorrect partitioning after LIMIT operator > -

[jira] [Updated] (SPARK-3349) Incorrect partitioning after LIMIT operator

[ https://issues.apache.org/jira/browse/SPARK-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3349: --- Description: Reproduced by the following example: {code} import org.apache.spark.sql.catalyst.plans.In

[jira] [Commented] (SPARK-3361) Expand PEP 8 checks to include EC2 script and Python examples

[ https://issues.apache.org/jira/browse/SPARK-3361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14124293#comment-14124293 ] Apache Spark commented on SPARK-3361: - User 'nchammas' has created a pull request for

[jira] [Updated] (SPARK-3411) Improve load-balancing of concurrently-submitted drivers across workers

[ https://issues.apache.org/jira/browse/SPARK-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangTaoTheTonic updated SPARK-3411: --- Description: If the waiting driver array is too big, the drivers in it will be dispatched to

[jira] [Updated] (SPARK-3411) Improve load-balancing of concurrently-submitted drivers across workers

[ https://issues.apache.org/jira/browse/SPARK-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangTaoTheTonic updated SPARK-3411: --- Summary: Improve load-balancing of concurrently-submitted drivers across workers (was: Optimi

[jira] [Created] (SPARK-3424) KMeans Plus Plus is too slow

Derrick Burns created SPARK-3424: Summary: KMeans Plus Plus is too slow Key: SPARK-3424 URL: https://issues.apache.org/jira/browse/SPARK-3424 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-3211) .take() is OOM-prone when there are empty partitions

[ https://issues.apache.org/jira/browse/SPARK-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3211. -- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 > .take() is OOM-prone wh

[jira] [Updated] (SPARK-3211) .take() is OOM-prone when there are empty partitions

[ https://issues.apache.org/jira/browse/SPARK-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3211: - Assignee: Andrew Ash > .take() is OOM-prone when there are empty partitions >

[jira] [Commented] (SPARK-2334) Attribute Error calling PipelinedRDD.id() in pyspark

[ https://issues.apache.org/jira/browse/SPARK-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14124239#comment-14124239 ] Apache Spark commented on SPARK-2334: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-3423) Implement BETWEEN support for regular SQL parser

[ https://issues.apache.org/jira/browse/SPARK-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14124238#comment-14124238 ] Apache Spark commented on SPARK-3423: - User 'willb' has created a pull request for thi

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123883#comment-14123883 ] Kostas Sakellis commented on SPARK-1239: [~pwendell] I'd like to take a crack at t

[jira] [Commented] (SPARK-3423) Implement BETWEEN support for regular SQL parser

[ https://issues.apache.org/jira/browse/SPARK-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123814#comment-14123814 ] William Benton commented on SPARK-3423: --- (PR is here: https://github.com/apache/spa

[jira] [Commented] (SPARK-3215) Add remote interface for SparkContext

[ https://issues.apache.org/jira/browse/SPARK-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123749#comment-14123749 ] Marcelo Vanzin commented on SPARK-3215: --- I updated the prototype to include a Java A

[jira] [Created] (SPARK-3423) Implement BETWEEN support for regular SQL parser

William Benton created SPARK-3423: - Summary: Implement BETWEEN support for regular SQL parser Key: SPARK-3423 URL: https://issues.apache.org/jira/browse/SPARK-3423 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-3082) yarn.Client.logClusterResourceDetails throws NPE if requested queue doesn't exist

[ https://issues.apache.org/jira/browse/SPARK-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved SPARK-3082. --- Resolution: Fixed Fix Version/s: 1.1.0 > yarn.Client.logClusterResourceDetails throws NPE if re

[jira] [Commented] (SPARK-3416) Add matrix operations for large data set

[ https://issues.apache.org/jira/browse/SPARK-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123695#comment-14123695 ] Yu Ishikawa commented on SPARK-3416: We discuss about this issue on the thread. http:

[jira] [Updated] (SPARK-3211) .take() is OOM-prone when there are empty partitions

[ https://issues.apache.org/jira/browse/SPARK-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-3211: - Target Version/s: 1.1.1, 1.2.0 > .take() is OOM-prone when there are empty partitions > --

[jira] [Commented] (SPARK-3418) [MLlib] Additional BLAS and Local Sparse Matrix support

[ https://issues.apache.org/jira/browse/SPARK-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123634#comment-14123634 ] Apache Spark commented on SPARK-3418: - User 'brkyvz' has created a pull request for th

[jira] [Commented] (SPARK-2099) Report TaskMetrics for running tasks

[ https://issues.apache.org/jira/browse/SPARK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123587#comment-14123587 ] Sandy Ryza commented on SPARK-2099: --- Yeah, unfortunately I haven't had the chance to add

[jira] [Commented] (SPARK-2099) Report TaskMetrics for running tasks

[ https://issues.apache.org/jira/browse/SPARK-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123584#comment-14123584 ] Andrew Ash commented on SPARK-2099: --- I just gave this a runthrough and most of the metri

[jira] [Resolved] (SPARK-2537) Workaround Timezone specific Hive tests

[ https://issues.apache.org/jira/browse/SPARK-2537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-2537. --- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 > Workaround Timezone

[jira] [Commented] (SPARK-2537) Workaround Timezone specific Hive tests

[ https://issues.apache.org/jira/browse/SPARK-2537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123556#comment-14123556 ] Cheng Lian commented on SPARK-2537: --- PR [#1440|https://github.com/apache/spark/pull/1440

[jira] [Created] (SPARK-3422) JavaAPISuite.getHadoopInputSplits isn't used anywhere

Sandy Ryza created SPARK-3422: - Summary: JavaAPISuite.getHadoopInputSplits isn't used anywhere Key: SPARK-3422 URL: https://issues.apache.org/jira/browse/SPARK-3422 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3421) StructField.toString should quote the name field to allow arbitrary character as struct field name

[ https://issues.apache.org/jira/browse/SPARK-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123537#comment-14123537 ] Apache Spark commented on SPARK-3421: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with uppercase letters in their names

[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123538#comment-14123538 ] Apache Spark commented on SPARK-3414: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123504#comment-14123504 ] Sandy Ryza commented on SPARK-3174: --- Posted a high-level design doc. > Under YARN, add

[jira] [Updated] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-3174: -- Attachment: SPARK-3174design.pdf > Under YARN, add and remove executors based on load >

[jira] [Commented] (SPARK-2491) When an OOM is thrown,the executor does not stop properly.

[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123447#comment-14123447 ] Andrew Or commented on SPARK-2491: -- [~witgo] This doesn't seem to be specific to YARN. I'

[jira] [Updated] (SPARK-2491) When an OOM is thrown,the executor does not stop properly.

[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2491: - Affects Version/s: 1.1.0 > When an OOM is thrown,the executor does not stop properly. > --

[jira] [Updated] (SPARK-2491) When an OOM is thrown,the executor does not stop properly.

[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-2491: - Component/s: (was: YARN) Spark Core > When an OOM is thrown,the executor does not sto

[jira] [Updated] (SPARK-2714) DAGScheduler should log jobid when runJob finishes

[ https://issues.apache.org/jira/browse/SPARK-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-2714: -- Description: When DAGScheduler concurrently runs multiple jobs, SparkContext only logs "Job finished" an

[jira] [Updated] (SPARK-2714) DAGScheduler should log jobid when runJob finishes

[ https://issues.apache.org/jira/browse/SPARK-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-2714: -- Summary: DAGScheduler should log jobid when runJob finishes (was: DAGScheduler logs jobid when runJob f

[jira] [Created] (SPARK-3421) StructField.toString should quote the name field to allow arbitrary character as struct field name

Cheng Lian created SPARK-3421: - Summary: StructField.toString should quote the name field to allow arbitrary character as struct field name Key: SPARK-3421 URL: https://issues.apache.org/jira/browse/SPARK-3421

[jira] [Commented] (SPARK-3280) Made sort-based shuffle the default implementation

[ https://issues.apache.org/jira/browse/SPARK-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123390#comment-14123390 ] Andrew Ash commented on SPARK-3280: --- [~joshrosen] do you have a theory for the cause of

[jira] [Updated] (SPARK-3160) Simplify DecisionTree data structure for training

[ https://issues.apache.org/jira/browse/SPARK-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3160: - Description: Improvement: code clarity Currently, we maintain a tree structure, a flat ar

[jira] [Created] (SPARK-3420) Using Sphinx to generate API docs for PySpark

Davies Liu created SPARK-3420: - Summary: Using Sphinx to generate API docs for PySpark Key: SPARK-3420 URL: https://issues.apache.org/jira/browse/SPARK-3420 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3419) Scheduler shouldn't delay running a task when executors don't reside at any of its preferred locations

Sandy Ryza created SPARK-3419: - Summary: Scheduler shouldn't delay running a task when executors don't reside at any of its preferred locations Key: SPARK-3419 URL: https://issues.apache.org/jira/browse/SPARK-3419

[jira] [Commented] (SPARK-2491) When an OOM is thrown,the executor does not stop properly.

[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123326#comment-14123326 ] Josh Rosen commented on SPARK-2491: --- Ah, I see. It looks like we don't want to display

[jira] [Resolved] (SPARK-3399) Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3399. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2270 [https://github.com/

[jira] [Commented] (SPARK-2892) Socket Receiver does not stop when streaming context is stopped

[ https://issues.apache.org/jira/browse/SPARK-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123253#comment-14123253 ] Tathagata Das commented on SPARK-2892: -- I intended it to be ERROR to catch such issue

[jira] [Commented] (SPARK-3399) Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123237#comment-14123237 ] Kousuke Saruta commented on SPARK-3399: --- Yes I meant like what you mentioned. > Tes

[jira] [Commented] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123238#comment-14123238 ] Sandy Ryza commented on SPARK-3174: --- I've been putting a little bit of thought into this

[jira] [Commented] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

[ https://issues.apache.org/jira/browse/SPARK-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123235#comment-14123235 ] Andrew Or commented on SPARK-1823: -- Thanks Andrew, I have updated the versions. > Extern

[jira] [Updated] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

[ https://issues.apache.org/jira/browse/SPARK-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-1823: - Target Version/s: 1.2.0 Affects Version/s: (was: 1.0.0) 1.1.0

[jira] [Commented] (SPARK-3399) Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123214#comment-14123214 ] Davies Liu commented on SPARK-3399: --- Given fs.defaultFs as hdfs://, saveAsTextFile() wi

[jira] [Commented] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

[ https://issues.apache.org/jira/browse/SPARK-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123213#comment-14123213 ] Andrew Ash commented on SPARK-1823: --- // This was not fixed in Spark 1.1 and should be bu

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123203#comment-14123203 ] Nicholas Chammas commented on SPARK-3369: - {quote} The API change is unlikely to h

[jira] [Commented] (SPARK-3399) Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123200#comment-14123200 ] Kousuke Saruta commented on SPARK-3399: --- Ah, did you set fs.defaultFs to like hdfs:/

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with uppercase letters in their names

[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Description: Paste the following snippet to {{spark-shell}} (need Hive support) to reproduce this issue

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with uppercase letters in their names

[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Description: Paste the following snippet to {{spark-shell}} (need Hive support) to reproduce this issue

[jira] [Commented] (SPARK-3399) Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123195#comment-14123195 ] Davies Liu commented on SPARK-3399: --- Thanks for the explain. I still can not reproduce t

[jira] [Updated] (SPARK-3418) [MLlib] Additional BLAS and Local Sparse Matrix support

[ https://issues.apache.org/jira/browse/SPARK-3418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-3418: --- Summary: [MLlib] Additional BLAS and Local Sparse Matrix support (was: Additional BLAS and Local Spar

[jira] [Created] (SPARK-3418) Additional BLAS and Local Sparse Matrix support

Burak Yavuz created SPARK-3418: -- Summary: Additional BLAS and Local Sparse Matrix support Key: SPARK-3418 URL: https://issues.apache.org/jira/browse/SPARK-3418 Project: Spark Issue Type: New Fea

[jira] [Commented] (SPARK-3399) Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123167#comment-14123167 ] Kousuke Saruta commented on SPARK-3399: --- Some test for pyspark, for instance rdd.py,

[jira] [Commented] (SPARK-3399) Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123143#comment-14123143 ] Davies Liu commented on SPARK-3399: --- Could you give an example to show the problem? > T

[jira] [Commented] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123137#comment-14123137 ] Patrick Wendell commented on SPARK-3174: We should come up with a design doc for t

[jira] [Updated] (SPARK-3174) Under YARN, add and remove executors based on load

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3174: --- Assignee: Andrew Or > Under YARN, add and remove executors based on load > ---

[jira] [Updated] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3369: --- Priority: Critical (was: Major) Target Version/s: 1.2.0 > Java mapPartitions Iter

[jira] [Commented] (SPARK-2966) Add an approximation algorithm for hierarchical clustering to MLlib

[ https://issues.apache.org/jira/browse/SPARK-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123077#comment-14123077 ] RJ Nowling commented on SPARK-2966: --- Wonderful! If I can help or when you're ready for

[jira] [Commented] (SPARK-3417) Use of old-style classes in pyspark

[ https://issues.apache.org/jira/browse/SPARK-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123059#comment-14123059 ] Apache Spark commented on SPARK-3417: - User 'mrocklin' has created a pull request for

[jira] [Created] (SPARK-3417) Use of old-style classes in pyspark

Matthew Rocklin created SPARK-3417: -- Summary: Use of old-style classes in pyspark Key: SPARK-3417 URL: https://issues.apache.org/jira/browse/SPARK-3417 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3416) Add matrix operations for large data set

Yu Ishikawa created SPARK-3416: -- Summary: Add matrix operations for large data set Key: SPARK-3416 URL: https://issues.apache.org/jira/browse/SPARK-3416 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-2966) Add an approximation algorithm for hierarchical clustering to MLlib

[ https://issues.apache.org/jira/browse/SPARK-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123029#comment-14123029 ] Yu Ishikawa commented on SPARK-2966: Hi [~rnowling], {quote} Based on my reading of t

[jira] [Resolved] (SPARK-3375) spark on yarn container allocation issues

[ https://issues.apache.org/jira/browse/SPARK-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3375. -- Resolution: Fixed Fix Version/s: 1.2.0 > spark on yarn container allocation issues >

[jira] [Resolved] (SPARK-3260) Yarn - pass acls along with executor launch

[ https://issues.apache.org/jira/browse/SPARK-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3260. -- Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 (was: 1.1.1, 1.2

[jira] [Updated] (SPARK-3377) Don't mix metrics from different applications otherwise we cannot distinguish

[ https://issues.apache.org/jira/browse/SPARK-3377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3377: -- Summary: Don't mix metrics from different applications otherwise we cannot distinguish (was: Do

[jira] [Commented] (SPARK-2430) Standarized Clustering Algorithm API and Framework

[ https://issues.apache.org/jira/browse/SPARK-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123005#comment-14123005 ] Yu Ishikawa commented on SPARK-2430: Hi [~rnowling], > The community had suggested l

[jira] [Updated] (SPARK-3415) Using sys.stderr in pyspark results in error

[ https://issues.apache.org/jira/browse/SPARK-3415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ward Viaene updated SPARK-3415: --- Affects Version/s: 1.1.0 1.0.2 > Using sys.stderr in pyspark results in error >

[jira] [Updated] (SPARK-3415) Using sys.stderr in pyspark results in error

[ https://issues.apache.org/jira/browse/SPARK-3415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ward Viaene updated SPARK-3415: --- Labels: python (was: ) > Using sys.stderr in pyspark results in error > -

[jira] [Updated] (SPARK-3415) Using sys.stderr in pyspark results in error

[ https://issues.apache.org/jira/browse/SPARK-3415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ward Viaene updated SPARK-3415: --- Component/s: PySpark > Using sys.stderr in pyspark results in error >

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122921#comment-14122921 ] Sean Owen commented on SPARK-3369: -- The API change is unlikely to happen. Making a bunch

[jira] [Created] (SPARK-3415) Using sys.stderr in pyspark results in error

Ward Viaene created SPARK-3415: -- Summary: Using sys.stderr in pyspark results in error Key: SPARK-3415 URL: https://issues.apache.org/jira/browse/SPARK-3415 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2594) Add CACHE TABLE AS SELECT ...

[ https://issues.apache.org/jira/browse/SPARK-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122860#comment-14122860 ] Ravindra Pesala commented on SPARK-2594: Thank you Michael for review and suggesti

[jira] [Comment Edited] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122611#comment-14122611 ] Lukas Nalezenec edited comment on SPARK-3369 at 9/5/14 11:59 AM: ---

[jira] [Comment Edited] (SPARK-3403) NaiveBayes crashes with blas/lapack native libraries for breeze (netlib-java)

[ https://issues.apache.org/jira/browse/SPARK-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122699#comment-14122699 ] Alexander Ulanov edited comment on SPARK-3403 at 9/5/14 9:53 AM: ---

[jira] [Commented] (SPARK-3412) Add Missing Types for Row API

[ https://issues.apache.org/jira/browse/SPARK-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122729#comment-14122729 ] Apache Spark commented on SPARK-3412: - User 'chenghao-intel' has created a pull reques

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with upper case letter in their names

[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Description: Paste the following snippet to {{spark-shell}} (need Hive support) to reproduce this issue

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with uppercase letters in their names

[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Summary: Case insensitivity breaks when unresolved relation contains attributes with uppercase letters i

[jira] [Comment Edited] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122611#comment-14122611 ] Lukas Nalezenec edited comment on SPARK-3369 at 9/5/14 9:17 AM:

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with upper case letter in their names

[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Description: Paste the following snippet to {{spark-shell}} (need Hive support) to reproduce this issue

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with upper case letter in their names

[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Description: Paste the following snippet to {{spark-shell}} (need Hive support) to reproduce this issue

[jira] [Created] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with upper case letter in their names

Cheng Lian created SPARK-3414: - Summary: Case insensitivity breaks when unresolved relation contains attributes with upper case letter in their names Key: SPARK-3414 URL: https://issues.apache.org/jira/browse/SPARK-34

[jira] [Comment Edited] (SPARK-2491) When an OOM is thrown,the executor does not stop properly.

[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122579#comment-14122579 ] Guoqiang Li edited comment on SPARK-2491 at 9/5/14 9:00 AM: Ex

[jira] [Commented] (SPARK-3403) NaiveBayes crashes with blas/lapack native libraries for breeze (netlib-java)

[ https://issues.apache.org/jira/browse/SPARK-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122699#comment-14122699 ] Alexander Ulanov commented on SPARK-3403: - I managed to compile OpenBLAS with MING

[jira] [Commented] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122611#comment-14122611 ] Lukas Nalezenec commented on SPARK-3369: Hi, it looks like serious issue for me. H

[jira] [Comment Edited] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122611#comment-14122611 ] Lukas Nalezenec edited comment on SPARK-3369 at 9/5/14 8:44 AM:

[jira] [Created] (SPARK-3413) Spark Blocked due to Executor lost in FIFO MODE

Patrick Liu created SPARK-3413: -- Summary: Spark Blocked due to Executor lost in FIFO MODE Key: SPARK-3413 URL: https://issues.apache.org/jira/browse/SPARK-3413 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2491) When an OOM is thrown,the executor does not stop properly.

[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122579#comment-14122579 ] Guoqiang Li commented on SPARK-2491: Executor running multiple tasks at the same time,

[jira] [Created] (SPARK-3412) Add Missing Types for Row API

Cheng Hao created SPARK-3412: Summary: Add Missing Types for Row API Key: SPARK-3412 URL: https://issues.apache.org/jira/browse/SPARK-3412 Project: Spark Issue Type: Bug Components: SQL