[jira] [Closed] (SPARK-3921) WorkerWatcher in Standalone mode fail to come up due to invalid workerUrl

[ https://issues.apache.org/jira/browse/SPARK-3921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3921. Resolution: Fixed Fix Version/s: 1.2.0 WorkerWatcher in Standalone mode fail to come up due to

[jira] [Resolved] (SPARK-3894) Scala style: line length increase to 120 for standard

[ https://issues.apache.org/jira/browse/SPARK-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3894. Resolution: Won't Fix As I said before - send an e-mail to dev@ if you want to start a

[jira] [Updated] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1209: --- Reporter: Sandy Ryza (was: Sandy Pérez González) SparkHadoopUtil should not use package

[jira] [Updated] (SPARK-3870) EOL character enforcement

[ https://issues.apache.org/jira/browse/SPARK-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3870: --- Component/s: Project Infra EOL character enforcement -

[jira] [Updated] (SPARK-3906) Support joins of multiple tables in SparkSQL (SQLContext, not HiveQL)

[ https://issues.apache.org/jira/browse/SPARK-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3906: --- Component/s: SQL Support joins of multiple tables in SparkSQL (SQLContext, not HiveQL)

[jira] [Updated] (SPARK-3426) Sort-based shuffle compression behavior is inconsistent

[ https://issues.apache.org/jira/browse/SPARK-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3426: --- Component/s: Spark Core Sort-based shuffle compression behavior is inconsistent

[jira] [Commented] (SPARK-1307) don't use term 'standalone' to refer to a Spark Application

[ https://issues.apache.org/jira/browse/SPARK-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170591#comment-14170591 ] Patrick Wendell commented on SPARK-1307: Yeah this is a good idea - there is no

[jira] [Updated] (SPARK-3936) Incorrect result in GraphX BytecodeUtils with closures + class/object methods

[ https://issues.apache.org/jira/browse/SPARK-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3936: --- Assignee: Ankur Dave Incorrect result in GraphX BytecodeUtils with closures + class/object methods

[jira] [Updated] (SPARK-1849) sc.textFile does not support non UTF-8 encodings

[ https://issues.apache.org/jira/browse/SPARK-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1849: --- Summary: sc.textFile does not support non UTF-8 encodings (was: Broken UTF-8 encoded data

[jira] [Updated] (SPARK-3826) Support JDBC/ODBC server with Hive 0.13.1

[ https://issues.apache.org/jira/browse/SPARK-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3826: --- Summary: Support JDBC/ODBC server with Hive 0.13.1 (was: enable hive-thriftserver support

[jira] [Updated] (SPARK-3826) Support JDBC/ODBC server with Hive 0.13.1

[ https://issues.apache.org/jira/browse/SPARK-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3826: --- Assignee: wangfei Support JDBC/ODBC server with Hive 0.13.1

[jira] [Commented] (SPARK-3895) Scala style: Indentation of method

[ https://issues.apache.org/jira/browse/SPARK-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170609#comment-14170609 ] Sean Owen commented on SPARK-3895: -- The rule is already in style guide. There seems to be

[jira] [Commented] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170617#comment-14170617 ] Sean Owen commented on SPARK-1209: -- Hm, it's {{private[apache]}} though. Couldn't this

[jira] [Created] (SPARK-3942) LogisticRegressionWithLBFGS should not use SquaredL2Updater

fuminglin created SPARK-3942: Summary: LogisticRegressionWithLBFGS should not use SquaredL2Updater Key: SPARK-3942 URL: https://issues.apache.org/jira/browse/SPARK-3942 Project: Spark Issue

[jira] [Commented] (SPARK-3741) ConnectionManager.sendMessage may not propagate errors to MessageStatus

[ https://issues.apache.org/jira/browse/SPARK-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170723#comment-14170723 ] Apache Spark commented on SPARK-3741: - User 'zsxwing' has created a pull request for

[jira] [Created] (SPARK-3944) Utils.resolveURI is not consistent on upgrading scalaVersion to 2.11.2

Shiti Saxena created SPARK-3944: --- Summary: Utils.resolveURI is not consistent on upgrading scalaVersion to 2.11.2 Key: SPARK-3944 URL: https://issues.apache.org/jira/browse/SPARK-3944 Project: Spark

[jira] [Updated] (SPARK-3944) Utils.resolveURI is not consistent on upgrading scalaVersion to 2.11.2

[ https://issues.apache.org/jira/browse/SPARK-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shiti Saxena updated SPARK-3944: Description: The method tries to match a String (`uri.getScheme`) to obtain the result. The value

[jira] [Commented] (SPARK-3944) Utils.resolveURI is not consistent on upgrading scalaVersion to 2.11.2

[ https://issues.apache.org/jira/browse/SPARK-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170759#comment-14170759 ] Apache Spark commented on SPARK-3944: - User 'Shiti' has created a pull request for

[jira] [Updated] (SPARK-3944) Utils.resolveURI is not consistent on upgrading scalaVersion to 2.11.2

[ https://issues.apache.org/jira/browse/SPARK-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-3944: --- Description: The method tries to match a String (`uri.getScheme`) to obtain the result. The

[jira] [Updated] (SPARK-3944) Utils.resolveURI fails on upgrading scalaVersion to 2.11.2

[ https://issues.apache.org/jira/browse/SPARK-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-3944: --- Summary: Utils.resolveURI fails on upgrading scalaVersion to 2.11.2 (was: Utils.resolveURI

[jira] [Created] (SPARK-3946) .gitignore in /python includes wrong directory.

Masayoshi TSUZUKI created SPARK-3946: Summary: .gitignore in /python includes wrong directory. Key: SPARK-3946 URL: https://issues.apache.org/jira/browse/SPARK-3946 Project: Spark Issue

[jira] [Commented] (SPARK-3946) .gitignore in /python includes wrong directory.

[ https://issues.apache.org/jira/browse/SPARK-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170778#comment-14170778 ] Apache Spark commented on SPARK-3946: - User 'tsudukim' has created a pull request for

[jira] [Commented] (SPARK-3943) Some scripts bin\*.cmd pollutes environment variables in Windows.

[ https://issues.apache.org/jira/browse/SPARK-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170782#comment-14170782 ] Masayoshi TSUZUKI commented on SPARK-3943: -- This should be modified after the

[jira] [Resolved] (SPARK-3480) Throws out Not a valid command 'yarn-alpha/scalastyle' in dev/scalastyle for sbt build tool during 'Running Scala style checks'

[ https://issues.apache.org/jira/browse/SPARK-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3480. -- Resolution: Cannot Reproduce Throws out Not a valid command 'yarn-alpha/scalastyle' in dev/scalastyle

[jira] [Commented] (SPARK-3943) Some scripts bin\*.cmd pollutes environment variables in Windows.

[ https://issues.apache.org/jira/browse/SPARK-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170789#comment-14170789 ] Apache Spark commented on SPARK-3943: - User 'tsudukim' has created a pull request for

[jira] [Commented] (SPARK-3822) Expose a mechanism for SparkContext to ask for / remove Yarn containers

[ https://issues.apache.org/jira/browse/SPARK-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170792#comment-14170792 ] Apache Spark commented on SPARK-3822: - User 'PraveenSeluka' has created a pull request

[jira] [Created] (SPARK-3947) [Spark SQL] UDAF Support

Pei-Lun Lee created SPARK-3947: -- Summary: [Spark SQL] UDAF Support Key: SPARK-3947 URL: https://issues.apache.org/jira/browse/SPARK-3947 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3945) Write properties of hive-site.xml to HiveContext when initilize session state In SparkSQLEnv.scala

[ https://issues.apache.org/jira/browse/SPARK-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170802#comment-14170802 ] Apache Spark commented on SPARK-3945: - User 'luogankun' has created a pull request for

[jira] [Commented] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

[ https://issues.apache.org/jira/browse/SPARK-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170925#comment-14170925 ] SUMANTH B B N commented on SPARK-3717: -- i'm working on the implementation of this

[jira] [Created] (SPARK-3948) Potential file append bugs in ExternalSorter which leads to sort-based shuffle unexpected exception

Saisai Shao created SPARK-3948: -- Summary: Potential file append bugs in ExternalSorter which leads to sort-based shuffle unexpected exception Key: SPARK-3948 URL: https://issues.apache.org/jira/browse/SPARK-3948

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170998#comment-14170998 ] Praveen Seluka commented on SPARK-3174: --- Posted a detailed autoscaling design doc =

[jira] [Updated] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Seluka updated SPARK-3174: -- Attachment: SparkElasticScalingDesignB.pdf Provide elastic scaling within a Spark application

[jira] [Commented] (SPARK-3904) HQL doesn't support the ConstantObjectInspector to pass into UDFs

[ https://issues.apache.org/jira/browse/SPARK-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171011#comment-14171011 ] Apache Spark commented on SPARK-3904: - User 'chenghao-intel' has created a pull

[jira] [Commented] (SPARK-3891) Support Hive Percentile UDAF with array of percentile values

[ https://issues.apache.org/jira/browse/SPARK-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171017#comment-14171017 ] Apache Spark commented on SPARK-3891: - User 'gvramana' has created a pull request for

[jira] [Commented] (SPARK-3891) Support Hive Percentile UDAF with array of percentile values

[ https://issues.apache.org/jira/browse/SPARK-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171019#comment-14171019 ] Venkata Ramana G commented on SPARK-3891: - Following problems need to be fixed to

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171024#comment-14171024 ] Sandy Ryza commented on SPARK-3174: --- bq. If I understand correctly, your concern with

[jira] [Comment Edited] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171024#comment-14171024 ] Sandy Ryza edited comment on SPARK-3174 at 10/14/14 3:03 PM: -

[jira] [Commented] (SPARK-3874) Provide stable TaskContext API

[ https://issues.apache.org/jira/browse/SPARK-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171095#comment-14171095 ] Apache Spark commented on SPARK-3874: - User 'pwendell' has created a pull request for

[jira] [Commented] (SPARK-3822) Expose a mechanism for SparkContext to ask for / remove Yarn containers

[ https://issues.apache.org/jira/browse/SPARK-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171221#comment-14171221 ] Andrew Or commented on SPARK-3822: -- Hey [~praveenseluka] thanks for your work on this.

[jira] [Commented] (SPARK-3360) Add RowMatrix.multiply(Vector)

[ https://issues.apache.org/jira/browse/SPARK-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171238#comment-14171238 ] Sandy Ryza commented on SPARK-3360: --- bq. You don't need Vector.multiply(RowMatrix)

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171247#comment-14171247 ] Patrick Wendell commented on SPARK-3630: It would be good to see if this only

[jira] [Updated] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3630: --- Assignee: (was: Ankur Dave) Identify cause of Kryo+Snappy PARSING_ERROR

[jira] [Assigned] (SPARK-3434) Distributed block matrix

[ https://issues.apache.org/jira/browse/SPARK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman reassigned SPARK-3434: Assignee: Shivaram Venkataraman Distributed block matrix

[jira] [Commented] (SPARK-3434) Distributed block matrix

[ https://issues.apache.org/jira/browse/SPARK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171272#comment-14171272 ] Shivaram Venkataraman commented on SPARK-3434: -- Sorry for the delay in

[jira] [Commented] (SPARK-3948) Potential file append bugs in ExternalSorter which leads to sort-based shuffle unexpected exception

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171295#comment-14171295 ] Josh Rosen commented on SPARK-3948: --- Hi [~jerryshao]], Could you share the changes that

[jira] [Commented] (SPARK-3736) Workers should reconnect to Master if disconnected

[ https://issues.apache.org/jira/browse/SPARK-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171358#comment-14171358 ] Nan Zhu commented on SPARK-3736: if the worker itself timeout, the Master will remove the

[jira] [Commented] (SPARK-3736) Workers should reconnect to Master if disconnected

[ https://issues.apache.org/jira/browse/SPARK-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171360#comment-14171360 ] Nan Zhu commented on SPARK-3736: BTW, master will not send heartbeat to Worker proactively

[jira] [Created] (SPARK-3949) Use IAMRole in lieu of static access key-id/secret

Rangarajan Sreenivasan created SPARK-3949: - Summary: Use IAMRole in lieu of static access key-id/secret Key: SPARK-3949 URL: https://issues.apache.org/jira/browse/SPARK-3949 Project: Spark

[jira] [Resolved] (SPARK-3178) setting SPARK_WORKER_MEMORY to a value without a label (m or g) sets the worker memory limit to zero

[ https://issues.apache.org/jira/browse/SPARK-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3178. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2309

[jira] [Commented] (SPARK-3822) Expose a mechanism for SparkContext to ask for / remove Yarn containers

[ https://issues.apache.org/jira/browse/SPARK-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171382#comment-14171382 ] Praveen Seluka commented on SPARK-3822: --- [~andrewor] Thanks for your comment. I do

[jira] [Commented] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171418#comment-14171418 ] Sean Owen commented on SPARK-3926: -- Oops, embarrassed to say I didn't realize

[jira] [Commented] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171421#comment-14171421 ] Apache Spark commented on SPARK-3926: - User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-3916) recognize appended data in textFileStream()

[ https://issues.apache.org/jira/browse/SPARK-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171524#comment-14171524 ] Apache Spark commented on SPARK-3916: - User 'davies' has created a pull request for

[jira] [Closed] (SPARK-3946) .gitignore in /python includes wrong directory.

[ https://issues.apache.org/jira/browse/SPARK-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3946. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Masayoshi TSUZUKI Target

[jira] [Updated] (SPARK-3944) Utils.resolveURI fails on upgrading scalaVersion to 2.11.2

[ https://issues.apache.org/jira/browse/SPARK-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3944: - Component/s: Spark Core Target Version/s: 1.2.0 Affects Version/s: 1.1.0 Fix

[jira] [Closed] (SPARK-3944) Utils.resolveURI fails on upgrading scalaVersion to 2.11.2

[ https://issues.apache.org/jira/browse/SPARK-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3944. Resolution: Fixed Utils.resolveURI fails on upgrading scalaVersion to 2.11.2

[jira] [Created] (SPARK-3950) Completed time is blank for some successful tasks

Aaron Davidson created SPARK-3950: - Summary: Completed time is blank for some successful tasks Key: SPARK-3950 URL: https://issues.apache.org/jira/browse/SPARK-3950 Project: Spark Issue

[jira] [Commented] (SPARK-3950) Completed time is blank for some successful tasks

[ https://issues.apache.org/jira/browse/SPARK-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171563#comment-14171563 ] Aaron Davidson commented on SPARK-3950: --- [~andrewor14] Completed time is blank for

[jira] [Resolved] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3803. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2801

[jira] [Updated] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3803: - Assignee: Sean Owen ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171580#comment-14171580 ] Marcelo Vanzin commented on SPARK-3174: --- [~andrewor14] still regarding the shuffle

[jira] [Created] (SPARK-3951) Make the external-* jars fat jars

Hari Shreedharan created SPARK-3951: --- Summary: Make the external-* jars fat jars Key: SPARK-3951 URL: https://issues.apache.org/jira/browse/SPARK-3951 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3952) Python examples in Streaming Programming Guide

Davies Liu created SPARK-3952: - Summary: Python examples in Streaming Programming Guide Key: SPARK-3952 URL: https://issues.apache.org/jira/browse/SPARK-3952 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171654#comment-14171654 ] Zhan Zhang edited comment on SPARK-2883 at 10/14/14 10:41 PM: --

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171654#comment-14171654 ] Zhan Zhang commented on SPARK-2883: --- I almost finished the prototype, and following is

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171654#comment-14171654 ] Zhan Zhang edited comment on SPARK-2883 at 10/14/14 10:44 PM: --

[jira] [Created] (SPARK-3953) Confusable variable name.

Kousuke Saruta created SPARK-3953: - Summary: Confusable variable name. Key: SPARK-3953 URL: https://issues.apache.org/jira/browse/SPARK-3953 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3953) Confusable variable name.

[ https://issues.apache.org/jira/browse/SPARK-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171662#comment-14171662 ] Apache Spark commented on SPARK-3953: - User 'sarutak' has created a pull request for

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171654#comment-14171654 ] Zhan Zhang edited comment on SPARK-2883 at 10/14/14 11:04 PM: --

[jira] [Commented] (SPARK-3257) Enable :cp to add JARs in spark-shell (Scala 2.11)

[ https://issues.apache.org/jira/browse/SPARK-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171686#comment-14171686 ] Heather Miller commented on SPARK-3257: --- PR'ed to scala/scala

[jira] [Commented] (SPARK-3795) Add scheduler hooks/heuristics for adding and removing executors

[ https://issues.apache.org/jira/browse/SPARK-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171692#comment-14171692 ] Apache Spark commented on SPARK-3795: - User 'andrewor14' has created a pull request

[jira] [Comment Edited] (SPARK-2883) Spark Support for ORCFile format

[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171654#comment-14171654 ] Zhan Zhang edited comment on SPARK-2883 at 10/14/14 11:59 PM: --

[jira] [Commented] (SPARK-3952) Python examples in Streaming Programming Guide

[ https://issues.apache.org/jira/browse/SPARK-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171753#comment-14171753 ] Apache Spark commented on SPARK-3952: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-3453) Refactor Netty module to use BlockTransferService

[ https://issues.apache.org/jira/browse/SPARK-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171782#comment-14171782 ] Apache Spark commented on SPARK-3453: - User 'aarondav' has created a pull request for

[jira] [Commented] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171817#comment-14171817 ] Josh Rosen commented on SPARK-3926: --- I opened a Scala JIRA to try to fix this:

[jira] [Commented] (SPARK-3948) Potential file append bugs in ExternalSorter which leads to sort-based shuffle unexpected exception

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171834#comment-14171834 ] Saisai Shao commented on SPARK-3948: Hi Josh, according to my observation, this bug is

[jira] [Updated] (SPARK-3948) Potential file append bugs in ExternalSorter which leads to sort-based shuffle unexpected exception

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-3948: --- Description: Several exceptions occurred when running TPC-DS queries against latest master branch

[jira] [Updated] (SPARK-3948) Potential file append bugs in ExternalSorter which leads to sort-based shuffle unexpected exception

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-3948: --- Description: Several exceptions occurred when running TPC-DS queries against latest master branch

[jira] [Updated] (SPARK-3948) Potential file append bugs in ExternalSorter which leads to sort-based shuffle unexpected exception

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3948: -- Assignee: Saisai Shao Potential file append bugs in ExternalSorter which leads to sort-based shuffle

[jira] [Commented] (SPARK-3948) Potential file append bugs in ExternalSorter which leads to sort-based shuffle unexpected exception

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171859#comment-14171859 ] Josh Rosen commented on SPARK-3948: --- Thanks for narrowing the problem down to a single

[jira] [Commented] (SPARK-3948) Potential file append bugs in ExternalSorter which leads to sort-based shuffle unexpected exception

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171861#comment-14171861 ] Saisai Shao commented on SPARK-3948: Thanks for your help :). Potential file append

[jira] [Comment Edited] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171885#comment-14171885 ] Saisai Shao edited comment on SPARK-3630 at 10/15/14 2:07 AM: --

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171885#comment-14171885 ] Saisai Shao commented on SPARK-3630: Hi Pactrick, this problem is still existed after

[jira] [Updated] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3948: --- Summary: Sort-based shuffle can lead to assorted stream-corruption exceptions (was:

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171920#comment-14171920 ] Patrick Wendell commented on SPARK-3948: Great thanks [~jerryshao] - I updated the

[jira] [Updated] (SPARK-1307) don't use term 'standalone' to refer to a Spark Application

[ https://issues.apache.org/jira/browse/SPARK-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1307: - Assignee: Sean Owen don't use term 'standalone' to refer to a Spark Application

[jira] [Resolved] (SPARK-1307) don't use term 'standalone' to refer to a Spark Application

[ https://issues.apache.org/jira/browse/SPARK-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-1307. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2787

[jira] [Commented] (SPARK-3896) checkSpeculatableTasks fask quit loop, invoking checkSpeculatableTasks is expensive

[ https://issues.apache.org/jira/browse/SPARK-3896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171987#comment-14171987 ] Apache Spark commented on SPARK-3896: - User 'shijinkui' has created a pull request for

[jira] [Commented] (SPARK-3569) Add metadata field to StructField

[ https://issues.apache.org/jira/browse/SPARK-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171999#comment-14171999 ] Xiangrui Meng commented on SPARK-3569: -- I put the design doc here:

[jira] [Closed] (SPARK-2098) All Spark processes should support spark-defaults.conf, config file

[ https://issues.apache.org/jira/browse/SPARK-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-2098. Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.2.0 All Spark processes should

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172024#comment-14172024 ] Josh Rosen commented on SPARK-3948: --- I've been looking over the code in

[jira] [Commented] (SPARK-3702) Standardize MLlib classes for learners, models

[ https://issues.apache.org/jira/browse/SPARK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172037#comment-14172037 ] Christoph Sawade commented on SPARK-3702: - Okay. I will follow it. Standardize