[jira] [Updated] (SPARK-1849) sc.textFile does not support non UTF-8 encodings

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1849: --- Summary: sc.textFile does not support non UTF-8 encodings (was: Broken UTF-8 encoded data get

[jira] [Updated] (SPARK-3936) Incorrect result in GraphX BytecodeUtils with closures + class/object methods

2014-10-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3936: --- Assignee: Ankur Dave > Incorrect result in GraphX BytecodeUtils with closures + class/object methods >

[jira] [Commented] (SPARK-1307) don't use term 'standalone' to refer to a Spark Application

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170591#comment-14170591 ] Patrick Wendell commented on SPARK-1307: Yeah this is a good idea - there is no re

[jira] [Updated] (SPARK-3426) Sort-based shuffle compression behavior is inconsistent

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3426: --- Component/s: Spark Core > Sort-based shuffle compression behavior is inconsistent > --

[jira] [Updated] (SPARK-3906) Support joins of multiple tables in SparkSQL (SQLContext, not HiveQL)

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3906: --- Component/s: SQL > Support joins of multiple tables in SparkSQL (SQLContext, not HiveQL) > ---

[jira] [Updated] (SPARK-3870) EOL character enforcement

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3870: --- Component/s: Project Infra > EOL character enforcement > - > >

[jira] [Updated] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1209: --- Reporter: Sandy Ryza (was: Sandy Pérez González) > SparkHadoopUtil should not use package org

[jira] [Resolved] (SPARK-3894) Scala style: line length increase to 120 for standard

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3894. Resolution: Won't Fix As I said before - send an e-mail to dev@ if you want to start a discu

[jira] [Closed] (SPARK-3921) WorkerWatcher in Standalone mode fail to come up due to invalid workerUrl

2014-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3921. Resolution: Fixed Fix Version/s: 1.2.0 > WorkerWatcher in Standalone mode fail to come up due to inva

[jira] [Resolved] (SPARK-3912) FlumeStreamSuite is flaky, fails either with port binding issues or data not being reliably sent

2014-10-13 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-3912. -- Resolution: Fixed > FlumeStreamSuite is flaky, fails either with port binding issues or data not

[jira] [Updated] (SPARK-3912) FlumeStreamSuite is flaky, fails either with port binding issues or data not being reliably sent

2014-10-13 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-3912: - Fix Version/s: 1.2.0 > FlumeStreamSuite is flaky, fails either with port binding issues or data no

[jira] [Resolved] (SPARK-2377) Create a Python API for Spark Streaming

2014-10-13 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-2377. -- Resolution: Fixed Many thanks to [~davies] and [~giwa] for making this happen! It has been merge

[jira] [Commented] (SPARK-3941) _remainingMem should not increase twice when updateBlockInfo

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170550#comment-14170550 ] Apache Spark commented on SPARK-3941: - User 'liyezhang556520' has created a pull reque

[jira] [Updated] (SPARK-2377) Create a Python API for Spark Streaming

2014-10-13 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2377: - Fix Version/s: 1.2.0 > Create a Python API for Spark Streaming > -

[jira] [Reopened] (SPARK-3895) Scala style: Indentation of method

2014-10-13 Thread sjk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sjk reopened SPARK-3895: > Scala style: Indentation of method > -- > > Key: SPARK-3895 >

[jira] [Commented] (SPARK-3895) Scala style: Indentation of method

2014-10-13 Thread sjk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170541#comment-14170541 ] sjk commented on SPARK-3895: i have close the pr that code format changes are much. but this

[jira] [Updated] (SPARK-3895) Scala style: Indentation of method

2014-10-13 Thread sjk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sjk updated SPARK-3895: --- Description: {code:title=core/src/main/scala/org/apache/spark/Aggregator.scala|borderStyle=solid} // for example de

[jira] [Created] (SPARK-3941) _remainingMem should not increase twice when updateBlockInfo

2014-10-13 Thread Zhang, Liye (JIRA)
Zhang, Liye created SPARK-3941: -- Summary: _remainingMem should not increase twice when updateBlockInfo Key: SPARK-3941 URL: https://issues.apache.org/jira/browse/SPARK-3941 Project: Spark Issue

[jira] [Closed] (SPARK-3897) Scala style: format example code

2014-10-13 Thread sjk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sjk closed SPARK-3897. -- > Scala style: format example code > > > Key: SPARK-3897 >

[jira] [Commented] (SPARK-3935) Unused variable in PairRDDFunctions.scala

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170521#comment-14170521 ] Apache Spark commented on SPARK-3935: - User 'jackylk' has created a pull request for t

[jira] [Updated] (SPARK-3939) NPE caused by SessionState.out not set in thriftserver2

2014-10-13 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-3939: --- Component/s: SQL > NPE caused by SessionState.out not set in thriftserver2 > -

[jira] [Commented] (SPARK-3940) Print the error code three times

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170487#comment-14170487 ] Apache Spark commented on SPARK-3940: - User 'wangxiaojing' has created a pull request

[jira] [Created] (SPARK-3940) Print the error code three times

2014-10-13 Thread wangxj (JIRA)
wangxj created SPARK-3940: - Summary: Print the error code three times Key: SPARK-3940 URL: https://issues.apache.org/jira/browse/SPARK-3940 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-3938) Set RDD name to table name during cache operations

2014-10-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170454#comment-14170454 ] Cheng Lian commented on SPARK-3938: --- A problem here is that after PR [#2501|https://git

[jira] [Created] (SPARK-3939) NPE caused by SessionState.out not set in thriftserver2

2014-10-13 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-3939: -- Summary: NPE caused by SessionState.out not set in thriftserver2 Key: SPARK-3939 URL: https://issues.apache.org/jira/browse/SPARK-3939 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3161) Cache example-node map for DecisionTree training

2014-10-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170443#comment-14170443 ] Joseph K. Bradley commented on SPARK-3161: -- Your summary for option (1) sounds go

[jira] [Reopened] (SPARK-3894) Scala style: line length increase to 120 for standard

2014-10-13 Thread sjk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sjk reopened SPARK-3894: use 120 instead scala function parameter effect the code readability > Scala style: line length increase to 120 for st

[jira] [Updated] (SPARK-3894) Scala style: line length increase to 120 for standard

2014-10-13 Thread sjk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sjk updated SPARK-3894: --- Summary: Scala style: line length increase to 120 for standard (was: Scala style: line length increase to 160) > Sca

[jira] [Commented] (SPARK-3894) Scala style: line length increase to 160

2014-10-13 Thread sjk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170414#comment-14170414 ] sjk commented on SPARK-3894: too much function's parameters are more than four, the length is

[jira] [Commented] (SPARK-3426) Sort-based shuffle compression behavior is inconsistent

2014-10-13 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170395#comment-14170395 ] Saisai Shao commented on SPARK-3426: Sorry about that, I just saw the PR (https://gith

[jira] [Commented] (SPARK-3814) Bitwise & does not work in Hive

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170393#comment-14170393 ] Apache Spark commented on SPARK-3814: - User 'ravipesala' has created a pull request fo

[jira] [Commented] (SPARK-3426) Sort-based shuffle compression behavior is inconsistent

2014-10-13 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170386#comment-14170386 ] Saisai Shao commented on SPARK-3426: Hi [~andrewor14], are you going to fix this issue

[jira] [Commented] (SPARK-3161) Cache example-node map for DecisionTree training

2014-10-13 Thread Sung Chung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170346#comment-14170346 ] Sung Chung commented on SPARK-3161: --- The basic idea is : * Have a separate class/object

[jira] [Commented] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170302#comment-14170302 ] Patrick Wendell commented on SPARK-1209: Actually a full set of forwarders might b

[jira] [Commented] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170298#comment-14170298 ] Patrick Wendell commented on SPARK-1209: It's possible this previously used packag

[jira] [Comment Edited] (SPARK-3257) Enable :cp to add JARs in spark-shell (Scala 2.11)

2014-10-13 Thread Heather Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170263#comment-14170263 ] Heather Miller edited comment on SPARK-3257 at 10/14/14 12:11 AM: --

[jira] [Commented] (SPARK-3161) Cache example-node map for DecisionTree training

2014-10-13 Thread Sung Chung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170262#comment-14170262 ] Sung Chung commented on SPARK-3161: --- On a second thought, for (option 2) to work, it wou

[jira] [Commented] (SPARK-3257) Enable :cp to add JARs in spark-shell (Scala 2.11)

2014-10-13 Thread Heather Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170263#comment-14170263 ] Heather Miller commented on SPARK-3257: --- Current status (waiting to PR to scala/scal

[jira] [Created] (SPARK-3938) Set RDD name to table name during cache operations

2014-10-13 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-3938: -- Summary: Set RDD name to table name during cache operations Key: SPARK-3938 URL: https://issues.apache.org/jira/browse/SPARK-3938 Project: Spark Issue Ty

[jira] [Commented] (SPARK-3161) Cache example-node map for DecisionTree training

2014-10-13 Thread Sung Chung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170254#comment-14170254 ] Sung Chung commented on SPARK-3161: --- (option 2) sounds intriguing but is this easy to do

[jira] [Created] (SPARK-3937) Unsafe memory access inside of Snappy library

2014-10-13 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-3937: -- Summary: Unsafe memory access inside of Snappy library Key: SPARK-3937 URL: https://issues.apache.org/jira/browse/SPARK-3937 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170244#comment-14170244 ] Andrew Or commented on SPARK-3174: -- @[~sandyr] by quick-start do you mean adding all exec

[jira] [Commented] (SPARK-3934) RandomForest bug in sanity check in DTStatsAggregator

2014-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170220#comment-14170220 ] Sean Owen commented on SPARK-3934: -- Yep that fixes the issue I was seeing. Thanks! I can

[jira] [Created] (SPARK-3936) Incorrect result in GraphX BytecodeUtils with closures + class/object methods

2014-10-13 Thread Pedro Rodriguez (JIRA)
Pedro Rodriguez created SPARK-3936: -- Summary: Incorrect result in GraphX BytecodeUtils with closures + class/object methods Key: SPARK-3936 URL: https://issues.apache.org/jira/browse/SPARK-3936 Proje

[jira] [Commented] (SPARK-2493) SBT gen-idea doesn't generate correct Intellij project

2014-10-13 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170080#comment-14170080 ] DB Tsai commented on SPARK-2493: sbt gen-idea will add extra meso dependency other than me

[jira] [Closed] (SPARK-2493) SBT gen-idea doesn't generate correct Intellij project

2014-10-13 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai closed SPARK-2493. -- Resolution: Won't Fix > SBT gen-idea doesn't generate correct Intellij project > ---

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-13 Thread Timothy St. Clair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170033#comment-14170033 ] Timothy St. Clair commented on SPARK-3174: -- [~pwendell] are you talking about res

[jira] [Updated] (SPARK-3935) Unused variable in PairRDDFunctions.scala

2014-10-13 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-3935: --- Description: There is a unused variable (count) in saveAsHadoopDataset function in PairRDDFunctions.scala. I

[jira] [Commented] (SPARK-3912) FlumeStreamSuite is flaky, fails either with port binding issues or data not being reliably sent

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170006#comment-14170006 ] Apache Spark commented on SPARK-3912: - User 'tdas' has created a pull request for this

[jira] [Commented] (SPARK-3906) Support joins of multiple tables in SparkSQL (SQLContext, not HiveQL)

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169948#comment-14169948 ] Apache Spark commented on SPARK-3906: - User 'liancheng' has created a pull request for

[jira] [Resolved] (SPARK-3809) make HiveThriftServer2Suite work correctly

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3809. - Resolution: Fixed Issue resolved by pull request 2675 [https://github.com/apache/spark/pul

[jira] [Resolved] (SPARK-3807) SparkSql does not work for tables created using custom serde

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3807. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2674 [https:/

[jira] [Created] (SPARK-3935) Unused variable in PairRDDFunctions.scala

2014-10-13 Thread wangfei (JIRA)
wangfei created SPARK-3935: -- Summary: Unused variable in PairRDDFunctions.scala Key: SPARK-3935 URL: https://issues.apache.org/jira/browse/SPARK-3935 Project: Spark Issue Type: Bug Compone

[jira] [Resolved] (SPARK-3559) appendReadColumnIDs and appendReadColumnNames introduce unnecessary columns in the lists of needed column ids and column names stored in hiveConf

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3559. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2713 [https:/

[jira] [Comment Edited] (SPARK-3923) All Standalone Mode services time out with each other

2014-10-13 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169901#comment-14169901 ] Jianshi Huang edited comment on SPARK-3923 at 10/13/14 8:45 PM:

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-13 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169931#comment-14169931 ] Sandy Ryza commented on SPARK-3174: --- bq. Slow-start is actually not slow at all if we lo

[jira] [Resolved] (SPARK-3771) AppendingParquetOutputFormat should use reflection to prevent from breaking binary-compatibility.

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3771. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2638 [https:/

[jira] [Resolved] (SPARK-2066) Better error message for non-aggregated attributes with aggregates

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2066. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2774 [https:/

[jira] [Resolved] (SPARK-3529) Delete the temporal files after test exit

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3529. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2393 [https:/

[jira] [Commented] (SPARK-2066) Better error message for non-aggregated attributes with aggregates

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169902#comment-14169902 ] Apache Spark commented on SPARK-2066: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-3923) All Standalone Mode services time out with each other

2014-10-13 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169901#comment-14169901 ] Jianshi Huang commented on SPARK-3923: -- I have similar problem in YARN-client mode. S

[jira] [Commented] (SPARK-3892) Map type do not need simpleName

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169897#comment-14169897 ] Apache Spark commented on SPARK-3892: - User 'adrian-wang' has created a pull request f

[jira] [Resolved] (SPARK-3407) Add Date type support

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3407. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2344 [https:/

[jira] [Resolved] (SPARK-3892) Map type do not need simpleName

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3892. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2747 [https:/

[jira] [Commented] (SPARK-3266) JavaDoubleRDD doesn't contain max()

2014-10-13 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169873#comment-14169873 ] Josh Rosen commented on SPARK-3266: --- I believe the erasure to {{Object}} is due to a Sca

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169856#comment-14169856 ] Andrew Or commented on SPARK-3174: -- [~sandyr] bq. Consider the (common) case of a user ke

[jira] [Commented] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

2014-10-13 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169846#comment-14169846 ] Sandy Ryza commented on SPARK-1209: --- Definitely worth changing, in my opinion. This has

[jira] [Commented] (SPARK-1209) SparkHadoopUtil should not use package org.apache.hadoop

2014-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169835#comment-14169835 ] Sean Owen commented on SPARK-1209: -- Yes, I wonder too, does SparkHadoopMapRedUtil and Spa

[jira] [Comment Edited] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169830#comment-14169830 ] Andrew Or edited comment on SPARK-3174 at 10/13/14 7:45 PM: [~

[jira] [Comment Edited] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169830#comment-14169830 ] Andrew Or edited comment on SPARK-3174 at 10/13/14 7:45 PM: [~

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169830#comment-14169830 ] Andrew Or commented on SPARK-3174: -- [~vanzin] bq. My first question I think is similar t

[jira] [Commented] (SPARK-1307) don't use term 'standalone' to refer to a Spark Application

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169831#comment-14169831 ] Apache Spark commented on SPARK-1307: - User 'srowen' has created a pull request for th

[jira] [Comment Edited] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169830#comment-14169830 ] Andrew Or edited comment on SPARK-3174 at 10/13/14 7:45 PM: [~

[jira] [Comment Edited] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-13 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169830#comment-14169830 ] Andrew Or edited comment on SPARK-3174 at 10/13/14 7:45 PM: [~

[jira] [Updated] (SPARK-3925) Do not consider the ordering of qualifiers during comparison

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3925: --- Component/s: SQL > Do not consider the ordering of qualifiers during comparison >

[jira] [Resolved] (SPARK-2707) Upgrade to Akka 2.3

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2707. Resolution: Fixed This was fixed in SPARK-2805 > Upgrade to Akka 2.3 > ---

[jira] [Updated] (SPARK-1010) Update all unit tests to use SparkConf instead of system properties

2014-10-13 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1010: --- Reporter: Patrick Wendell (was: Patrick Cogan) > Update all unit tests to use SparkConf inste

[jira] [Updated] (SPARK-3708) Backticks aren't handled correctly is aliases

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3708: Assignee: Ravindra Pesala > Backticks aren't handled correctly is aliases >

[jira] [Updated] (SPARK-3371) Spark SQL: Renaming a function expression with group by gives error

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3371: Assignee: Ravindra Pesala > Spark SQL: Renaming a function expression with group by gives er

[jira] [Updated] (SPARK-2594) Add CACHE TABLE AS SELECT ...

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2594: Assignee: Ravindra Pesala > Add CACHE TABLE AS SELECT ... > ---

[jira] [Updated] (SPARK-3654) Implement all extended HiveQL statements/commands with a separate parser combinator

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3654: Assignee: Ravindra Pesala > Implement all extended HiveQL statements/commands with a separat

[jira] [Updated] (SPARK-3813) Support "case when" conditional functions in Spark SQL

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3813: Assignee: Ravindra Pesala > Support "case when" conditional functions in Spark SQL > ---

[jira] [Commented] (SPARK-3913) Spark Yarn Client API change to expose Yarn Resource Capacity, Yarn Application Listener and killApplication() API

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169760#comment-14169760 ] Apache Spark commented on SPARK-3913: - User 'chesterxgchen' has created a pull request

[jira] [Commented] (SPARK-3918) Forget Unpersist in RandomForest.scala(train Method)

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169753#comment-14169753 ] Apache Spark commented on SPARK-3918: - User 'jkbradley' has created a pull request for

[jira] [Commented] (SPARK-3934) RandomForest bug in sanity check in DTStatsAggregator

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169752#comment-14169752 ] Apache Spark commented on SPARK-3934: - User 'jkbradley' has created a pull request for

[jira] [Resolved] (SPARK-3861) Avoid rebuilding hash tables on each partition

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3861. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2727 [https:/

[jira] [Created] (SPARK-3934) RandomForest bug in sanity check in DTStatsAggregator

2014-10-13 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-3934: Summary: RandomForest bug in sanity check in DTStatsAggregator Key: SPARK-3934 URL: https://issues.apache.org/jira/browse/SPARK-3934 Project: Spark I

[jira] [Commented] (SPARK-1010) Update all unit tests to use SparkConf instead of system properties

2014-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169724#comment-14169724 ] Sean Owen commented on SPARK-1010: -- Yes, lots of usage in tests still. A lot looks intent

[jira] [Commented] (SPARK-1463) cleanup unnecessary dependency jars in the spark assembly jars

2014-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169709#comment-14169709 ] Sean Owen commented on SPARK-1463: -- FWIW I do not see these packages in the final assembl

[jira] [Resolved] (SPARK-1409) Flaky Test: "actor input stream" test in org.apache.spark.streaming.InputStreamsSuite

2014-10-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-1409. - Resolution: Won't Fix > Flaky Test: "actor input stream" test in > org.apache.spark.strea

[jira] [Commented] (SPARK-3923) All Standalone Mode services time out with each other

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169692#comment-14169692 ] Apache Spark commented on SPARK-3923: - User 'aarondav' has created a pull request for

[jira] [Resolved] (SPARK-1017) Set the permgen even if we are calling the users sbt (via SBT_OPTS)

2014-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1017. -- Resolution: Won't Fix As I understand, only {{sbt/sbt}} is supported for building Spark with SBT, rathe

[jira] [Resolved] (SPARK-1083) Build fail

2014-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1083. -- Resolution: Cannot Reproduce This looks like a git error, and is ancient at this point. I presume that s

[jira] [Commented] (SPARK-1192) Around 30 parameters in Spark are used but undocumented and some are having confusing name

2014-10-13 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169663#comment-14169663 ] Nan Zhu commented on SPARK-1192: yes, I resubmitted https://github.com/apache/spark/pull/2

[jira] [Resolved] (SPARK-1149) Bad partitioners can cause Spark to hang

2014-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1149. -- Resolution: Fixed Looks like Patrick merged this into master in March. It might have been fixed for ...

[jira] [Commented] (SPARK-3251) Clarify learning interfaces

2014-10-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169659#comment-14169659 ] Joseph K. Bradley commented on SPARK-3251: -- I agree it's hard to say. Based on t

[jira] [Commented] (SPARK-1192) Around 30 parameters in Spark are used but undocumented and some are having confusing name

2014-10-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169658#comment-14169658 ] Apache Spark commented on SPARK-1192: - User 'CodingCat' has created a pull request for

[jira] [Commented] (SPARK-1192) Around 30 parameters in Spark are used but undocumented and some are having confusing name

2014-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169649#comment-14169649 ] Sean Owen commented on SPARK-1192: -- PR is actually at https://github.com/apache/spark/pul

[jira] [Resolved] (SPARK-1234) clean up typos and grammar issues in Spark on YARN page

2014-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1234. -- Resolution: Won't Fix Given the discussion in https://github.com/apache/spark/pull/130 , this was aband

[jira] [Resolved] (SPARK-1306) no instructions provided for sbt assembly with Hadoop 2.2

2014-10-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1306. -- Resolution: Fixed I think this was obviated by subsequent changes to this documentation. SBT is no long

  1   2   >