[jira] [Commented] (SPARK-6923) Get invalid hive table columns after save DataFrame to hive table

2015-04-21 Thread pin_zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504409#comment-14504409 ] pin_zhang commented on SPARK-6923: -- Hi, Michael We run spark app in Spark1.3, and use

[jira] [Assigned] (SPARK-1442) Add Window function support

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-1442: --- Assignee: (was: Apache Spark) Add Window function support ---

[jira] [Updated] (SPARK-7026) LeftSemiJoin can not work when it has not equal condition.

2015-04-21 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-7026: -- Description: Run sql like that {quote} select * from web_sales ws1 left semi join web_sales

[jira] [Updated] (SPARK-7026) LeftSemiJoin can not work when it has equal condition and not equal condition.

2015-04-21 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-7026: -- Summary: LeftSemiJoin can not work when it has equal condition and not equal condition.

[jira] [Resolved] (SPARK-6949) Support Date/Timestamp in Column expression of DataFrame Python API

2015-04-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6949. Resolution: Fixed Fix Version/s: 1.4.0 Support Date/Timestamp in Column expression of

[jira] [Resolved] (SPARK-6867) Dropout regularization

2015-04-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6867. -- Resolution: Later Rakesh, thanks for sharing the papers! Per discussion on the PR page, we need

[jira] [Resolved] (SPARK-5081) Shuffle write increases

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5081. Resolution: Duplicate I'm pretty sure this is fixed via SPARK-6905. Closing this and we can

[jira] [Comment Edited] (SPARK-5081) Shuffle write increases

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504442#comment-14504442 ] Patrick Wendell edited comment on SPARK-5081 at 4/21/15 6:36 AM:

[jira] [Resolved] (SPARK-6490) Deprecate configurations for askWithReply and use new configuration names

2015-04-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6490. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Shixiong Zhu Deprecate

[jira] [Updated] (SPARK-6490) Deprecate configurations for askWithReply and use new configuration names

2015-04-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6490: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-5293 Deprecate configurations for

[jira] [Resolved] (SPARK-6719) Update spark.apache.org/mllib page to 1.3

2015-04-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6719. -- Resolution: Done Update spark.apache.org/mllib page to 1.3

[jira] [Updated] (SPARK-7026) LeftSemiJoin can not work when it has not equal condition.

2015-04-21 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-7026: -- Description: Run sql like that {panel} select * from web_sales ws1 left semi join web_sales

[jira] [Updated] (SPARK-7026) LeftSemiJoin can not work when it has not equal condition.

2015-04-21 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-7026: -- Description: Run sql like that {panel} select * from web_sales ws1 left semi join web_sales

[jira] [Updated] (SPARK-7026) LeftSemiJoin can not work when it has equal condition and not equal condition.

2015-04-21 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-7026: -- Description: Run sql like that {panel} select * from web_sales ws1 left semi join web_sales

[jira] [Updated] (SPARK-6845) Add isTranposed flag to DenseMatrix SerDe

2015-04-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6845: - Priority: Minor (was: Major) Target Version/s: 1.4.0 Summary: Add

[jira] [Updated] (SPARK-6845) Add isTranposed flag to DenseMatrix

2015-04-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6845: - Assignee: Manoj Kumar Add isTranposed flag to DenseMatrix ---

[jira] [Commented] (SPARK-6932) A Prototype of Parameter Server

2015-04-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504403#comment-14504403 ] Xiangrui Meng commented on SPARK-6932: -- [~chouqin] Could you list the changes to core

[jira] [Updated] (SPARK-7008) An Implement of Factorization Machine (LibFM)

2015-04-21 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-7008: --- Attachment: FM_convergence_rate.xlsx QQ20150421-1.png QQ20150421-2.png

[jira] [Commented] (SPARK-1442) Add Window function support

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504407#comment-14504407 ] Apache Spark commented on SPARK-1442: - User 'guowei2' has created a pull request for

[jira] [Assigned] (SPARK-1442) Add Window function support

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-1442: --- Assignee: Apache Spark Add Window function support ---

[jira] [Updated] (SPARK-7026) LeftSemiJoin can not work when it has not equal condition and equal condition.

2015-04-21 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-7026: -- Summary: LeftSemiJoin can not work when it has not equal condition and equal condition.

[jira] [Resolved] (SPARK-2044) Pluggable interface for shuffles

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2044. Resolution: Fixed Fix Version/s: 1.1.2 Pluggable interface for shuffles

[jira] [Updated] (SPARK-2044) Pluggable interface for shuffles

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2044: --- Fix Version/s: (was: 1.1.2) 1.1.0 Pluggable interface for shuffles

[jira] [Commented] (SPARK-7001) Partitions for a long single line file

2015-04-21 Thread Victor Bashurov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504452#comment-14504452 ] Victor Bashurov commented on SPARK-7001: Need to try InputFormat then or just

[jira] [Created] (SPARK-7026) LeftSemiJoin can not work when it has not equal condition.

2015-04-21 Thread Zhongshuai Pei (JIRA)
Zhongshuai Pei created SPARK-7026: - Summary: LeftSemiJoin can not work when it has not equal condition. Key: SPARK-7026 URL: https://issues.apache.org/jira/browse/SPARK-7026 Project: Spark

[jira] [Updated] (SPARK-7028) Add filterNot to RDD

2015-04-21 Thread Marius Soutier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marius Soutier updated SPARK-7028: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) Add filterNot to RDD

[jira] [Created] (SPARK-7028) Add filterNot to RDD

2015-04-21 Thread Marius Soutier (JIRA)
Marius Soutier created SPARK-7028: - Summary: Add filterNot to RDD Key: SPARK-7028 URL: https://issues.apache.org/jira/browse/SPARK-7028 Project: Spark Issue Type: Bug Reporter:

[jira] [Created] (SPARK-7039) JdbcRdd doesn't support java.sql.Types.NVARCHAR

2015-04-21 Thread Shuai Zheng (JIRA)
Shuai Zheng created SPARK-7039: -- Summary: JdbcRdd doesn't support java.sql.Types.NVARCHAR Key: SPARK-7039 URL: https://issues.apache.org/jira/browse/SPARK-7039 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6980: --- Assignee: Harsh Gupta Akka timeout exceptions indicate which conf controls them

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505404#comment-14505404 ] Patrick Wendell commented on SPARK-6980: [~harshg] I just assigned you Akka

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-21 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505493#comment-14505493 ] Bryan Cutler commented on SPARK-6980: - Hi [~pwendell], I already have been working on

[jira] [Assigned] (SPARK-7037) Inconsistent behavior for non-spark config properties in spark-shell and spark-submit

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7037: --- Assignee: (was: Apache Spark) Inconsistent behavior for non-spark config properties in

[jira] [Commented] (SPARK-7037) Inconsistent behavior for non-spark config properties in spark-shell and spark-submit

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505553#comment-14505553 ] Apache Spark commented on SPARK-7037: - User 'piaozhexiu' has created a pull request

[jira] [Assigned] (SPARK-7037) Inconsistent behavior for non-spark config properties in spark-shell and spark-submit

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7037: --- Assignee: Apache Spark Inconsistent behavior for non-spark config properties in spark-shell

[jira] [Commented] (SPARK-7035) Drop __getattr__ on pyspark.sql.DataFrame

2015-04-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505327#comment-14505327 ] Reynold Xin commented on SPARK-7035: Pandas also support this, doesn't it? Drop

[jira] [Updated] (SPARK-7027) Spark 1.2.2 Hadoop 2.4 download is missing

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7027: --- Summary: Spark 1.2.2 Hadoop 2.4 download is missing (was: Spark 1.2.2 Hadoop 2.4 download

[jira] [Updated] (SPARK-7027) Spark 1.2.2 Hadoop 2.4 download is missing

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7027: --- Priority: Critical (was: Major) Spark 1.2.2 Hadoop 2.4 download is missing

[jira] [Resolved] (SPARK-7030) spark-submit in cluster mode with supervisor flag, not able to transfer application jar to driver node

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-7030. Resolution: Invalid This is filed as a bug report, but this is not currently a feature (in

[jira] [Assigned] (SPARK-7027) Spark 1.2.2 Hadoop 2.4 download is missing

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reassigned SPARK-7027: -- Assignee: Patrick Wendell Spark 1.2.2 Hadoop 2.4 download is missing

[jira] [Updated] (SPARK-6958) Add Pandas style sort operator

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6958: --- Summary: Add Pandas style sort operator (was: sort) Add Pandas style sort operator

[jira] [Updated] (SPARK-6957) Add Pandas style group by operator

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6957: --- Summary: Add Pandas style group by operator (was: groupby) Add Pandas style group by

[jira] [Created] (SPARK-7036) ALS.train should support DataFrames in PySpark

2015-04-21 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-7036: Summary: ALS.train should support DataFrames in PySpark Key: SPARK-7036 URL: https://issues.apache.org/jira/browse/SPARK-7036 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-21 Thread Harsh Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505530#comment-14505530 ] Harsh Gupta commented on SPARK-6980: [~bryanc] I need something to start with so that

[jira] [Updated] (SPARK-6774) Implement Parquet complex types backwards-compatiblity rules

2015-04-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6774: -- Assignee: (was: Cheng Lian) Implement Parquet complex types backwards-compatiblity rules

[jira] [Updated] (SPARK-7016) Refactor dev/run-tests(-jenkins) from Bash to Python

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7016: --- Priority: Critical (was: Major) Refactor dev/run-tests(-jenkins) from Bash to Python

[jira] [Commented] (SPARK-7039) JdbcRdd doesn't support java.sql.Types.NVARCHAR

2015-04-21 Thread Shuai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505597#comment-14505597 ] Shuai Zheng commented on SPARK-7039: I have created a pull request. And the related

[jira] [Comment Edited] (SPARK-7039) JdbcRdd doesn't support java.sql.Types.NVARCHAR

2015-04-21 Thread Shuai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505597#comment-14505597 ] Shuai Zheng edited comment on SPARK-7039 at 4/21/15 7:43 PM: -

[jira] [Commented] (SPARK-7035) Drop __getattr__ on pyspark.sql.DataFrame

2015-04-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505362#comment-14505362 ] Reynold Xin commented on SPARK-7035: It'd be great to understand the delta between

[jira] [Comment Edited] (SPARK-7035) Drop __getattr__ on pyspark.sql.DataFrame

2015-04-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505362#comment-14505362 ] Reynold Xin edited comment on SPARK-7035 at 4/21/15 6:03 PM: -

[jira] [Created] (SPARK-7037) Inconsistent behavior for non-spark config properties in spark-shell and spark-submit

2015-04-21 Thread Cheolsoo Park (JIRA)
Cheolsoo Park created SPARK-7037: Summary: Inconsistent behavior for non-spark config properties in spark-shell and spark-submit Key: SPARK-7037 URL: https://issues.apache.org/jira/browse/SPARK-7037

[jira] [Resolved] (SPARK-5360) For CoGroupedRDD, rdds for narrow dependencies and shuffle handles are included twice in serialized task

2015-04-21 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-5360. --- Resolution: Fixed Fix Version/s: 1.4.0 For CoGroupedRDD, rdds for narrow dependencies

[jira] [Resolved] (SPARK-6995) Spark driver program throws exception cant get Master Kerberose principal for use as renewer

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-6995. Resolution: Invalid Can you reach out on the user list with this question? Thanks! Spark

[jira] [Assigned] (SPARK-6829) Support math functions in DataFrames

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6829: --- Assignee: Apache Spark Support math functions in DataFrames

[jira] [Commented] (SPARK-6829) Support math functions in DataFrames

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505526#comment-14505526 ] Apache Spark commented on SPARK-6829: - User 'brkyvz' has created a pull request for

[jira] [Assigned] (SPARK-6829) Support math functions in DataFrames

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6829: --- Assignee: (was: Apache Spark) Support math functions in DataFrames

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505527#comment-14505527 ] Sean Owen commented on SPARK-6980: -- [~harsh.gup...@gmail.com] can you coordinate with

[jira] [Updated] (SPARK-7020) Restrict module testing based on commit contents

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7020: --- Priority: Critical (was: Major) Restrict module testing based on commit contents

[jira] [Resolved] (SPARK-7027) Spark 1.2.2 Hadoop 2.4 download is missing

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-7027. Resolution: Fixed Fix Version/s: 1.2.2 It's fixed now, but will take several hours

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-21 Thread Harsh Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505543#comment-14505543 ] Harsh Gupta commented on SPARK-6980: [~bryanc] Can we discuss over it if you are ok

[jira] [Created] (SPARK-7038) [Streaming] Spark Sink requires spark assembly in classpath

2015-04-21 Thread Hari Shreedharan (JIRA)
Hari Shreedharan created SPARK-7038: --- Summary: [Streaming] Spark Sink requires spark assembly in classpath Key: SPARK-7038 URL: https://issues.apache.org/jira/browse/SPARK-7038 Project: Spark

[jira] [Updated] (SPARK-3860) Improve dimension joins

2015-04-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3860: Assignee: (was: Michael Armbrust) Improve dimension joins ---

[jira] [Updated] (SPARK-3440) HiveServer2 and CLI should retrieve Hive result set schema

2015-04-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-3440: Shepherd: (was: Michael Armbrust) HiveServer2 and CLI should retrieve Hive result set

[jira] [Updated] (SPARK-6990) Add Java linting script

2015-04-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6990: --- Labels: starter (was: ) Add Java linting script ---

[jira] [Commented] (SPARK-5081) Shuffle write increases

2015-04-21 Thread Dr. Christian Betz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504592#comment-14504592 ] Dr. Christian Betz commented on SPARK-5081: --- Hi [~pwendell], I hope this is

[jira] [Updated] (SPARK-5423) ExternalAppendOnlyMap won't delete temp spilled file if some exception happens during using it

2015-04-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5423: - Target Version/s: (was: 1.1.2, 1.2.2, 1.3.0) ExternalAppendOnlyMap won't delete temp spilled file if

[jira] [Commented] (SPARK-6932) A Prototype of Parameter Server

2015-04-21 Thread Qiping Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504541#comment-14504541 ] Qiping Li commented on SPARK-6932: -- Hi, Xiangrui. Our implementation has the following

[jira] [Assigned] (SPARK-3386) Reuse serializer and serializer buffer in shuffle block iterator

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3386: --- Assignee: Apache Spark Reuse serializer and serializer buffer in shuffle block iterator

[jira] [Updated] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-04-21 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-7008: Summary: An implementation of Factorization Machine (LibFM) (was: An Implement of Factorization

[jira] [Commented] (SPARK-6567) Large linear model parallelism via a join and reduceByKey

2015-04-21 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504583#comment-14504583 ] Reza Zadeh commented on SPARK-6567: --- Hi Hucheng, Yes this is what I have in mind. Do you

[jira] [Updated] (SPARK-6567) Large linear model parallelism via a join and reduceByKey

2015-04-21 Thread hucheng zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hucheng zhou updated SPARK-6567: Attachment: (was: model-parallelism.pptx) Large linear model parallelism via a join and

[jira] [Updated] (SPARK-6567) Large linear model parallelism via a join and reduceByKey

2015-04-21 Thread hucheng zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hucheng zhou updated SPARK-6567: Attachment: model-parallelism.pptx This slides illustrates the model parallelism for logistic

[jira] [Issue Comment Deleted] (SPARK-6567) Large linear model parallelism via a join and reduceByKey

2015-04-21 Thread hucheng zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hucheng zhou updated SPARK-6567: Comment: was deleted (was: This slides illustrates the model parallelism for logistic regression.

[jira] [Updated] (SPARK-6567) Large linear model parallelism via a join and reduceByKey

2015-04-21 Thread hucheng zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hucheng zhou updated SPARK-6567: Attachment: model-parallelism.pptx This slides illustrates the model parallelism for logistic

[jira] [Comment Edited] (SPARK-6567) Large linear model parallelism via a join and reduceByKey

2015-04-21 Thread hucheng zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504473#comment-14504473 ] hucheng zhou edited comment on SPARK-6567 at 4/21/15 7:37 AM: --

[jira] [Commented] (SPARK-7009) Build assembly JAR via ant to avoid zip64 problems

2015-04-21 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504639#comment-14504639 ] Steve Loughran commented on SPARK-7009: --- yes, what we are trying to do is come up

[jira] [Commented] (SPARK-6932) A Prototype of Parameter Server

2015-04-21 Thread Qiping Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504572#comment-14504572 ] Qiping Li commented on SPARK-6932: -- [~gq], {quote} `. If the reduceFunc rely on other

[jira] [Comment Edited] (SPARK-6932) A Prototype of Parameter Server

2015-04-21 Thread Qiping Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504572#comment-14504572 ] Qiping Li edited comment on SPARK-6932 at 4/21/15 8:17 AM: ---

[jira] [Created] (SPARK-7029) Unable to use hive built-in functions in sparkSQL

2015-04-21 Thread Aditya Parmar (JIRA)
Aditya Parmar created SPARK-7029: Summary: Unable to use hive built-in functions in sparkSQL Key: SPARK-7029 URL: https://issues.apache.org/jira/browse/SPARK-7029 Project: Spark Issue Type:

[jira] [Created] (SPARK-7030) spark-submit in cluster mode with supervisor flag, not able to transfer application jar to driver node

2015-04-21 Thread pankaj (JIRA)
pankaj created SPARK-7030: - Summary: spark-submit in cluster mode with supervisor flag, not able to transfer application jar to driver node Key: SPARK-7030 URL: https://issues.apache.org/jira/browse/SPARK-7030

[jira] [Assigned] (SPARK-3386) Reuse serializer and serializer buffer in shuffle block iterator

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3386: --- Assignee: (was: Apache Spark) Reuse serializer and serializer buffer in shuffle block

[jira] [Commented] (SPARK-3386) Reuse serializer and serializer buffer in shuffle block iterator

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504540#comment-14504540 ] Apache Spark commented on SPARK-3386: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-6490) Deprecate configurations for askWithReply and use new configuration names

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504635#comment-14504635 ] Apache Spark commented on SPARK-6490: - User 'zsxwing' has created a pull request for

[jira] [Updated] (SPARK-7030) spark-submit in cluster mode with supervisor flag, not able to transfer application jar to driver node

2015-04-21 Thread pankaj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pankaj updated SPARK-7030: -- Affects Version/s: 1.3.0 Fix Version/s: (was: 1.3.0) spark-submit in cluster mode with supervisor

[jira] [Comment Edited] (SPARK-6567) Large linear model parallelism via a join and reduceByKey

2015-04-21 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504583#comment-14504583 ] Reza Zadeh edited comment on SPARK-6567 at 4/21/15 8:32 AM: Hi

[jira] [Commented] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-04-21 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504596#comment-14504596 ] zhengruifeng commented on SPARK-7008: - I had not considered of the size of model,

[jira] [Commented] (SPARK-6738) EstimateSize is difference with spill file size

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504638#comment-14504638 ] Apache Spark commented on SPARK-6738: - User 'shenh062326' has created a pull request

[jira] [Commented] (SPARK-1153) Generalize VertexId in GraphX so that UUIDs can be used as vertex IDs.

2015-04-21 Thread Carlos Balduz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505071#comment-14505071 ] Carlos Balduz commented on SPARK-1153: -- I am currently using zipWithUniqueId() to get

[jira] [Assigned] (SPARK-6827) Wrap FPGrowthModel.freqItemsets with namedtuples (or document the return type) in PySpark

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6827: --- Assignee: Apache Spark (was: Yanbo Liang) Wrap FPGrowthModel.freqItemsets with namedtuples

[jira] [Assigned] (SPARK-6827) Wrap FPGrowthModel.freqItemsets with namedtuples (or document the return type) in PySpark

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6827: --- Assignee: Yanbo Liang (was: Apache Spark) Wrap FPGrowthModel.freqItemsets with namedtuples

[jira] [Commented] (SPARK-6827) Wrap FPGrowthModel.freqItemsets with namedtuples (or document the return type) in PySpark

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505109#comment-14505109 ] Apache Spark commented on SPARK-6827: - User 'yanboliang' has created a pull request

[jira] [Created] (SPARK-7035) Drop __getattr__ on pyspark.sql.DataFrame

2015-04-21 Thread Kalle Jepsen (JIRA)
Kalle Jepsen created SPARK-7035: --- Summary: Drop __getattr__ on pyspark.sql.DataFrame Key: SPARK-7035 URL: https://issues.apache.org/jira/browse/SPARK-7035 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4977) spark-ec2 start resets all the spark/conf configurations

2015-04-21 Thread Brad Willard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504977#comment-14504977 ] Brad Willard commented on SPARK-4977: - I would love to see this addressed. I have

[jira] [Commented] (SPARK-6567) Large linear model parallelism via a join and reduceByKey

2015-04-21 Thread hucheng zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505039#comment-14505039 ] hucheng zhou commented on SPARK-6567: - @Reza, Yes, we have implemented it for

[jira] [Commented] (SPARK-3276) Provide a API to specify MIN_REMEMBER_DURATION for files to consider as input in streaming

2015-04-21 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504962#comment-14504962 ] Emre Sevinç commented on SPARK-3276: [~tdas] did you have the opportunity to check my

[jira] [Created] (SPARK-7034) Support escaped double quotes on data source options

2015-04-21 Thread Santiago M. Mola (JIRA)
Santiago M. Mola created SPARK-7034: --- Summary: Support escaped double quotes on data source options Key: SPARK-7034 URL: https://issues.apache.org/jira/browse/SPARK-7034 Project: Spark

[jira] [Created] (SPARK-7033) Use JavaRDD.partitions() instead of JavaRDD.splits()

2015-04-21 Thread Sun Rui (JIRA)
Sun Rui created SPARK-7033: -- Summary: Use JavaRDD.partitions() instead of JavaRDD.splits() Key: SPARK-7033 URL: https://issues.apache.org/jira/browse/SPARK-7033 Project: Spark Issue Type:

[jira] [Commented] (SPARK-7026) LeftSemiJoin can not work when it has both equal condition and not equal condition.

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504976#comment-14504976 ] Apache Spark commented on SPARK-7026: - User 'scwf' has created a pull request for this

[jira] [Assigned] (SPARK-6852) Accept numeric as numPartitions in SparkR

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6852: --- Assignee: Apache Spark Accept numeric as numPartitions in SparkR

[jira] [Commented] (SPARK-6852) Accept numeric as numPartitions in SparkR

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504984#comment-14504984 ] Apache Spark commented on SPARK-6852: - User 'sun-rui' has created a pull request for

[jira] [Assigned] (SPARK-6852) Accept numeric as numPartitions in SparkR

2015-04-21 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6852: --- Assignee: (was: Apache Spark) Accept numeric as numPartitions in SparkR

  1   2   3   >