[jira] [Commented] (SPARK-5472) Add support for reading from and writing to a JDBC database

2015-02-03 Thread Tor Myklebust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304588#comment-14304588 ] Tor Myklebust commented on SPARK-5472: -- If the data in the underlying table changes,

[jira] [Commented] (SPARK-5583) Support unique join in hive context

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304624#comment-14304624 ] Apache Spark commented on SPARK-5583: - User 'scwf' has created a pull request for this

[jira] [Updated] (SPARK-5584) Add Maven Enforcer Plugin dependencyConvergence rule (fail false)

2015-02-03 Thread Markus Dale (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Markus Dale updated SPARK-5584: --- Description: The Spark Maven build uses the Maven Enforcer plugin but does not have a rule for

[jira] [Resolved] (SPARK-4969) Add binaryRecords support to streaming

2015-02-03 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4969. -- Resolution: Fixed Fix Version/s: 1.3.0 Add binaryRecords support to streaming

[jira] [Updated] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5585: --- Labels: flaky-test (was: ) Flaky test: Python regression -

[jira] [Closed] (SPARK-5526) expression [date '2011-01-01' = cast(timestamp('2011-01-01 23:24:25') as date)] return false

2015-02-03 Thread xukun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xukun closed SPARK-5526. Resolution: Fixed the issue is fixed by #4325 expression [date '2011-01-01' = cast(timestamp('2011-01-01

[jira] [Created] (SPARK-5583) Support unique join in hive context

2015-02-03 Thread wangfei (JIRA)
wangfei created SPARK-5583: -- Summary: Support unique join in hive context Key: SPARK-5583 URL: https://issues.apache.org/jira/browse/SPARK-5583 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-4795) Redesign the primitive type = Writable implicit APIs to make them be activated automatically

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-4795. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Shixiong Zhu Redesign the

[jira] [Created] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-5585: -- Summary: Flaky test: Python regression Key: SPARK-5585 URL: https://issues.apache.org/jira/browse/SPARK-5585 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5585: --- Affects Version/s: 1.3.0 Flaky test: Python regression -

[jira] [Updated] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5585: --- Priority: Critical (was: Major) Flaky test: Python regression

[jira] [Commented] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304509#comment-14304509 ] Lianhui Wang commented on SPARK-5529: - the phenomenon is: blockManagerSlave is timeout

[jira] [Commented] (SPARK-5140) Two RDDs which are scheduled concurrently should be able to wait on parent in all cases

2015-02-03 Thread Corey J. Nolet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304510#comment-14304510 ] Corey J. Nolet commented on SPARK-5140: --- I think the problem is that when actions

[jira] [Updated] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-5529: --- Description: When I run a spark job, one executor is hold, after 120s, blockManager is removed by

[jira] [Created] (SPARK-5582) History server does not list anything if log root contains an empty directory

2015-02-03 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-5582: - Summary: History server does not list anything if log root contains an empty directory Key: SPARK-5582 URL: https://issues.apache.org/jira/browse/SPARK-5582

[jira] [Created] (SPARK-5584) Add Maven Enforcer Plugin dependencyConvergence rule (fail false)

2015-02-03 Thread Markus Dale (JIRA)
Markus Dale created SPARK-5584: -- Summary: Add Maven Enforcer Plugin dependencyConvergence rule (fail false) Key: SPARK-5584 URL: https://issues.apache.org/jira/browse/SPARK-5584 Project: Spark

[jira] [Comment Edited] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304509#comment-14304509 ] Lianhui Wang edited comment on SPARK-5529 at 2/4/15 2:39 AM: -

[jira] [Comment Edited] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304509#comment-14304509 ] Lianhui Wang edited comment on SPARK-5529 at 2/4/15 2:40 AM: -

[jira] [Created] (SPARK-5581) When writing sorted map output file, avoid open / close between each partition

2015-02-03 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-5581: - Summary: When writing sorted map output file, avoid open / close between each partition Key: SPARK-5581 URL: https://issues.apache.org/jira/browse/SPARK-5581 Project:

[jira] [Resolved] (SPARK-2440) Enable HistoryServer to display lots of Application History

2015-02-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-2440. --- Resolution: Fixed I'll mark this as fixed since the current history server doesn't have that

[jira] [Commented] (SPARK-5475) Java 8 tests are like maintenance overhead.

2015-02-03 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304662#comment-14304662 ] Prashant Sharma commented on SPARK-5475: And this is how it looks after running in

[jira] [Comment Edited] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304509#comment-14304509 ] Lianhui Wang edited comment on SPARK-5529 at 2/4/15 2:37 AM: -

[jira] [Commented] (SPARK-5582) History server does not list anything if log root contains an empty directory

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304599#comment-14304599 ] Apache Spark commented on SPARK-5582: - User 'vanzin' has created a pull request for

[jira] [Commented] (SPARK-5577) Create a convenient way for Python users to register SQL UDFs

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304494#comment-14304494 ] Apache Spark commented on SPARK-5577: - User 'davies' has created a pull request for

[jira] [Resolved] (SPARK-5578) Provide a convenient way for Scala users to use UDFs

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5578. Resolution: Fixed Fix Version/s: 1.3.0 Provide a convenient way for Scala users to use UDFs

[jira] [Resolved] (SPARK-5237) UDTF don't work with multi-alias of multi-columns as output on SparK SQL

2015-02-03 Thread Adrian Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang resolved SPARK-5237. Resolution: Duplicate SPARK-5383 should solved this. UDTF don't work with multi-alias of

[jira] [Created] (SPARK-5580) Grep bug in compute-classpath.sh

2015-02-03 Thread Yadong Qi (JIRA)
Yadong Qi created SPARK-5580: Summary: Grep bug in compute-classpath.sh Key: SPARK-5580 URL: https://issues.apache.org/jira/browse/SPARK-5580 Project: Spark Issue Type: Bug Reporter:

[jira] [Updated] (SPARK-5580) Grep bug in compute-classpath.sh

2015-02-03 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi updated SPARK-5580: - Affects Version/s: 1.2.0 Grep bug in compute-classpath.sh

[jira] [Updated] (SPARK-5580) Grep bug in compute-classpath.sh

2015-02-03 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi updated SPARK-5580: - Fix Version/s: 1.3.0 Grep bug in compute-classpath.sh

[jira] [Comment Edited] (SPARK-5529) Executor is still hold while BlockManager has been removed

2015-02-03 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304509#comment-14304509 ] Lianhui Wang edited comment on SPARK-5529 at 2/4/15 2:27 AM: -

[jira] [Closed] (SPARK-5580) Grep bug in compute-classpath.sh

2015-02-03 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yadong Qi closed SPARK-5580. Resolution: Fixed Grep bug in compute-classpath.sh

[jira] [Commented] (SPARK-5260) Expose JsonRDD.allKeysWithValueTypes() in a utility class

2015-02-03 Thread Corey J. Nolet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304577#comment-14304577 ] Corey J. Nolet commented on SPARK-5260: --- I'm thinking all the schema-specific

[jira] [Commented] (SPARK-5367) support star expression in udf

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304611#comment-14304611 ] Apache Spark commented on SPARK-5367: - User 'scwf' has created a pull request for this

[jira] [Closed] (SPARK-5583) Support unique join in hive context

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-5583. -- Resolution: Won't Fix Going to close this one as won't fix since it is a weird syntax that only Hive

[jira] [Updated] (SPARK-5341) Support maven coordinates in spark-shell and spark-submit

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5341: --- Assignee: Burak Yavuz Support maven coordinates in spark-shell and spark-submit

[jira] [Resolved] (SPARK-5341) Support maven coordinates in spark-shell and spark-submit

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5341. Resolution: Fixed Fix Version/s: 1.3.0 Support maven coordinates in spark-shell and

[jira] [Updated] (SPARK-5586) Automatically provide sqlContext in Spark shell

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5586: --- Priority: Critical (was: Major) Automatically provide sqlContext in Spark shell

[jira] [Created] (SPARK-5586) Automatically provide sqlContext in Spark shell

2015-02-03 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-5586: -- Summary: Automatically provide sqlContext in Spark shell Key: SPARK-5586 URL: https://issues.apache.org/jira/browse/SPARK-5586 Project: Spark Issue

[jira] [Updated] (SPARK-5586) Automatically provide sqlContext in Spark shell

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5586: --- Fix Version/s: (was: 1.3.0) Automatically provide sqlContext in Spark shell

[jira] [Commented] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304724#comment-14304724 ] Davies Liu commented on SPARK-5585: --- [~pwendell] I can not reproduce it locally, will

[jira] [Commented] (SPARK-5068) When the path not found in the hdfs,we can't get the result

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304710#comment-14304710 ] Apache Spark commented on SPARK-5068: - User 'chenghao-intel' has created a pull

[jira] [Created] (SPARK-5587) Support change database owner

2015-02-03 Thread wangfei (JIRA)
wangfei created SPARK-5587: -- Summary: Support change database owner Key: SPARK-5587 URL: https://issues.apache.org/jira/browse/SPARK-5587 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-5587) Support change database owner

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304721#comment-14304721 ] Apache Spark commented on SPARK-5587: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-5585) Flaky test: Python regression

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304729#comment-14304729 ] Apache Spark commented on SPARK-5585: - User 'davies' has created a pull request for

[jira] [Updated] (SPARK-5558) pySpark zip function unexpected errors

2015-02-03 Thread Charles Hayden (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Hayden updated SPARK-5558: -- Description: Example: {quote} x = sc.parallelize(range(0,5)) y = x.map(lambda x: x+1000,

[jira] [Commented] (SPARK-3203) ClassNotFoundException in spark-shell with Cassandra

2015-02-03 Thread Helena Edelson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303215#comment-14303215 ] Helena Edelson commented on SPARK-3203: --- Are you both using the spark shell and open

[jira] [Updated] (SPARK-5558) pySpark zip function unexpected errors

2015-02-03 Thread Charles Hayden (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Hayden updated SPARK-5558: -- Description: Example: x = sc.parallelize(range(0,5)) y = x.map(lambda x: x+1000,

[jira] [Commented] (SPARK-4986) Graceful shutdown for Spark Streaming does not work in Standalone cluster mode

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303358#comment-14303358 ] Apache Spark commented on SPARK-4986: - User 'cleaton' has created a pull request for

[jira] [Resolved] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2015-02-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-1405. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 4047

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

2015-02-03 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302932#comment-14302932 ] Debasish Das commented on SPARK-2426: - [~mengxr] [~coderxiang] David is out in Feb and

[jira] [Created] (SPARK-5553) Reimplement SQL binary type with more efficient data structure

2015-02-03 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-5553: - Summary: Reimplement SQL binary type with more efficient data structure Key: SPARK-5553 URL: https://issues.apache.org/jira/browse/SPARK-5553 Project: Spark

[jira] [Commented] (SPARK-5013) User guide for Gaussian Mixture Model

2015-02-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302942#comment-14302942 ] Xiangrui Meng commented on SPARK-5013: -- The Python API was merged. We can add

[jira] [Resolved] (SPARK-5551) Create type alias for SchemaRDD for source backward compatibility

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5551. Resolution: Fixed Fix Version/s: 1.3.0 Create type alias for SchemaRDD for source backward

[jira] [Updated] (SPARK-5345) Fix unstable test case in FsHistoryProviderSuite

2015-02-03 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-5345: -- Labels: flaky-test (was: ) Fix unstable test case in FsHistoryProviderSuite

[jira] [Resolved] (SPARK-5549) Define TaskContext interface in Scala

2015-02-03 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5549. Resolution: Fixed Fix Version/s: 1.3.0 Define TaskContext interface in Scala

[jira] [Updated] (SPARK-2004) Automate QA of Spark Build/Deploy Matrix

2015-02-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2004: - Assignee: Nicholas Chammas Automate QA of Spark Build/Deploy Matrix

[jira] [Commented] (SPARK-2004) Automate QA of Spark Build/Deploy Matrix

2015-02-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302972#comment-14302972 ] Xiangrui Meng commented on SPARK-2004: -- Done. Automate QA of Spark Build/Deploy

[jira] [Updated] (SPARK-2004) Automate QA of Spark Build/Deploy Matrix

2015-02-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2004: - Target Version/s: 1.4.0 (was: 1.1.0) Automate QA of Spark Build/Deploy Matrix

[jira] [Updated] (SPARK-2004) Automate QA of Spark Build/Deploy Matrix

2015-02-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2004: - Summary: Automate QA of Spark Build/Deploy Matrix (was: QA Automation) Automate QA of Spark

[jira] [Created] (SPARK-5554) Add more tests and docs for DataFrame Python API

2015-02-03 Thread Davies Liu (JIRA)
Davies Liu created SPARK-5554: - Summary: Add more tests and docs for DataFrame Python API Key: SPARK-5554 URL: https://issues.apache.org/jira/browse/SPARK-5554 Project: Spark Issue Type:

[jira] [Commented] (SPARK-5554) Add more tests and docs for DataFrame Python API

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302928#comment-14302928 ] Apache Spark commented on SPARK-5554: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2015-02-03 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302952#comment-14302952 ] yuhao yang commented on SPARK-1405: --- Hi everyone, I'm sharing an implementation of

[jira] [Comment Edited] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2015-02-03 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302952#comment-14302952 ] yuhao yang edited comment on SPARK-1405 at 2/3/15 8:35 AM: --- Hi

[jira] [Commented] (SPARK-2945) Allow specifying num of executors in the context configuration

2015-02-03 Thread WangTaoTheTonic (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303092#comment-14303092 ] WangTaoTheTonic commented on SPARK-2945: I also tested on master and branch 1.2,

[jira] [Updated] (SPARK-5557) spark-shell failed to start

2015-02-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-5557: --- Component/s: Spark Core spark-shell failed to start ---

[jira] [Updated] (SPARK-5557) spark-shell failed to start

2015-02-03 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-5557: --- Affects Version/s: 1.3.0 spark-shell failed to start ---

[jira] [Commented] (SPARK-5475) Java 8 tests are like maintenance overhead.

2015-02-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303100#comment-14303100 ] Sean Owen commented on SPARK-5475: -- Running those commands gives me an error about not

[jira] [Commented] (SPARK-5021) GaussianMixtureEM should be faster for SparseVector input

2015-02-03 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304099#comment-14304099 ] Manoj Kumar commented on SPARK-5021: Hi. I'm almost there. I have one last question.

[jira] [Comment Edited] (SPARK-5021) GaussianMixtureEM should be faster for SparseVector input

2015-02-03 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304099#comment-14304099 ] Manoj Kumar edited comment on SPARK-5021 at 2/3/15 10:01 PM: -

[jira] [Comment Edited] (SPARK-5021) GaussianMixtureEM should be faster for SparseVector input

2015-02-03 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304099#comment-14304099 ] Manoj Kumar edited comment on SPARK-5021 at 2/3/15 10:02 PM: -

[jira] [Created] (SPARK-5573) Support explode in DataFrame DSL

2015-02-03 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-5573: -- Summary: Support explode in DataFrame DSL Key: SPARK-5573 URL: https://issues.apache.org/jira/browse/SPARK-5573 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-5140) Two RDDs which are scheduled concurrently should be able to wait on parent in all cases

2015-02-03 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5140: --- Fix Version/s: (was: 1.2.1) (was: 1.3.0) Two RDDs which are

[jira] [Commented] (SPARK-5140) Two RDDs which are scheduled concurrently should be able to wait on parent in all cases

2015-02-03 Thread Corey J. Nolet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303548#comment-14303548 ] Corey J. Nolet commented on SPARK-5140: --- Is anyone against this behavior for any

[jira] [Updated] (SPARK-5558) pySpark zip function unexpected errors

2015-02-03 Thread Charles Hayden (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Hayden updated SPARK-5558: -- Description: Example: {quote} x = sc.parallelize(range(0,5)) y = x.map(lambda x: x+1000,

[jira] [Commented] (SPARK-5520) Make FP-Growth implementation take generic item types

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303613#comment-14303613 ] Apache Spark commented on SPARK-5520: - User 'jackylk' has created a pull request for

[jira] [Commented] (SPARK-5013) User guide for Gaussian Mixture Model

2015-02-03 Thread Travis Galoppo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303522#comment-14303522 ] Travis Galoppo commented on SPARK-5013: --- Great! I will submit a PR soon. User

[jira] [Updated] (SPARK-5559) Remove oppotunity we met flakiness when running FlumeStreamSuite

2015-02-03 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-5559: -- Labels: flaky-test (was: ) Remove oppotunity we met flakiness when running FlumeStreamSuite

[jira] [Commented] (SPARK-4897) Python 3 support

2015-02-03 Thread Ian Ozsvald (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303154#comment-14303154 ] Ian Ozsvald commented on SPARK-4897: If I can cast a vote... I note that Python 2.6

[jira] [Created] (SPARK-5558) pySpark zip function unexpected errors

2015-02-03 Thread Charles Hayden (JIRA)
Charles Hayden created SPARK-5558: - Summary: pySpark zip function unexpected errors Key: SPARK-5558 URL: https://issues.apache.org/jira/browse/SPARK-5558 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5557) spark-shell failed to start

2015-02-03 Thread Nathan McCarthy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303332#comment-14303332 ] Nathan McCarthy commented on SPARK-5557: I am seeing similar when trying to launch

[jira] [Commented] (SPARK-5548) Flaky test: org.apache.spark.util.AkkaUtilsSuite.remote fetch ssl on - untrusted server

2015-02-03 Thread Jacek Lewandowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303252#comment-14303252 ] Jacek Lewandowski commented on SPARK-5548: -- I tried also on ubuntu with Java 7 -

[jira] [Updated] (SPARK-5558) pySpark zip function unexpected errors

2015-02-03 Thread Charles Hayden (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Hayden updated SPARK-5558: -- Description: Example: {{x = sc.parallelize(range(0,5)) y = x.map(lambda x: x+1000,

[jira] [Updated] (SPARK-5558) pySpark zip function unexpected errors

2015-02-03 Thread Charles Hayden (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Hayden updated SPARK-5558: -- Description: Example: {{quote}} x = sc.parallelize(range(0,5)) y = x.map(lambda x: x+1000,

[jira] [Created] (SPARK-5559) Remove oppotunity we met flakiness when running FlumeStreamSuite

2015-02-03 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-5559: - Summary: Remove oppotunity we met flakiness when running FlumeStreamSuite Key: SPARK-5559 URL: https://issues.apache.org/jira/browse/SPARK-5559 Project: Spark

[jira] [Updated] (SPARK-5559) Remove oppotunity we met flakiness when running FlumeStreamSuite

2015-02-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-5559: -- Description: When we run FlumeStreamSuite on Jenkins, sometimes we get error like as follows.

[jira] [Commented] (SPARK-5559) Remove oppotunity we met flakiness when running FlumeStreamSuite

2015-02-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303304#comment-14303304 ] Apache Spark commented on SPARK-5559: - User 'sarutak' has created a pull request for

[jira] [Resolved] (SPARK-5547) With assembly jar to run example throws an exception

2015-02-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5547. -- Resolution: Duplicate This is a real problem I think and almost certainly the same as SPARK-5557 With

[jira] [Commented] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2015-02-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303737#comment-14303737 ] Joseph K. Bradley commented on SPARK-5556: -- I believe [~mengxr] and [~witgo] have

[jira] [Created] (SPARK-5562) LDA should handle empty documents

2015-02-03 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-5562: Summary: LDA should handle empty documents Key: SPARK-5562 URL: https://issues.apache.org/jira/browse/SPARK-5562 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-5563) LDA with online variational inference

2015-02-03 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5563: - Issue Type: Improvement (was: Test) LDA with online variational inference

[jira] [Created] (SPARK-5563) LDA with online variational inference

2015-02-03 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-5563: Summary: LDA with online variational inference Key: SPARK-5563 URL: https://issues.apache.org/jira/browse/SPARK-5563 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-5388) Provide a stable application submission gateway in standalone cluster mode

2015-02-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303702#comment-14303702 ] Marcelo Vanzin commented on SPARK-5388: --- HI [~pwendell], Let me try to write a

[jira] [Created] (SPARK-5561) Generalize PeriodicGraphCheckpointer for RDDs

2015-02-03 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-5561: Summary: Generalize PeriodicGraphCheckpointer for RDDs Key: SPARK-5561 URL: https://issues.apache.org/jira/browse/SPARK-5561 Project: Spark Issue

[jira] [Commented] (SPARK-5548) Flaky test: org.apache.spark.util.AkkaUtilsSuite.remote fetch ssl on - untrusted server

2015-02-03 Thread Jacek Lewandowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303648#comment-14303648 ] Jacek Lewandowski commented on SPARK-5548: -- Java 6 test passed... except kinesis

[jira] [Commented] (SPARK-5501) Write support for the data source API

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303722#comment-14303722 ] Yin Huai commented on SPARK-5501: - h3. Interfaces introduced to the data source API The PR

[jira] [Commented] (SPARK-4705) Driver retries in yarn-cluster mode always fail if event logging is enabled

2015-02-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303742#comment-14303742 ] Marcelo Vanzin commented on SPARK-4705: --- Hi [~twinkle], a few comments. I'm not

[jira] [Comment Edited] (SPARK-5501) Write support for the data source API

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303722#comment-14303722 ] Yin Huai edited comment on SPARK-5501 at 2/3/15 6:43 PM: - h3.

[jira] [Created] (SPARK-5564) Support sparse LDA solutions

2015-02-03 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-5564: Summary: Support sparse LDA solutions Key: SPARK-5564 URL: https://issues.apache.org/jira/browse/SPARK-5564 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-5565) LDA wrapper for spark.ml package

2015-02-03 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-5565: Summary: LDA wrapper for spark.ml package Key: SPARK-5565 URL: https://issues.apache.org/jira/browse/SPARK-5565 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

2015-02-03 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-5420. - Resolution: Fixed This JIRA is fixed by the attached PR. A summary of added interfaces can be found in

  1   2   >