[jira] [Commented] (SPARK-6320) Adding new query plan strategy to SQLContext

2015-03-17 Thread Youssef Hatem (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365274#comment-14365274 ] Youssef Hatem commented on SPARK-6320: -- Thank you Michael for your response,

[jira] [Commented] (SPARK-6250) Types are now reserved words in DDL parser.

2015-03-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365307#comment-14365307 ] Yin Huai commented on SPARK-6250: - I see. The problem is that the data type string

[jira] [Created] (SPARK-6394) cleanup BlockManager companion object and improve the getCacheLocs method in DAGScheduler

2015-03-17 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-6394: -- Summary: cleanup BlockManager companion object and improve the getCacheLocs method in DAGScheduler Key: SPARK-6394 URL: https://issues.apache.org/jira/browse/SPARK-6394

[jira] [Commented] (SPARK-6200) Support dialect in SQL

2015-03-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366575#comment-14366575 ] Michael Armbrust commented on SPARK-6200: - Thanks for taking the time to explain

[jira] [Commented] (SPARK-6394) cleanup BlockManager companion object and improve the getCacheLocs method in DAGScheduler

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366622#comment-14366622 ] Apache Spark commented on SPARK-6394: - User 'cloud-fan' has created a pull request for

[jira] [Updated] (SPARK-6145) ORDER BY fails to resolve nested fields

2015-03-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6145: Target Version/s: 1.3.1 (was: 1.3.0) ORDER BY fails to resolve nested fields

[jira] [Resolved] (SPARK-6299) ClassNotFoundException in standalone mode when running groupByKey with class defined in REPL.

2015-03-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6299. Resolution: Fixed Fix Version/s: 1.3.1 1.4.0 Assignee: Kevin

[jira] [Updated] (SPARK-6299) ClassNotFoundException in standalone mode when running groupByKey with class defined in REPL.

2015-03-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6299: --- Description: Anyone can reproduce this issue by the code below (runs well in local mode, got

[jira] [Resolved] (SPARK-6044) RDD.aggregate() should not use the closure serializer on the zero value

2015-03-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6044. Resolution: Fixed Assignee: Sean Owen RDD.aggregate() should not use the closure serializer

[jira] [Commented] (SPARK-6085) Increase default value for memory overhead

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364740#comment-14364740 ] Apache Spark commented on SPARK-6085: - User 'jongyoul' has created a pull request for

[jira] [Commented] (SPARK-6334) spark-local dir not getting cleared during ALS

2015-03-17 Thread Antony Mayi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364764#comment-14364764 ] Antony Mayi commented on SPARK-6334: users: 12.5 millions ratings: 3.3 billions rank:

[jira] [Commented] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

2015-03-17 Thread Marko Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364700#comment-14364700 ] Marko Bonaci commented on SPARK-6370: - Isn't the {{fraction}} parameter the size of

[jira] [Commented] (SPARK-5954) Add topByKey to pair RDDs

2015-03-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364703#comment-14364703 ] Reynold Xin commented on SPARK-5954: Shouldn't this just be an API in dataframe? Add

[jira] [Comment Edited] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

2015-03-17 Thread Marko Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364700#comment-14364700 ] Marko Bonaci edited comment on SPARK-6370 at 3/17/15 7:17 AM: --

[jira] [Updated] (SPARK-6380) Resolution of equi-join key in post-join projection

2015-03-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6380: --- Description: {code} df1.join(df2, df1(key) === df2(key)).select(key) {code} It would be great to

[jira] [Commented] (SPARK-6281) Support incremental updates for Graph

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364787#comment-14364787 ] Apache Spark commented on SPARK-6281: - User 'maropu' has created a pull request for

[jira] [Created] (SPARK-6380) Resolution of equi-join key in post-join projection

2015-03-17 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-6380: -- Summary: Resolution of equi-join key in post-join projection Key: SPARK-6380 URL: https://issues.apache.org/jira/browse/SPARK-6380 Project: Spark Issue Type:

[jira] [Updated] (SPARK-6350) Make mesosExecutorCores configurable in mesos fine-grained mode

2015-03-17 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-6350: Fix Version/s: 1.3.1 1.4.0 Make mesosExecutorCores configurable in mesos

[jira] [Commented] (SPARK-6350) Make mesosExecutorCores configurable in mesos fine-grained mode

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364661#comment-14364661 ] Apache Spark commented on SPARK-6350: - User 'jongyoul' has created a pull request for

[jira] [Comment Edited] (SPARK-6304) Checkpointing doesn't retain driver port

2015-03-17 Thread Marius Soutier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364713#comment-14364713 ] Marius Soutier edited comment on SPARK-6304 at 3/17/15 7:36 AM:

[jira] [Commented] (SPARK-6304) Checkpointing doesn't retain driver port

2015-03-17 Thread Marius Soutier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364713#comment-14364713 ] Marius Soutier commented on SPARK-6304: --- Got it, thanks. Checkpointing doesn't

[jira] [Commented] (SPARK-5523) TaskMetrics and TaskInfo have innumerable copies of the hostname string

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364712#comment-14364712 ] Apache Spark commented on SPARK-5523: - User 'jerryshao' has created a pull request for

[jira] [Updated] (SPARK-6390) Add MatrixUDT in PySpark

2015-03-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6390: - Component/s: PySpark Add MatrixUDT in PySpark Key:

[jira] [Created] (SPARK-6390) Add MatrixUDT in PySpark

2015-03-17 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-6390: Summary: Add MatrixUDT in PySpark Key: SPARK-6390 URL: https://issues.apache.org/jira/browse/SPARK-6390 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-5836) Highlight in Spark documentation that by default Spark does not delete its temporary files

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365897#comment-14365897 ] Apache Spark commented on SPARK-5836: - User 'ilganeli' has created a pull request for

[jira] [Commented] (SPARK-6192) Enhance MLlib's Python API (GSoC 2015)

2015-03-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365918#comment-14365918 ] Xiangrui Meng commented on SPARK-6192: -- [~MechCoder] Please be a little (but not too)

[jira] [Issue Comment Deleted] (SPARK-6116) Making DataFrame API non-experimental

2015-03-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6116: --- Comment: was deleted (was: User 'azagrebin' has created a pull request for this issue:

[jira] [Updated] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

2015-03-17 Thread Marko Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marko Bonaci updated SPARK-6370: Priority: Minor (was: Major) Issue Type: Documentation (was: Bug) RDD sampling with

[jira] [Commented] (SPARK-5954) Add topByKey to pair RDDs

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366081#comment-14366081 ] Apache Spark commented on SPARK-5954: - User 'coderxiang' has created a pull request

[jira] [Commented] (SPARK-5955) Add checkpointInterval to ALS

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366127#comment-14366127 ] Apache Spark commented on SPARK-5955: - User 'mengxr' has created a pull request for

[jira] [Updated] (SPARK-6382) withUDF(...) {...} for supporting temporary UDF definitions in the scope

2015-03-17 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianshi Huang updated SPARK-6382: - Description: Currently the scope of UDF registration is global. It's unsuitable for libraries

[jira] [Updated] (SPARK-6382) withUDF(...) {...} for supporting temporary UDF definitions in the scope

2015-03-17 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianshi Huang updated SPARK-6382: - Description: Currently the scope of UDF registration is global. It's unsuitable for libraries

[jira] [Updated] (SPARK-6381) add Apriori algorithm to MLLib

2015-03-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6381: - Target Version/s: (was: 1.4.0) Affects Version/s: (was: 1.3.1) add Apriori algorithm to MLLib

[jira] [Updated] (SPARK-6381) add Apriori algorithm to MLLib

2015-03-17 Thread zhangyouhua (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangyouhua updated SPARK-6381: --- Target Version/s: 1.3.1 Affects Version/s: 1.3.0 Fix Version/s: 1.3.1 add Apriori

[jira] [Commented] (SPARK-6381) add Apriori algorithm to MLLib

2015-03-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364847#comment-14364847 ] Sean Owen commented on SPARK-6381: -- See SPARK-2432. I don't think apriori is as useful or

[jira] [Created] (SPARK-6385) ISO 8601 timestamp parsing does not support arbitrary precision second fractions

2015-03-17 Thread Nick Bruun (JIRA)
Nick Bruun created SPARK-6385: - Summary: ISO 8601 timestamp parsing does not support arbitrary precision second fractions Key: SPARK-6385 URL: https://issues.apache.org/jira/browse/SPARK-6385 Project:

[jira] [Commented] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

2015-03-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364859#comment-14364859 ] Sean Owen commented on SPARK-6370: -- Ah. The docs don't explain this behavior indeed.

[jira] [Commented] (SPARK-4590) Early investigation of parameter server

2015-03-17 Thread Qiping Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364803#comment-14364803 ] Qiping Li commented on SPARK-4590: -- Hi, [~rezazadeh], thanks for your investigation. It

[jira] [Resolved] (SPARK-6381) add Apriori algorithm to MLLib

2015-03-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6381. -- Resolution: Duplicate Fix Version/s: (was: 1.4.0) (Don't set fix version, and 1.3.1 does not

[jira] [Commented] (SPARK-6383) Few examples on Dataframe operation give compiler errors

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364825#comment-14364825 ] Apache Spark commented on SPARK-6383: - User 'tijoparacka' has created a pull request

[jira] [Created] (SPARK-6384) saveAsParquet doesn't clean up attempt_* folders

2015-03-17 Thread Rex Xiong (JIRA)
Rex Xiong created SPARK-6384: Summary: saveAsParquet doesn't clean up attempt_* folders Key: SPARK-6384 URL: https://issues.apache.org/jira/browse/SPARK-6384 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-6384) saveAsParquet doesn't clean up attempt_* folders

2015-03-17 Thread Rex Xiong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rex Xiong updated SPARK-6384: - Description: After calling SchemaRDD.saveAsParquet, it runs well and generate *.parquet, _SUCCESS,

[jira] [Created] (SPARK-6382) withUDF(...) {...} for supporting temporary UDF definitions in the scope

2015-03-17 Thread Jianshi Huang (JIRA)
Jianshi Huang created SPARK-6382: Summary: withUDF(...) {...} for supporting temporary UDF definitions in the scope Key: SPARK-6382 URL: https://issues.apache.org/jira/browse/SPARK-6382 Project:

[jira] [Updated] (SPARK-6381) add Apriori algorithm to MLLib

2015-03-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6381: - Target Version/s: (was: 1.3.1) Fix Version/s: (was: 1.3.1) @zhangyouhua please do not set

[jira] [Issue Comment Deleted] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

2015-03-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6370: - Comment: was deleted (was: What's the bug? Each element is sampled with probability 0.5. I think the

[jira] [Commented] (SPARK-6377) Set the number of shuffle partitions automatically based on the size of input tables and the reduce-side operation.

2015-03-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364901#comment-14364901 ] Sean Owen commented on SPARK-6377: -- I think this is the same basic suggestion as

[jira] [Created] (SPARK-6381) add Apriori algorithm to MLLib

2015-03-17 Thread zhangyouhua (JIRA)
zhangyouhua created SPARK-6381: -- Summary: add Apriori algorithm to MLLib Key: SPARK-6381 URL: https://issues.apache.org/jira/browse/SPARK-6381 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-6383) Few examples on Dataframe operation give compiler errors

2015-03-17 Thread Tijo Thomas (JIRA)
Tijo Thomas created SPARK-6383: -- Summary: Few examples on Dataframe operation give compiler errors Key: SPARK-6383 URL: https://issues.apache.org/jira/browse/SPARK-6383 Project: Spark Issue

[jira] [Comment Edited] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

2015-03-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364859#comment-14364859 ] Sean Owen edited comment on SPARK-6370 at 3/17/15 10:09 AM:

[jira] [Updated] (SPARK-6384) saveAsParquet doesn't clean up attempt_* folders

2015-03-17 Thread Rex Xiong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rex Xiong updated SPARK-6384: - Affects Version/s: 1.2.1 saveAsParquet doesn't clean up attempt_* folders

[jira] [Commented] (SPARK-6349) Add probability estimates in SVMModel predict result

2015-03-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364888#comment-14364888 ] Sean Owen commented on SPARK-6349: -- One is just a monotonic function of the other. It

[jira] [Commented] (SPARK-5081) Shuffle write increases

2015-03-17 Thread Dr. Christian Betz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364813#comment-14364813 ] Dr. Christian Betz commented on SPARK-5081: --- [~pwendell] Hi, obviously there's

[jira] [Resolved] (SPARK-5332) Efficient way to deal with ExecutorLost

2015-03-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5332. -- Resolution: Won't Fix Efficient way to deal with ExecutorLost ---

[jira] [Created] (SPARK-6386) add association rule mining algorithm to MLLib

2015-03-17 Thread zhangyouhua (JIRA)
zhangyouhua created SPARK-6386: -- Summary: add association rule mining algorithm to MLLib Key: SPARK-6386 URL: https://issues.apache.org/jira/browse/SPARK-6386 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-6117) describe function for summary statistics

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365781#comment-14365781 ] Apache Spark commented on SPARK-6117: - User 'azagrebin' has created a pull request for

[jira] [Updated] (SPARK-6336) LBFGS should document what convergenceTol means

2015-03-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6336: - Target Version/s: 1.4.0, 1.3.1 (was: 1.4.0) LBFGS should document what convergenceTol means

[jira] [Commented] (SPARK-6388) Spark 1.3 + Hadoop 2.6 Can't work on Java 8_40

2015-03-17 Thread Michael Malak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365758#comment-14365758 ] Michael Malak commented on SPARK-6388: -- Isn't it Hadoop 2.7 that is supposed to

[jira] [Closed] (SPARK-6358) Spark-submit error when using PYSPARK_PYTHON enviromnental variable

2015-03-17 Thread dustin davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dustin davidson closed SPARK-6358. -- Resolution: Invalid Found a work around. The issue was with group permissions on the '/home'

[jira] [Commented] (SPARK-6250) Types are now reserved words in DDL parser.

2015-03-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365735#comment-14365735 ] Yin Huai commented on SPARK-6250: - Update: Seems this problem affects struct fields when a

[jira] [Commented] (SPARK-5313) Create simple framework for highlighting changes introduced in a PR

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365561#comment-14365561 ] Apache Spark commented on SPARK-5313: - User 'brennonyork' has created a pull request

[jira] [Commented] (SPARK-6363) make scala 2.11 default language

2015-03-17 Thread Jianshi Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365726#comment-14365726 ] Jianshi Huang commented on SPARK-6363: -- My two cents. The only module that's not

[jira] [Closed] (SPARK-6204) GenerateProjection's equals should check length equality

2015-03-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust closed SPARK-6204. --- Resolution: Won't Fix GenerateProjection's equals should check length equality

[jira] [Created] (SPARK-6392) [SQL]class not found exception thows when `add jar` use spark cli

2015-03-17 Thread jeanlyn (JIRA)
jeanlyn created SPARK-6392: -- Summary: [SQL]class not found exception thows when `add jar` use spark cli Key: SPARK-6392 URL: https://issues.apache.org/jira/browse/SPARK-6392 Project: Spark Issue

[jira] [Resolved] (SPARK-5651) Support 'create db.table' in HiveContext

2015-03-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5651. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4427

[jira] [Updated] (SPARK-6366) In Python API, the default save mode for save and saveAsTable should be error instead of append.

2015-03-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6366: -- Fix Version/s: (was: 1.3.0) 1.3.1 In Python API, the default save mode for save

[jira] [Resolved] (SPARK-5404) Statistic of Logical Plan is too aggresive

2015-03-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5404. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4914

[jira] [Resolved] (SPARK-6247) Certain self joins cannot be analyzed

2015-03-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6247. - Resolution: Fixed Fix Version/s: 1.3.1 Issue resolved by pull request 5062

[jira] [Commented] (SPARK-6392) [SQL]class not found exception thows when `add jar` use spark cli

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366506#comment-14366506 ] Apache Spark commented on SPARK-6392: - User 'jeanlyn' has created a pull request for

[jira] [Created] (SPARK-6393) Extra RPC to the AM during killExecutor invocation

2015-03-17 Thread Sandy Ryza (JIRA)
Sandy Ryza created SPARK-6393: - Summary: Extra RPC to the AM during killExecutor invocation Key: SPARK-6393 URL: https://issues.apache.org/jira/browse/SPARK-6393 Project: Spark Issue Type:

[jira] [Commented] (SPARK-6200) Support dialect in SQL

2015-03-17 Thread haiyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366569#comment-14366569 ] haiyang commented on SPARK-6200: Thank you for your comments. The core idea of this

[jira] [Resolved] (SPARK-6366) In Python API, the default save mode for save and saveAsTable should be error instead of append.

2015-03-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6366. --- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 5053

[jira] [Resolved] (SPARK-6383) Few examples on Dataframe operation give compiler errors

2015-03-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6383. Resolution: Fixed Fix Version/s: 1.3.0 Few examples on Dataframe operation give compiler

[jira] [Updated] (SPARK-6383) Few examples on Dataframe operation give compiler errors

2015-03-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6383: --- Fix Version/s: (was: 1.3.0) 1.3.1 1.4.0 Few examples on

[jira] [Updated] (SPARK-6383) Few examples on Dataframe operation give compiler errors

2015-03-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6383: --- Labels: DataFrame (was: ) Few examples on Dataframe operation give compiler errors

[jira] [Resolved] (SPARK-5908) Hive udtf with single alias should be resolved correctly

2015-03-17 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5908. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4692

[jira] [Commented] (SPARK-6356) Support the ROLLUP/CUBE/GROUPING SETS/grouping() in SQLContext

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366551#comment-14366551 ] Apache Spark commented on SPARK-6356: - User 'watermen' has created a pull request for

[jira] [Commented] (SPARK-6330) newParquetRelation gets incorrect FileSystem

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366394#comment-14366394 ] Apache Spark commented on SPARK-6330: - User 'ypcat' has created a pull request for

[jira] [Commented] (SPARK-5654) Integrate SparkR into Apache Spark

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366237#comment-14366237 ] Apache Spark commented on SPARK-5654: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-6250) Types are now reserved words in DDL parser.

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366269#comment-14366269 ] Apache Spark commented on SPARK-6250: - User 'yhuai' has created a pull request for

[jira] [Commented] (SPARK-6250) Types are now reserved words in DDL parser.

2015-03-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366284#comment-14366284 ] Yin Huai commented on SPARK-6250: - [~nitay] Can you try

[jira] [Updated] (SPARK-6248) LocalRelation needs to implement statistics

2015-03-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6248: Target Version/s: 1.3.1 (was: 1.4.0) LocalRelation needs to implement statistics

[jira] [Commented] (SPARK-6248) LocalRelation needs to implement statistics

2015-03-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366333#comment-14366333 ] Yin Huai commented on SPARK-6248: - https://github.com/apache/spark/pull/5062 will also fix

[jira] [Commented] (SPARK-6154) Support Kafka, JDBC in Scala 2.11

2015-03-17 Thread Olivier Toupin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366295#comment-14366295 ] Olivier Toupin commented on SPARK-6154: --- I made an unperfect fix for

[jira] [Updated] (SPARK-6248) LocalRelation needs to implement statistics

2015-03-17 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6248: Description: We need to implement statistics for LocalRelation. Otherwise, we cannot join a LocalRelation

[jira] [Created] (SPARK-6391) Update Tachyon version compatibility documentation

2015-03-17 Thread Calvin Jia (JIRA)
Calvin Jia created SPARK-6391: - Summary: Update Tachyon version compatibility documentation Key: SPARK-6391 URL: https://issues.apache.org/jira/browse/SPARK-6391 Project: Spark Issue Type:

[jira] [Updated] (SPARK-6391) Update Tachyon version compatibility documentation

2015-03-17 Thread Haoyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haoyuan Li updated SPARK-6391: -- Component/s: (was: Spark Core) Documentation Update Tachyon version compatibility

[jira] [Updated] (SPARK-6391) Update Tachyon version compatibility documentation

2015-03-17 Thread Haoyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haoyuan Li updated SPARK-6391: -- Fix Version/s: 1.4.0 Update Tachyon version compatibility documentation

[jira] [Updated] (SPARK-6391) Update Tachyon version compatibility documentation

2015-03-17 Thread Haoyuan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haoyuan Li updated SPARK-6391: -- Affects Version/s: 1.3.0 Update Tachyon version compatibility documentation

[jira] [Commented] (SPARK-6334) spark-local dir not getting cleared during ALS

2015-03-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366021#comment-14366021 ] Xiangrui Meng commented on SPARK-6334: -- Couple suggestions before SPARK-5955 is

[jira] [Commented] (SPARK-4590) Early investigation of parameter server

2015-03-17 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365581#comment-14365581 ] Reza Zadeh commented on SPARK-4590: --- Hi Qiping, We are waiting for IndexedRDD to be

[jira] [Closed] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

2015-03-17 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-3351. Resolution: Invalid Yarn YarnRMClientImpl.shutdown can be called before register - NPE

[jira] [Commented] (SPARK-3351) Yarn YarnRMClientImpl.shutdown can be called before register - NPE

2015-03-17 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365607#comment-14365607 ] Thomas Graves commented on SPARK-3351: -- ok lets close this and reopen if we see it

[jira] [Updated] (SPARK-6336) LBFGS should document what convergenceTol means

2015-03-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6336: - Fix Version/s: (was: 1.3.0) 1.3.1 1.4.0 LBFGS should

[jira] [Resolved] (SPARK-6336) LBFGS should document what convergenceTol means

2015-03-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6336. -- Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 5033

[jira] [Updated] (SPARK-6336) LBFGS should document what convergenceTol means

2015-03-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6336: - Assignee: Kai Sasaki LBFGS should document what convergenceTol means

[jira] [Commented] (SPARK-4416) Support Mesos framework authentication

2015-03-17 Thread Adam B (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366011#comment-14366011 ] Adam B commented on SPARK-4416: --- Can close as duplicate of SPARK-6284 Support Mesos

[jira] [Assigned] (SPARK-5955) Add checkpointInterval to ALS

2015-03-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-5955: Assignee: Xiangrui Meng Add checkpointInterval to ALS -

[jira] [Commented] (SPARK-5750) Document that ordering of elements in shuffled partitions is not deterministic across runs

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365895#comment-14365895 ] Apache Spark commented on SPARK-5750: - User 'ilganeli' has created a pull request for

[jira] [Resolved] (SPARK-6226) Support model save/load in Python's KMeans

2015-03-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6226. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5049

[jira] [Commented] (SPARK-3441) Explain in docs that repartitionAndSortWithinPartitions enacts Hadoop style shuffle

2015-03-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365896#comment-14365896 ] Apache Spark commented on SPARK-3441: - User 'ilganeli' has created a pull request for

  1   2   >