[jira] [Commented] (SPARK-14503) spark.ml API for FPGrowth

2016-07-10 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370262#comment-15370262 ] yuhao yang commented on SPARK-14503: Link two related issue about adding "support" to

[jira] [Commented] (SPARK-16318) xpath_int, xpath_short, xpath_long, xpath_float, xpath_double, xpath_string, and xpath

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370203#comment-15370203 ] Apache Spark commented on SPARK-16318: -- User 'petermaxlee' has created a pull reques

[jira] [Resolved] (SPARK-16318) xpath_int, xpath_short, xpath_long, xpath_float, xpath_double, xpath_string, and xpath

2016-07-10 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-16318. - Resolution: Fixed Fix Version/s: 2.1.0 > xpath_int, xpath_short, xpath_long, xpath_float,

[jira] [Updated] (SPARK-16318) xpath_int, xpath_short, xpath_long, xpath_float, xpath_double, xpath_string, and xpath

2016-07-10 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-16318: Assignee: Peter Lee > xpath_int, xpath_short, xpath_long, xpath_float, xpath_double, xpath_string,

[jira] [Assigned] (SPARK-16477) Bump master version to 2.1.0-SNAPSHOT

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16477: Assignee: Apache Spark (was: Reynold Xin) > Bump master version to 2.1.0-SNAPSHOT > -

[jira] [Assigned] (SPARK-16477) Bump master version to 2.1.0-SNAPSHOT

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16477: Assignee: Reynold Xin (was: Apache Spark) > Bump master version to 2.1.0-SNAPSHOT > -

[jira] [Commented] (SPARK-16477) Bump master version to 2.1.0-SNAPSHOT

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370180#comment-15370180 ] Apache Spark commented on SPARK-16477: -- User 'rxin' has created a pull request for t

[jira] [Updated] (SPARK-16477) Bump master version to 2.1.0-SNAPSHOT

2016-07-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16477: Description: This should now be doable with SPARK-16476. > Bump master version to 2.1.0-SNAPSHOT

[jira] [Created] (SPARK-16477) Bump master version to 2.1.0-SNAPSHOT

2016-07-10 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16477: --- Summary: Bump master version to 2.1.0-SNAPSHOT Key: SPARK-16477 URL: https://issues.apache.org/jira/browse/SPARK-16477 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-16476) Restructure MimaExcludes for easier union excludes

2016-07-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-16476. - Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 2.0.0 > Restructure MimaEx

[jira] [Comment Edited] (SPARK-16258) Automatically append the grouping keys in SparkR's gapply

2016-07-10 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370136#comment-15370136 ] Narine Kokhlikyan edited comment on SPARK-16258 at 7/11/16 3:52 AM: ---

[jira] [Comment Edited] (SPARK-16258) Automatically append the grouping keys in SparkR's gapply

2016-07-10 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370136#comment-15370136 ] Narine Kokhlikyan edited comment on SPARK-16258 at 7/11/16 3:53 AM: ---

[jira] [Commented] (SPARK-16258) Automatically append the grouping keys in SparkR's gapply

2016-07-10 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370136#comment-15370136 ] Narine Kokhlikyan commented on SPARK-16258: --- Thanks [~shivaram]! I also vote fo

[jira] [Commented] (SPARK-16370) Union queries should not be executed eagerly

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370129#comment-15370129 ] Dongjoon Hyun commented on SPARK-16370: --- Current PR is not enough and it's not wort

[jira] [Closed] (SPARK-16370) Union queries should not be executed eagerly

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-16370. - Resolution: Won't Fix > Union queries should not be executed eagerly > --

[jira] [Comment Edited] (SPARK-16475) Broadcast Hint for SQL Queries

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370041#comment-15370041 ] Dongjoon Hyun edited comment on SPARK-16475 at 7/11/16 3:04 AM: ---

[jira] [Assigned] (SPARK-16280) Implement histogram_numeric SQL function

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16280: Assignee: (was: Apache Spark) > Implement histogram_numeric SQL function > ---

[jira] [Commented] (SPARK-16280) Implement histogram_numeric SQL function

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370074#comment-15370074 ] Apache Spark commented on SPARK-16280: -- User 'tilumi' has created a pull request for

[jira] [Assigned] (SPARK-16280) Implement histogram_numeric SQL function

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16280: Assignee: Apache Spark > Implement histogram_numeric SQL function > --

[jira] [Commented] (SPARK-16467) After importing R data.frame, although DataFrame columns show . replaced by _, the describe() function gives warnings on . in the name

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370066#comment-15370066 ] Dongjoon Hyun commented on SPARK-16467: --- Thank YOU for reporting! > After importin

[jira] [Updated] (SPARK-16475) Broadcast Hint for SQL Queries

2016-07-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16475: Description: Broadcast hint is a way for users to manually annotate a query and suggest to the que

[jira] [Commented] (SPARK-16467) After importing R data.frame, although DataFrame columns show . replaced by _, the describe() function gives warnings on . in the name

2016-07-10 Thread Neil Dewar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370056#comment-15370056 ] Neil Dewar commented on SPARK-16467: Thank you Sir - user error! > After importing R

[jira] [Commented] (SPARK-16283) Implement percentile_approx SQL function

2016-07-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370051#comment-15370051 ] Reynold Xin commented on SPARK-16283: - [~thunterdb] can we use your implementation fo

[jira] [Commented] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Neil Dewar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370049#comment-15370049 ] Neil Dewar commented on SPARK-16464: Thank you Dongjoon, Let me try to explain a litt

[jira] [Assigned] (SPARK-16476) Restructure MimaExcludes for easier union excludes

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16476: Assignee: Apache Spark > Restructure MimaExcludes for easier union excludes >

[jira] [Commented] (SPARK-16476) Restructure MimaExcludes for easier union excludes

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370048#comment-15370048 ] Apache Spark commented on SPARK-16476: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-16476) Restructure MimaExcludes for easier union excludes

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16476: Assignee: (was: Apache Spark) > Restructure MimaExcludes for easier union excludes > -

[jira] [Created] (SPARK-16476) Restructure MimaExcludes for easier version transition

2016-07-10 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16476: --- Summary: Restructure MimaExcludes for easier version transition Key: SPARK-16476 URL: https://issues.apache.org/jira/browse/SPARK-16476 Project: Spark Issue Ty

[jira] [Updated] (SPARK-16476) Restructure MimaExcludes for easier union excludes

2016-07-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16476: Summary: Restructure MimaExcludes for easier union excludes (was: Restructure MimaExcludes for eas

[jira] [Commented] (SPARK-16475) Broadcast Hint for SQL Queries

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370041#comment-15370041 ] Dongjoon Hyun commented on SPARK-16475: --- Of course. It's not finished yet. I'm work

[jira] [Resolved] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-07-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15467. - Resolution: Fixed Assignee: Kazuaki Ishizaki Fix Version/s: 2.1.0 > Getting stack

[jira] [Commented] (SPARK-16475) Broadcast Hint for SQL Queries

2016-07-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370039#comment-15370039 ] Reynold Xin commented on SPARK-16475: - BTW let's also make sure we finish the informa

[jira] [Commented] (SPARK-16475) Broadcast Hint for SQL Queries

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370037#comment-15370037 ] Dongjoon Hyun commented on SPARK-16475: --- Thank you for important issues! I'll start

[jira] [Commented] (SPARK-16475) Broadcast Hint for SQL Queries

2016-07-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370035#comment-15370035 ] Reynold Xin commented on SPARK-16475: - Yes - we would need to update the parser to su

[jira] [Commented] (SPARK-16475) Broadcast Hint for SQL Queries

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370032#comment-15370032 ] Dongjoon Hyun commented on SPARK-16475: --- Oh, Spark supports `Hint` really? Amazing.

[jira] [Resolved] (SPARK-16467) After importing R data.frame, although DataFrame columns show . replaced by _, the describe() function gives warnings on . in the name

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-16467. --- Resolution: Not A Problem > After importing R data.frame, although DataFrame columns show . r

[jira] [Commented] (SPARK-16475) Broadcast Hint for SQL Queries

2016-07-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370026#comment-15370026 ] Reynold Xin commented on SPARK-16475: - cc [~dongjoon] want to take this? > Broadcas

[jira] [Updated] (SPARK-16475) Broadcast Hint for SQL Queries

2016-07-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16475: Attachment: BroadcastHintinSparkSQL.pdf > Broadcast Hint for SQL Queries >

[jira] [Commented] (SPARK-16467) After importing R data.frame, although DataFrame columns show . replaced by _, the describe() function gives warnings on . in the name

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370028#comment-15370028 ] Dongjoon Hyun commented on SPARK-16467: --- Hi, you missed `"` in the last command. :)

[jira] [Created] (SPARK-16475) Broadcast Hint for SQL Queries

2016-07-10 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-16475: --- Summary: Broadcast Hint for SQL Queries Key: SPARK-16475 URL: https://issues.apache.org/jira/browse/SPARK-16475 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-16466) names() function allows creation of column name containing "-". filter() function subsequently fails

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370024#comment-15370024 ] Dongjoon Hyun commented on SPARK-16466: --- I hope this example resolves your problem.

[jira] [Commented] (SPARK-16466) names() function allows creation of column name containing "-". filter() function subsequently fails

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370022#comment-15370022 ] Dongjoon Hyun commented on SPARK-16466: --- IMO, this is not a problem. > names() fun

[jira] [Commented] (SPARK-16466) names() function allows creation of column name containing "-". filter() function subsequently fails

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370021#comment-15370021 ] Dongjoon Hyun commented on SPARK-16466: --- Here is the result of Spark 1.6.2. https:

[jira] [Commented] (SPARK-16466) names() function allows creation of column name containing "-". filter() function subsequently fails

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370020#comment-15370020 ] Dongjoon Hyun commented on SPARK-16466: --- You can use like this. The following is th

[jira] [Commented] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370016#comment-15370016 ] Dongjoon Hyun commented on SPARK-16464: --- Since 1.6.2 was released recently on 2016-

[jira] [Commented] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370011#comment-15370011 ] Dongjoon Hyun commented on SPARK-16464: --- Yep. I checked that 1.6.2 still have the s

[jira] [Commented] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370009#comment-15370009 ] Dongjoon Hyun commented on SPARK-16464: --- Other languages also give reasonable error

[jira] [Commented] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370008#comment-15370008 ] Dongjoon Hyun commented on SPARK-16464: --- FYI, here is the result of current master.

[jira] [Issue Comment Deleted] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16464: -- Comment: was deleted (was: Hi, [~n...@dewar-us.com]. I agree with you. This seems an interestin

[jira] [Issue Comment Deleted] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16464: -- Comment: was deleted (was: Hmm, for current master branch, it seems to work reasonably. {code}

[jira] [Issue Comment Deleted] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16464: -- Comment: was deleted (was: For PySpark, {code} >>> df = spark.range(10) >>> df.withColumn("id",

[jira] [Commented] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370005#comment-15370005 ] Dongjoon Hyun commented on SPARK-16464: --- For PySpark, {code} >>> df = spark.range(1

[jira] [Commented] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15370001#comment-15370001 ] Dongjoon Hyun commented on SPARK-16464: --- Hmm, for current master branch, it seems t

[jira] [Commented] (SPARK-16464) withColumn() allows illegal creation of duplicate column names on DataFrame

2016-07-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369991#comment-15369991 ] Dongjoon Hyun commented on SPARK-16464: --- Hi, [~n...@dewar-us.com]. I agree with you

[jira] [Commented] (SPARK-16465) Add nonnegative flag to mllib ALS

2016-07-10 Thread Roberto Pagliari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369967#comment-15369967 ] Roberto Pagliari commented on SPARK-16465: -- yes, but it would be nice to do some

[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369856#comment-15369856 ] Apache Spark commented on SPARK-15467: -- User 'kiszk' has created a pull request for

[jira] [Assigned] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15467: Assignee: (was: Apache Spark) > Getting stack overflow when attempting to query a wide

[jira] [Assigned] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15467: Assignee: Apache Spark > Getting stack overflow when attempting to query a wide Dataset (>

[jira] [Closed] (SPARK-16474) Global Aggregation doesn't seem to work at all

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Sela closed SPARK-16474. - Resolution: Not A Problem It seems as if the right way to use the agg() API directly on Dataset/DataFrame

[jira] [Commented] (SPARK-16474) Global Aggregation doesn't seem to work at all

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369829#comment-15369829 ] Amit Sela commented on SPARK-16474: --- I thought the bufferEncoder is supposed to take ca

[jira] [Commented] (SPARK-16474) Global Aggregation doesn't seem to work at all

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369706#comment-15369706 ] Amit Sela commented on SPARK-16474: --- Thanks [~koert] that works. > Global Aggregation

[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-07-10 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369703#comment-15369703 ] Kazuaki Ishizaki commented on SPARK-15467: -- [Janino 3.0.0|https://mvnrepository

[jira] [Commented] (SPARK-16474) Global Aggregation doesn't seem to work at all

2016-07-10 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369693#comment-15369693 ] koert kuipers commented on SPARK-16474: --- try ds.select(aggregator) instead of ds.ag

[jira] [Commented] (SPARK-16474) Global Aggregation doesn't seem to work at all

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369686#comment-15369686 ] Sean Owen commented on SPARK-16474: --- I am not sure that is expected to work. You have d

[jira] [Comment Edited] (SPARK-15810) Aggregator doesn't play nice with Option

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369496#comment-15369496 ] Amit Sela edited comment on SPARK-15810 at 7/10/16 2:53 PM: J

[jira] [Commented] (SPARK-15144) option nullValue for CSV data source not working for several types.

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369659#comment-15369659 ] Apache Spark commented on SPARK-15144: -- User 'lw-lin' has created a pull request for

[jira] [Created] (SPARK-16474) Global Aggregation doesn't seem to work at all

2016-07-10 Thread Amit Sela (JIRA)
Amit Sela created SPARK-16474: - Summary: Global Aggregation doesn't seem to work at all Key: SPARK-16474 URL: https://issues.apache.org/jira/browse/SPARK-16474 Project: Spark Issue Type: Sub-tas

[jira] [Updated] (SPARK-16469) Long running Driver task while multiplying big matrices

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16469: -- Fix Version/s: (was: 2.0.0) > Long running Driver task while multiplying big matrices > ---

[jira] [Resolved] (SPARK-16361) It takes a long time for gc when building cube with many fields

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16361. --- Resolution: Not A Problem > It takes a long time for gc when building cube with many fields > --

[jira] [Resolved] (SPARK-15937) Spark declares a succeeding job to be failed in yarn-cluster mode if the job takes very small time (~ < 10 seconds) to finish

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15937. --- Resolution: Not A Problem Per JIRA discussion > Spark declares a succeeding job to be failed in yarn

[jira] [Updated] (SPARK-16470) ml.regression.LinearRegression training data do not check whether the result actually reach convergence

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16470: -- Affects Version/s: (was: 2.0.1) (was: 2.1.0) 2.0.0

[jira] [Created] (SPARK-16473) BisectingKMeans Algorithm failing with java.util.NoSuchElementException: key not found

2016-07-10 Thread Alok Bhandari (JIRA)
Alok Bhandari created SPARK-16473: - Summary: BisectingKMeans Algorithm failing with java.util.NoSuchElementException: key not found Key: SPARK-16473 URL: https://issues.apache.org/jira/browse/SPARK-16473

[jira] [Commented] (SPARK-16465) Add nonnegative flag to mllib ALS

2016-07-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369545#comment-15369545 ] Sean Owen commented on SPARK-16465: --- What are you referring to -- there has been a setN

[jira] [Comment Edited] (SPARK-15810) Aggregator doesn't play nice with Option

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369509#comment-15369509 ] Amit Sela edited comment on SPARK-15810 at 7/10/16 9:01 AM: R

[jira] [Comment Edited] (SPARK-15810) Aggregator doesn't play nice with Option

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369509#comment-15369509 ] Amit Sela edited comment on SPARK-15810 at 7/10/16 8:59 AM: R

[jira] [Comment Edited] (SPARK-15810) Aggregator doesn't play nice with Option

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369509#comment-15369509 ] Amit Sela edited comment on SPARK-15810 at 7/10/16 8:59 AM: R

[jira] [Commented] (SPARK-15810) Aggregator doesn't play nice with Option

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369509#comment-15369509 ] Amit Sela commented on SPARK-15810: --- Running the (sort of) same Java code: {code} S

[jira] [Comment Edited] (SPARK-15810) Aggregator doesn't play nice with Option

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369496#comment-15369496 ] Amit Sela edited comment on SPARK-15810 at 7/10/16 8:28 AM: J

[jira] [Comment Edited] (SPARK-15810) Aggregator doesn't play nice with Option

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369496#comment-15369496 ] Amit Sela edited comment on SPARK-15810 at 7/10/16 8:28 AM: J

[jira] [Commented] (SPARK-15810) Aggregator doesn't play nice with Option

2016-07-10 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369496#comment-15369496 ] Amit Sela commented on SPARK-15810: --- Just ran this exact code, prefixed by: {code} val

[jira] [Commented] (SPARK-16472) Inconsistent nullability in schema after being read in SQL API.

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369489#comment-15369489 ] Apache Spark commented on SPARK-16472: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-16472) Inconsistent nullability in schema after being read in SQL API.

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16472: Assignee: (was: Apache Spark) > Inconsistent nullability in schema after being read in

[jira] [Assigned] (SPARK-16472) Inconsistent nullability in schema after being read in SQL API.

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16472: Assignee: Apache Spark > Inconsistent nullability in schema after being read in SQL API. >

[jira] [Updated] (SPARK-16472) Inconsistent nullability in schema after being read in SQL API.

2016-07-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-16472: - Description: It seems the data sources implementing {{FileFormat}} seems loading the data by for

[jira] [Created] (SPARK-16472) Inconsistent nullability in schema after being read in SQL API.

2016-07-10 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-16472: Summary: Inconsistent nullability in schema after being read in SQL API. Key: SPARK-16472 URL: https://issues.apache.org/jira/browse/SPARK-16472 Project: Spark

[jira] [Updated] (SPARK-16472) Inconsistent nullability in schema after being read in SQL API.

2016-07-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-16472: - Priority: Minor (was: Major) > Inconsistent nullability in schema after being read in SQL API. >

[jira] [Comment Edited] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

2016-07-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369481#comment-15369481 ] Cheng Lian edited comment on SPARK-16344 at 7/10/16 8:07 AM: -

[jira] [Commented] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

2016-07-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369481#comment-15369481 ] Cheng Lian commented on SPARK-16344: Thanks to [~rdblue]'s comment about why there're

[jira] [Assigned] (SPARK-16471) Remove Hive-specific CreateHiveTableAsSelectLogicalPlan

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16471: Assignee: Apache Spark > Remove Hive-specific CreateHiveTableAsSelectLogicalPlan > ---

[jira] [Commented] (SPARK-16471) Remove Hive-specific CreateHiveTableAsSelectLogicalPlan

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369463#comment-15369463 ] Apache Spark commented on SPARK-16471: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-16471) Remove Hive-specific CreateHiveTableAsSelectLogicalPlan

2016-07-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16471: Assignee: (was: Apache Spark) > Remove Hive-specific CreateHiveTableAsSelectLogicalPla

[jira] [Created] (SPARK-16471) Remove Hive-specific CreateHiveTableAsSelectLogicalPlan

2016-07-10 Thread Xiao Li (JIRA)
Xiao Li created SPARK-16471: --- Summary: Remove Hive-specific CreateHiveTableAsSelectLogicalPlan Key: SPARK-16471 URL: https://issues.apache.org/jira/browse/SPARK-16471 Project: Spark Issue Type: Imp