[jira] [Updated] (SPARK-24359) SPIP: ML Pipelines in R

2018-05-26 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-24359: --- Description: h1. Background and motivation SparkR supports calling MLlib functionality with

[jira] [Updated] (SPARK-24359) SPIP: ML Pipelines in R

2018-05-26 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hossein Falaki updated SPARK-24359: --- Attachment: SparkML_ ML Pipelines in R-v3.pdf > SPIP: ML Pipelines in R >

[jira] [Commented] (SPARK-23435) R tests should support latest testthat

2018-05-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491908#comment-16491908 ] Felix Cheung commented on SPARK-23435: -- Work was stopped for a while. Hopefully I can come by in a

[jira] [Commented] (SPARK-24359) SPIP: ML Pipelines in R

2018-05-26 Thread Hossein Falaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491907#comment-16491907 ] Hossein Falaki commented on SPARK-24359: Thank you [~josephkb] and [~felixcheung] for reviewing.

[jira] [Comment Edited] (SPARK-24359) SPIP: ML Pipelines in R

2018-05-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491867#comment-16491867 ] Felix Cheung edited comment on SPARK-24359 at 5/27/18 3:24 AM: --- # are the

[jira] [Assigned] (SPARK-24307) Support sending messages over 2GB from memory

2018-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24307: Assignee: (was: Apache Spark) > Support sending messages over 2GB from memory >

[jira] [Commented] (SPARK-24307) Support sending messages over 2GB from memory

2018-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491876#comment-16491876 ] Apache Spark commented on SPARK-24307: -- User 'squito' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24307) Support sending messages over 2GB from memory

2018-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24307: Assignee: Apache Spark > Support sending messages over 2GB from memory >

[jira] [Commented] (SPARK-24359) SPIP: ML Pipelines in R

2018-05-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491867#comment-16491867 ] Felix Cheung commented on SPARK-24359: -- # are the set_input_col / set_output_col methods going to

[jira] [Commented] (SPARK-24356) Duplicate strings in File.path managed by FileSegmentManagedBuffer

2018-05-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491788#comment-16491788 ] Felix Cheung commented on SPARK-24356: -- Interesting. we will definitely look into this. Is the plan

[jira] [Issue Comment Deleted] (SPARK-24391) to_json/from_json should support arrays of primitives, and more generally all JSON

2018-05-26 Thread Teng Peng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-24391: -- Comment: was deleted (was: My plan is to follow the Spark-19849 & Spark-21513 to support more

[jira] [Commented] (SPARK-24383) spark on k8s: "driver-svc" are not getting deleted

2018-05-26 Thread Lenin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491753#comment-16491753 ] Lenin commented on SPARK-24383: --- Not sure about how k8s runs it GC. But when there are objects to be gc-ed,

[jira] [Commented] (SPARK-24391) to_json/from_json should support arrays of primitives, and more generally all JSON

2018-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491750#comment-16491750 ] Apache Spark commented on SPARK-24391: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24391) to_json/from_json should support arrays of primitives, and more generally all JSON

2018-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24391: Assignee: Apache Spark > to_json/from_json should support arrays of primitives, and more

[jira] [Assigned] (SPARK-24391) to_json/from_json should support arrays of primitives, and more generally all JSON

2018-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24391: Assignee: (was: Apache Spark) > to_json/from_json should support arrays of

[jira] [Commented] (SPARK-24391) to_json/from_json should support arrays of primitives, and more generally all JSON

2018-05-26 Thread Teng Peng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491745#comment-16491745 ] Teng Peng commented on SPARK-24391: --- My plan is to follow the Spark-19849 & Spark-21513 to support more

[jira] [Updated] (SPARK-24398) SQLAppStatusListener.aggregateMetrics() invocation is too slow

2018-05-26 Thread furongbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] furongbin updated SPARK-24398: -- Description: RROR 18/05/24 14:08:25 AsyncEventQueue: Dropping event from queue appStatus. This likely

[jira] [Updated] (SPARK-24398) SQLAppStatusListener.aggregateMetrics() invocation is too slow

2018-05-26 Thread furongbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] furongbin updated SPARK-24398: -- Description: RROR 18/05/24 14:08:25 AsyncEventQueue: Dropping event from queue appStatus. This likely

[jira] [Created] (SPARK-24398) SQLAppStatusListener.aggregateMetrics() invocation is too slow

2018-05-26 Thread furongbin (JIRA)
furongbin created SPARK-24398: - Summary: SQLAppStatusListener.aggregateMetrics() invocation is too slow Key: SPARK-24398 URL: https://issues.apache.org/jira/browse/SPARK-24398 Project: Spark

[jira] [Updated] (SPARK-24395) Fix Behavior of NOT IN with Literals Containing NULL

2018-05-26 Thread Miles Yucht (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miles Yucht updated SPARK-24395: Issue Type: Bug (was: Improvement) > Fix Behavior of NOT IN with Literals Containing NULL >

[jira] [Updated] (SPARK-24395) Fix Behavior of NOT IN with Literals Containing NULL

2018-05-26 Thread Miles Yucht (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miles Yucht updated SPARK-24395: Description: Spark does not return the correct answer when evaluating NOT IN in some cases. For