[jira] [Commented] (SPARK-24421) Accessing sun.misc.Cleaner in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682915#comment-16682915 ] Assaf Mendelson commented on SPARK-24421: - Would it be possible to add {{Add-Opens

[jira] [Created] (SPARK-25709) Can't append to a data source V2 unless target already has schema information or fixed schema matching dataframe

Assaf Mendelson created SPARK-25709: --- Summary: Can't append to a data source V2 unless target already has schema information or fixed schema matching dataframe Key: SPARK-25709 URL:

[jira] [Comment Edited] (SPARK-17333) Make pyspark interface friendly with static analysis

[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091210#comment-16091210 ] Assaf Mendelson edited comment on SPARK-17333 at 7/19/17 7:18 PM: --

[jira] [Commented] (SPARK-17333) Make pyspark interface friendly with static analysis

[ https://issues.apache.org/jira/browse/SPARK-17333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091210#comment-16091210 ] Assaf Mendelson commented on SPARK-17333: - Originally when I suggested this, I envisioned adding

[jira] [Commented] (SPARK-20706) Spark-shell not overriding method definition

[ https://issues.apache.org/jira/browse/SPARK-20706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005986#comment-16005986 ] Assaf Mendelson commented on SPARK-20706: - I don't think the priority is right. This shouldn't be

[jira] [Commented] (SPARK-20706) Spark-shell not overriding method definition

[ https://issues.apache.org/jira/browse/SPARK-20706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005985#comment-16005985 ] Assaf Mendelson commented on SPARK-20706: - Actually it seems like the use of ';' to do more than

[jira] [Commented] (SPARK-19046) Dataset checkpoint consumes too much disk space

[ https://issues.apache.org/jira/browse/SPARK-19046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15795004#comment-15795004 ] Assaf Mendelson commented on SPARK-19046: - Currently, behind the scenes we use RDD serialization

[jira] [Commented] (SPARK-19045) irrelevant warning when creating a checkpoint dir

[ https://issues.apache.org/jira/browse/SPARK-19045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15791511#comment-15791511 ] Assaf Mendelson commented on SPARK-19045: - I 100% agree standalone is not local, however,

[jira] [Commented] (SPARK-19046) Dataset checkpoint consumes too much disk space

[ https://issues.apache.org/jira/browse/SPARK-19046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15791468#comment-15791468 ] Assaf Mendelson commented on SPARK-19046: - This is an easily created example. I see a factor of

[jira] [Created] (SPARK-19046) Dataset checkpoint consumes too much disk space

Assaf Mendelson created SPARK-19046: --- Summary: Dataset checkpoint consumes too much disk space Key: SPARK-19046 URL: https://issues.apache.org/jira/browse/SPARK-19046 Project: Spark Issue

[jira] [Created] (SPARK-19045) irrelevant warning when creating a checkpoint dir

Assaf Mendelson created SPARK-19045: --- Summary: irrelevant warning when creating a checkpoint dir Key: SPARK-19045 URL: https://issues.apache.org/jira/browse/SPARK-19045 Project: Spark

[jira] [Commented] (SPARK-9850) Adaptive execution in Spark

[ https://issues.apache.org/jira/browse/SPARK-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15666806#comment-15666806 ] Assaf Mendelson commented on SPARK-9850: I like the overall idea. What I am trying to figure out

[jira] [Commented] (SPARK-17691) Add aggregate function to collect list with maximum number of elements

[ https://issues.apache.org/jira/browse/SPARK-17691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653344#comment-15653344 ] Assaf Mendelson commented on SPARK-17691: - While you can use mutable buffers with the aggregator

[jira] [Commented] (SPARK-17691) Add aggregate function to collect list with maximum number of elements

[ https://issues.apache.org/jira/browse/SPARK-17691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15649792#comment-15649792 ] Assaf Mendelson commented on SPARK-17691: - I don't believe UDF (or rather UDAF in this case) is

[jira] [Created] (SPARK-17691) Add aggregate function to collect list with maximum number of elements

Assaf Mendelson created SPARK-17691: --- Summary: Add aggregate function to collect list with maximum number of elements Key: SPARK-17691 URL: https://issues.apache.org/jira/browse/SPARK-17691

[jira] [Created] (SPARK-17334) Provide management tools for broadcasted variables

Assaf Mendelson created SPARK-17334: --- Summary: Provide management tools for broadcasted variables Key: SPARK-17334 URL: https://issues.apache.org/jira/browse/SPARK-17334 Project: Spark

[jira] [Created] (SPARK-17333) Make pyspark interface friendly with static analysis

Assaf Mendelson created SPARK-17333: --- Summary: Make pyspark interface friendly with static analysis Key: SPARK-17333 URL: https://issues.apache.org/jira/browse/SPARK-17333 Project: Spark