[jira] [Comment Edited] (SPARK-25299) Use remote storage for persisting shuffle data

2020-10-03 Thread BoYang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17207093#comment-17207093 ] BoYang edited comment on SPARK-25299 at 10/4/20, 5:27 AM: -- While people work on 

[jira] [Comment Edited] (SPARK-25299) Use remote storage for persisting shuffle data

2020-10-03 Thread BoYang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17207093#comment-17207093 ] BoYang edited comment on SPARK-25299 at 10/4/20, 5:23 AM: -- While people work on 

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

2020-10-03 Thread BoYang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17207093#comment-17207093 ] BoYang commented on SPARK-25299: While people work on remote storage for persisting shuffle data, and

[jira] [Updated] (SPARK-31257) Unify create table syntax to fix ambiguous two different CREATE TABLE syntaxes

2020-10-03 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31257: Target Version/s: 3.1.0 > Unify create table syntax to fix ambiguous two different CREATE TABLE syntaxes

[jira] [Updated] (SPARK-32346) Support filters pushdown in Avro datasource

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32346: -- Fix Version/s: 3.1.0 > Support filters pushdown in Avro datasource >

[jira] [Commented] (SPARK-32346) Support filters pushdown in Avro datasource

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206894#comment-17206894 ] Dongjoon Hyun commented on SPARK-32346: --- I added the fix version, [~Gengliang.Wang]. > Support

[jira] [Updated] (SPARK-12312) Support JDBC Kerberos w/ keytab

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-12312: -- Summary: Support JDBC Kerberos w/ keytab (was: JDBC connection to Kerberos secured databases

[jira] [Updated] (SPARK-29250) Upgrade to Hadoop 3.2.1 Upgrade to Hadoop 3.2.1 and move to shaded client

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29250: -- Summary: Upgrade to Hadoop 3.2.1 Upgrade to Hadoop 3.2.1 and move to shaded client (was:

[jira] [Updated] (SPARK-32058) Use Apache Hadoop 3.2.0 dependency by default

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32058: -- Target Version/s: 3.1.0 > Use Apache Hadoop 3.2.0 dependency by default >

[jira] [Assigned] (SPARK-29250) Upgrade to Hadoop 3.2.1

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29250: - Assignee: Chao Sun > Upgrade to Hadoop 3.2.1 > --- > >

[jira] [Updated] (SPARK-32688) LiteralGenerator for float and double does not generate special values

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32688: -- Fix Version/s: (was: 3.02) 3.0.2 > LiteralGenerator for float and

[jira] [Updated] (SPARK-32981) Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32981: -- Target Version/s: 3.1.0 > Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution >

[jira] [Assigned] (SPARK-20202) Remove references to org.spark-project.hive

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-20202: - Assignee: Dongjoon Hyun > Remove references to org.spark-project.hive >

[jira] [Updated] (SPARK-20202) Remove references to org.spark-project.hive

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-20202: -- Affects Version/s: 3.1.0 > Remove references to org.spark-project.hive >

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206870#comment-17206870 ] Apache Spark commented on SPARK-20202: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206869#comment-17206869 ] Apache Spark commented on SPARK-20202: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33049) Decommission Core Integration Test is flaky.

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33049: - Assignee: Holden Karau > Decommission Core Integration Test is flaky. >

[jira] [Resolved] (SPARK-33049) Decommission Core Integration Test is flaky.

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33049. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29929

[jira] [Updated] (SPARK-32067) Executor pod template ConfigMap of ongoing submission got inadvertently altered by subsequent submission

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32067: -- Summary: Executor pod template ConfigMap of ongoing submission got inadvertently altered by

[jira] [Updated] (SPARK-32067) Executor pod template ConfigMap of ongoing submission got inadvertently altered by subsequent submission

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32067: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Bug) > Executor pod template

[jira] [Comment Edited] (SPARK-26043) Make SparkHadoopUtil private to Spark

2020-10-03 Thread Ryan Pifer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206418#comment-17206418 ] Ryan Pifer edited comment on SPARK-26043 at 10/3/20, 10:03 PM: --- [~srowen] 

[jira] [Resolved] (SPARK-32958) Column pruning for JsonToStructs

2020-10-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32958. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29900

[jira] [Resolved] (SPARK-33043) RowMatrix is incompatible with spark.driver.maxResultSize=0

2020-10-03 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-33043. -- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-33043) RowMatrix is incompatible with spark.driver.maxResultSize=0

2020-10-03 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-33043: Assignee: Sean R. Owen > RowMatrix is incompatible with spark.driver.maxResultSize=0 >

[jira] [Commented] (SPARK-32949) Add timestamp_seconds to SparkR

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206830#comment-17206830 ] Apache Spark commented on SPARK-32949: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33062) Make DataFrameReader.jdbc work for DataSource V2

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33062: Assignee: (was: Apache Spark) > Make DataFrameReader.jdbc work for DataSource V2 >

[jira] [Assigned] (SPARK-33062) Make DataFrameReader.jdbc work for DataSource V2

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33062: Assignee: Apache Spark > Make DataFrameReader.jdbc work for DataSource V2 >

[jira] [Commented] (SPARK-33062) Make DataFrameReader.jdbc work for DataSource V2

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206819#comment-17206819 ] Apache Spark commented on SPARK-33062: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Created] (SPARK-33062) Make DataFrameReader.jdbc work for DataSource V2

2020-10-03 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-33062: -- Summary: Make DataFrameReader.jdbc work for DataSource V2 Key: SPARK-33062 URL: https://issues.apache.org/jira/browse/SPARK-33062 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-33058) Fetch topic partitions once when getting offset ranges

2020-10-03 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-33058. - Resolution: Duplicate > Fetch topic partitions once when getting offset ranges >

[jira] [Assigned] (SPARK-33061) Expose inverse hyperbolic functions through sql.functions API

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33061: Assignee: (was: Apache Spark) > Expose inverse hyperbolic functions through

[jira] [Commented] (SPARK-33061) Expose inverse hyperbolic functions through sql.functions API

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206785#comment-17206785 ] Apache Spark commented on SPARK-33061: -- User 'rwpenney' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33061) Expose inverse hyperbolic functions through sql.functions API

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33061: Assignee: Apache Spark > Expose inverse hyperbolic functions through sql.functions API >

[jira] [Updated] (SPARK-33061) Expose inverse hyperbolic functions through sql.functions API

2020-10-03 Thread Richard Penney (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Penney updated SPARK-33061: --- Description: Recent changes elsewhere have added inverse hyperbolic functions acosh, asinh,

[jira] [Created] (SPARK-33061) Expose inverse hyperbolic functions through sql.functions API

2020-10-03 Thread Richard Penney (Jira)
Richard Penney created SPARK-33061: -- Summary: Expose inverse hyperbolic functions through sql.functions API Key: SPARK-33061 URL: https://issues.apache.org/jira/browse/SPARK-33061 Project: Spark

[jira] [Updated] (SPARK-33060) approxSimilarityJoin in Structured Stream causes state to explode in size

2020-10-03 Thread Bram van den Akker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bram van den Akker updated SPARK-33060: --- Attachment: Screenshot 2020-10-01 at 16.03.26.png > approxSimilarityJoin in

[jira] [Commented] (SPARK-33060) approxSimilarityJoin in Structured Stream causes state to explode in size

2020-10-03 Thread Bram van den Akker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206693#comment-17206693 ] Bram van den Akker commented on SPARK-33060: !Screenshot 2020-10-01 at 16.03.26.png! >

[jira] [Created] (SPARK-33060) approxSimilarityJoin in Structured Stream causes state to explode in size

2020-10-03 Thread Bram van den Akker (Jira)
Bram van den Akker created SPARK-33060: -- Summary: approxSimilarityJoin in Structured Stream causes state to explode in size Key: SPARK-33060 URL: https://issues.apache.org/jira/browse/SPARK-33060

[jira] [Created] (SPARK-33059) Bug with self joined tables after posexploded column

2020-10-03 Thread Struchkov Lookuut Fedorovich (Jira)
Struchkov Lookuut Fedorovich created SPARK-33059: Summary: Bug with self joined tables after posexploded column Key: SPARK-33059 URL: https://issues.apache.org/jira/browse/SPARK-33059

[jira] [Commented] (SPARK-32067) [K8S] Executor pod template ConfigMap of ongoing submission got inadvertently altered by subsequent submission

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206641#comment-17206641 ] Apache Spark commented on SPARK-32067: -- User 'stijndehaes' has created a pull request for this

[jira] [Commented] (SPARK-32067) [K8S] Executor pod template ConfigMap of ongoing submission got inadvertently altered by subsequent submission

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206640#comment-17206640 ] Apache Spark commented on SPARK-32067: -- User 'stijndehaes' has created a pull request for this

[jira] [Assigned] (SPARK-32067) [K8S] Executor pod template ConfigMap of ongoing submission got inadvertently altered by subsequent submission

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32067: Assignee: (was: Apache Spark) > [K8S] Executor pod template ConfigMap of ongoing

[jira] [Assigned] (SPARK-32067) [K8S] Executor pod template ConfigMap of ongoing submission got inadvertently altered by subsequent submission

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32067: Assignee: Apache Spark > [K8S] Executor pod template ConfigMap of ongoing submission got

[jira] [Updated] (SPARK-33058) Fetch topic partitions once when getting offset ranges

2020-10-03 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-33058: Priority: Minor (was: Major) > Fetch topic partitions once when getting offset ranges >

[jira] [Commented] (SPARK-33058) Fetch topic partitions once when getting offset ranges

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206623#comment-17206623 ] Apache Spark commented on SPARK-33058: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-33058) Fetch topic partitions once when getting offset ranges

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206622#comment-17206622 ] Apache Spark commented on SPARK-33058: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33058) Fetch topic partitions once when getting offset ranges

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33058: Assignee: L. C. Hsieh (was: Apache Spark) > Fetch topic partitions once when getting

[jira] [Assigned] (SPARK-33058) Fetch topic partitions once when getting offset ranges

2020-10-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33058: Assignee: Apache Spark (was: L. C. Hsieh) > Fetch topic partitions once when getting

[jira] [Created] (SPARK-33058) Fetch topic partitions once when getting offset ranges

2020-10-03 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-33058: --- Summary: Fetch topic partitions once when getting offset ranges Key: SPARK-33058 URL: https://issues.apache.org/jira/browse/SPARK-33058 Project: Spark Issue