[
https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim reassigned SPARK-34471:
Assignee: Bo Zhang
> Document DataStreamReader/Writer table APIs in Structured Streaming
[
https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim resolved SPARK-34471.
--
Fix Version/s: 3.1.1
Resolution: Fixed
Issue resolved by pull request 31590
[
https://issues.apache.org/jira/browse/SPARK-33602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287485#comment-17287485
]
jiaan.geng commented on SPARK-33602:
I'm working on.
> Group exception messages in
[
https://issues.apache.org/jira/browse/SPARK-33542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiaan.geng updated SPARK-33542:
---
Comment: was deleted
(was: I'm working on.)
> Group exception messages in catalyst/catalog
>
[
https://issues.apache.org/jira/browse/SPARK-33599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiaan.geng updated SPARK-33599:
---
Comment: was deleted
(was: I'm working on.)
> Group exception messages in catalyst/analysis
>
Lichuanliang created SPARK-34480:
Summary: Module launcher build failed with profile hadoop-3.2
activated
Key: SPARK-34480
URL: https://issues.apache.org/jira/browse/SPARK-34480
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-34478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287479#comment-17287479
]
Apache Spark commented on SPARK-34478:
--
User 'AngersZh' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34478:
Assignee: (was: Apache Spark)
> Ignore or reject wrong config when start
[
https://issues.apache.org/jira/browse/SPARK-33602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287492#comment-17287492
]
jiaan.geng commented on SPARK-33602:
ping [~allisonwang-db] Should we put AnalysisException in
Yuchen Huo created SPARK-34481:
--
Summary: Refactor dataframe reader/writer path option logic
Key: SPARK-34481
URL: https://issues.apache.org/jira/browse/SPARK-34481
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-34481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287583#comment-17287583
]
Apache Spark commented on SPARK-34481:
--
User 'yuchenhuo' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-34481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34481:
Assignee: Apache Spark
> Refactor dataframe reader/writer path option logic
>
[
https://issues.apache.org/jira/browse/SPARK-34481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34481:
Assignee: (was: Apache Spark)
> Refactor dataframe reader/writer path option logic
>
[
https://issues.apache.org/jira/browse/SPARK-34449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-34449.
--
Fix Version/s: 3.1.2
3.0.3
2.4.8
Resolution:
[
https://issues.apache.org/jira/browse/SPARK-33541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiaan.geng updated SPARK-33541:
---
Comment: was deleted
(was: I'm working on.)
> Group exception messages in catalyst/expressions
>
[
https://issues.apache.org/jira/browse/SPARK-33601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiaan.geng updated SPARK-33601:
---
Comment: was deleted
(was: I'm working on.)
> Group exception messages in catalyst/parser
>
[
https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287585#comment-17287585
]
Yuanjian Li commented on SPARK-10816:
-
Great thanks for your heads up! [~viirya] [~kabhwan]
[
https://issues.apache.org/jira/browse/SPARK-34478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34478:
Assignee: Apache Spark
> Ignore or reject wrong config when start sparksession
>
[
https://issues.apache.org/jira/browse/SPARK-34480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lichuanliang closed SPARK-34480.
Please ignore this issue
> Module launcher build failed with profile hadoop-3.2 activated
>
[
https://issues.apache.org/jira/browse/SPARK-34480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lichuanliang resolved SPARK-34480.
--
Resolution: Fixed
> Module launcher build failed with profile hadoop-3.2 activated
>
[
https://issues.apache.org/jira/browse/SPARK-34478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287478#comment-17287478
]
Apache Spark commented on SPARK-34478:
--
User 'AngersZh' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuchen Huo updated SPARK-34481:
---
Priority: Trivial (was: Major)
> Refactor dataframe reader/writer path option logic
>
Shardul Mahadik created SPARK-34477:
---
Summary: Kryo NPEs when serializing Avro GenericData objects
(except GenericRecord)
Key: SPARK-34477
URL: https://issues.apache.org/jira/browse/SPARK-34477
[
https://issues.apache.org/jira/browse/SPARK-34478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287459#comment-17287459
]
angerszhu commented on SPARK-34478:
---
Raise a pr soon
> Ignore or reject wrong config when start
angerszhu created SPARK-34478:
-
Summary: Ignore or reject wrong config when start sparksession
Key: SPARK-34478
URL: https://issues.apache.org/jira/browse/SPARK-34478
Project: Spark
Issue Type:
Yuming Wang created SPARK-34479:
---
Summary: Add zstandard codec to spark.sql.avro.compression.codec
Key: SPARK-34479
URL: https://issues.apache.org/jira/browse/SPARK-34479
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang updated SPARK-34479:
Description: Avro add AVRO-2195
> Add zstandard codec to spark.sql.avro.compression.codec
>
[
https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang updated SPARK-34479:
Description: Avro add zstandard codec since AVRO-2195. (was: Avro add
AVRO-2195)
> Add
[
https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287463#comment-17287463
]
Yuming Wang commented on SPARK-34479:
-
But zstd 1.4.5-12 is not compatible with 1.4.8-4.
[
https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287463#comment-17287463
]
Yuming Wang edited comment on SPARK-34479 at 2/20/21, 3:02 AM:
---
But zstd
[
https://issues.apache.org/jira/browse/SPARK-34479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287463#comment-17287463
]
Yuming Wang edited comment on SPARK-34479 at 2/20/21, 3:02 AM:
---
But zstd
[
https://issues.apache.org/jira/browse/SPARK-34477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287474#comment-17287474
]
Apache Spark commented on SPARK-34477:
--
User 'shardulm94' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34477:
Assignee: Apache Spark
> Kryo NPEs when serializing Avro GenericData objects (except
[
https://issues.apache.org/jira/browse/SPARK-34477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34477:
Assignee: (was: Apache Spark)
> Kryo NPEs when serializing Avro GenericData objects
[
https://issues.apache.org/jira/browse/SPARK-34424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-34424.
-
Fix Version/s: 3.1.1
3.0.2
Assignee: Maxim Gekk
Resolution:
Wenchen Fan created SPARK-34473:
---
Summary: avoid NPE in DataFrameReader.schema(StructType)
Key: SPARK-34473
URL: https://issues.apache.org/jira/browse/SPARK-34473
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287109#comment-17287109
]
Guillaume Martres commented on SPARK-25075:
---
[~dongjoon] I think something is wrong with the
[
https://issues.apache.org/jira/browse/SPARK-34421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287057#comment-17287057
]
Apache Spark commented on SPARK-34421:
--
User 'peter-toth' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-34283.
-
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 31404
[
https://issues.apache.org/jira/browse/SPARK-28123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-28123:
---
Assignee: jiaan.geng
> String Functions: Add support btrim
>
[
https://issues.apache.org/jira/browse/SPARK-28123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-28123.
-
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 31390
[
https://issues.apache.org/jira/browse/SPARK-34473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34473:
Assignee: (was: Apache Spark)
> avoid NPE in DataFrameReader.schema(StructType)
>
[
https://issues.apache.org/jira/browse/SPARK-34473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287096#comment-17287096
]
Apache Spark commented on SPARK-34473:
--
User 'cloud-fan' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-34473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34473:
Assignee: Apache Spark
> avoid NPE in DataFrameReader.schema(StructType)
>
[
https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-34283:
---
Assignee: Zhichao Zhang
> Combines all adjacent 'Union' operators into a single 'Union'
[
https://issues.apache.org/jira/browse/SPARK-34421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-34421:
---
Assignee: Peter Toth
> Custom functions can't be used in temporary views with CTEs
>
Bo Zhang created SPARK-34471:
Summary: Document DataStreamReader/Writer table APIs in Structured
Streaming Programming Guide
Key: SPARK-34471
URL: https://issues.apache.org/jira/browse/SPARK-34471
[
https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34472:
Assignee: (was: Apache Spark)
> SparkContext.addJar with an ivy path fails in
[
https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287009#comment-17287009
]
Apache Spark commented on SPARK-34472:
--
User 'shardulm94' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34472:
Assignee: Apache Spark
> SparkContext.addJar with an ivy path fails in cluster mode with
[
https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287008#comment-17287008
]
Apache Spark commented on SPARK-34472:
--
User 'shardulm94' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286926#comment-17286926
]
Apache Spark commented on SPARK-34424:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34471:
Assignee: Apache Spark
> Document DataStreamReader/Writer table APIs in Structured
[
https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286951#comment-17286951
]
Apache Spark commented on SPARK-34471:
--
User 'bozhang2820' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34471:
Assignee: (was: Apache Spark)
> Document DataStreamReader/Writer table APIs in
[
https://issues.apache.org/jira/browse/SPARK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286953#comment-17286953
]
Apache Spark commented on SPARK-34471:
--
User 'bozhang2820' has created a pull request for this
Shardul Mahadik created SPARK-34472:
---
Summary: SparkContext.addJar with an ivy path fails in cluster
mode with a custom ivySettings file
Key: SPARK-34472
URL: https://issues.apache.org/jira/browse/SPARK-34472
[
https://issues.apache.org/jira/browse/SPARK-34421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan updated SPARK-34421:
Description:
The following query works in Spark 3.0 not Spark 3.1.
Start with:
[
https://issues.apache.org/jira/browse/SPARK-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286916#comment-17286916
]
Weichen Xu commented on SPARK-21770:
[~rishi-aga]
Could you create a new ticket for this with
[
https://issues.apache.org/jira/browse/SPARK-34314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-34314:
---
Assignee: Maxim Gekk
> Wrong discovered partition value
>
[
https://issues.apache.org/jira/browse/SPARK-34314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-34314.
-
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 31549
[
https://issues.apache.org/jira/browse/SPARK-34421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-34421.
-
Fix Version/s: 3.1.1
Resolution: Fixed
Issue resolved by pull request 31550
[
https://issues.apache.org/jira/browse/SPARK-34472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286977#comment-17286977
]
Shardul Mahadik commented on SPARK-34472:
-
I will be sending a PR for this soon.
>
[
https://issues.apache.org/jira/browse/SPARK-34469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-34469:
-
Assignee: Dongjoon Hyun
> Ignore RegisterExecutor when SparkContext is stopped
>
[
https://issues.apache.org/jira/browse/SPARK-34469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-34469.
---
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 31587
[
https://issues.apache.org/jira/browse/SPARK-34475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Gekk updated SPARK-34475:
---
Description: Rename v2 logical nodes for simplicity in the form: +
(was: To be consistent with
[
https://issues.apache.org/jira/browse/SPARK-34468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287274#comment-17287274
]
Apache Spark commented on SPARK-34468:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-34468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34468:
Assignee: (was: Apache Spark)
> Fix v2 ALTER TABLE .. RENAME TO
>
[
https://issues.apache.org/jira/browse/SPARK-34468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34468:
Assignee: Apache Spark
> Fix v2 ALTER TABLE .. RENAME TO
>
L. C. Hsieh created SPARK-34474:
---
Summary: Remove unnecessary Union under Distinct like operators
Key: SPARK-34474
URL: https://issues.apache.org/jira/browse/SPARK-34474
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-34424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-34424:
--
Fix Version/s: (was: 3.0.2)
3.0.3
> HiveOrcHadoopFsRelationSuite fails
[
https://issues.apache.org/jira/browse/SPARK-34475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287340#comment-17287340
]
Apache Spark commented on SPARK-34475:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-34475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34475:
Assignee: Apache Spark (was: Maxim Gekk)
> Rename v2 logical nodes
>
[
https://issues.apache.org/jira/browse/SPARK-34475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34475:
Assignee: Maxim Gekk (was: Apache Spark)
> Rename v2 logical nodes
>
[
https://issues.apache.org/jira/browse/SPARK-34474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34474:
Assignee: Apache Spark (was: L. C. Hsieh)
> Remove unnecessary Union under Distinct
[
https://issues.apache.org/jira/browse/SPARK-34474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287320#comment-17287320
]
Apache Spark commented on SPARK-34474:
--
User 'viirya' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-34474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34474:
Assignee: L. C. Hsieh (was: Apache Spark)
> Remove unnecessary Union under Distinct
Maxim Gekk created SPARK-34475:
--
Summary: Rename v2 logical nodes
Key: SPARK-34475
URL: https://issues.apache.org/jira/browse/SPARK-34475
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287437#comment-17287437
]
Dongjoon Hyun commented on SPARK-25075:
---
[~MasseGuillaume]. Feel free to create a new independent
[
https://issues.apache.org/jira/browse/SPARK-34476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated SPARK-34476:
---
Description:
When running test with Spark extension that converts custom function to json
path expression,
[
https://issues.apache.org/jira/browse/SPARK-24818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mridul Muralidharan resolved SPARK-24818.
-
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request
Ted Yu created SPARK-34476:
--
Summary: Duplicate referenceNames are given for ambiguousReferences
Key: SPARK-34476
URL: https://issues.apache.org/jira/browse/SPARK-34476
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-24818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mridul Muralidharan reassigned SPARK-24818:
---
Assignee: wuyi
> Ensure all the barrier tasks in the same stage are
[
https://issues.apache.org/jira/browse/SPARK-34476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287403#comment-17287403
]
Ted Yu commented on SPARK-34476:
The basic jsonb test is here:
84 matches
Mail list logo