[jira] [Commented] (SPARK-32779) Spark/Hive3 interaction potentially causes deadlock

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194619#comment-17194619 ] Apache Spark commented on SPARK-32779: -- User 'sandeep-katta' has created a pull request for this

[jira] [Commented] (SPARK-32779) Spark/Hive3 interaction potentially causes deadlock

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194617#comment-17194617 ] Apache Spark commented on SPARK-32779: -- User 'sandeep-katta' has created a pull request for this

[jira] [Commented] (SPARK-32180) Getting Started - Installation

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194577#comment-17194577 ] Apache Spark commented on SPARK-32180: -- User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-32180) Getting Started - Installation

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194578#comment-17194578 ] Apache Spark commented on SPARK-32180: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32861) GenerateExec should require output column ordering

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32861: Assignee: Apache Spark > GenerateExec should require output column ordering >

[jira] [Assigned] (SPARK-32861) GenerateExec should require output column ordering

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32861: Assignee: (was: Apache Spark) > GenerateExec should require output column ordering >

[jira] [Commented] (SPARK-32861) GenerateExec should require output column ordering

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194576#comment-17194576 ] Apache Spark commented on SPARK-32861: -- User 'allisonwang-db' has created a pull request for this

[jira] [Commented] (SPARK-32180) Getting Started - Installation

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194571#comment-17194571 ] Apache Spark commented on SPARK-32180: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-32180) Getting Started - Installation

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194570#comment-17194570 ] Apache Spark commented on SPARK-32180: -- User 'sunchao' has created a pull request for this issue:

[jira] [Updated] (SPARK-32861) GenerateExec should require output column ordering

2020-09-11 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-32861: - Summary: GenerateExec should require output column ordering (was: GeneratorExec should require

[jira] [Updated] (SPARK-32861) GeneratorExec should require output column ordering

2020-09-11 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-32861: - Description: {{GenerateExec}} should require column ordering since it binds its input rows

[jira] [Updated] (SPARK-32861) GeneratorExec should require output column ordering

2020-09-11 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-32861: - Description: {{ GeneratorExec }} should require column ordering since it binds its input rows

[jira] [Updated] (SPARK-32861) GeneratorExec should require output column ordering

2020-09-11 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-32861: - Description: {{GenerateExec}} should require column ordering since it binds its input rows

[jira] [Updated] (SPARK-32861) GeneratorExec should require output column ordering

2020-09-11 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-32861: - Description: {{GeneratorExec }}should require column ordering since it binds its input rows

[jira] [Created] (SPARK-32861) GeneratorExec should require output column ordering

2020-09-11 Thread Allison Wang (Jira)
Allison Wang created SPARK-32861: Summary: GeneratorExec should require output column ordering Key: SPARK-32861 URL: https://issues.apache.org/jira/browse/SPARK-32861 Project: Spark Issue

[jira] [Resolved] (SPARK-32851) Tests should fail if errors happen when generating expr code

2020-09-11 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32851. -- Fix Version/s: 3.1.0 Assignee: Takeshi Yamamuro Resolution: Fixed

[jira] [Commented] (SPARK-27872) Driver and executors use a different service account breaking pull secrets

2020-09-11 Thread Neelesh Srinivas Salian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194557#comment-17194557 ] Neelesh Srinivas Salian commented on SPARK-27872: - I have a patch to add this fix to the

[jira] [Updated] (SPARK-32860) Encoders::bean doc incorrectly states maps are not supported

2020-09-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-32860: Issue Type: Documentation (was: Bug) > Encoders::bean doc incorrectly states maps are not

[jira] [Updated] (SPARK-32860) Encoders::bean doc incorrectly states maps are not supported

2020-09-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-32860: Affects Version/s: 3.1.0 > Encoders::bean doc incorrectly states maps are not supported >

[jira] [Commented] (SPARK-32860) Encoders::bean doc incorrectly states maps are not supported

2020-09-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194551#comment-17194551 ] L. C. Hsieh commented on SPARK-32860: - I think so. Java bean encoder supports map. Would you like to

[jira] [Created] (SPARK-32860) Encoders::bean doc incorrectly states maps are not supported

2020-09-11 Thread Dan Ziemba (Jira)
Dan Ziemba created SPARK-32860: -- Summary: Encoders::bean doc incorrectly states maps are not supported Key: SPARK-32860 URL: https://issues.apache.org/jira/browse/SPARK-32860 Project: Spark

[jira] [Commented] (SPARK-24528) Add support to read multiple sorted bucket files for data source v1

2020-09-11 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194505#comment-17194505 ] Cheng Su commented on SPARK-24528: -- After discussing with [~cloud_fan] , it would be better to have a

[jira] [Updated] (SPARK-32845) Add sinkParameter to check sink options robustly in DataStreamReaderWriterSuite

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32845: -- Fix Version/s: (was: 2.4.7) 2.4.8 > Add sinkParameter to check sink

[jira] [Updated] (SPARK-32794) Rare corner case error in micro-batch engine with some stateful queries + no-data-batches + V1 streaming sources

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32794: -- Fix Version/s: (was: 2.4.7) 2.4.8 > Rare corner case error in

[jira] [Updated] (SPARK-32845) Add sinkParameter to check sink options robustly in DataStreamReaderWriterSuite

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32845: -- Affects Version/s: 2.4.6 3.0.1 > Add sinkParameter to check sink

[jira] [Resolved] (SPARK-32845) Add sinkParameter to check sink options robustly in DataStreamReaderWriterSuite

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32845. --- Fix Version/s: 3.0.2 3.1.0 2.4.7 Resolution:

[jira] [Commented] (SPARK-32411) GPU Cluster Fail

2020-09-11 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194485#comment-17194485 ] Thomas Graves commented on SPARK-32411: --- [~chitralverma] if you are still having an issue please

[jira] [Commented] (SPARK-32859) Introduce SQL physical plan rule to decide enable/disable bucketing

2020-09-11 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194481#comment-17194481 ] Cheng Su commented on SPARK-32859: -- Will raise a PR in next couple of days. > Introduce SQL physical

[jira] [Created] (SPARK-32859) Introduce SQL physical plan rule to decide enable/disable bucketing

2020-09-11 Thread Cheng Su (Jira)
Cheng Su created SPARK-32859: Summary: Introduce SQL physical plan rule to decide enable/disable bucketing Key: SPARK-32859 URL: https://issues.apache.org/jira/browse/SPARK-32859 Project: Spark

[jira] [Updated] (SPARK-32859) Introduce SQL physical plan rule to decide enable/disable bucketing

2020-09-11 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32859: - Description: Discussed with [~cloud_fan] offline, it would be better that we can decide enable/disable

[jira] [Updated] (SPARK-32858) UnwrapCastInBinaryComparison: support more types other than Integral

2020-09-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-32858: Affects Version/s: (was: 3.0.1) 3.1.0 > UnwrapCastInBinaryComparison:

[jira] [Commented] (SPARK-32838) Connot overwite different partition with same table

2020-09-11 Thread CHC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194448#comment-17194448 ] CHC commented on SPARK-32838: - [~dongjoon] Yes, I have tried this, this is right, but there are too many sql

[jira] [Commented] (SPARK-32838) Connot overwite different partition with same table

2020-09-11 Thread CHC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194406#comment-17194406 ] CHC commented on SPARK-32838: - [~rohitmishr1484] Ok, I get it. Thanks :) > Connot overwite different

[jira] [Commented] (SPARK-32857) Flaky o.a.s.scheduler.BarrierTaskContextSuite.throw exception if the number of barrier() calls are not the same on every task

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194362#comment-17194362 ] Apache Spark commented on SPARK-32857: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32857) Flaky o.a.s.scheduler.BarrierTaskContextSuite.throw exception if the number of barrier() calls are not the same on every task

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32857: Assignee: (was: Apache Spark) > Flaky o.a.s.scheduler.BarrierTaskContextSuite.throw

[jira] [Commented] (SPARK-32857) Flaky o.a.s.scheduler.BarrierTaskContextSuite.throw exception if the number of barrier() calls are not the same on every task

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194361#comment-17194361 ] Apache Spark commented on SPARK-32857: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32857) Flaky o.a.s.scheduler.BarrierTaskContextSuite.throw exception if the number of barrier() calls are not the same on every task

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32857: Assignee: Apache Spark > Flaky o.a.s.scheduler.BarrierTaskContextSuite.throw exception

[jira] [Updated] (SPARK-24994) Support cast pushdown for integral types

2020-09-11 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-24994: - Summary: Support cast pushdown for integral types (was: Support pushdown casts for ) > Support cast

[jira] [Updated] (SPARK-24994) Support pushdown casts for

2020-09-11 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-24994: - Summary: Support pushdown casts for (was: When the data type of the field is converted to other

[jira] [Commented] (SPARK-32790) FINISHED state of application is not final

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194351#comment-17194351 ] Dongjoon Hyun commented on SPARK-32790: --- Thank you for reporting and analysis, [~rosieb]. +1 for

[jira] [Created] (SPARK-32858) UnwrapCastInBinaryComparison: support more types other than Integral

2020-09-11 Thread Chao Sun (Jira)
Chao Sun created SPARK-32858: Summary: UnwrapCastInBinaryComparison: support more types other than Integral Key: SPARK-32858 URL: https://issues.apache.org/jira/browse/SPARK-32858 Project: Spark

[jira] [Commented] (SPARK-32795) ApplicationInfo#removedExecutors can cause OOM

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194331#comment-17194331 ] Dongjoon Hyun commented on SPARK-32795: --- Hi, [~victor.tso]. Could you provide a reproducible

[jira] [Resolved] (SPARK-32180) Getting Started - Installation

2020-09-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-32180. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29640

[jira] [Updated] (SPARK-32821) cannot group by with window in sql statement for structured streaming with watermark

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32821: -- Affects Version/s: (was: 2.4.0) (was: 2.3.0)

[jira] [Created] (SPARK-32857) Flaky o.a.s.scheduler.BarrierTaskContextSuite.throw exception if the number of barrier() calls are not the same on every task

2020-09-11 Thread wuyi (Jira)
wuyi created SPARK-32857: Summary: Flaky o.a.s.scheduler.BarrierTaskContextSuite.throw exception if the number of barrier() calls are not the same on every task Key: SPARK-32857 URL:

[jira] [Updated] (SPARK-32180) Getting Started - Installation

2020-09-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-32180: - Priority: Minor (was: Major) > Getting Started - Installation > --

[jira] [Commented] (SPARK-32838) Connot overwite different partition with same table

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194326#comment-17194326 ] Dongjoon Hyun commented on SPARK-32838: --- [~CHENXCHEN] and [~lithiumlee-_-]. Could you try the

[jira] [Updated] (SPARK-32840) Invalid interval value can happen to be just adhesive with the unit

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32840: -- Affects Version/s: 3.0.0 > Invalid interval value can happen to be just adhesive with the

[jira] [Commented] (SPARK-32706) Poor performance when casting invalid decimal string to decimal type

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194315#comment-17194315 ] Apache Spark commented on SPARK-32706: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32706) Poor performance when casting invalid decimal string to decimal type

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32706: Assignee: (was: Apache Spark) > Poor performance when casting invalid decimal string

[jira] [Assigned] (SPARK-32706) Poor performance when casting invalid decimal string to decimal type

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32706: Assignee: Apache Spark > Poor performance when casting invalid decimal string to decimal

[jira] [Commented] (SPARK-32706) Poor performance when casting invalid decimal string to decimal type

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194316#comment-17194316 ] Apache Spark commented on SPARK-32706: -- User 'wangyum' has created a pull request for this issue:

[jira] [Updated] (SPARK-32845) Add sinkParameter to check sink options robustly in DataStreamReaderWriterSuite

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32845: -- Priority: Minor (was: Major) > Add sinkParameter to check sink options robustly in >

[jira] [Updated] (SPARK-32845) Add sinkParameter to check sink options robustly in DataStreamReaderWriterSuite

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32845: -- Issue Type: Bug (was: Test) > Add sinkParameter to check sink options robustly in >

[jira] [Commented] (SPARK-32845) Add sinkParameter to check sink options robustly in DataStreamReaderWriterSuite

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194312#comment-17194312 ] Apache Spark commented on SPARK-32845: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-32845) Add sinkParameter to check sink options robustly in DataStreamReaderWriterSuite

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194311#comment-17194311 ] Apache Spark commented on SPARK-32845: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-32845) Add sinkParameter to check sink options in DataStreamReaderWriterSuite

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32845: -- Summary: Add sinkParameter to check sink options in DataStreamReaderWriterSuite (was: flaky

[jira] [Updated] (SPARK-32845) Add sinkParameter to check sink options robustly in DataStreamReaderWriterSuite

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32845: -- Summary: Add sinkParameter to check sink options robustly in DataStreamReaderWriterSuite

[jira] [Updated] (SPARK-32706) Poor performance when casting invalid decimal string to decimal type

2020-09-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-32706: Description: How to reproduce this issue: {code:java}

[jira] [Updated] (SPARK-32706) Poor performance when casting invalid decimal string to decimal type

2020-09-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-32706: Attachment: part-0.parquet > Poor performance when casting invalid decimal string to decimal

[jira] [Commented] (SPARK-32852) spark.sql.hive.metastore.jars support HDFS location

2020-09-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194286#comment-17194286 ] Yuming Wang commented on SPARK-32852: - [~leoluan] Could you provide the patch? >

[jira] [Assigned] (SPARK-32852) spark.sql.hive.metastore.jars support HDFS location

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32852: - Assignee: (was: Xuedong Luan) > spark.sql.hive.metastore.jars support HDFS

[jira] [Commented] (SPARK-32032) Eliminate deprecated poll(long) API calls to avoid infinite wait in driver

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194277#comment-17194277 ] Apache Spark commented on SPARK-32032: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Assigned] (SPARK-32032) Eliminate deprecated poll(long) API calls to avoid infinite wait in driver

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32032: Assignee: (was: Apache Spark) > Eliminate deprecated poll(long) API calls to avoid

[jira] [Commented] (SPARK-32032) Eliminate deprecated poll(long) API calls to avoid infinite wait in driver

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194278#comment-17194278 ] Apache Spark commented on SPARK-32032: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Assigned] (SPARK-32032) Eliminate deprecated poll(long) API calls to avoid infinite wait in driver

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32032: Assignee: Apache Spark > Eliminate deprecated poll(long) API calls to avoid infinite

[jira] [Commented] (SPARK-32032) Eliminate deprecated poll(long) API calls to avoid infinite wait in driver

2020-09-11 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194269#comment-17194269 ] Gabor Somogyi commented on SPARK-32032: --- [~Bartalos] query is not progressing. I've just finished

[jira] [Resolved] (SPARK-32853) consecutive load/save calls should be allowed

2020-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32853. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29723

[jira] [Commented] (SPARK-32838) Connot overwite different partition with same table

2020-09-11 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194232#comment-17194232 ] Rohit Mishra commented on SPARK-32838: -- [~CHENXCHEN], Please refrain from marking priority as

[jira] [Updated] (SPARK-32838) Connot overwite different partition with same table

2020-09-11 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32838: - Priority: Major (was: Critical) > Connot overwite different partition with same table >

[jira] [Assigned] (SPARK-32856) Prohibit binary comparisons chain

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32856: Assignee: (was: Apache Spark) > Prohibit binary comparisons chain >

[jira] [Assigned] (SPARK-32856) Prohibit binary comparisons chain

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32856: Assignee: Apache Spark > Prohibit binary comparisons chain >

[jira] [Commented] (SPARK-32856) Prohibit binary comparisons chain

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194204#comment-17194204 ] Apache Spark commented on SPARK-32856: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-32032) Eliminate deprecated poll(long) API calls to avoid infinite wait in driver

2020-09-11 Thread Tomas Bartalos (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194201#comment-17194201 ] Tomas Bartalos commented on SPARK-32032: We've encountered this issue. Kafka started with

[jira] [Created] (SPARK-32856) Prohibit binary comparisons chain

2020-09-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-32856: --- Summary: Prohibit binary comparisons chain Key: SPARK-32856 URL: https://issues.apache.org/jira/browse/SPARK-32856 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32730) Improve LeftSemi SortMergeJoin right side buffering

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194063#comment-17194063 ] Apache Spark commented on SPARK-32730: -- User 'peter-toth' has created a pull request for this

[jira] [Assigned] (SPARK-32855) Improve DPP for some join type do not support broadcast filtering side

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32855: Assignee: Apache Spark > Improve DPP for some join type do not support broadcast

[jira] [Commented] (SPARK-32855) Improve DPP for some join type do not support broadcast filtering side

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194058#comment-17194058 ] Apache Spark commented on SPARK-32855: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32855) Improve DPP for some join type do not support broadcast filtering side

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32855: Assignee: (was: Apache Spark) > Improve DPP for some join type do not support

[jira] [Commented] (SPARK-32096) Improve sorting performance for Spark SQL rank window function​

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194057#comment-17194057 ] Apache Spark commented on SPARK-32096: -- User 'xuzikun2003' has created a pull request for this

[jira] [Assigned] (SPARK-32096) Improve sorting performance for Spark SQL rank window function​

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32096: Assignee: (was: Apache Spark) > Improve sorting performance for Spark SQL rank

[jira] [Commented] (SPARK-32096) Improve sorting performance for Spark SQL rank window function​

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194056#comment-17194056 ] Apache Spark commented on SPARK-32096: -- User 'xuzikun2003' has created a pull request for this

[jira] [Assigned] (SPARK-32096) Improve sorting performance for Spark SQL rank window function​

2020-09-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32096: Assignee: Apache Spark > Improve sorting performance for Spark SQL rank window function​

[jira] [Resolved] (SPARK-32794) Rare corner case error in micro-batch engine with some stateful queries + no-data-batches + V1 streaming sources

2020-09-11 Thread Tathagata Das (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-32794. --- Fix Version/s: (was: 2.4.8) 2.4.7 Resolution: Fixed Issue

[jira] [Created] (SPARK-32855) Improve DPP for some join type do not support broadcast filtering side

2020-09-11 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-32855: --- Summary: Improve DPP for some join type do not support broadcast filtering side Key: SPARK-32855 URL: https://issues.apache.org/jira/browse/SPARK-32855 Project: Spark