[jira] [Commented] (SPARK-32312) Upgrade Apache Arrow to 1.0.0

2020-09-03 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190553#comment-17190553 ] Bryan Cutler commented on SPARK-32312: -- Sorry for the delay, I was holding off for a couple of

[jira] [Commented] (SPARK-32764) compare of -0.0 < 0.0 return true

2020-09-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190543#comment-17190543 ] Dongjoon Hyun commented on SPARK-32764: --- Thank you for analysis, [~sandeep.katta2007]. Also, cc

[jira] [Updated] (SPARK-32795) ApplicationInfo#removedExecutors can cause OOM

2020-09-03 Thread Victor Tso (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Tso updated SPARK-32795: --- Description: !image-2020-09-03-23-27-11-809.png! In my case, the Standalone Spark master process

[jira] [Updated] (SPARK-32795) ApplicationInfo#removedExecutors can cause OOM

2020-09-03 Thread Victor Tso (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Tso updated SPARK-32795: --- Description: !image-2020-09-03-23-27-11-809.png|width=840,height=439! In my case, the Standalone

[jira] [Updated] (SPARK-32795) ApplicationInfo#removedExecutors can cause OOM

2020-09-03 Thread Victor Tso (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Tso updated SPARK-32795: --- Attachment: image-2020-09-03-23-27-11-809.png > ApplicationInfo#removedExecutors can cause OOM >

[jira] [Created] (SPARK-32795) ApplicationInfo#removedExecutors can cause OOM

2020-09-03 Thread Victor Tso (Jira)
Victor Tso created SPARK-32795: -- Summary: ApplicationInfo#removedExecutors can cause OOM Key: SPARK-32795 URL: https://issues.apache.org/jira/browse/SPARK-32795 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-32778) Accidental Data Deletion on calling saveAsTable

2020-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32778. -- Resolution: Incomplete Leaving this resolved for now since 2.2.0 is EOL and we won't land a

[jira] [Commented] (SPARK-32793) Expose assert_true in Python/Scala APIs and add error message parameter

2020-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190533#comment-17190533 ] Hyukjin Kwon commented on SPARK-32793: -- We'd probably expose this in R side as well > Expose

[jira] [Resolved] (SPARK-29422) test failed: org.apache.spark.deploy.history.HistoryServerSuite ajax rendered relative links are prefixed with uiRoot

2020-09-03 Thread huangtianhua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtianhua resolved SPARK-29422. -- Resolution: Not A Problem > test failed: org.apache.spark.deploy.history.HistoryServerSuite

[jira] [Commented] (SPARK-32598) Not able to see driver logs in spark history server in standalone mode

2020-09-03 Thread Kevin Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190513#comment-17190513 ] Kevin Wang commented on SPARK-32598: Facing this bug too, I'm trying to fix it at 

[jira] [Assigned] (SPARK-32598) Not able to see driver logs in spark history server in standalone mode

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32598: Assignee: Apache Spark > Not able to see driver logs in spark history server in

[jira] [Assigned] (SPARK-32598) Not able to see driver logs in spark history server in standalone mode

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32598: Assignee: (was: Apache Spark) > Not able to see driver logs in spark history server

[jira] [Commented] (SPARK-32598) Not able to see driver logs in spark history server in standalone mode

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190510#comment-17190510 ] Apache Spark commented on SPARK-32598: -- User 'KevinSmile' has created a pull request for this

[jira] [Updated] (SPARK-32186) Development - Debugging

2020-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32186: - Summary: Development - Debugging (was: Developement - Debugging) > Development - Debugging >

[jira] [Updated] (SPARK-32186) Developement - Debugging

2020-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32186: - Summary: Developement - Debugging (was: User Guide - Debugging) > Developement - Debugging >

[jira] [Commented] (SPARK-32603) CREATE/REPLACE TABLE AS SELECT not support multi-part identifiers

2020-09-03 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190480#comment-17190480 ] angerszhu commented on SPARK-32603: --- In current grammar {code:java} | replaceTableHeader ('('

[jira] [Commented] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called by user program

2020-09-03 Thread Victor Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190470#comment-17190470 ] Victor Wong commented on SPARK-24981: - My mistake:D, thanks for reply. > ShutdownHook timeout

[jira] [Updated] (SPARK-32786) Improve performance for some slow DPP tests

2020-09-03 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32786: - Fix Version/s: 3.0.2 > Improve performance for some slow DPP tests >

[jira] [Resolved] (SPARK-32762) Enhance the verification of sql-expression-schema.md

2020-09-03 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32762. -- Fix Version/s: 3.1.0 Assignee: Yang Jie Resolution: Fixed Resolved by 

[jira] [Resolved] (SPARK-32786) Improve performance for some slow DPP tests

2020-09-03 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32786. -- Fix Version/s: 3.1.0 Assignee: Zhenhua Wang Resolution: Fixed

[jira] [Commented] (SPARK-32603) CREATE/REPLACE TABLE AS SELECT not support multi-part identifiers

2020-09-03 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190450#comment-17190450 ] Huaxin Gao commented on SPARK-32603: [~angerszhuuu] I opened this ticket when working on this PR

[jira] [Assigned] (SPARK-32767) Bucket join should work if spark.sql.shuffle.partitions larger than bucket number

2020-09-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-32767: --- Assignee: Yuming Wang > Bucket join should work if spark.sql.shuffle.partitions larger

[jira] [Resolved] (SPARK-32767) Bucket join should work if spark.sql.shuffle.partitions larger than bucket number

2020-09-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-32767. - Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-32794) Rare corner case error in micro-batch engine with some stateful queries + no-data-batches + V1 streaming sources

2020-09-03 Thread Tathagata Das (Jira)
Tathagata Das created SPARK-32794: - Summary: Rare corner case error in micro-batch engine with some stateful queries + no-data-batches + V1 streaming sources Key: SPARK-32794 URL:

[jira] [Updated] (SPARK-32793) Expose assert_true in Python/Scala APIs and add error message parameter

2020-09-03 Thread Karen Feng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Feng updated SPARK-32793: --- Fix Version/s: (was: 3.1.0) > Expose assert_true in Python/Scala APIs and add error message

[jira] [Created] (SPARK-32793) Expose assert_true in Python/Scala APIs and add error message parameter

2020-09-03 Thread Karen Feng (Jira)
Karen Feng created SPARK-32793: -- Summary: Expose assert_true in Python/Scala APIs and add error message parameter Key: SPARK-32793 URL: https://issues.apache.org/jira/browse/SPARK-32793 Project: Spark

[jira] [Commented] (SPARK-32638) WidenSetOperationTypes in subquery attribute missing

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190341#comment-17190341 ] Apache Spark commented on SPARK-32638: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-32638) WidenSetOperationTypes in subquery attribute missing

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190340#comment-17190340 ] Apache Spark commented on SPARK-32638: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called by user program

2020-09-03 Thread Hieu Tri Huynh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190336#comment-17190336 ] Hieu Tri Huynh commented on SPARK-24981: As far as I know, InterruptedException is considered as

[jira] [Commented] (SPARK-32764) compare of -0.0 < 0.0 return true

2020-09-03 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190333#comment-17190333 ] Sandeep Katta commented on SPARK-32764: --- So it's edge case scenario, how should we fix it or we

[jira] [Comment Edited] (SPARK-32764) compare of -0.0 < 0.0 return true

2020-09-03 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190331#comment-17190331 ] Sandeep Katta edited comment on SPARK-32764 at 9/3/20, 5:47 PM:

[jira] [Comment Edited] (SPARK-32764) compare of -0.0 < 0.0 return true

2020-09-03 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190331#comment-17190331 ] Sandeep Katta edited comment on SPARK-32764 at 9/3/20, 5:45 PM:

[jira] [Commented] (SPARK-32764) compare of -0.0 < 0.0 return true

2020-09-03 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190331#comment-17190331 ] Sandeep Katta commented on SPARK-32764: --- [~cloud_fan] and [~smilegator] I did some analysis on

[jira] [Comment Edited] (SPARK-28955) Support for LocalDateTime semantics

2020-09-03 Thread Bill Schneider (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190322#comment-17190322 ] Bill Schneider edited comment on SPARK-28955 at 9/3/20, 5:25 PM: -

[jira] [Updated] (SPARK-28955) Support for LocalDateTime semantics

2020-09-03 Thread Bill Schneider (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Schneider updated SPARK-28955: --- Issue Type: New Feature (was: Wish) Changing to new feature; this is a recurring issue

[jira] [Assigned] (SPARK-32792) Improve in filter pushdown for ParquetFilters

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32792: Assignee: (was: Apache Spark) > Improve in filter pushdown for ParquetFilters >

[jira] [Commented] (SPARK-32792) Improve in filter pushdown for ParquetFilters

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190300#comment-17190300 ] Apache Spark commented on SPARK-32792: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32792) Improve in filter pushdown for ParquetFilters

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32792: Assignee: Apache Spark > Improve in filter pushdown for ParquetFilters >

[jira] [Created] (SPARK-32792) Improve in filter pushdown for ParquetFilters

2020-09-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-32792: --- Summary: Improve in filter pushdown for ParquetFilters Key: SPARK-32792 URL: https://issues.apache.org/jira/browse/SPARK-32792 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-32791) non-partitioned table metric should not have dynamic partition pruning time

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32791: Assignee: (was: Apache Spark) > non-partitioned table metric should not have dynamic

[jira] [Commented] (SPARK-32791) non-partitioned table metric should not have dynamic partition pruning time

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190283#comment-17190283 ] Apache Spark commented on SPARK-32791: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32791) non-partitioned table metric should not have dynamic partition pruning time

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32791: Assignee: Apache Spark > non-partitioned table metric should not have dynamic partition

[jira] [Updated] (SPARK-32791) non-partitioned table metric should not have dynamic partition pruning time

2020-09-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-32791: Attachment: image-2020-09-04-00-22-07-944.png > non-partitioned table metric should not have

[jira] [Updated] (SPARK-32791) non-partitioned table metric should not have dynamic partition pruning time

2020-09-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-32791: Description: default.t3 is not a partition table, but it has dynamic partition pruning time:

[jira] [Created] (SPARK-32791) non-partitioned table metric should not have dynamic partition pruning time

2020-09-03 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-32791: --- Summary: non-partitioned table metric should not have dynamic partition pruning time Key: SPARK-32791 URL: https://issues.apache.org/jira/browse/SPARK-32791 Project:

[jira] [Commented] (SPARK-32687) CostBasedJoinReorder should have deterministic Optimization result

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190256#comment-17190256 ] Apache Spark commented on SPARK-32687: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-32687) CostBasedJoinReorder should have deterministic Optimization result

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32687: Assignee: (was: Apache Spark) > CostBasedJoinReorder should have deterministic

[jira] [Assigned] (SPARK-32687) CostBasedJoinReorder should have deterministic Optimization result

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32687: Assignee: Apache Spark > CostBasedJoinReorder should have deterministic Optimization

[jira] [Commented] (SPARK-32687) CostBasedJoinReorder should have deterministic Optimization result

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190255#comment-17190255 ] Apache Spark commented on SPARK-32687: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-32180) Getting Started - Installation

2020-09-03 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190247#comment-17190247 ] Rohit Mishra commented on SPARK-32180: -- Hi [~hyukjin.kwon], I have created a new pull request as

[jira] [Updated] (SPARK-32779) Spark/Hive3 interaction potentially causes deadlock

2020-09-03 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-32779: -- Summary: Spark/Hive3 interaction potentially causes deadlock (was: Spark/Hive3 HMS

[jira] [Resolved] (SPARK-32788) non-partitioned table scan should not have partition filter

2020-09-03 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-32788. - Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-32180) Getting Started - Installation

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190225#comment-17190225 ] Apache Spark commented on SPARK-32180: -- User 'rohitmishr1484' has created a pull request for this

[jira] [Commented] (SPARK-32180) Getting Started - Installation

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190224#comment-17190224 ] Apache Spark commented on SPARK-32180: -- User 'rohitmishr1484' has created a pull request for this

[jira] [Created] (SPARK-32790) FINISHED state of application is not final

2020-09-03 Thread Rosie Bloxsom (Jira)
Rosie Bloxsom created SPARK-32790: - Summary: FINISHED state of application is not final Key: SPARK-32790 URL: https://issues.apache.org/jira/browse/SPARK-32790 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-32779) Spark/Hive3 HMS interaction potentially causes deadlock

2020-09-03 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-32779: -- Summary: Spark/Hive3 HMS interaction potentially causes deadlock (was: Spark/Hive3

[jira] [Commented] (SPARK-32323) Javascript/HTML bug in spark application UI

2020-09-03 Thread Ihor Bobak (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190193#comment-17190193 ] Ihor Bobak commented on SPARK-32323: I logged on here from another PC, and I see that the image is

[jira] [Resolved] (SPARK-32638) WidenSetOperationTypes in subquery attribute missing

2020-09-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32638. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29485

[jira] [Assigned] (SPARK-32638) WidenSetOperationTypes in subquery attribute missing

2020-09-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32638: --- Assignee: Takeshi Yamamuro > WidenSetOperationTypes in subquery attribute missing >

[jira] [Assigned] (SPARK-25200) Allow setting HADOOP_CONF_DIR as a spark property

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25200: Assignee: Apache Spark > Allow setting HADOOP_CONF_DIR as a spark property >

[jira] [Assigned] (SPARK-25200) Allow setting HADOOP_CONF_DIR as a spark property

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25200: Assignee: (was: Apache Spark) > Allow setting HADOOP_CONF_DIR as a spark property >

[jira] [Resolved] (SPARK-32730) Improve LeftSemi SortMergeJoin right side buffering

2020-09-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32730. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29572

[jira] [Assigned] (SPARK-32730) Improve LeftSemi SortMergeJoin right side buffering

2020-09-03 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32730: --- Assignee: Peter Toth > Improve LeftSemi SortMergeJoin right side buffering >

[jira] [Commented] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called by user program

2020-09-03 Thread Victor Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190153#comment-17190153 ] Victor Wong commented on SPARK-24981: - `ERROR Utils: Uncaught exception in thread pool-4-thread-1`

[jira] [Assigned] (SPARK-32186) User Guide - Debugging

2020-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32186: Assignee: Hyukjin Kwon > User Guide - Debugging > -- > >

[jira] [Assigned] (SPARK-32186) User Guide - Debugging

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32186: Assignee: Apache Spark > User Guide - Debugging > -- > >

[jira] [Commented] (SPARK-32186) User Guide - Debugging

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190114#comment-17190114 ] Apache Spark commented on SPARK-32186: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-32186) User Guide - Debugging

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32186: Assignee: (was: Apache Spark) > User Guide - Debugging > -- > >

[jira] [Commented] (SPARK-32783) Development - Testing PySpark

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190100#comment-17190100 ] Apache Spark commented on SPARK-32783: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-32755) Maintain the order of expressions in AttributeSet and ExpressionSet

2020-09-03 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-32755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-32755. --- Fix Version/s: 3.1.0 Assignee: Ali Afroozeh Resolution: Fixed >

[jira] [Updated] (SPARK-32789) Wildcards not working in get_json_object

2020-09-03 Thread Thomas Uhren (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Uhren updated SPARK-32789: - Attachment: image-2020-09-03-13-22-38-569.png > Wildcards not working in get_json_object >

[jira] [Created] (SPARK-32789) Wildcards not working in get_json_object

2020-09-03 Thread Thomas Uhren (Jira)
Thomas Uhren created SPARK-32789: Summary: Wildcards not working in get_json_object Key: SPARK-32789 URL: https://issues.apache.org/jira/browse/SPARK-32789 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-32789) Wildcards not working in get_json_object

2020-09-03 Thread Thomas Uhren (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Uhren updated SPARK-32789: - Description: It seems that wildcards (star) are not supported when using {{get_json_object}}:

[jira] [Updated] (SPARK-32786) Improve performance for some slow DPP tests

2020-09-03 Thread Zhenhua Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-32786: - Description: The whole `DynamicPartitionPruningSuite` takes about 2 min on my laptop (either

[jira] [Updated] (SPARK-32186) User Guide - Debugging

2020-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32186: - Description: 1. Python Profiler:

[jira] [Commented] (SPARK-32788) non-partitioned table scan should not have partition filter

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190044#comment-17190044 ] Apache Spark commented on SPARK-32788: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-32788) non-partitioned table scan should not have partition filter

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190041#comment-17190041 ] Apache Spark commented on SPARK-32788: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32788) non-partitioned table scan should not have partition filter

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32788: Assignee: Wenchen Fan (was: Apache Spark) > non-partitioned table scan should not have

[jira] [Assigned] (SPARK-32788) non-partitioned table scan should not have partition filter

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32788: Assignee: Apache Spark (was: Wenchen Fan) > non-partitioned table scan should not have

[jira] [Created] (SPARK-32788) non-partitioned table scan should not have partition filter

2020-09-03 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-32788: --- Summary: non-partitioned table scan should not have partition filter Key: SPARK-32788 URL: https://issues.apache.org/jira/browse/SPARK-32788 Project: Spark

[jira] [Resolved] (SPARK-32181) Getting Started - Package Overview

2020-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32181. -- Resolution: Duplicate This is basically done at SPARK-32507 > Getting Started - Package

[jira] [Updated] (SPARK-32786) Improve performance for some slow DPP tests

2020-09-03 Thread Zhenhua Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-32786: - Description: The whole `DynamicPartitionPruningSuite` takes about 2 min in my laptop (either

[jira] [Assigned] (SPARK-32786) Improve performance for some slow DPP tests

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32786: Assignee: Apache Spark > Improve performance for some slow DPP tests >

[jira] [Assigned] (SPARK-32786) Improve performance for some slow DPP tests

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32786: Assignee: (was: Apache Spark) > Improve performance for some slow DPP tests >

[jira] [Commented] (SPARK-32786) Improve performance for some slow DPP tests

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190017#comment-17190017 ] Apache Spark commented on SPARK-32786: -- User 'wzhfy' has created a pull request for this issue:

[jira] [Created] (SPARK-32786) Improve performance for some slow DPP tests

2020-09-03 Thread Zhenhua Wang (Jira)
Zhenhua Wang created SPARK-32786: Summary: Improve performance for some slow DPP tests Key: SPARK-32786 URL: https://issues.apache.org/jira/browse/SPARK-32786 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32779) Spark/Hive3 interaction potentially causes deadlock

2020-09-03 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190014#comment-17190014 ] JinxinTang commented on SPARK-32779: Hi [~bersprockets], Seems spark cannot work with hive-3.1.x

[jira] [Commented] (SPARK-32785) interval with dangling part should not results null

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189998#comment-17189998 ] Apache Spark commented on SPARK-32785: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32785) interval with dangling part should not results null

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32785: Assignee: (was: Apache Spark) > interval with dangling part should not results null

[jira] [Assigned] (SPARK-32785) interval with dangling part should not results null

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32785: Assignee: Apache Spark > interval with dangling part should not results null >

[jira] [Created] (SPARK-32785) interval with dangling part should not results null

2020-09-03 Thread Kent Yao (Jira)
Kent Yao created SPARK-32785: Summary: interval with dangling part should not results null Key: SPARK-32785 URL: https://issues.apache.org/jira/browse/SPARK-32785 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-32783) Development - Testing PySpark

2020-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32783: Assignee: Hyukjin Kwon > Development - Testing PySpark > - >

[jira] [Commented] (SPARK-32783) Development - Testing PySpark

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189935#comment-17189935 ] Apache Spark commented on SPARK-32783: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-32783) Development - Testing PySpark

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189932#comment-17189932 ] Apache Spark commented on SPARK-32783: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-32783) Development - Testing PySpark

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32783: Assignee: (was: Apache Spark) > Development - Testing PySpark >

[jira] [Assigned] (SPARK-32783) Development - Testing PySpark

2020-09-03 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32783: Assignee: Apache Spark > Development - Testing PySpark > - >

[jira] [Commented] (SPARK-32354) Fix & re-enable failing R K8s tests

2020-09-03 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189918#comment-17189918 ] Hyukjin Kwon commented on SPARK-32354: -- [~holden]can you share the Jenkins link or error messages?

[jira] [Comment Edited] (SPARK-32135) Show Spark Driver name on Spark history web page

2020-09-03 Thread Gaurangi Saxena (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189668#comment-17189668 ] Gaurangi Saxena edited comment on SPARK-32135 at 9/3/20, 7:40 AM: --

[jira] [Commented] (SPARK-32603) CREATE/REPLACE TABLE AS SELECT not support multi-part identifiers

2020-09-03 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189902#comment-17189902 ] angerszhu commented on SPARK-32603: --- | replaceTableHeader ('(' colTypeList ')')? tableProvider

[jira] [Updated] (SPARK-32603) CREATE/REPLACE TABLE AS SELECT not support multi-part identifiers

2020-09-03 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32603: -- Description: {code:java} == SQL == CREATE TABLE h2.test.abc AS SELECT * FROM h2.test.people ^^^

  1   2   >