[jira] [Updated] (SPARK-31802) Format Java date-time types in Row.jsonValue directly

2020-05-22 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31802: --- Description: Row.jsonValue has to convert incoming Java date/timestamps types to days/microseconds

[jira] [Updated] (SPARK-31802) Format Java date-time types in Row.jsonValue directly

2020-05-22 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31802: --- Description: Row.jsonValue has to convert incoming Java date/timestamps types to days/microseconds

[jira] [Commented] (SPARK-21040) On executor/worker decommission consider speculatively re-launching current tasks

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114535#comment-17114535 ] Apache Spark commented on SPARK-21040: -- User 'prakharjain09' has created a pull request for this

[jira] [Created] (SPARK-31802) Format Java date-time types in Row.jsonValue directly

2020-05-22 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31802: -- Summary: Format Java date-time types in Row.jsonValue directly Key: SPARK-31802 URL: https://issues.apache.org/jira/browse/SPARK-31802 Project: Spark Issue

[jira] [Commented] (SPARK-31802) Format Java date-time types in Row.jsonValue directly

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114550#comment-17114550 ] Apache Spark commented on SPARK-31802: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31802) Format Java date-time types in Row.jsonValue directly

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31802: Assignee: Maxim Gekk (was: Apache Spark) > Format Java date-time types in Row.jsonValue

[jira] [Assigned] (SPARK-31802) Format Java date-time types in Row.jsonValue directly

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31802: Assignee: Apache Spark (was: Maxim Gekk) > Format Java date-time types in Row.jsonValue

[jira] [Commented] (SPARK-31802) Format Java date-time types in Row.jsonValue directly

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114551#comment-17114551 ] Apache Spark commented on SPARK-31802: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Updated] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-22 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-31792: - Fix Version/s: (was: 3.0.0) > Introduce the structured streaming UI in the Web UI page >

[jira] [Commented] (SPARK-31612) SQL Reference clean up

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113799#comment-17113799 ] Apache Spark commented on SPARK-31612: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113806#comment-17113806 ] Apache Spark commented on SPARK-31792: -- User 'xccui' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31792: Assignee: (was: Apache Spark) > Introduce the structured streaming UI in the Web UI

[jira] [Resolved] (SPARK-31755) allow missing year/hour when parsing date/timestamp

2020-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31755. -- Fix Version/s: 3.1.0 Resolution: Fixed Fixed in SPARK-31755 > allow missing year/hour

[jira] [Updated] (SPARK-31793) Reduce the memory usage in file scan location metadata

2020-05-22 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-31793: --- Summary: Reduce the memory usage in file scan location metadata (was: Reduce the memory

[jira] [Updated] (SPARK-31789) cancel

2020-05-22 Thread Dylan Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dylan Yao updated SPARK-31789: -- Summary: cancel (was: SparkSubmitOperator could not get Exit Code after log stream interrupted by

[jira] [Created] (SPARK-31794) Incorrect distribution with repartitionByRange and repartition column expression

2020-05-22 Thread Ramesha Bhatta (Jira)
Ramesha Bhatta created SPARK-31794: -- Summary: Incorrect distribution with repartitionByRange and repartition column expression Key: SPARK-31794 URL: https://issues.apache.org/jira/browse/SPARK-31794

[jira] [Commented] (SPARK-31769) Add support of MDC in spark driver logs.

2020-05-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113874#comment-17113874 ] Wenchen Fan commented on SPARK-31769: - SGTM. We can include driver side information like app id. >

[jira] [Commented] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-22 Thread Xingcan Cui (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113798#comment-17113798 ] Xingcan Cui commented on SPARK-31792: - Ah yes, thanks for the reminder! > Introduce the structured

[jira] [Commented] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113805#comment-17113805 ] Apache Spark commented on SPARK-31792: -- User 'xccui' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31792: Assignee: Apache Spark > Introduce the structured streaming UI in the Web UI page >

[jira] [Commented] (SPARK-31755) allow missing year/hour when parsing date/timestamp

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113835#comment-17113835 ] Apache Spark commented on SPARK-31755: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-31757) Improve HistoryServerDiskManager.updateAccessTime()

2020-05-22 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113794#comment-17113794 ] Xingbo Jiang commented on SPARK-31757: -- Fixed by https://github.com/apache/spark/pull/28579 >

[jira] [Resolved] (SPARK-31757) Improve HistoryServerDiskManager.updateAccessTime()

2020-05-22 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-31757. -- Assignee: Xingbo Jiang Resolution: Fixed > Improve

[jira] [Assigned] (SPARK-31793) Reduce the memory usage in file scan location metadata

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31793: Assignee: Apache Spark (was: Gengliang Wang) > Reduce the memory usage in file scan

[jira] [Assigned] (SPARK-31793) Reduce the memory usage in file scan location metadata

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31793: Assignee: Gengliang Wang (was: Apache Spark) > Reduce the memory usage in file scan

[jira] [Commented] (SPARK-31793) Reduce the memory usage in file scan location metadata

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113817#comment-17113817 ] Apache Spark commented on SPARK-31793: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-31793) Reduce the memory usage in file scan location metadata

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113818#comment-17113818 ] Apache Spark commented on SPARK-31793: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-31755) allow missing year/hour when parsing date/timestamp

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113838#comment-17113838 ] Apache Spark commented on SPARK-31755: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-24634) Add a new metric regarding number of rows later than watermark

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113777#comment-17113777 ] Apache Spark commented on SPARK-24634: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Created] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-22 Thread Xingcan Cui (Jira)
Xingcan Cui created SPARK-31792: --- Summary: Introduce the structured streaming UI in the Web UI page Key: SPARK-31792 URL: https://issues.apache.org/jira/browse/SPARK-31792 Project: Spark Issue

[jira] [Commented] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-22 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113791#comment-17113791 ] Jungtaek Lim commented on SPARK-31792: -- Fix version is for tracking the version which contains the

[jira] [Updated] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

2020-05-22 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-31784: - Affects Version/s: (was: 3.1.0) 3.0.0 > Fix test

[jira] [Resolved] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

2020-05-22 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-31784. -- Fix Version/s: 3.0.0 Target Version/s: 3.0.0 Assignee: wuyi

[jira] [Commented] (SPARK-31784) Fix test BarrierTaskContextSuite."share messages with allGather() call"

2020-05-22 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113796#comment-17113796 ] Xingbo Jiang commented on SPARK-31784: -- Fixed by https://github.com/apache/spark/pull/28596 > Fix

[jira] [Created] (SPARK-31793) Reduce the memory usage in data source scan metadata

2020-05-22 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-31793: -- Summary: Reduce the memory usage in data source scan metadata Key: SPARK-31793 URL: https://issues.apache.org/jira/browse/SPARK-31793 Project: Spark

[jira] [Created] (SPARK-31795) Stream Data with API to ServiceNow

2020-05-22 Thread Dominic Wetenkamp (Jira)
Dominic Wetenkamp created SPARK-31795: - Summary: Stream Data with API to ServiceNow Key: SPARK-31795 URL: https://issues.apache.org/jira/browse/SPARK-31795 Project: Spark Issue Type:

[jira] [Commented] (SPARK-26631) Issue while reading Parquet data from Hadoop Archive files (.har)

2020-05-22 Thread Jiri Humpolicek (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114013#comment-17114013 ] Jiri Humpolicek commented on SPARK-26631: - Currently I tried same thing with same result, it

[jira] [Resolved] (SPARK-31790) cast scenarios may generate different results between Hive and Spark

2020-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31790. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28605

[jira] [Assigned] (SPARK-31790) cast scenarios may generate different results between Hive and Spark

2020-05-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31790: Assignee: philipse > cast scenarios may generate different results between Hive and

[jira] [Commented] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-05-22 Thread Geet Bhalshankar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113909#comment-17113909 ] Geet Bhalshankar commented on SPARK-31337: -- [~gsomogyi]  by any chance if this is solved ,

[jira] [Commented] (SPARK-21040) On executor/worker decommission consider speculatively re-launching current tasks

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114534#comment-17114534 ] Apache Spark commented on SPARK-21040: -- User 'prakharjain09' has created a pull request for this

[jira] [Assigned] (SPARK-21040) On executor/worker decommission consider speculatively re-launching current tasks

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21040: Assignee: Apache Spark > On executor/worker decommission consider speculatively

[jira] [Assigned] (SPARK-21040) On executor/worker decommission consider speculatively re-launching current tasks

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21040: Assignee: (was: Apache Spark) > On executor/worker decommission consider

[jira] [Resolved] (SPARK-31756) Add real headless browser support for UI test

2020-05-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31756. -- Fix Version/s: 3.1.0 Resolution: Fixed Resolved by

[jira] [Commented] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114117#comment-17114117 ] Apache Spark commented on SPARK-31797: -- User 'TJX2014' has created a pull request for this issue:

[jira] [Created] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread JinxinTang (Jira)
JinxinTang created SPARK-31797: -- Summary: Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions Key: SPARK-31797 URL: https://issues.apache.org/jira/browse/SPARK-31797 Project: Spark

[jira] [Assigned] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31797: Assignee: Apache Spark > Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS

[jira] [Assigned] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31797: Assignee: (was: Apache Spark) > Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and

[jira] [Commented] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114116#comment-17114116 ] Apache Spark commented on SPARK-31797: -- User 'TJX2014' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31797: --- Assignee: JinxinTang > Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS

[jira] [Updated] (SPARK-31756) Add real headless browser support for UI test

2020-05-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-31756: - Component/s: Tests Priority: Minor (was: Major) > Add real headless browser support for

[jira] [Updated] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31797: --- Description: 1.Add and register three new functions: {{TIMESTAMP_SECONDS}},  {{TIMESTAMP_MILLIS}} 

[jira] [Created] (SPARK-31796) Download or Install Spark On windows

2020-05-22 Thread Jielan123 (Jira)
Jielan123 created SPARK-31796: - Summary: Download or Install Spark On windows Key: SPARK-31796 URL: https://issues.apache.org/jira/browse/SPARK-31796 Project: Spark Issue Type: Question

[jira] [Updated] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31797: --- Description: (was: ### What changes were proposed in this pull request? Add and register three

[jira] [Commented] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114139#comment-17114139 ] Wenchen Fan commented on SPARK-31797: - resolved by https://github.com/apache/spark/pull/28534 >

[jira] [Commented] (SPARK-31694) Add SupportsPartitions Catalog APIs on DataSourceV2

2020-05-22 Thread Jackey Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114467#comment-17114467 ] Jackey Lee commented on SPARK-31694: Hi [~younggyuchun] Thank you for your attention Basically, a

[jira] [Commented] (SPARK-31694) Add SupportsPartitions Catalog APIs on DataSourceV2

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114473#comment-17114473 ] Apache Spark commented on SPARK-31694: -- User 'stczwd' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31791) Improve cache block migration test reliability

2020-05-22 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau reassigned SPARK-31791: Assignee: Holden Karau > Improve cache block migration test reliability >

[jira] [Assigned] (SPARK-31694) Add SupportsPartitions Catalog APIs on DataSourceV2

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31694: Assignee: (was: Apache Spark) > Add SupportsPartitions Catalog APIs on DataSourceV2

[jira] [Assigned] (SPARK-31694) Add SupportsPartitions Catalog APIs on DataSourceV2

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31694: Assignee: Apache Spark > Add SupportsPartitions Catalog APIs on DataSourceV2 >

[jira] [Commented] (SPARK-31694) Add SupportsPartitions Catalog APIs on DataSourceV2

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114472#comment-17114472 ] Apache Spark commented on SPARK-31694: -- User 'stczwd' has created a pull request for this issue:

[jira] [Updated] (SPARK-31791) Improve cache block migration test reliability

2020-05-22 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-31791: - Fix Version/s: 3.1.0 > Improve cache block migration test reliability >

[jira] [Resolved] (SPARK-31791) Improve cache block migration test reliability

2020-05-22 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-31791. -- Resolution: Fixed > Improve cache block migration test reliability >

[jira] [Created] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
Anoop Johnson created SPARK-31799: - Summary: Spark Datasource Tables Creating Incorrect Hive Metadata Key: SPARK-31799 URL: https://issues.apache.org/jira/browse/SPARK-31799 Project: Spark

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: If you create a CSV or JSON table using Spark SQL, it writes the  wrong Hive

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: If you create a CSV or JSON table using Spark SQL, it writes the  wrong Hive

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: If you create a CSV or JSON table using Spark SQL, it writes the wrong Hive

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: If you create a CSV or JSON table using Spark SQL, it writes the  wrong Hive

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: If you create a CSV or JSON table using Spark SQL, it writes the wrong Hive

[jira] [Created] (SPARK-31800) Unable to disable Kerberos when submitting jobs to Kubernetes

2020-05-22 Thread James Boylan (Jira)
James Boylan created SPARK-31800: Summary: Unable to disable Kerberos when submitting jobs to Kubernetes Key: SPARK-31800 URL: https://issues.apache.org/jira/browse/SPARK-31800 Project: Spark

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: If you create a CSV or JSON table using Spark SQL, it writes the wrong Hive

[jira] [Created] (SPARK-31801) Register shuffle map output metadata with a shuffle output tracker

2020-05-22 Thread Matt Cheah (Jira)
Matt Cheah created SPARK-31801: -- Summary: Register shuffle map output metadata with a shuffle output tracker Key: SPARK-31801 URL: https://issues.apache.org/jira/browse/SPARK-31801 Project: Spark

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: If you create a CSV or JSON table using Spark SQL, it writes the wrong Hive

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: If you create a CSV or JSON table using Spark SQL, it writes the wrong Hive

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: I found that if I create a CSV or JSON table using Spark SQL, it writes the

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: I found that if I create a CSV or JSON table using Spark SQL, it writes the

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: I found that if I create a CSV or JSON table using Spark SQL, it writes the

[jira] [Updated] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-05-22 Thread Anoop Johnson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Johnson updated SPARK-31799: -- Description: If you create a CSV or JSON table using Spark SQL, it writes the wrong Hive

[jira] [Assigned] (SPARK-31801) Register shuffle map output metadata with a shuffle output tracker

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31801: Assignee: Apache Spark > Register shuffle map output metadata with a shuffle output

[jira] [Assigned] (SPARK-31801) Register shuffle map output metadata with a shuffle output tracker

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31801: Assignee: (was: Apache Spark) > Register shuffle map output metadata with a shuffle

[jira] [Commented] (SPARK-31801) Register shuffle map output metadata with a shuffle output tracker

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114477#comment-17114477 ] Apache Spark commented on SPARK-31801: -- User 'mccheah' has created a pull request for this issue:

[jira] [Commented] (SPARK-31786) Exception on submitting Spark-Pi to Kubernetes 1.17.3

2020-05-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114201#comment-17114201 ] Dongjoon Hyun commented on SPARK-31786: --- Thank you for the addition info, [~felixzheng]. >

[jira] [Updated] (SPARK-31770) spark double count issue when table is recreated from hive

2020-05-22 Thread Nithin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nithin updated SPARK-31770: --- Affects Version/s: 2.4.5 > spark double count issue when table is recreated from hive >

[jira] [Updated] (SPARK-31751) spark serde property path overwrites table property location

2020-05-22 Thread Nithin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nithin updated SPARK-31751: --- Affects Version/s: 2.4.5 > spark serde property path overwrites table property location >

[jira] [Commented] (SPARK-31755) allow missing year/hour when parsing date/timestamp

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114212#comment-17114212 ] Apache Spark commented on SPARK-31755: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-31755) allow missing year/hour when parsing date/timestamp

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114211#comment-17114211 ] Apache Spark commented on SPARK-31755: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31778) Support cross-building docker images

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31778: Assignee: Apache Spark > Support cross-building docker images >

[jira] [Assigned] (SPARK-31778) Support cross-building docker images

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31778: Assignee: (was: Apache Spark) > Support cross-building docker images >

[jira] [Commented] (SPARK-31778) Support cross-building docker images

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114250#comment-17114250 ] Apache Spark commented on SPARK-31778: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-31696) Support spark.kubernetes.driver.service.annotation

2020-05-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114203#comment-17114203 ] Dongjoon Hyun commented on SPARK-31696: --- You can choose one of pod or service annotations or use

[jira] [Assigned] (SPARK-31791) Improve cache block migration test reliability

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31791: Assignee: Apache Spark > Improve cache block migration test reliability >

[jira] [Assigned] (SPARK-31791) Improve cache block migration test reliability

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31791: Assignee: (was: Apache Spark) > Improve cache block migration test reliability >

[jira] [Commented] (SPARK-31791) Improve cache block migration test reliability

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114245#comment-17114245 ] Apache Spark commented on SPARK-31791: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31798) Return map output metadata from shuffle writers

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31798: Assignee: Apache Spark > Return map output metadata from shuffle writers >

[jira] [Commented] (SPARK-31798) Return map output metadata from shuffle writers

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114374#comment-17114374 ] Apache Spark commented on SPARK-31798: -- User 'mccheah' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31798) Return map output metadata from shuffle writers

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31798: Assignee: (was: Apache Spark) > Return map output metadata from shuffle writers >

[jira] [Commented] (SPARK-31798) Return map output metadata from shuffle writers

2020-05-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114373#comment-17114373 ] Apache Spark commented on SPARK-31798: -- User 'mccheah' has created a pull request for this issue:

[jira] [Updated] (SPARK-31798) Return map output metadata from shuffle writers

2020-05-22 Thread Matt Cheah (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-31798: --- Description: Part of the overall sub-push for shuffle metadata management as proposed in [this

[jira] [Created] (SPARK-31798) Return map output metadata from shuffle writers

2020-05-22 Thread Matt Cheah (Jira)
Matt Cheah created SPARK-31798: -- Summary: Return map output metadata from shuffle writers Key: SPARK-31798 URL: https://issues.apache.org/jira/browse/SPARK-31798 Project: Spark Issue Type:

[jira] [Updated] (SPARK-31798) Return map output metadata from shuffle writers

2020-05-22 Thread Matt Cheah (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-31798: --- Description: Part of the overall sub-push for shuffle metadata management as proposed in [this

  1   2   >