[jira] [Updated] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30951: Priority: Blocker (was: Major) > Potential data loss for legacy applications after switch to proleptic

[jira] [Updated] (SPARK-31102) spark-sql fails to parse when contains comment

[ https://issues.apache.org/jira/browse/SPARK-31102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31102: Labels: (was: 3.0.0) > spark-sql fails to parse when contains comment >

[jira] [Updated] (SPARK-31102) spark-sql fails to parse when contains comment

[ https://issues.apache.org/jira/browse/SPARK-31102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31102: Labels: 3.0.0 (was: ) > spark-sql fails to parse when contains comment >

[jira] [Commented] (SPARK-30707) Lead/Lag window function throws AnalysisException without ORDER BY clause

[ https://issues.apache.org/jira/browse/SPARK-30707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1709#comment-1709 ] angerszhu commented on SPARK-30707: --- our production meet this problem too when hive sql run in spark

[jira] [Commented] (SPARK-31102) spark-sql fails to parse when contains comment

[ https://issues.apache.org/jira/browse/SPARK-31102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055546#comment-17055546 ] Yuming Wang commented on SPARK-31102: - cc [~javier_ivanov] > spark-sql fails to parse when contains

[jira] [Created] (SPARK-31102) spark-sql fails to parse when contains comment

Yuming Wang created SPARK-31102: --- Summary: spark-sql fails to parse when contains comment Key: SPARK-31102 URL: https://issues.apache.org/jira/browse/SPARK-31102 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-31101) Upgrade Janino to 3.1.1

Jungtaek Lim created SPARK-31101: Summary: Upgrade Janino to 3.1.1 Key: SPARK-31101 URL: https://issues.apache.org/jira/browse/SPARK-31101 Project: Spark Issue Type: Dependency upgrade

[jira] [Updated] (SPARK-31010) forbid untyped scala UDF API by default

[ https://issues.apache.org/jira/browse/SPARK-31010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31010: Labels: release-notes (was: ) > forbid untyped scala UDF API by default >

[jira] [Updated] (SPARK-31100) Detect namespace existence when setting namespace

[ https://issues.apache.org/jira/browse/SPARK-31100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee updated SPARK-31100: --- Description: We should check if the namespace exists while calling "use namespace", and throw

[jira] [Created] (SPARK-31100) Detect namespace existence when setting namespace

Jackey Lee created SPARK-31100: -- Summary: Detect namespace existence when setting namespace Key: SPARK-31100 URL: https://issues.apache.org/jira/browse/SPARK-31100 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31072) Default to ParquetOutputCommitter even after configuring s3a committer as "partitioned"

[ https://issues.apache.org/jira/browse/SPARK-31072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055435#comment-17055435 ] Felix Kizhakkel Jose commented on SPARK-31072: -- Could you please provide some insights? >

[jira] [Commented] (SPARK-31099) Create migration script for metastore_db

[ https://issues.apache.org/jira/browse/SPARK-31099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055413#comment-17055413 ] Gengliang Wang commented on SPARK-31099: cc [~yumwang] > Create migration script for

[jira] [Created] (SPARK-31099) Create migration script for metastore_db

Gengliang Wang created SPARK-31099: -- Summary: Create migration script for metastore_db Key: SPARK-31099 URL: https://issues.apache.org/jira/browse/SPARK-31099 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055404#comment-17055404 ] Michael Heuer commented on SPARK-31097: --- Ah, don't worry about it. My comment was only meant to

[jira] [Commented] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055398#comment-17055398 ] Shane Knapp commented on SPARK-31097: - it seems that i am unable to change the close reason, or

[jira] [Updated] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp updated SPARK-31097: Issue Type: Bug (was: Task) > Expired SSL cert on Spark 2.4.5 download >

[jira] [Commented] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055396#comment-17055396 ] Michael Heuer commented on SPARK-31097: --- Thank you, [~shaneknapp]! > Expired SSL cert on Spark

[jira] [Closed] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp closed SPARK-31097. --- > Expired SSL cert on Spark 2.4.5 download > > >

[jira] [Commented] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055393#comment-17055393 ] Michael Heuer commented on SPARK-31097: --- Please close as CannotReproduce.   > Expired SSL cert

[jira] [Commented] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055392#comment-17055392 ] Shane Knapp commented on SPARK-31097: - i am unable to reproduce this failure, and curl's verbose

[jira] [Assigned] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp reassigned SPARK-31097: --- Assignee: Shane Knapp > Expired SSL cert on Spark 2.4.5 download >

[jira] [Resolved] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp resolved SPARK-31097. - Resolution: Fixed > Expired SSL cert on Spark 2.4.5 download >

[jira] [Commented] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055360#comment-17055360 ] Shane Knapp commented on SPARK-31097: - same worker, same curl command, different build:

[jira] [Comment Edited] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055337#comment-17055337 ] Shane Knapp edited comment on SPARK-31097 at 3/9/20, 8:13 PM: -- odd.  i

[jira] [Comment Edited] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055337#comment-17055337 ] Shane Knapp edited comment on SPARK-31097 at 3/9/20, 8:12 PM: -- odd.  i

[jira] [Commented] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055339#comment-17055339 ] Michael Heuer commented on SPARK-31097: --- Will do, thank you for your help, [~shaneknapp]   >

[jira] [Comment Edited] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055337#comment-17055337 ] Shane Knapp edited comment on SPARK-31097 at 3/9/20, 8:08 PM: -- odd.  i

[jira] [Commented] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

[ https://issues.apache.org/jira/browse/SPARK-31097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055337#comment-17055337 ] Shane Knapp commented on SPARK-31097: - odd.  i replicated the environment that jenkins uses when

[jira] [Commented] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-31098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055336#comment-17055336 ] Gengliang Wang commented on SPARK-31098: cc [~dongjoon] Do you have any idea of this? >

[jira] [Updated] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-31098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-31098: --- Attachment: files.tar > Reading ORC files throws IndexOutOfBoundsException >

[jira] [Created] (SPARK-31098) Reading ORC files throws IndexOutOfBoundsException

Gengliang Wang created SPARK-31098: -- Summary: Reading ORC files throws IndexOutOfBoundsException Key: SPARK-31098 URL: https://issues.apache.org/jira/browse/SPARK-31098 Project: Spark Issue

[jira] [Resolved] (SPARK-31055) Update config docs for shuffle local host reads to have dep on external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-31055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31055. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27812

[jira] [Assigned] (SPARK-31055) Update config docs for shuffle local host reads to have dep on external shuffle service

[ https://issues.apache.org/jira/browse/SPARK-31055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31055: - Assignee: Thomas Graves > Update config docs for shuffle local host reads to have dep

[jira] [Created] (SPARK-31097) Expired SSL cert on Spark 2.4.5 download

Michael Heuer created SPARK-31097: - Summary: Expired SSL cert on Spark 2.4.5 download Key: SPARK-31097 URL: https://issues.apache.org/jira/browse/SPARK-31097 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-31075) Add documentation for ALTER TABLE ... ADD PARTITION

[ https://issues.apache.org/jira/browse/SPARK-31075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-31075. -- Resolution: Duplicate > Add documentation for ALTER TABLE ... ADD PARTITION >

[jira] [Created] (SPARK-31096) Replace `Array` with `Seq` in AQE `CustomShuffleReaderExec`

Wei Xue created SPARK-31096: --- Summary: Replace `Array` with `Seq` in AQE `CustomShuffleReaderExec` Key: SPARK-31096 URL: https://issues.apache.org/jira/browse/SPARK-31096 Project: Spark Issue

[jira] [Updated] (SPARK-30941) PySpark Row can be instantiated with duplicate field names

[ https://issues.apache.org/jira/browse/SPARK-30941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30941: -- Environment: (was: Ubuntu 18.04 Python 3.6.8 Spark 2.4.4 (installed via binary from

[jira] [Assigned] (SPARK-30941) PySpark Row can be instantiated with duplicate field names

[ https://issues.apache.org/jira/browse/SPARK-30941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30941: - Assignee: Hyukjin Kwon > PySpark Row can be instantiated with duplicate field names >

[jira] [Resolved] (SPARK-30941) PySpark Row can be instantiated with duplicate field names

[ https://issues.apache.org/jira/browse/SPARK-30941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30941. --- Fix Version/s: 3.0.0 2.4.6 Resolution: Fixed Issue resolved by

[jira] [Commented] (SPARK-30941) PySpark Row can be instantiated with duplicate field names

[ https://issues.apache.org/jira/browse/SPARK-30941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055277#comment-17055277 ] Dongjoon Hyun commented on SPARK-30941: --- I updated the Affected Versions because this is a

[jira] [Updated] (SPARK-30941) PySpark Row can be instantiated with duplicate field names

[ https://issues.apache.org/jira/browse/SPARK-30941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30941: -- Affects Version/s: (was: 2.4.4) 3.0.0 1.6.3

[jira] [Updated] (SPARK-30941) PySpark Row can be instantiated with duplicate field names

[ https://issues.apache.org/jira/browse/SPARK-30941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30941: -- Description: It is possible to create a Row that has fields with the same name when calling

[jira] [Updated] (SPARK-30941) PySpark Row can be instantiated with duplicate field names

[ https://issues.apache.org/jira/browse/SPARK-30941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30941: -- Labels: correctness (was: ) > PySpark Row can be instantiated with duplicate field names >

[jira] [Updated] (SPARK-31065) Empty string values cause schema_of_json() to return a schema not usable by from_json()

[ https://issues.apache.org/jira/browse/SPARK-31065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31065: -- Affects Version/s: 2.4.0 2.4.1 2.4.2

[jira] [Commented] (SPARK-31065) Empty string values cause schema_of_json() to return a schema not usable by from_json()

[ https://issues.apache.org/jira/browse/SPARK-31065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055226#comment-17055226 ] Dongjoon Hyun commented on SPARK-31065: --- Thank you, [~nchammas] and [~hyukjin.kwon]. > Empty

[jira] [Created] (SPARK-31095) Upgrade netty version to fix security vulnerabilities

Vishwas Vijaya Kumar created SPARK-31095: Summary: Upgrade netty version to fix security vulnerabilities Key: SPARK-31095 URL: https://issues.apache.org/jira/browse/SPARK-31095 Project: Spark

[jira] [Commented] (SPARK-19256) Hive bucketing support

[ https://issues.apache.org/jira/browse/SPARK-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055007#comment-17055007 ] Felix Kizhakkel Jose commented on SPARK-19256: -- [~chengsu] Any updates on this feature

[jira] [Resolved] (SPARK-30929) ML, GraphX 3.0 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-30929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30929. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27818

[jira] [Assigned] (SPARK-30929) ML, GraphX 3.0 QA: API: New Scala APIs, docs

[ https://issues.apache.org/jira/browse/SPARK-30929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30929: Assignee: Huaxin Gao > ML, GraphX 3.0 QA: API: New Scala APIs, docs >

[jira] [Updated] (SPARK-31093) Fast fail while fetching shuffle data unsuccessfully

[ https://issues.apache.org/jira/browse/SPARK-31093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-31093: Description: When fetching shuffle data unsuccessfully, we put a FailureFetchResult into results(a

[jira] [Created] (SPARK-31094) Removing redundant rules in the output of Frequent Pattern Growth Algorithm

Aditya Addepalli created SPARK-31094: Summary: Removing redundant rules in the output of Frequent Pattern Growth Algorithm Key: SPARK-31094 URL: https://issues.apache.org/jira/browse/SPARK-31094

[jira] [Updated] (SPARK-31093) Fast fail while fetching shuffle data unsuccessfully

[ https://issues.apache.org/jira/browse/SPARK-31093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-31093: Summary: Fast fail while fetching shuffle data unsuccessfully (was: Fast fail while fetching shuffle

[jira] [Comment Edited] (SPARK-31074) Avro serializer should not fail when a nullable Spark field is written to a non-null Avro column

[ https://issues.apache.org/jira/browse/SPARK-31074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054859#comment-17054859 ] Kyrill Alyoshin edited comment on SPARK-31074 at 3/9/20, 11:57 AM: ---

[jira] [Created] (SPARK-31093) Fast fail while fetching shuffle data from a remote block unsuccessfully

feiwang created SPARK-31093: --- Summary: Fast fail while fetching shuffle data from a remote block unsuccessfully Key: SPARK-31093 URL: https://issues.apache.org/jira/browse/SPARK-31093 Project: Spark

[jira] [Assigned] (SPARK-31082) MapOutputTrackerMaster.getMapLocation can't handle last mapIndex

[ https://issues.apache.org/jira/browse/SPARK-31082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31082: --- Assignee: wuyi > MapOutputTrackerMaster.getMapLocation can't handle last mapIndex >

[jira] [Resolved] (SPARK-31082) MapOutputTrackerMaster.getMapLocation can't handle last mapIndex

[ https://issues.apache.org/jira/browse/SPARK-31082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31082. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27850

[jira] [Commented] (SPARK-31074) Avro serializer should not fail when a nullable Spark field is written to a non-null Avro column

[ https://issues.apache.org/jira/browse/SPARK-31074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054859#comment-17054859 ] Kyrill Alyoshin commented on SPARK-31074: - The first issue was about controlling _nullability_

[jira] [Created] (SPARK-31092) Add version information to the configuration of Yarn

jiaan.geng created SPARK-31092: -- Summary: Add version information to the configuration of Yarn Key: SPARK-31092 URL: https://issues.apache.org/jira/browse/SPARK-31092 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-31022) group by alias should fail if there are name conflicts

[ https://issues.apache.org/jira/browse/SPARK-31022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-31022. - Resolution: Won't Fix > group by alias should fail if there are name conflicts >

[jira] [Commented] (SPARK-31083) .ClassNotFoundException CoarseGrainedClusterMessages$RetrieveDelegationTokens

[ https://issues.apache.org/jira/browse/SPARK-31083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054703#comment-17054703 ] L. C. Hsieh commented on SPARK-31083: - Can you tell more about this issue? I think

[jira] [Commented] (SPARK-31074) Avro serializer should not fail when a nullable Spark field is written to a non-null Avro column

[ https://issues.apache.org/jira/browse/SPARK-31074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054697#comment-17054697 ] L. C. Hsieh commented on SPARK-31074: - Based on the description, is this the same issue as you

[jira] [Commented] (SPARK-30983) Support more than 5 typed column in typed Dataset.select API

[ https://issues.apache.org/jira/browse/SPARK-30983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054692#comment-17054692 ] L. C. Hsieh commented on SPARK-30983: - For option 2, adding more overloading typed select APIs, one

[jira] [Commented] (SPARK-31065) Empty string values cause schema_of_json() to return a schema not usable by from_json()

[ https://issues.apache.org/jira/browse/SPARK-31065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054683#comment-17054683 ] Hyukjin Kwon commented on SPARK-31065: -- Oh, nice catch. Yes, {{null}} types inferred for each field