[jira] [Assigned] (SPARK-34211) Benchmark TPC-DS with 1GB scale factor

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34211: Assignee: (was: Apache Spark) > Benchmark TPC-DS with 1GB scale factor >

[jira] [Assigned] (SPARK-34211) Benchmark TPC-DS with 1GB scale factor

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34211: Assignee: Apache Spark > Benchmark TPC-DS with 1GB scale factor >

[jira] [Assigned] (SPARK-34211) Benchmark TPC-DS with 1GB scale factor

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34211: Assignee: (was: Apache Spark) > Benchmark TPC-DS with 1GB scale factor >

[jira] [Commented] (SPARK-34211) Benchmark TPC-DS with 1GB scale factor

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270552#comment-17270552 ] Apache Spark commented on SPARK-34211: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-34210) Cannot create a record reader because of a previous error when spark accesses the hive on HBase table

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270551#comment-17270551 ] Apache Spark commented on SPARK-34210: -- User 'zzccctv' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34210) Cannot create a record reader because of a previous error when spark accesses the hive on HBase table

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34210: Assignee: (was: Apache Spark) > Cannot create a record reader because of a previous

[jira] [Commented] (SPARK-34210) Cannot create a record reader because of a previous error when spark accesses the hive on HBase table

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270550#comment-17270550 ] Apache Spark commented on SPARK-34210: -- User 'zzccctv' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34210) Cannot create a record reader because of a previous error when spark accesses the hive on HBase table

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34210: Assignee: Apache Spark > Cannot create a record reader because of a previous error when

[jira] [Created] (SPARK-34211) Benchmark TPC-DS with 1GB scale factor

2021-01-22 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-34211: --- Summary: Benchmark TPC-DS with 1GB scale factor Key: SPARK-34211 URL: https://issues.apache.org/jira/browse/SPARK-34211 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-34210) Cannot create a record reader because of a previous error when spark accesses the hive on HBase table

2021-01-22 Thread zhangzhanchang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangzhanchang updated SPARK-34210: --- Description: It is normal for version 2.4.6 to use spark SQL to access hive on HBase

[jira] [Created] (SPARK-34210) Cannot create a record reader because of a previous error when spark accesses the hive on HBase table

2021-01-22 Thread zhangzhanchang (Jira)
zhangzhanchang created SPARK-34210: -- Summary: Cannot create a record reader because of a previous error when spark accesses the hive on HBase table Key: SPARK-34210 URL:

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-22 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/23/21, 3:39 AM: -- [~angerszhuuu] and

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-22 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary

[jira] [Assigned] (SPARK-34206) Make Guava Cache to ExecutorPodsLifecycleManager private field

2021-01-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34206: - Assignee: Dongjoon Hyun > Make Guava Cache to ExecutorPodsLifecycleManager private

[jira] [Resolved] (SPARK-34206) Make Guava Cache to ExecutorPodsLifecycleManager private field

2021-01-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34206. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31297

[jira] [Commented] (SPARK-34209) Allow multiple namespaces with session catalog

2021-01-22 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270508#comment-17270508 ] Holden Karau commented on SPARK-34209: -- This was broken in  # SPARK-30885 > Allow multiple

[jira] [Updated] (SPARK-34209) Allow multiple namespaces with session catalog

2021-01-22 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-34209: - Summary: Allow multiple namespaces with session catalog (was: Delegate table name validation

[jira] [Created] (SPARK-34209) Delegate table name validation to the catalog

2021-01-22 Thread Holden Karau (Jira)
Holden Karau created SPARK-34209: Summary: Delegate table name validation to the catalog Key: SPARK-34209 URL: https://issues.apache.org/jira/browse/SPARK-34209 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34208) Upgrade ORC to 1.6.7

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270489#comment-17270489 ] Apache Spark commented on SPARK-34208: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-34208) Upgrade ORC to 1.6.7

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34208: Assignee: Apache Spark > Upgrade ORC to 1.6.7 > > >

[jira] [Assigned] (SPARK-34208) Upgrade ORC to 1.6.7

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34208: Assignee: (was: Apache Spark) > Upgrade ORC to 1.6.7 > > >

[jira] [Commented] (SPARK-34208) Upgrade ORC to 1.6.7

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270488#comment-17270488 ] Apache Spark commented on SPARK-34208: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-34208) Upgrade ORC to 1.6.7

2021-01-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-34208: - Summary: Upgrade ORC to 1.6.7 Key: SPARK-34208 URL: https://issues.apache.org/jira/browse/SPARK-34208 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-34052) A cached view should become invalid after a table is dropped

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270444#comment-17270444 ] Apache Spark commented on SPARK-34052: -- User 'sunchao' has created a pull request for this issue:

[jira] [Updated] (SPARK-34207) Rename `isTemporaryTable` to `isTempView` in `SessionCatalog`

2021-01-22 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34207: --- Priority: Trivial (was: Major) > Rename `isTemporaryTable` to `isTempView` in `SessionCatalog` >

[jira] [Commented] (SPARK-34207) Rename `isTemporaryTable` to `isTempView` in `SessionCatalog`

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270422#comment-17270422 ] Apache Spark commented on SPARK-34207: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34207) Rename `isTemporaryTable` to `isTempView` in `SessionCatalog`

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34207: Assignee: (was: Apache Spark) > Rename `isTemporaryTable` to `isTempView` in

[jira] [Assigned] (SPARK-34207) Rename `isTemporaryTable` to `isTempView` in `SessionCatalog`

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34207: Assignee: Apache Spark > Rename `isTemporaryTable` to `isTempView` in `SessionCatalog` >

[jira] [Created] (SPARK-34207) Rename `isTemporaryTable` to `isTempView` in `SessionCatalog`

2021-01-22 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34207: -- Summary: Rename `isTemporaryTable` to `isTempView` in `SessionCatalog` Key: SPARK-34207 URL: https://issues.apache.org/jira/browse/SPARK-34207 Project: Spark

[jira] [Commented] (SPARK-32866) Docker buildx now requires --push

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270413#comment-17270413 ] Apache Spark commented on SPARK-32866: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32866) Docker buildx now requires --push

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32866: Assignee: (was: Apache Spark) > Docker buildx now requires --push >

[jira] [Commented] (SPARK-32866) Docker buildx now requires --push

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270412#comment-17270412 ] Apache Spark commented on SPARK-32866: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32866) Docker buildx now requires --push

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32866: Assignee: Apache Spark > Docker buildx now requires --push >

[jira] [Assigned] (SPARK-34193) Potential race condition during decommissioning with TorrentBroadcast

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34193: Assignee: Apache Spark > Potential race condition during decommissioning with

[jira] [Commented] (SPARK-34193) Potential race condition during decommissioning with TorrentBroadcast

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270408#comment-17270408 ] Apache Spark commented on SPARK-34193: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34193) Potential race condition during decommissioning with TorrentBroadcast

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34193: Assignee: (was: Apache Spark) > Potential race condition during decommissioning with

[jira] [Updated] (SPARK-34167) Reading parquet with Decimal(8,2) written as a Decimal64 blows up

2021-01-22 Thread Raza Jafri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raza Jafri updated SPARK-34167: --- Description: When reading a parquet file written with Decimals with precision < 10 as a 64-bit

[jira] [Commented] (SPARK-34206) Make Guava Cache to ExecutorPodsLifecycleManager private field

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270376#comment-17270376 ] Apache Spark commented on SPARK-34206: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-34206) Make Guava Cache to ExecutorPodsLifecycleManager private field

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34206: Assignee: Apache Spark > Make Guava Cache to ExecutorPodsLifecycleManager private field

[jira] [Assigned] (SPARK-34206) Make Guava Cache to ExecutorPodsLifecycleManager private field

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34206: Assignee: (was: Apache Spark) > Make Guava Cache to ExecutorPodsLifecycleManager

[jira] [Created] (SPARK-34206) Make Guava Cache to ExecutorPodsLifecycleManager private field

2021-01-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-34206: - Summary: Make Guava Cache to ExecutorPodsLifecycleManager private field Key: SPARK-34206 URL: https://issues.apache.org/jira/browse/SPARK-34206 Project: Spark

[jira] [Commented] (SPARK-34167) Reading parquet with Decimal(8,2) written as a Decimal64 blows up

2021-01-22 Thread Raza Jafri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270371#comment-17270371 ] Raza Jafri commented on SPARK-34167: [~tgraves] I have updated the description to help trace the

[jira] [Updated] (SPARK-34167) Reading parquet with Decimal(8,2) written as a Decimal64 blows up

2021-01-22 Thread Raza Jafri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raza Jafri updated SPARK-34167: --- Description: When reading a parquet file written with Decimals with precision < 10 as a 64-bit

[jira] [Updated] (SPARK-34167) Reading parquet with Decimal(8,2) written as a Decimal64 blows up

2021-01-22 Thread Raza Jafri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raza Jafri updated SPARK-34167: --- Description: When reading a parquet file written with Decimals with precision < 10 as a 64-bit

[jira] [Commented] (SPARK-34196) Improve error message when folks try and install in Python 2

2021-01-22 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270354#comment-17270354 ] Holden Karau commented on SPARK-34196: -- I tried to fix this, but because of the way Python works I

[jira] [Resolved] (SPARK-34196) Improve error message when folks try and install in Python 2

2021-01-22 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-34196. -- Resolution: Won't Fix > Improve error message when folks try and install in Python 2 >

[jira] [Updated] (SPARK-32866) Docker buildx now requires --push

2021-01-22 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-32866: - Affects Version/s: 3.1.1 3.2.0 > Docker buildx now requires --push >

[jira] [Assigned] (SPARK-34205) Add pipe API to Dataset

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34205: Assignee: L. C. Hsieh (was: Apache Spark) > Add pipe API to Dataset >

[jira] [Commented] (SPARK-34205) Add pipe API to Dataset

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270335#comment-17270335 ] Apache Spark commented on SPARK-34205: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-34205) Add pipe API to Dataset

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270336#comment-17270336 ] Apache Spark commented on SPARK-34205: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-34205) Add pipe API to Dataset

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34205: Assignee: Apache Spark (was: L. C. Hsieh) > Add pipe API to Dataset >

[jira] [Updated] (SPARK-34205) Add pipe API to Dataset

2021-01-22 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-34205: Description: Dataset doesn't have pipe API but RDD has it. Although for normal Dataset, user can

[jira] [Created] (SPARK-34205) Add pipe API to Dataset

2021-01-22 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-34205: --- Summary: Add pipe API to Dataset Key: SPARK-34205 URL: https://issues.apache.org/jira/browse/SPARK-34205 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-34204) When use input_file_name() func all column from file appeared in physical plan of query, not only projection.

2021-01-22 Thread Nick Hryhoriev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270326#comment-17270326 ] Nick Hryhoriev commented on SPARK-34204: Look like `spark_partition_id` have same issue. > When

[jira] [Commented] (SPARK-34194) Queries that only touch partition columns shouldn't scan through all files

2021-01-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270263#comment-17270263 ] Steve Loughran commented on SPARK-34194: * if you can also use any of the incremental list calls

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

2021-01-22 Thread Simeon H.K. Fitch (Jira)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270256#comment-17270256 ] Simeon H.K. Fitch commented on SPARK-7768: -- [~xkrogen] ‍♂️ > Make user-defined type (UDT)

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

2021-01-22 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270235#comment-17270235 ] Erik Krogen commented on SPARK-7768: [~metasim] you might want to start a discussion on

[jira] [Commented] (SPARK-34167) Reading parquet with Decimal(8,2) written as a Decimal64 blows up

2021-01-22 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270199#comment-17270199 ] Thomas Graves commented on SPARK-34167: --- [~razajafri] can you finish detailing your finding as to

[jira] [Commented] (SPARK-21770) ProbabilisticClassificationModel: Improve normalization of all-zero raw predictions

2021-01-22 Thread Rishi A (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270179#comment-17270179 ] Rishi A commented on SPARK-21770: - [~davidmav86] I am facing same issue with 3.0 version. Did you find

[jira] [Updated] (SPARK-34204) When use input_file_name() func all column from file appeared in physical plan of query, not only projection.

2021-01-22 Thread Nick Hryhoriev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Hryhoriev updated SPARK-34204: --- Summary: When use input_file_name() func all column from file appeared in physical plan of

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

2021-01-22 Thread Simeon H.K. Fitch (Jira)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270148#comment-17270148 ] Simeon H.K. Fitch commented on SPARK-7768: -- Not being familiar with the Spark issue triage and

[jira] [Updated] (SPARK-34204) input_file_name() func damage applying projection to physical plan of query.

2021-01-22 Thread Nick Hryhoriev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Hryhoriev updated SPARK-34204: --- Description: input_file_name() function damage applying projection to the physical plan of

[jira] [Updated] (SPARK-34204) input_file_name() func damage applying projection to physical plan of query.

2021-01-22 Thread Nick Hryhoriev (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Hryhoriev updated SPARK-34204: --- Description: input_file_name() function damage applying projection to the physical plan of

[jira] [Created] (SPARK-34204) input_file_name() func damage applying projection to physical plan of query.

2021-01-22 Thread Nick Hryhoriev (Jira)
Nick Hryhoriev created SPARK-34204: -- Summary: input_file_name() func damage applying projection to physical plan of query. Key: SPARK-34204 URL: https://issues.apache.org/jira/browse/SPARK-34204

[jira] [Assigned] (SPARK-34191) udf type hint should allow dectorator with named returnType

2021-01-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34191: Assignee: Piotr Grzegorski > udf type hint should allow dectorator with named returnType

[jira] [Commented] (SPARK-34191) udf type hint should allow dectorator with named returnType

2021-01-22 Thread Piotr Grzegorski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270101#comment-17270101 ] Piotr Grzegorski commented on SPARK-34191: -- Leaving a comment as requested > udf type hint

[jira] [Resolved] (SPARK-34191) udf type hint should allow dectorator with named returnType

2021-01-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34191. -- Fix Version/s: 3.1.2 Resolution: Fixed Issue resolved by pull request 31282

[jira] [Commented] (SPARK-34202) Let HiveExternalCatalogVersionsSuite can run in orgs internal environment

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270075#comment-17270075 ] Apache Spark commented on SPARK-34202: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-34202) Let HiveExternalCatalogVersionsSuite can run in orgs internal environment

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34202: Assignee: (was: Apache Spark) > Let HiveExternalCatalogVersionsSuite can run in orgs

[jira] [Assigned] (SPARK-34202) Let HiveExternalCatalogVersionsSuite can run in orgs internal environment

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34202: Assignee: Apache Spark > Let HiveExternalCatalogVersionsSuite can run in orgs internal

[jira] [Commented] (SPARK-34202) Let HiveExternalCatalogVersionsSuite can run in orgs internal environment

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270074#comment-17270074 ] Apache Spark commented on SPARK-34202: -- User 'LuciferYang' has created a pull request for this

[jira] [Resolved] (SPARK-34200) ambiguous column reference should consider attribute availability

2021-01-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34200. -- Fix Version/s: 3.1.2 3.0.2 Resolution: Fixed Fixed in

[jira] [Created] (SPARK-34203) FileSource table null partition can not be dropped

2021-01-22 Thread Kent Yao (Jira)
Kent Yao created SPARK-34203: Summary: FileSource table null partition can not be dropped Key: SPARK-34203 URL: https://issues.apache.org/jira/browse/SPARK-34203 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-33601) Group exception messages in catalyst/parser

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270070#comment-17270070 ] Apache Spark commented on SPARK-33601: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-33601) Group exception messages in catalyst/parser

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270071#comment-17270071 ] Apache Spark commented on SPARK-33601: -- User 'beliefer' has created a pull request for this issue:

[jira] [Created] (SPARK-34202) Let HiveExternalCatalogVersionsSuite can run in orgs internal environment

2021-01-22 Thread Yang Jie (Jira)
Yang Jie created SPARK-34202: Summary: Let HiveExternalCatalogVersionsSuite can run in orgs internal environment Key: SPARK-34202 URL: https://issues.apache.org/jira/browse/SPARK-34202 Project: Spark

[jira] [Commented] (SPARK-32333) Drop references to Master

2021-01-22 Thread Jacek Laskowski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270058#comment-17270058 ] Jacek Laskowski commented on SPARK-32333: - Just today when I was reading about a new ASF project

[jira] [Commented] (SPARK-33601) Group exception messages in catalyst/parser

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270023#comment-17270023 ] Apache Spark commented on SPARK-33601: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33601) Group exception messages in catalyst/parser

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33601: Assignee: Apache Spark > Group exception messages in catalyst/parser >

[jira] [Assigned] (SPARK-33601) Group exception messages in catalyst/parser

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33601: Assignee: Apache Spark > Group exception messages in catalyst/parser >

[jira] [Assigned] (SPARK-33601) Group exception messages in catalyst/parser

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33601: Assignee: (was: Apache Spark) > Group exception messages in catalyst/parser >

[jira] [Assigned] (SPARK-34201) Fix the test code build error for docker-integration-tests in branch-3.0

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34201: Assignee: Kousuke Saruta (was: Apache Spark) > Fix the test code build error for

[jira] [Assigned] (SPARK-34201) Fix the test code build error for docker-integration-tests in branch-3.0

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34201: Assignee: Apache Spark (was: Kousuke Saruta) > Fix the test code build error for

[jira] [Commented] (SPARK-34201) Fix the test code build error for docker-integration-tests in branch-3.0

2021-01-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270018#comment-17270018 ] Apache Spark commented on SPARK-34201: -- User 'sarutak' has created a pull request for this issue:

[jira] [Updated] (SPARK-34201) Fix the test code build error for docker-integration-tests in branch-3.0

2021-01-22 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34201: --- Summary: Fix the test code build error for docker-integration-tests in branch-3.0 (was:

[jira] [Created] (SPARK-34201) Fix the test code build error for docker-integration-tests

2021-01-22 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-34201: -- Summary: Fix the test code build error for docker-integration-tests Key: SPARK-34201 URL: https://issues.apache.org/jira/browse/SPARK-34201 Project: Spark