[jira] [Updated] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

2022-06-21 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-39543: - Description: The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1, to

[jira] [Updated] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

2022-06-21 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-39543: - Description: The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1, to

[jira] [Updated] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1

2022-06-21 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-39543: - Summary: The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1 (was: The

[jira] [Updated] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage Properties if fallback to v1

2022-06-21 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-39543: - Summary: The option of DataFrameWriterV2 should be passed to storage Properties if fallback to v1 (was: The

[jira] [Updated] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage Properties if fallback to v1

2022-06-21 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-39543: - Description: The option of DataFrameWriterV2 should be passed to storage properties if fallback to v1, to

[jira] [Created] (SPARK-39543) The option of DataFrameWriterV2 should be passed to storage Properties

2022-06-21 Thread yikf (Jira)
yikf created SPARK-39543: Summary: The option of DataFrameWriterV2 should be passed to storage Properties Key: SPARK-39543 URL: https://issues.apache.org/jira/browse/SPARK-39543 Project: Spark

[jira] [Created] (SPARK-38482) Migrate legacy.keepCommandOutputSchema related to KeepLegacyOutputs

2022-03-09 Thread yikf (Jira)
yikf created SPARK-38482: Summary: Migrate legacy.keepCommandOutputSchema related to KeepLegacyOutputs Key: SPARK-38482 URL: https://issues.apache.org/jira/browse/SPARK-38482 Project: Spark Issue

[jira] [Resolved] (SPARK-38247) Unify the output of df.explain and "explain " if plan is command

2022-02-17 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf resolved SPARK-38247. -- Resolution: Invalid > Unify the output of df.explain and "explain " if plan is command >

[jira] [Created] (SPARK-38247) Unify the output of df.explain and "explain " if plan is command

2022-02-17 Thread yikf (Jira)
yikf created SPARK-38247: Summary: Unify the output of df.explain and "explain " if plan is command Key: SPARK-38247 URL: https://issues.apache.org/jira/browse/SPARK-38247 Project: Spark Issue

[jira] [Created] (SPARK-38229) Should't check temp/external/ifNotExists with visitReplaceTable when parser

2022-02-16 Thread yikf (Jira)
yikf created SPARK-38229: Summary: Should't check temp/external/ifNotExists with visitReplaceTable when parser Key: SPARK-38229 URL: https://issues.apache.org/jira/browse/SPARK-38229 Project: Spark

[jira] [Created] (SPARK-38216) When creating a Hive table, fail early if all the columns are partitioned columns

2022-02-15 Thread yikf (Jira)
yikf created SPARK-38216: Summary: When creating a Hive table, fail early if all the columns are partitioned columns Key: SPARK-38216 URL: https://issues.apache.org/jira/browse/SPARK-38216 Project: Spark

[jira] [Updated] (SPARK-38216) When creating a Hive table, fail early if all the columns are partitioned columns

2022-02-15 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-38216: - Description: In Hive the schema and partition columns must be disjoint sets, if hive table which all columns

[jira] [Created] (SPARK-38053) Report driver side metric from datasource V2

2022-01-28 Thread yikf (Jira)
yikf created SPARK-38053: Summary: Report driver side metric from datasource V2 Key: SPARK-38053 URL: https://issues.apache.org/jira/browse/SPARK-38053 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-38020) Clear unnecessary branch switch in VectorizedColumnReader.readBatch

2022-01-25 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-38020: - Summary: Clear unnecessary branch switch in VectorizedColumnReader.readBatch (was: Clear unnecessary branch

[jira] [Created] (SPARK-38020) Clear unnecessary branch switch

2022-01-25 Thread yikf (Jira)
yikf created SPARK-38020: Summary: Clear unnecessary branch switch Key: SPARK-38020 URL: https://issues.apache.org/jira/browse/SPARK-38020 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-38011) Remove useless and duplicated configuration in ParquetFileFormat builderReader

2022-01-24 Thread yikf (Jira)
yikf created SPARK-38011: Summary: Remove useless and duplicated configuration in ParquetFileFormat builderReader Key: SPARK-38011 URL: https://issues.apache.org/jira/browse/SPARK-38011 Project: Spark

[jira] [Resolved] (SPARK-37918) Specified construct when instance SessionStateBuilder

2022-01-15 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf resolved SPARK-37918. -- Resolution: Invalid > Specified construct when instance SessionStateBuilder >

[jira] [Created] (SPARK-37918) Specified construct when instance []SessionStateBuilder

2022-01-15 Thread yikf (Jira)
yikf created SPARK-37918: Summary: Specified construct when instance []SessionStateBuilder Key: SPARK-37918 URL: https://issues.apache.org/jira/browse/SPARK-37918 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-37918) Specified construct when instance SessionStateBuilder

2022-01-15 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-37918: - Summary: Specified construct when instance SessionStateBuilder (was: Specified construct when instance

[jira] [Created] (SPARK-37911) Unify the output of spark.version and select version()

2022-01-14 Thread yikf (Jira)
yikf created SPARK-37911: Summary: Unify the output of spark.version and select version() Key: SPARK-37911 URL: https://issues.apache.org/jira/browse/SPARK-37911 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-37764) Reserve bucket information when relation conversion from metastore relations to data source relations

2021-12-28 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf resolved SPARK-37764. -- Resolution: Invalid > Reserve bucket information when relation conversion from metastore relations > to data

[jira] [Updated] (SPARK-37764) Reserve bucket information when relation conversion from metastore relations to data source relations

2021-12-28 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-37764: - Description: Currently, we discarded bucket information when relation conversion from metastore relations to

[jira] [Updated] (SPARK-37764) Reserve bucket information when relation conversion from metastore relations to data source relations

2021-12-28 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-37764: - Description: Currently, we discarded bucket information when  > Reserve bucket information when relation

[jira] [Created] (SPARK-37764) Reserve bucket information when relation conversion from metastore relations to data source relations

2021-12-28 Thread yikf (Jira)
yikf created SPARK-37764: Summary: Reserve bucket information when relation conversion from metastore relations to data source relations Key: SPARK-37764 URL: https://issues.apache.org/jira/browse/SPARK-37764

[jira] [Updated] (SPARK-37243) Fix the format of the document

2021-11-08 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-37243: - Description: Currently, the format of `sql-ref-syntax-dml-insert-table.md` is: [currently

[jira] [Updated] (SPARK-37243) Fix the format of the document

2021-11-08 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-37243: - Attachment: currently format.png > Fix the format of the document > -- > >

[jira] [Updated] (SPARK-37243) Fix the format of the document

2021-11-08 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-37243: - Description: Currently, the format of `sql-ref-syntax-dml-insert-overwrite-table.md` is: [currently

[jira] [Updated] (SPARK-37243) Fix the format of the document

2021-11-08 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-37243: - Description: Currently, the format of `sql-ref-syntax-dml-insert-overwrite-table.md` is: [currently

[jira] [Updated] (SPARK-37243) Fix the format of the document

2021-11-08 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-37243: - Description: Currently, the format of `sql-ref-syntax-dml-insert-overwrite-table.md` is:   it should be

[jira] [Created] (SPARK-37243) Fix the format of the document

2021-11-08 Thread yikf (Jira)
yikf created SPARK-37243: Summary: Fix the format of the document Key: SPARK-37243 URL: https://issues.apache.org/jira/browse/SPARK-37243 Project: Spark Issue Type: Improvement Components:

[jira] [Updated] (SPARK-35722) wait until something does get queued

2021-06-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35722: - Description:  If nothing has been added to the queue, should wait until something does get queued instead of

[jira] [Updated] (SPARK-35722) wait until something does get queued

2021-06-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35722: - Description:  If nothing has been added to the queue, should wait until something does get queued instead of

[jira] [Updated] (SPARK-35722) wait until something does get queued

2021-06-10 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35722: - Summary: wait until something does get queued (was: wait until until something does get queued) > wait until

[jira] [Created] (SPARK-35722) wait until until something does get queued

2021-06-10 Thread yikf (Jira)
yikf created SPARK-35722: Summary: wait until until something does get queued Key: SPARK-35722 URL: https://issues.apache.org/jira/browse/SPARK-35722 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35444) Improve createTable logic if table exists

2021-05-19 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35444: - Description: It should return directly if table exists and ignoreIfExists=true   current logic: {code:java}

[jira] [Created] (SPARK-35444) Improve createTable logic if table exists

2021-05-19 Thread yikf (Jira)
yikf created SPARK-35444: Summary: Improve createTable logic if table exists Key: SPARK-35444 URL: https://issues.apache.org/jira/browse/SPARK-35444 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35356: - Description: In the case of externalCatalog is InMemoryCatalog, Using a different SparkSession will report the

[jira] [Updated] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35356: - Description: In the case of externalCatalog is InMemoryCatalog, Using a different SparkSession will report the

[jira] [Commented] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342949#comment-17342949 ] yikf commented on SPARK-35356: -- Sorry, my version of Spark is 3.0.0, Please visit

[jira] [Created] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-09 Thread yikf (Jira)
yikf created SPARK-35356: Summary: Fix issue of the createTable when externalCatalog is InMemoryCatalog Key: SPARK-35356 URL: https://issues.apache.org/jira/browse/SPARK-35356 Project: Spark Issue

[jira] [Updated] (SPARK-35355) improve execution performance in insert...select...limit case

2021-05-09 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35355: - Description: In the case of `insert into...select...limit` , `CollectLimitExec` has better execution

[jira] [Created] (SPARK-35355) improve execution performance in insert...select...limit case

2021-05-09 Thread yikf (Jira)
yikf created SPARK-35355: Summary: improve execution performance in insert...select...limit case Key: SPARK-35355 URL: https://issues.apache.org/jira/browse/SPARK-35355 Project: Spark Issue Type:

[jira] [Created] (SPARK-34541) Fixed an issue where data could not be cleaned up when unregisterShuffle

2021-02-25 Thread yikf (Jira)
yikf created SPARK-34541: Summary: Fixed an issue where data could not be cleaned up when unregisterShuffle Key: SPARK-34541 URL: https://issues.apache.org/jira/browse/SPARK-34541 Project: Spark

[jira] [Updated] (SPARK-34395) Clean up unused code for code simplifications

2021-02-07 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-34395: - Description: Currently, we pass the default value `EmptyRow` to method `checkEvaluation` in the 

[jira] [Updated] (SPARK-34395) Clean up unused code for code simplifications

2021-02-07 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-34395: - Description: Currently, we pass the default value `EmptyRow` to method `checkEvaluation` in the 

[jira] [Created] (SPARK-34395) Clean up unused code for code simplifications

2021-02-07 Thread yikf (Jira)
yikf created SPARK-34395: Summary: Clean up unused code for code simplifications Key: SPARK-34395 URL: https://issues.apache.org/jira/browse/SPARK-34395 Project: Spark Issue Type: Improvement