[
https://issues.apache.org/jira/browse/SPARK-48429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849690#comment-17849690
]
Zhen Wang commented on SPARK-48429:
---
Should we keep pushed filters like `PruneHiveTablePartitions`? cc
[
https://issues.apache.org/jira/browse/SPARK-48429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-48429:
--
Description:
org.apache.spark.sql.execution.datasources.v2.V2ScanRelationPushDown removes
pushed
[
https://issues.apache.org/jira/browse/SPARK-48429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-48429:
--
Description:
*org.apache.spark.sql.execution.datasources.v2.V2ScanRelationPushDown* removes
pushed
[
https://issues.apache.org/jira/browse/SPARK-48429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-48429:
--
Affects Version/s: 4.0.0
(was: 3.5.0)
> CleanupDynamicPruningFilters does
Zhen Wang created SPARK-48429:
-
Summary: CleanupDynamicPruningFilters does not take effect for
V2ScanRelation
Key: SPARK-48429
URL: https://issues.apache.org/jira/browse/SPARK-48429
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-47912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-47912:
--
Description:
File format of insert overwrite dir does not take effect.
Reproduce:
{code:java}
set
Zhen Wang created SPARK-47912:
-
Summary: File format of insert overwrite dir does not take effect
Key: SPARK-47912
URL: https://issues.apache.org/jira/browse/SPARK-47912
Project: Spark
Issue
Zhen Wang created SPARK-47897:
-
Summary: ExpressionSet performance regression in scala 2.12
Key: SPARK-47897
URL: https://issues.apache.org/jira/browse/SPARK-47897
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835282#comment-17835282
]
Zhen Wang edited comment on SPARK-47463 at 4/10/24 2:22 AM:
new case:
[
https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-47463:
--
Issue Type: Bug (was: Test)
> An error occurred while pushing down the filter of if expression for
[
https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-47463:
--
Description:
Reproduce:
{code:java}
create table t1(c1 int) using iceberg;
select * from
(select
[
https://issues.apache.org/jira/browse/SPARK-47463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-47463:
--
Description:
Reproduce:
{code:java}
create table t1(c1 int) using iceberg;select * from
(select if(c1
Zhen Wang created SPARK-47463:
-
Summary: An error occurred while pushing down the filter of if
expression for iceberg datasource.
Key: SPARK-47463
URL: https://issues.apache.org/jira/browse/SPARK-47463
[
https://issues.apache.org/jira/browse/SPARK-47315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824259#comment-17824259
]
Zhen Wang commented on SPARK-47315:
---
https://github.com/apache/spark/pull/45417
> tempView created in
[
https://issues.apache.org/jira/browse/SPARK-47315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-47315:
--
Description: tempView created in createTempView UT (which is in
DatasetSuite.scala) is not cleaned up
Zhen Wang created SPARK-47315:
-
Summary: tempView created in createTempView UT is not cleaned up
Key: SPARK-47315
URL: https://issues.apache.org/jira/browse/SPARK-47315
Project: Spark
Issue
Zhen Wang created SPARK-47270:
-
Summary: Dataset.isEmpty should not trigger job execution on
CommandResults
Key: SPARK-47270
URL: https://issues.apache.org/jira/browse/SPARK-47270
Project: Spark
Zhen Wang created SPARK-45975:
-
Summary: STORE_ASSIGNMENT_POLICY should be reset in
HiveCompatibilitySuite
Key: SPARK-45975
URL: https://issues.apache.org/jira/browse/SPARK-45975
Project: Spark
Zhen Wang created SPARK-45974:
-
Summary: Add scan.filterAttributes non-empty judgment for
RowLevelOperationRuntimeGroupFiltering
Key: SPARK-45974
URL: https://issues.apache.org/jira/browse/SPARK-45974
[
https://issues.apache.org/jira/browse/SPARK-45943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786654#comment-17786654
]
Zhen Wang commented on SPARK-45943:
---
[~ashahid7] I haven't thought of a good solution yet, if you have
[
https://issues.apache.org/jira/browse/SPARK-45943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786584#comment-17786584
]
Zhen Wang edited comment on SPARK-45943 at 11/16/23 3:24 AM:
-
I encountered
[
https://issues.apache.org/jira/browse/SPARK-45943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786584#comment-17786584
]
Zhen Wang commented on SPARK-45943:
---
I encountered the same problem, and after debugging, I found that
[
https://issues.apache.org/jira/browse/SPARK-44902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759854#comment-17759854
]
Zhen Wang commented on SPARK-44902:
---
I tried changing it to 19 but some test cases failed.
[
https://issues.apache.org/jira/browse/SPARK-44998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-44998:
--
Description:
I found a lot of retry parsing inprogress event log records in history server
log. The
[
https://issues.apache.org/jira/browse/SPARK-44998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-44998:
--
Attachment: image-2023-08-29-10-47-08-027.png
> No need to retry parsing event log path again when
[
https://issues.apache.org/jira/browse/SPARK-44998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-44998:
--
Attachment: image-2023-08-29-10-47-43-567.png
> No need to retry parsing event log path again when
Zhen Wang created SPARK-44998:
-
Summary: No need to retry parsing event log path again when
FileNotFoundException occurs
Key: SPARK-44998
URL: https://issues.apache.org/jira/browse/SPARK-44998
Project:
[
https://issues.apache.org/jira/browse/SPARK-44902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-44902:
--
Description:
The precision of LongDecimal in Hive is 19 but it is 20 in Spark. This leads to
type
Zhen Wang created SPARK-44902:
-
Summary: The precision of LongDecimal is inconsistent with Hive.
Key: SPARK-44902
URL: https://issues.apache.org/jira/browse/SPARK-44902
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-44065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733271#comment-17733271
]
Zhen Wang commented on SPARK-44065:
---
https://github.com/apache/spark/pull/41609
> Optimize
Zhen Wang created SPARK-44065:
-
Summary: Optimize BroadcastHashJoin skew when localShuffleReader
is disabled
Key: SPARK-44065
URL: https://issues.apache.org/jira/browse/SPARK-44065
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-43900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-43900:
--
Summary: Support optimize skewed partitions even if introduce extra shuffle
(was: Support optimize
[
https://issues.apache.org/jira/browse/SPARK-43900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17727949#comment-17727949
]
Zhen Wang commented on SPARK-43900:
---
cc [~ulysses]
> Support optimize skewed partitions if introduce
Zhen Wang created SPARK-43900:
-
Summary: Support optimize skewed partitions if introduce extra
shuffle
Key: SPARK-43900
URL: https://issues.apache.org/jira/browse/SPARK-43900
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-41344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644087#comment-17644087
]
Zhen Wang commented on SPARK-41344:
---
[~planga82] Thanks for your reply, I have submitted a PR
[
https://issues.apache.org/jira/browse/SPARK-41344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643131#comment-17643131
]
Zhen Wang commented on SPARK-41344:
---
[~ganaakruti] As far as I understand, V1 and V2 only have
[
https://issues.apache.org/jira/browse/SPARK-41344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642298#comment-17642298
]
Zhen Wang commented on SPARK-41344:
---
I want to work on this and will send a PR later.
> Reading V2
[
https://issues.apache.org/jira/browse/SPARK-40600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17611315#comment-17611315
]
Zhen Wang commented on SPARK-40600:
---
There is a related implementation in
Zhen Wang created SPARK-40600:
-
Summary: Support recursiveFileLookup for partitioned datasource
Key: SPARK-40600
URL: https://issues.apache.org/jira/browse/SPARK-40600
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603901#comment-17603901
]
Zhen Wang commented on SPARK-32563:
---
In Spark 3.1.1 and hive 3.1.2, this issue also occurs when I set
[
https://issues.apache.org/jira/browse/SPARK-39997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-39997:
--
Description:
{code:scala}
test("SPARK-38094: absence of field ids: reading nested schema struct
Zhen Wang created SPARK-39997:
-
Summary: ParquetSchemaConverter fails match schema by id
Key: SPARK-39997
URL: https://issues.apache.org/jira/browse/SPARK-39997
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572754#comment-17572754
]
Zhen Wang edited comment on SPARK-37210 at 7/29/22 6:27 AM:
{quote}I'm not
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572754#comment-17572754
]
Zhen Wang commented on SPARK-37210:
---
> BTW, could you please reopen the PR freshly? I'll try to ping
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-37210:
--
Description:
An error occurred while concurrently writing to different static partitions.
For
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572408#comment-17572408
]
Zhen Wang commented on SPARK-37210:
---
The concurrent writes in INSERT INTO mode seems to have the same
Zhen Wang created SPARK-39362:
-
Summary: Datasource v2 scan not pruned in tpcds q10
Key: SPARK-39362
URL: https://issues.apache.org/jira/browse/SPARK-39362
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-28098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511562#comment-17511562
]
Zhen Wang commented on SPARK-28098:
---
> set spark.sql.hive.convertMetastoreParquet=false
With this
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17451582#comment-17451582
]
Zhen Wang commented on SPARK-37210:
---
I made some adjustments to the test cases to make it more likely
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17451580#comment-17451580
]
Zhen Wang commented on SPARK-37210:
---
There seem to be two bugs:
# For the insert overwrite fully
[
https://issues.apache.org/jira/browse/SPARK-37382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446367#comment-17446367
]
Zhen Wang commented on SPARK-37382:
---
https://issues.apache.org/jira/browse/SPARK-36447 related?
>
[
https://issues.apache.org/jira/browse/SPARK-37387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446365#comment-17446365
]
Zhen Wang commented on SPARK-37387:
---
[~yoda-mon] what is the use case behind, random doesn't seem
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439944#comment-17439944
]
Zhen Wang edited comment on SPARK-37210 at 11/7/21, 10:28 AM:
--
Yes
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439944#comment-17439944
]
Zhen Wang edited comment on SPARK-37210 at 11/7/21, 10:24 AM:
--
Yes
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439944#comment-17439944
]
Zhen Wang commented on SPARK-37210:
---
Yes [~dongjoon] , this error will occur when two tasks
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439061#comment-17439061
]
Zhen Wang edited comment on SPARK-37210 at 11/5/21, 7:55 AM:
-
In
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439061#comment-17439061
]
Zhen Wang edited comment on SPARK-37210 at 11/5/21, 7:28 AM:
-
In
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439061#comment-17439061
]
Zhen Wang commented on SPARK-37210:
---
In DataSourceAnalysis, static partition writing is converted to
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-37210:
--
Attachment:
[SPARK-37210]_Write_to_static_partition_in_dynamic_write_mode.patch
> An error occurred
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-37210:
--
Description:
An error occurred while concurrently writing to different static partitions.
For
[
https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439025#comment-17439025
]
Zhen Wang commented on SPARK-37210:
---
The test code can be executed normally in spark 2.4.3. I noticed
Zhen Wang created SPARK-37210:
-
Summary: An error occurred while concurrently writing to different
static partitions
Key: SPARK-37210
URL: https://issues.apache.org/jira/browse/SPARK-37210
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-35270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-35270:
--
Summary: Remove the use of guava in order to upgrade guava version to 27
(was: Remove the use of
[
https://issues.apache.org/jira/browse/SPARK-35270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335233#comment-17335233
]
Zhen Wang commented on SPARK-35270:
---
Modify with reference to
Zhen Wang created SPARK-35270:
-
Summary: Remove the use of guava to fix Hadoop 3.2.2 guava conflict
Key: SPARK-35270
URL: https://issues.apache.org/jira/browse/SPARK-35270
Project: Spark
Issue
65 matches
Mail list logo
