ming95 created SPARK-46211:
--
Summary: Directly writing the parquet table location will cause
the query to be null
Key: SPARK-46211
URL: https://issues.apache.org/jira/browse/SPARK-46211
Project:
[
https://issues.apache.org/jira/browse/SPARK-43149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762387#comment-17762387
]
ming95 commented on SPARK-43149:
pr:https://github.com/apache/spark/pull/42574
> When CTAS with
[
https://issues.apache.org/jira/browse/SPARK-44911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762386#comment-17762386
]
ming95 commented on SPARK-44911:
https://github.com/apache/spark/pull/42609
> create hive table
[
https://issues.apache.org/jira/browse/SPARK-45041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17762385#comment-17762385
]
ming95 commented on SPARK-45041:
https://github.com/apache/spark/pull/42760
> spark using
[
https://issues.apache.org/jira/browse/SPARK-45071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-45071:
---
Description:
Since `BinaryArithmetic#dataType` will recursively process the datatype of each
node,
ming95 created SPARK-45071:
--
Summary: Optimize the processing speed of
`BinaryArithmetic#dataType` when processing multi-column data
Key: SPARK-45071
URL: https://issues.apache.org/jira/browse/SPARK-45071
[
https://issues.apache.org/jira/browse/SPARK-45071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-45071:
---
Issue Type: Improvement (was: Bug)
> Optimize the processing speed of `BinaryArithmetic#dataType`
ming95 created SPARK-45041:
--
Summary: spark using --proxy-user GSS init failed when
`hive.metastore.token.signature` not empty
Key: SPARK-45041
URL: https://issues.apache.org/jira/browse/SPARK-45041
ming95 created SPARK-44911:
--
Summary: create hive table with invalid column should return error
class
Key: SPARK-44911
URL: https://issues.apache.org/jira/browse/SPARK-44911
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-40156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 resolved SPARK-40156.
Fix Version/s: 3.4.0
Resolution: Fixed
> url_decode() exposes a Java error
>
[
https://issues.apache.org/jira/browse/SPARK-43327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718060#comment-17718060
]
ming95 edited comment on SPARK-43327 at 8/11/23 12:58 PM:
--
pr :
[
https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749445#comment-17749445
]
ming95 edited comment on SPARK-44598 at 8/1/23 4:23 AM:
it seem is a
[
https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 resolved SPARK-44598.
Resolution: Fixed
> spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize
[
https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749445#comment-17749445
]
ming95 commented on SPARK-44598:
it seem is a hbase bug. and hbase fix this bug at 2.5+
[
https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749444#comment-17749444
]
ming95 commented on SPARK-44598:
[~ulysses] yes , it fix my case , thanks~
> spark 3.2+ can not
[
https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-44598:
---
Issue Type: Bug (was: Improvement)
> spark 3.2+ can not read hive table with hbase serde when
ming95 created SPARK-44598:
--
Summary: spark 3.2+ can not read hive table with hbase serde when
hbase StorefileSize is 0
Key: SPARK-44598
URL: https://issues.apache.org/jira/browse/SPARK-44598
ming95 created SPARK-43866:
--
Summary: Partition filter condition should pushed down to
metastore query if it is equivalence Predicate
Key: SPARK-43866
URL: https://issues.apache.org/jira/browse/SPARK-43866
[
https://issues.apache.org/jira/browse/SPARK-43866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-43866:
---
Description:
Typically, hive partition fields are created as string types.
{code:java}
```
CREATE
[
https://issues.apache.org/jira/browse/SPARK-43327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718060#comment-17718060
]
ming95 commented on SPARK-43327:
pr : https://github.com/apache/spark/pull/41000
> Trigger
[
https://issues.apache.org/jira/browse/SPARK-43326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718026#comment-17718026
]
ming95 commented on SPARK-43326:
Thank you for the record. I will follow up on this issue~
>
[
https://issues.apache.org/jira/browse/SPARK-43327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-43327:
---
Description:
In this jira, the case where `outputOrdering` might not work if AQE is enabled
has
ming95 created SPARK-43327:
--
Summary: Trigger `committer.setupJob` before plan execute in
`FileFormatWriter`
Key: SPARK-43327
URL: https://issues.apache.org/jira/browse/SPARK-43327
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-43287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716936#comment-17716936
]
ming95 commented on SPARK-43287:
i can try to fix this issue : -) [~WweiL]
> Connect JVM
[
https://issues.apache.org/jira/browse/SPARK-43074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-43074:
---
Description:
{code:java}
```
df = spark.sql("select 1")
catalyst_plan =
ming95 created SPARK-43074:
--
Summary: Add the function without constant parameters of
`SessionState#executePlan`
Key: SPARK-43074
URL: https://issues.apache.org/jira/browse/SPARK-43074
Project:
[
https://issues.apache.org/jira/browse/SPARK-43021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17709448#comment-17709448
]
ming95 commented on SPARK-43021:
PR：https://github.com/apache/spark/pull/40688
> Shuffle
[
https://issues.apache.org/jira/browse/SPARK-42503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 resolved SPARK-42503.
Resolution: Won't Do
> Spark SQL should do further validation on join condition fields
>
[
https://issues.apache.org/jira/browse/SPARK-42715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702088#comment-17702088
]
ming95 commented on SPARK-42715:
[~achong]
Can you provide the code to reproduce this issue
[
https://issues.apache.org/jira/browse/SPARK-42753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699148#comment-17699148
]
ming95 commented on SPARK-42753:
[~steven.chen]
Can you provide the reproduction code?
>
[ https://issues.apache.org/jira/browse/SPARK-42623 ]
ming95 deleted comment on SPARK-42623:
was (Author: zing):
i can try to fix this issue
> parameter markers not blocked in DDL
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-42623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696501#comment-17696501
]
ming95 commented on SPARK-42623:
i can try to fix this issue
> parameter markers not blocked in
[
https://issues.apache.org/jira/browse/SPARK-42638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696409#comment-17696409
]
ming95 commented on SPARK-42638:
Maybe we can use `insert as select` to achieve the same effect?
[
https://issues.apache.org/jira/browse/SPARK-40885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 resolved SPARK-40885.
Resolution: Fixed
> Spark will filter out data field sorting when dynamic partitions and data
>
[
https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 resolved SPARK-42594.
Resolution: Fixed
> spark can not read lastest view sql when run `create or replace view` by hive
[
https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693810#comment-17693810
]
ming95 commented on SPARK-42594:
OK ,Thanks~ [~yumwang]
> spark can not read lastest view sql
[
https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693795#comment-17693795
]
ming95 commented on SPARK-42594:
[~yumwang] [~gurwls223] gentel ping ~
> spark can not read
[
https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-42594:
---
Description:
1. Spark would save view schema as tabel param.
2. Spark will make tabel param as
[
https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-42594:
---
Attachment: image-2023-02-27-13-31-20-420.png
> spark can not read lastest view sql when run
ming95 created SPARK-42594:
--
Summary: spark can not read lastest view sql when run `create or
replace view` by hive
Key: SPARK-42594
URL: https://issues.apache.org/jira/browse/SPARK-42594
Project:
[
https://issues.apache.org/jira/browse/SPARK-42503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691391#comment-17691391
]
ming95 commented on SPARK-42503:
I test hive and mysql , they also do not have this validation .
[
https://issues.apache.org/jira/browse/SPARK-42503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-42503:
---
Description:
In Spark SQL, the conditions for the join use fields that are allowed to be
fields
[
https://issues.apache.org/jira/browse/SPARK-42503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691235#comment-17691235
]
ming95 commented on SPARK-42503:
[~yumwang] [~gurwls223] cc
> Spark SQL should do further
ming95 created SPARK-42503:
--
Summary: Spark SQL should do further validation on join condition
fields
Key: SPARK-42503
URL: https://issues.apache.org/jira/browse/SPARK-42503
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-42034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17686612#comment-17686612
]
ming95 commented on SPARK-42034:
I am interested in this pr, if no one else is in development I
[
https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655799#comment-17655799
]
ming95 commented on SPARK-39743:
[~euigeun_chung]
Currently, Spark uses `airCompressor` lib
[ https://issues.apache.org/jira/browse/SPARK-39743 ]
ming95 deleted comment on SPARK-39743:
was (Author: zing):
[~euigeun_chung]
see this Jira : https://issues.apache.org/jira/browse/SPARK-33978
> Unable to set zstd compression
[
https://issues.apache.org/jira/browse/SPARK-39743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655244#comment-17655244
]
ming95 commented on SPARK-39743:
[~euigeun_chung]
see this Jira :
[
https://issues.apache.org/jira/browse/SPARK-40808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 resolved SPARK-40808.
Resolution: Fixed
expected behavior
> Infer schema for CSV files - wrong behavior using header +
[ https://issues.apache.org/jira/browse/SPARK-40588 ]
ming95 deleted comment on SPARK-40588:
was (Author: zing):
Yes, I found the same problem. This should be a bug in Spark. When the sorting
field is the same as the dynamic partitioning
[
https://issues.apache.org/jira/browse/SPARK-40885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-40885:
---
Description:
When using dynamic partitions to write data and sort partitions and data
fields,
[
https://issues.apache.org/jira/browse/SPARK-40885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-40885:
---
Description:
When using dynamic partitions to write data and sort partitions and data
fields,
[
https://issues.apache.org/jira/browse/SPARK-40885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-40885:
---
Attachment: 1666494504884.jpg
> Spark will filter out data field sorting when dynamic partitions
[
https://issues.apache.org/jira/browse/SPARK-40885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-40885:
---
External issue URL: https://issues.apache.org/jira/browse/SPARK-40588
Description:
When
ming95 created SPARK-40885:
--
Summary: Spark will filter out data field sorting when dynamic
partitions and data fields are sorted at the same time
Key: SPARK-40885
URL:
[
https://issues.apache.org/jira/browse/SPARK-40588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17622728#comment-17622728
]
ming95 commented on SPARK-40588:
Yes, I found the same problem. This should be a bug in Spark.
[
https://issues.apache.org/jira/browse/SPARK-40808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17622594#comment-17622594
]
ming95 edited comment on SPARK-40808 at 10/22/22 9:39 AM:
--
[~ohadm]
[
https://issues.apache.org/jira/browse/SPARK-40808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17622594#comment-17622594
]
ming95 commented on SPARK-40808:
[~ohadm]
In spark , infer csv schema will skip first line
[
https://issues.apache.org/jira/browse/SPARK-40563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-40563:
---
Fix Version/s: 3.4.0
(was: 3.3.1)
> Error at where clause, when sql case
[
https://issues.apache.org/jira/browse/SPARK-40563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618973#comment-17618973
]
ming95 commented on SPARK-40563:
[~yumwang]
The problem was not fixed in branch-3.3.
> Error
[
https://issues.apache.org/jira/browse/SPARK-40808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618277#comment-17618277
]
ming95 commented on SPARK-40808:
[~ohadm]
Can you provide code to reproduce this issue.
>
[
https://issues.apache.org/jira/browse/SPARK-40588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17618265#comment-17618265
]
ming95 commented on SPARK-40588:
After my test, I think this is not a problem of AQE, because
[
https://issues.apache.org/jira/browse/SPARK-40588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-40588:
---
Attachment: image-2022-10-16-22-05-47-159.png
> Sorting issue with AQE turned on
>
[
https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612135#comment-17612135
]
ming95 edited comment on SPARK-40610 at 10/3/22 10:38 AM:
--
Spark will
[
https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612241#comment-17612241
]
ming95 commented on SPARK-40610:
[~icyjhl]
If `dt` is string type, you can cast
[
https://issues.apache.org/jira/browse/SPARK-40602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612166#comment-17612166
]
ming95 edited comment on SPARK-40602 at 10/3/22 4:11 AM:
-
It seems to be
[
https://issues.apache.org/jira/browse/SPARK-40602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612166#comment-17612166
]
ming95 commented on SPARK-40602:
It seems to be a problem with the hive metastore and S3. You
[
https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-40610:
---
Attachment: (was: image-2022-10-03-00-21-24-506.png)
> Spark fall back to use getPartitions
[
https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612135#comment-17612135
]
ming95 edited comment on SPARK-40610 at 10/2/22 4:26 PM:
-
Spark will
[
https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612135#comment-17612135
]
ming95 commented on SPARK-40610:
Spark will perform partition pruning when executing the sql of
[
https://issues.apache.org/jira/browse/SPARK-40610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-40610:
---
Attachment: image-2022-10-03-00-21-24-506.png
> Spark fall back to use getPartitions instead of
[ https://issues.apache.org/jira/browse/SPARK-40563 ]
ming95 deleted comment on SPARK-40563:
was (Author: zing):
The issue has been fixed in the latest master branch.
> Error at where clause, when sql case executes by else branch
>
[
https://issues.apache.org/jira/browse/SPARK-40563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 resolved SPARK-40563.
Fix Version/s: 3.3.1
Resolution: Fixed
The issue has been fixed in the latest master
[
https://issues.apache.org/jira/browse/SPARK-40563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17611343#comment-17611343
]
ming95 commented on SPARK-40563:
In my test , the issue has been fixed in the latest master
[
https://issues.apache.org/jira/browse/SPARK-40563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17609536#comment-17609536
]
ming95 commented on SPARK-40563:
I can reproduce this problem. I can try to fix this issue if no
[
https://issues.apache.org/jira/browse/SPARK-40388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602307#comment-17602307
]
ming95 commented on SPARK-40388:
Hi [~praetp] , can you provide a way to reproduce this bug?
>
[
https://issues.apache.org/jira/browse/SPARK-40140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ming95 updated SPARK-40140:
---
Attachment: running.png
> REST API for SQL level information does not show information on running
[
https://issues.apache.org/jira/browse/SPARK-40140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582460#comment-17582460
]
ming95 commented on SPARK-40140:
!running.png!
> REST API for SQL level information does not
[
https://issues.apache.org/jira/browse/SPARK-40140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582459#comment-17582459
]
ming95 edited comment on SPARK-40140 at 8/21/22 4:07 AM:
-
[~yeachan153]
[
https://issues.apache.org/jira/browse/SPARK-40140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582459#comment-17582459
]
ming95 edited comment on SPARK-40140 at 8/21/22 4:06 AM:
-
[~yeachan153]
[
https://issues.apache.org/jira/browse/SPARK-40140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582459#comment-17582459
]
ming95 commented on SPARK-40140:
[~yeachan153]
In my test I was able to get running state
[
https://issues.apache.org/jira/browse/SPARK-40156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582458#comment-17582458
]
ming95 commented on SPARK-40156:
{noformat}
It should the return an ERROR_CLASS{noformat}
[
https://issues.apache.org/jira/browse/SPARK-40156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582450#comment-17582450
]
ming95 commented on SPARK-40156:
[~srielau]
`url_decode` is not a udf inside spark, maybe it's
[
https://issues.apache.org/jira/browse/SPARK-40140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581580#comment-17581580
]
ming95 commented on SPARK-40140:
I'm interested in this issue and I can fix it, if no one else
[
https://issues.apache.org/jira/browse/SPARK-40062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579217#comment-17579217
]
ming95 commented on SPARK-40062:
I can't reproduce the problem.
Can you give the specific
85 matches
Mail list logo
