[
https://issues.apache.org/jira/browse/SPARK-34470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286887#comment-17286887
]
Apache Spark commented on SPARK-34470:
--
User 'zhengruifeng' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286885#comment-17286885
]
Apache Spark commented on SPARK-34470:
--
User 'zhengruifeng' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34470:
Assignee: (was: Apache Spark)
> VectorSlicer use ordering if possible
>
[
https://issues.apache.org/jira/browse/SPARK-34470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34470:
Assignee: Apache Spark
> VectorSlicer use ordering if possible
>
zhengruifeng created SPARK-34470:
Summary: VectorSlicer use ordering if possible
Key: SPARK-34470
URL: https://issues.apache.org/jira/browse/SPARK-34470
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-32027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286865#comment-17286865
]
Seulki jake Han commented on SPARK-32027:
-
I am getting the same error as [~kristopherkane] 's
[
https://issues.apache.org/jira/browse/SPARK-34462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286837#comment-17286837
]
Zhang Jianguo edited comment on SPARK-34462 at 2/19/21, 6:08 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-34302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286858#comment-17286858
]
Terry Kim commented on SPARK-34302:
---
[~maxgekk] Thanks for letting me know!
> Migrate ALTER TABLE ..
[
https://issues.apache.org/jira/browse/SPARK-34302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286842#comment-17286842
]
Maxim Gekk commented on SPARK-34302:
[~imback82] I don't plan to work on this in the near future.
[
https://issues.apache.org/jira/browse/SPARK-34466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-34466:
---
Assignee: Maxim Gekk
> Improve docs for ALTER TABLE .. RENAME TO
>
[
https://issues.apache.org/jira/browse/SPARK-34466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-34466.
-
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 31586
[
https://issues.apache.org/jira/browse/SPARK-34469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286836#comment-17286836
]
Apache Spark commented on SPARK-34469:
--
User 'dongjoon-hyun' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286837#comment-17286837
]
Zhang Jianguo commented on SPARK-34462:
---
[~yumwang] I updated the description.
> thriftserver
[
https://issues.apache.org/jira/browse/SPARK-34462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhang Jianguo updated SPARK-34462:
--
Description:
I found there are too many empty files generated by thriftserver in HDFS.
It
[
https://issues.apache.org/jira/browse/SPARK-34469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286835#comment-17286835
]
Apache Spark commented on SPARK-34469:
--
User 'dongjoon-hyun' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-34462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhang Jianguo updated SPARK-34462:
--
Description:
I found there are too many empty files generated by thriftserver in HDFS.
It
[
https://issues.apache.org/jira/browse/SPARK-34469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34469:
Assignee: (was: Apache Spark)
> Ignore RegisterExecutor when SparkContext is stopped
[
https://issues.apache.org/jira/browse/SPARK-34469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34469:
Assignee: Apache Spark
> Ignore RegisterExecutor when SparkContext is stopped
>
[
https://issues.apache.org/jira/browse/SPARK-34462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhang Jianguo updated SPARK-34462:
--
Attachment: empty_file.PNG
> thriftserver create many empty files
>
Dongjoon Hyun created SPARK-34469:
-
Summary: Ignore RegisterExecutor when SparkContext is stopped
Key: SPARK-34469
URL: https://issues.apache.org/jira/browse/SPARK-34469
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-34384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-34384:
-
Description: ResourceProfile APIs missed a few API docs, we should add them
for the better user experience.
>
[
https://issues.apache.org/jira/browse/SPARK-34384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuyi updated SPARK-34384:
-
Summary: Add missing docs for ResourceProfile APIs (was: API cleanup for
ResourceProfile)
> Add missing docs
[
https://issues.apache.org/jira/browse/SPARK-34459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-34459.
-
Resolution: Invalid
> map_from_arrays() throws UnsupportedOperationException with array of
>
[
https://issues.apache.org/jira/browse/SPARK-34459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-34459:
-
Description:
An example to reproduce this error:
{code:scala}
sql("select map(1, 2) as m_a,
[
https://issues.apache.org/jira/browse/SPARK-32385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shannon Carey updated SPARK-32385:
--
Comment: was deleted
(was: Here's another reason that either a BOM or a move away from
[
https://issues.apache.org/jira/browse/SPARK-34465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-34465.
---
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 31584
[
https://issues.apache.org/jira/browse/SPARK-34465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-34465:
-
Assignee: Maxim Gekk
> Rename alter table exec nodes
> -
>
[
https://issues.apache.org/jira/browse/SPARK-34467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-34467.
---
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 31585
[
https://issues.apache.org/jira/browse/SPARK-34467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-34467:
-
Assignee: Dongjoon Hyun
> Upgrade Zstd-jni to 1.4.8-4
> ---
>
[
https://issues.apache.org/jira/browse/SPARK-32385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286704#comment-17286704
]
Shannon Carey edited comment on SPARK-32385 at 2/18/21, 8:19 PM:
-
Here's
[
https://issues.apache.org/jira/browse/SPARK-32385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286704#comment-17286704
]
Shannon Carey edited comment on SPARK-32385 at 2/18/21, 8:12 PM:
-
Here's
[
https://issues.apache.org/jira/browse/SPARK-32385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286704#comment-17286704
]
Shannon Carey commented on SPARK-32385:
---
Here's another reason that either a BOM or a move away
Maxim Gekk created SPARK-34468:
--
Summary: Fix v2 ALTER TABLE .. RENAME TO
Key: SPARK-34468
URL: https://issues.apache.org/jira/browse/SPARK-34468
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-34466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34466:
Assignee: (was: Apache Spark)
> Improve docs for ALTER TABLE .. RENAME TO
>
[
https://issues.apache.org/jira/browse/SPARK-34466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34466:
Assignee: Apache Spark
> Improve docs for ALTER TABLE .. RENAME TO
>
[
https://issues.apache.org/jira/browse/SPARK-34466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286678#comment-17286678
]
Apache Spark commented on SPARK-34466:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-34467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34467:
Assignee: (was: Apache Spark)
> Upgrade Zstd-jni to 1.4.8-4
>
[
https://issues.apache.org/jira/browse/SPARK-34467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34467:
Assignee: Apache Spark
> Upgrade Zstd-jni to 1.4.8-4
> ---
>
>
[
https://issues.apache.org/jira/browse/SPARK-34467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286676#comment-17286676
]
Apache Spark commented on SPARK-34467:
--
User 'dongjoon-hyun' has created a pull request for this
Dongjoon Hyun created SPARK-34467:
-
Summary: Upgrade Zstd-jni to 1.4.8-4
Key: SPARK-34467
URL: https://issues.apache.org/jira/browse/SPARK-34467
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-32027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286661#comment-17286661
]
Kristopher Kane commented on SPARK-32027:
-
I take that back, my trace is different.
[
https://issues.apache.org/jira/browse/SPARK-34454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-34454.
---
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 31577
Maxim Gekk created SPARK-34466:
--
Summary: Improve docs for ALTER TABLE .. RENAME TO
Key: SPARK-34466
URL: https://issues.apache.org/jira/browse/SPARK-34466
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-32027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286658#comment-17286658
]
Kristopher Kane commented on SPARK-32027:
-
I am seeing this on Spark 3.1.1 (Google Dataproc
[
https://issues.apache.org/jira/browse/SPARK-34439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Gekk resolved SPARK-34439.
Resolution: Won't Fix
> Recognize `spark_catalog` in new identifier while view/table renaming
>
[
https://issues.apache.org/jira/browse/SPARK-34465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34465:
Assignee: (was: Apache Spark)
> Rename alter table exec nodes
>
[
https://issues.apache.org/jira/browse/SPARK-34465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34465:
Assignee: Apache Spark
> Rename alter table exec nodes
> -
>
[
https://issues.apache.org/jira/browse/SPARK-34465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286636#comment-17286636
]
Apache Spark commented on SPARK-34465:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-34465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34465:
Assignee: (was: Apache Spark)
> Rename alter table exec nodes
>
[
https://issues.apache.org/jira/browse/SPARK-34465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Gekk updated SPARK-34465:
---
Description:
To be consistent with other exec nodes, rename:
* AlterTableAddPartitionExec ->
Maxim Gekk created SPARK-34465:
--
Summary: Rename alter table exec nodes
Key: SPARK-34465
URL: https://issues.apache.org/jira/browse/SPARK-34465
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-33843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-33843:
--
Summary: Upgrade to Zstd 1.4.8 (was: Upgrade to Zstd 1.4.7)
> Upgrade to Zstd 1.4.8
>
[
https://issues.apache.org/jira/browse/SPARK-32703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean R. Owen updated SPARK-32703:
-
Priority: Minor (was: Major)
> Enable dictionary filtering for Parquet vectorized reader
>
[
https://issues.apache.org/jira/browse/SPARK-32703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean R. Owen resolved SPARK-32703.
--
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 29542
[
https://issues.apache.org/jira/browse/SPARK-32703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean R. Owen reassigned SPARK-32703:
Assignee: Chao Sun
> Enable dictionary filtering for Parquet vectorized reader
>
[
https://issues.apache.org/jira/browse/SPARK-34464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Louis Fruleux updated SPARK-34464:
--
Priority: Minor (was: Major)
> `first` function is sorting the dataset while sometimes it is
[
https://issues.apache.org/jira/browse/SPARK-34464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Louis Fruleux updated SPARK-34464:
--
Description:
When one wants to groupBy and take any value (not necessarily the first), one
Louis Fruleux created SPARK-34464:
-
Summary: `first` function is sorting the dataset while sometimes
it is used to get "any value"
Key: SPARK-34464
URL: https://issues.apache.org/jira/browse/SPARK-34464
[
https://issues.apache.org/jira/browse/SPARK-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286548#comment-17286548
]
David Mavashev commented on SPARK-21770:
No, we didnt resolve the issue yet.
>
[
https://issues.apache.org/jira/browse/SPARK-34461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286517#comment-17286517
]
Thomas Graves commented on SPARK-34461:
---
So the discussions in the original SPIP and prs went back
[
https://issues.apache.org/jira/browse/SPARK-33739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-33739.
---
Fix Version/s: 3.2.0
Assignee: Steve Loughran
Resolution: Fixed
> Jobs
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286478#comment-17286478
]
David Li commented on SPARK-34463:
--
I'll take a look.
The reason for the three options is as follows:
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286476#comment-17286476
]
Weichen Xu commented on SPARK-34463:
[~lidavidm]
Btw, could you tell me why "use_threads=false" is
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286475#comment-17286475
]
Weichen Xu commented on SPARK-34463:
[~lidavidm] OK. Could you help build a simple reproduce case
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286474#comment-17286474
]
David Li commented on SPARK-34463:
--
This is a "case by case" kind of thing - it'll depend on what you
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286473#comment-17286473
]
Weichen Xu commented on SPARK-34463:
[~lidavidm] Yes it works when selfDestruct disabled.
And the
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286470#comment-17286470
]
David Li commented on SPARK-34463:
--
[~weichenxu123] Does it work when selfDestruct is not enabled? If
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weichen Xu updated SPARK-34463:
---
Description:
Environment:
apache/spark master
pandas version > 1.0.5
Reproduce code:
{code}
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286469#comment-17286469
]
Weichen Xu commented on SPARK-34463:
[~bryanc] [~lidavidm] [~hyukjin.kwon] Any idea ?
This is bug
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286469#comment-17286469
]
Weichen Xu edited comment on SPARK-34463 at 2/18/21, 1:15 PM:
--
[~bryanc]
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weichen Xu updated SPARK-34463:
---
Description:
Environment:
apache/park master
pandas version > 1.0.5
Reproduce code:
{code}
[
https://issues.apache.org/jira/browse/SPARK-34463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weichen Xu updated SPARK-34463:
---
Description:
Environment:
apache/park master
pandas version > 1.0.5
Reproduce code:
{code}
Weichen Xu created SPARK-34463:
--
Summary: toPandas failed with error: buffer source array is
read-only
Key: SPARK-34463
URL: https://issues.apache.org/jira/browse/SPARK-34463
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-34394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-34394:
---
Assignee: jiaan.geng
> Unify output of SHOW FUNCTIONS and pass output attributes properly
[
https://issues.apache.org/jira/browse/SPARK-34394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-34394.
-
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 31519
[
https://issues.apache.org/jira/browse/SPARK-34393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-34393.
-
Fix Version/s: 3.2.0
Assignee: jiaan.geng
Resolution: Fixed
> Unify output of
[
https://issues.apache.org/jira/browse/SPARK-34449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286441#comment-17286441
]
Apache Spark commented on SPARK-34449:
--
User 'sarutak' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-34459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286414#comment-17286414
]
jiaan.geng edited comment on SPARK-34459 at 2/18/21, 11:03 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-34459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286414#comment-17286414
]
jiaan.geng edited comment on SPARK-34459 at 2/18/21, 10:57 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-34459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286414#comment-17286414
]
jiaan.geng commented on SPARK-34459:
[~allisonwang-db] I test and cannot reproduce it.
{code:java}
[
https://issues.apache.org/jira/browse/SPARK-34437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-34437:
Assignee: Maxim Gekk
> Update Spark SQL guide about rebase DS options and SQL configs
>
81 matches
Mail list logo
