[
https://issues.apache.org/jira/browse/SPARK-25021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
holdenk resolved SPARK-25021.
-
Resolution: Fixed
Fix Version/s: 3.0.0
Merged for 3 - open to the discussion around backporting.
[
https://issues.apache.org/jira/browse/SPARK-25021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
holdenk reassigned SPARK-25021:
---
Assignee: Ilan Filonenko
> Add spark.executor.pyspark.memory support to Kubernetes
>
[
https://issues.apache.org/jira/browse/SPARK-25072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608293#comment-16608293
]
Apache Spark commented on SPARK-25072:
--
User 'xuanyuanking' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-25072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608292#comment-16608292
]
Apache Spark commented on SPARK-25072:
--
User 'xuanyuanking' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608291#comment-16608291
]
Apache Spark commented on SPARK-25368:
--
User 'wangyum' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608290#comment-16608290
]
Apache Spark commented on SPARK-25368:
--
User 'wangyum' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25368:
Assignee: (was: Apache Spark)
> Incorrect predicate pushdown returns wrong result
>
[
https://issues.apache.org/jira/browse/SPARK-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25368:
Assignee: Apache Spark
> Incorrect predicate pushdown returns wrong result
>
[
https://issues.apache.org/jira/browse/SPARK-23880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li updated SPARK-23880:
Summary: table cache should be lazy and don't trigger any job (was: table
cache should be lazy and don't
[
https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608256#comment-16608256
]
Liang-Chi Hsieh commented on SPARK-25378:
-
Thanks for pinging me.
I agreed with [~hvanhovell].
[
https://issues.apache.org/jira/browse/SPARK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608217#comment-16608217
]
Apache Spark commented on SPARK-17916:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608216#comment-16608216
]
Apache Spark commented on SPARK-25241:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608215#comment-16608215
]
Apache Spark commented on SPARK-17916:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608189#comment-16608189
]
Apache Spark commented on SPARK-25384:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608188#comment-16608188
]
Apache Spark commented on SPARK-25384:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25384:
Assignee: Apache Spark
> Removing spark.sql.fromJsonForceNullableSchema
>
[
https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608187#comment-16608187
]
Apache Spark commented on SPARK-25384:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25384:
Assignee: (was: Apache Spark)
> Removing spark.sql.fromJsonForceNullableSchema
>
Maxim Gekk created SPARK-25384:
--
Summary: Removing spark.sql.fromJsonForceNullableSchema
Key: SPARK-25384
URL: https://issues.apache.org/jira/browse/SPARK-25384
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608171#comment-16608171
]
Xiangrui Meng commented on SPARK-25378:
---
Btw, thanks for suggesting the right way to get strings
[
https://issues.apache.org/jira/browse/SPARK-25375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-25375.
-
Resolution: Fixed
Assignee: Dongjoon Hyun
Fix Version/s: 2.4.0
> Reenable qualified
[
https://issues.apache.org/jira/browse/SPARK-20636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-20636.
-
Resolution: Fixed
Fix Version/s: 3.0.0
> Eliminate unnecessary shuffle with adjacent Window
[
https://issues.apache.org/jira/browse/SPARK-20636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li reassigned SPARK-20636:
---
Assignee: Michael Styles (was: Apache Spark)
> Eliminate unnecessary shuffle with adjacent Window
[
https://issues.apache.org/jira/browse/SPARK-23243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608169#comment-16608169
]
Bruce Robbins commented on SPARK-23243:
---
BTW, I took a stab at back porting it to 2.2, but to get
[
https://issues.apache.org/jira/browse/SPARK-23243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608167#comment-16608167
]
Bruce Robbins commented on SPARK-23243:
---
Any plans to back port this to 2.2?
>
[
https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608165#comment-16608165
]
Xiangrui Meng commented on SPARK-25378:
---
This is a breaking change anyway. The error comes from
[
https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608122#comment-16608122
]
Herman van Hovell commented on SPARK-25378:
---
I think this particular use is wrong.
[
https://issues.apache.org/jira/browse/SPARK-25381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608109#comment-16608109
]
Apache Spark commented on SPARK-25381:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25381:
Assignee: (was: Apache Spark)
> Stratified sampling by Column argument
>
[
https://issues.apache.org/jira/browse/SPARK-25381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608108#comment-16608108
]
Apache Spark commented on SPARK-25381:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25381:
Assignee: Apache Spark
> Stratified sampling by Column argument
>
[
https://issues.apache.org/jira/browse/SPARK-25382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25382:
--
Description: A follow-up task from SPARK-25345. We might need to support
sampling
Xiangrui Meng created SPARK-25383:
-
Summary: Image data source supports sample pushdown
Key: SPARK-25383
URL: https://issues.apache.org/jira/browse/SPARK-25383
Project: Spark
Issue Type: New
[
https://issues.apache.org/jira/browse/SPARK-25345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25345:
--
Description: After SPARK-22666, we can deprecate the public APIs in
ImageSchema
Xiangrui Meng created SPARK-25382:
-
Summary: Remove ImageSchema.readImages in 3.0
Key: SPARK-25382
URL: https://issues.apache.org/jira/browse/SPARK-25382
Project: Spark
Issue Type: Task
[
https://issues.apache.org/jira/browse/SPARK-25345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25345:
--
Summary: Deprecate readImages APIs from ImageSchema (was: Deprecate public
APIs from
[
https://issues.apache.org/jira/browse/SPARK-25345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-25345:
-
Assignee: Weichen Xu
> Deprecate public APIs from ImageSchema
>
[
https://issues.apache.org/jira/browse/SPARK-25345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-25345.
---
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 22349
Maxim Gekk created SPARK-25381:
--
Summary: Stratified sampling by Column argument
Key: SPARK-25381
URL: https://issues.apache.org/jira/browse/SPARK-25381
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608092#comment-16608092
]
Xiangrui Meng commented on SPARK-25378:
---
Seems caused by SPARK-23875. cc: [~viirya] [~hvanhovell]
[
https://issues.apache.org/jira/browse/SPARK-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608079#comment-16608079
]
Yuming Wang commented on SPARK-25368:
-
I think the root cause is
[
https://issues.apache.org/jira/browse/SPARK-23670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608047#comment-16608047
]
Michael Spector commented on SPARK-23670:
-
Reported: SPARK-25380
> Memory leak of
[
https://issues.apache.org/jira/browse/SPARK-25380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608045#comment-16608045
]
Michael Spector commented on SPARK-25380:
-
Originally reported in SPARK-23670
> Generated
Michael Spector created SPARK-25380:
---
Summary: Generated plans occupy over 50% of Spark driver memory
Key: SPARK-25380
URL: https://issues.apache.org/jira/browse/SPARK-25380
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-25380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Spector updated SPARK-25380:
Attachment: Screen Shot 2018-09-06 at 23.19.56.png
> Generated plans occupy over 50% of
[
https://issues.apache.org/jira/browse/SPARK-25380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Spector updated SPARK-25380:
Attachment: heapdump_OOM.png
> Generated plans occupy over 50% of Spark driver memory
>
[
https://issues.apache.org/jira/browse/SPARK-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608030#comment-16608030
]
Lev Katzav commented on SPARK-25368:
[~yumwang] yes, the code worked as expected on the master
[
https://issues.apache.org/jira/browse/SPARK-25368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608013#comment-16608013
]
Yuming Wang commented on SPARK-25368:
-
Can you try master, You provided test case can passed since
[
https://issues.apache.org/jira/browse/SPARK-25379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25379:
Assignee: (was: Apache Spark)
> Improve ColumnPruning performance
>
[
https://issues.apache.org/jira/browse/SPARK-25379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16607975#comment-16607975
]
Apache Spark commented on SPARK-25379:
--
User 'mgaido91' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16607977#comment-16607977
]
Apache Spark commented on SPARK-25379:
--
User 'mgaido91' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-25379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-25379:
Assignee: Apache Spark
> Improve ColumnPruning performance
>
[
https://issues.apache.org/jira/browse/SPARK-25091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16607970#comment-16607970
]
Liang-Chi Hsieh commented on SPARK-25091:
-
I think this is duplicate to SPARK-24889.
> UNCACHE
[
https://issues.apache.org/jira/browse/SPARK-25379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Gaido updated SPARK-25379:
Description:
The {{--}} operation on {{AttributeSet}} is quite expensive, especially where
many
Marco Gaido created SPARK-25379:
---
Summary: Improve ColumnPruning performance
Key: SPARK-25379
URL: https://issues.apache.org/jira/browse/SPARK-25379
Project: Spark
Issue Type: Sub-task
Xiangrui Meng created SPARK-25378:
-
Summary: ArrayData.toArray assume UTF8String
Key: SPARK-25378
URL: https://issues.apache.org/jira/browse/SPARK-25378
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-25377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Iverson Hu updated SPARK-25377:
---
Description:
When I use SQL dataframe in application, I found that dataframe.cache is
invalid,
Iverson Hu created SPARK-25377:
--
Summary: spark sql dataframe cache is invalid
Key: SPARK-25377
URL: https://issues.apache.org/jira/browse/SPARK-25377
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25376:
--
Summary: Scenarios we should handle but missed in 2.4 for barrier execution
mode (was:
[
https://issues.apache.org/jira/browse/SPARK-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25376:
--
Description:
[~irashid] You mentioned that there are couple scenarios we should handle in
60 matches
Mail list logo