[jira] [Updated] (SPARK-32060) Huber loss Convergence

[ https://issues.apache.org/jira/browse/SPARK-32060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-32060: - Description: |performace test in https://issues.apache.org/jira/browse/SPARK-31783, Huber loss

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang commented on SPARK-32130: [~lotus2you] Thank you for your feedback. Lots

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 7:39 AM:

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 7:58 AM:

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 7:59 AM:

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 8:05 AM:

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 8:05 AM:

[jira] [Commented] (SPARK-32115) Incorrect results for SUBSTRING when overflow

[ https://issues.apache.org/jira/browse/SPARK-32115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148409#comment-17148409 ] Yuanjian Li commented on SPARK-32115: - Thank you for verifying! [~dongjoon] > Incor

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 8:07 AM:

[jira] [Commented] (SPARK-32060) Huber loss Convergence

[ https://issues.apache.org/jira/browse/SPARK-32060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148412#comment-17148412 ] zhengruifeng commented on SPARK-32060: -- I found that the optimization of Huber Loss

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148416#comment-17148416 ] Jungtaek Lim commented on SPARK-32130: -- I've feeling that "opt-in" approach is corr

[jira] [Comment Edited] (SPARK-32060) Huber loss Convergence

[ https://issues.apache.org/jira/browse/SPARK-32060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148412#comment-17148412 ] zhengruifeng edited comment on SPARK-32060 at 6/30/20, 8:17 AM: --

[jira] [Comment Edited] (SPARK-32060) Huber loss Convergence

[ https://issues.apache.org/jira/browse/SPARK-32060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148412#comment-17148412 ] zhengruifeng edited comment on SPARK-32060 at 6/30/20, 8:20 AM: --

[jira] [Updated] (SPARK-32060) Huber loss Convergence

[ https://issues.apache.org/jira/browse/SPARK-32060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-32060: - Attachment: (was: image-2020-06-28-18-05-28-867.png) > Huber loss Convergence >

[jira] [Commented] (SPARK-28861) Jetty property handling: java.lang.NumberFormatException: For input string: "unknown".

[ https://issues.apache.org/jira/browse/SPARK-28861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148444#comment-17148444 ] Siddhant Chadha commented on SPARK-28861: - I’d like to take this up > Jetty pro

[jira] [Commented] (SPARK-3181) Add Robust Regression Algorithm with Huber Estimator

[ https://issues.apache.org/jira/browse/SPARK-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148452#comment-17148452 ] zhengruifeng commented on SPARK-3181: - I am working on blockify+gemv/gemm for better

[jira] [Created] (SPARK-32136) Spark producing incorrect groupBy results when key is a struct with nullable properties

Jason Moore created SPARK-32136: --- Summary: Spark producing incorrect groupBy results when key is a struct with nullable properties Key: SPARK-32136 URL: https://issues.apache.org/jira/browse/SPARK-32136

[jira] [Updated] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gourav updated SPARK-32130: --- Attachment: SPARK 32130 - replication and findings.ipynb > Spark 3.0 json load performance is unacceptable i

[jira] [Updated] (SPARK-32136) Spark producing incorrect groupBy results when key is a struct with nullable properties

[ https://issues.apache.org/jira/browse/SPARK-32136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Moore updated SPARK-32136: Description: I'm in the process of migrating from Spark 2.4.x to Spark 3.0.0 and I'm noticing a b

[jira] [Updated] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gourav updated SPARK-32130: --- Attachment: (was: SPARK 32130 - replication and findings.ipynb) > Spark 3.0 json load performance is una

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148459#comment-17148459 ] Gourav commented on SPARK-32130: [~JinxinTang] and [~lotus2you] I think that it is only

[jira] [Updated] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gourav updated SPARK-32130: --- Attachment: SPARK 32130 - replication and findings.ipynb > Spark 3.0 json load performance is unacceptable i

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148459#comment-17148459 ] Gourav edited comment on SPARK-32130 at 6/30/20, 9:02 AM: -- [~Ji

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148501#comment-17148501 ] JinxinTang commented on SPARK-32130: [~gourav.sengupta] Nice notebook, is seems the

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148546#comment-17148546 ] Jungtaek Lim commented on SPARK-32130: -- For me it's reproduced consistently. Please

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148559#comment-17148559 ] Jungtaek Lim commented on SPARK-32130: -- So anyone can just reproduce via running sp

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148570#comment-17148570 ] Jungtaek Lim commented on SPARK-32130: -- Looks like we already saw the difference bu

[jira] [Created] (SPARK-32137) AttributeError: Can only use .dt accessor with datetimelike values

David Lacalle Castillo created SPARK-32137: -- Summary: AttributeError: Can only use .dt accessor with datetimelike values Key: SPARK-32137 URL: https://issues.apache.org/jira/browse/SPARK-32137

[jira] [Updated] (SPARK-32137) AttributeError: Can only use .dt accessor with datetimelike values

[ https://issues.apache.org/jira/browse/SPARK-32137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Lacalle Castillo updated SPARK-32137: --- Priority: Critical (was: Major) > AttributeError: Can only use .dt accessor

[jira] [Commented] (SPARK-32083) Unnecessary tasks are launched when input is empty with AQE

[ https://issues.apache.org/jira/browse/SPARK-32083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148584#comment-17148584 ] Apache Spark commented on SPARK-32083: -- User 'manuzhang' has created a pull request

[jira] [Commented] (SPARK-32083) Unnecessary tasks are launched when input is empty with AQE

[ https://issues.apache.org/jira/browse/SPARK-32083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148585#comment-17148585 ] Apache Spark commented on SPARK-32083: -- User 'manuzhang' has created a pull request

[jira] [Commented] (SPARK-25556) Predicate Pushdown for Nested fields

[ https://issues.apache.org/jira/browse/SPARK-25556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148587#comment-17148587 ] Apache Spark commented on SPARK-25556: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148588#comment-17148588 ] Apache Spark commented on SPARK-17636: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-31026) Parquet predicate pushdown on columns with dots

[ https://issues.apache.org/jira/browse/SPARK-31026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148590#comment-17148590 ] Apache Spark commented on SPARK-31026: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148589#comment-17148589 ] Apache Spark commented on SPARK-17636: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-31060) Handle column names containing `dots` in data source `Filter`

[ https://issues.apache.org/jira/browse/SPARK-31060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148592#comment-17148592 ] Apache Spark commented on SPARK-31060: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-31026) Parquet predicate pushdown on columns with dots

[ https://issues.apache.org/jira/browse/SPARK-31026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148593#comment-17148593 ] Apache Spark commented on SPARK-31026: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-31060) Handle column names containing `dots` in data source `Filter`

[ https://issues.apache.org/jira/browse/SPARK-31060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148594#comment-17148594 ] Apache Spark commented on SPARK-31060: -- User 'HyukjinKwon' has created a pull reque

[jira] [Updated] (SPARK-29919) remove python2 test execution

[ https://issues.apache.org/jira/browse/SPARK-29919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29919: - Parent: (was: SPARK-27884) Issue Type: Bug (was: Sub-task) > remove python2 test ex

[jira] [Updated] (SPARK-29919) remove python2 test execution

[ https://issues.apache.org/jira/browse/SPARK-29919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29919: - Parent: SPARK-29909 Issue Type: Sub-task (was: Bug) > remove python2 test execution > -

[jira] [Updated] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjeev Mishra updated SPARK-32130: --- Description: We are planning to move to Spark 3 but the read performance of our json files

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148663#comment-17148663 ] Sanjeev Mishra commented on SPARK-32130: I tried to load entire dataset using ab

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148663#comment-17148663 ] Sanjeev Mishra edited comment on SPARK-32130 at 6/30/20, 1:35 PM:

[jira] [Commented] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148696#comment-17148696 ] Apache Spark commented on SPARK-31797: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148694#comment-17148694 ] Apache Spark commented on SPARK-31797: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-32068) Spark 3 UI task launch time show in error time zone

[ https://issues.apache.org/jira/browse/SPARK-32068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-32068: - Assignee: JinxinTang > Spark 3 UI task launch time show in error time zone > --

[jira] [Resolved] (SPARK-32068) Spark 3 UI task launch time show in error time zone

[ https://issues.apache.org/jira/browse/SPARK-32068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-32068. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed > Spark 3 UI task lau

[jira] [Updated] (SPARK-31816) Create high level description about JDBC connection providers for users/developers

[ https://issues.apache.org/jira/browse/SPARK-31816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-31816: -- Summary: Create high level description about JDBC connection providers for users/developers (

[jira] [Commented] (SPARK-31816) Create high level description about JDBC connection providers for users/developers

[ https://issues.apache.org/jira/browse/SPARK-31816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148712#comment-17148712 ] Gabor Somogyi commented on SPARK-31816: --- Originally I've thought it's enough to ad

[jira] [Commented] (SPARK-32135) Show Spark Driver name on Spark history web page

[ https://issues.apache.org/jira/browse/SPARK-32135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148742#comment-17148742 ] Thomas Graves commented on SPARK-32135: --- [~gaurangi]can you please clarify what yo

[jira] [Commented] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster

[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148746#comment-17148746 ] Thomas Graves commented on SPARK-32119: --- You can specify the jars in extraClassPat

[jira] [Resolved] (SPARK-29803) remove all instances of 'from __future__ import print_function'

[ https://issues.apache.org/jira/browse/SPARK-29803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29803. -- Resolution: Duplicate > remove all instances of 'from __future__ import print_function' > ---

[jira] [Updated] (SPARK-29803) remove all instances of 'from __future__ import print_function'

[ https://issues.apache.org/jira/browse/SPARK-29803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29803: - Parent: (was: SPARK-29909) Issue Type: Improvement (was: Sub-task) > remove all ins

[jira] [Commented] (SPARK-29803) remove all instances of 'from __future__ import print_function'

[ https://issues.apache.org/jira/browse/SPARK-29803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148755#comment-17148755 ] Hyukjin Kwon commented on SPARK-29803: -- I will do it at SPARK-29909 > remove all

[jira] [Updated] (SPARK-29919) remove python2 test execution

[ https://issues.apache.org/jira/browse/SPARK-29919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29919: - Parent: (was: SPARK-29909) Issue Type: Improvement (was: Sub-task) > remove python2

[jira] [Updated] (SPARK-29919) remove python2 test execution

[ https://issues.apache.org/jira/browse/SPARK-29919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29919: - Parent: SPARK-29909 Issue Type: Sub-task (was: Improvement) > remove python2 test execu

[jira] [Updated] (SPARK-29919) Remove python2 test execution in Jenkins environment

[ https://issues.apache.org/jira/browse/SPARK-29919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29919: - Summary: Remove python2 test execution in Jenkins environment (was: remove python2 test executi

[jira] [Comment Edited] (SPARK-29803) remove all instances of 'from __future__ import print_function'

[ https://issues.apache.org/jira/browse/SPARK-29803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148755#comment-17148755 ] Hyukjin Kwon edited comment on SPARK-29803 at 6/30/20, 3:07 PM: --

[jira] [Updated] (SPARK-29802) Update remaining python scripts in repo to python3 shebang

[ https://issues.apache.org/jira/browse/SPARK-29802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29802: - Summary: Update remaining python scripts in repo to python3 shebang (was: update remaining pyth

[jira] [Created] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in the main and dev codes

Hyukjin Kwon created SPARK-32138: Summary: Drop Python 2, 3.4 and 3.5 in the main and dev codes Key: SPARK-32138 URL: https://issues.apache.org/jira/browse/SPARK-32138 Project: Spark Issue Ty

[jira] [Updated] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in codes and documentation

[ https://issues.apache.org/jira/browse/SPARK-32138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32138: - Summary: Drop Python 2, 3.4 and 3.5 in codes and documentation (was: Drop Python 2, 3.4 and 3.5

[jira] [Commented] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in codes and documentation

[ https://issues.apache.org/jira/browse/SPARK-32138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148766#comment-17148766 ] Apache Spark commented on SPARK-32138: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in codes and documentation

[ https://issues.apache.org/jira/browse/SPARK-32138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32138: Assignee: Apache Spark > Drop Python 2, 3.4 and 3.5 in codes and documentation >

[jira] [Assigned] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in codes and documentation

[ https://issues.apache.org/jira/browse/SPARK-32138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32138: Assignee: (was: Apache Spark) > Drop Python 2, 3.4 and 3.5 in codes and documentation

[jira] [Commented] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster

[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148790#comment-17148790 ] Kousuke Saruta commented on SPARK-32119: Yeah I know it works with extraClassPat

[jira] [Updated] (SPARK-32026) Add PrometheusServletSuite

[ https://issues.apache.org/jira/browse/SPARK-32026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eren Avsarogullari updated SPARK-32026: --- Description: This Jira aims to be added _PrometheusServletSuite_. *Note:* This jira

[jira] [Commented] (SPARK-32132) Thriftserver interval returns "4 weeks 2 days" in 2.4 and "30 days" in 3.0

[ https://issues.apache.org/jira/browse/SPARK-32132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148836#comment-17148836 ] Wenchen Fan commented on SPARK-32132: - We did it intentionally in https://github.co

[jira] [Updated] (SPARK-31935) Hadoop file system config should be effective in data source options

[ https://issues.apache.org/jira/browse/SPARK-31935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31935: Issue Type: Bug (was: Improvement) > Hadoop file system config should be effective in data source

[jira] [Commented] (SPARK-32132) Thriftserver interval returns "4 weeks 2 days" in 2.4 and "30 days" in 3.0

[ https://issues.apache.org/jira/browse/SPARK-32132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148848#comment-17148848 ] Juliusz Sompolski commented on SPARK-32132: --- Also 2.4 adds "interval" at the s

[jira] [Updated] (SPARK-31935) Hadoop file system config should be effective in data source options

[ https://issues.apache.org/jira/browse/SPARK-31935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-31935: --- Affects Version/s: (was: 3.0.1) (was: 3.1.0) 2.

[jira] [Assigned] (SPARK-31336) Support Oracle Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31336: - Assignee: Gabor Somogyi > Support Oracle Kerberos login in JDBC connector > ---

[jira] [Resolved] (SPARK-31336) Support Oracle Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31336. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28863 [https://

[jira] [Updated] (SPARK-32121) ExternalShuffleBlockResolverSuite failed on Windows

[ https://issues.apache.org/jira/browse/SPARK-32121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-32121: -- Issue Type: Bug (was: Test) > ExternalShuffleBlockResolverSuite failed on Windows > -

[jira] [Commented] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148913#comment-17148913 ] Apache Spark commented on SPARK-32088: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148936#comment-17148936 ] Sean R. Owen commented on SPARK-32130: -- So, is the issue that it's trying and faili

[jira] [Created] (SPARK-32139) Unify Classification Training Summary

Huaxin Gao created SPARK-32139: -- Summary: Unify Classification Training Summary Key: SPARK-32139 URL: https://issues.apache.org/jira/browse/SPARK-32139 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-31893) Add a generic ClassificationSummary trait

[ https://issues.apache.org/jira/browse/SPARK-31893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-31893: --- Parent: SPARK-32139 Issue Type: Sub-task (was: Improvement) > Add a generic ClassificationS

[jira] [Created] (SPARK-32140) Add summary to FMClassificationModel

Huaxin Gao created SPARK-32140: -- Summary: Add summary to FMClassificationModel Key: SPARK-32140 URL: https://issues.apache.org/jira/browse/SPARK-32140 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-23631) Add summary to RandomForestClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-23631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-23631: --- Parent: SPARK-32139 Issue Type: Sub-task (was: New Feature) > Add summary to RandomForestCl

[jira] [Updated] (SPARK-20249) Add summary for LinearSVCModel

[ https://issues.apache.org/jira/browse/SPARK-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-20249: --- Parent: SPARK-32139 Issue Type: Sub-task (was: Improvement) > Add summary for LinearSVCMode

[jira] [Updated] (SPARK-28664) ORDER BY in aggregate function

[ https://issues.apache.org/jira/browse/SPARK-28664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Will Zimmerman updated SPARK-28664: --- Attachment: image-2020-06-30-15-49-46-796.png > ORDER BY in aggregate function > ---

[jira] [Commented] (SPARK-28664) ORDER BY in aggregate function

[ https://issues.apache.org/jira/browse/SPARK-28664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148952#comment-17148952 ] Will Zimmerman commented on SPARK-28664: [~yumwang] - Would this allow for the c

[jira] [Commented] (SPARK-32140) Add summary to FMClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-32140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148984#comment-17148984 ] Apache Spark commented on SPARK-32140: -- User 'huaxingao' has created a pull request

[jira] [Assigned] (SPARK-32140) Add summary to FMClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-32140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32140: Assignee: Apache Spark > Add summary to FMClassificationModel > -

[jira] [Commented] (SPARK-32140) Add summary to FMClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-32140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148985#comment-17148985 ] Apache Spark commented on SPARK-32140: -- User 'huaxingao' has created a pull request

[jira] [Assigned] (SPARK-32140) Add summary to FMClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-32140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32140: Assignee: (was: Apache Spark) > Add summary to FMClassificationModel > --

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149019#comment-17149019 ] Jungtaek Lim commented on SPARK-32130: -- There might be some tricks to make type inf

[jira] [Created] (SPARK-32141) Repartition leads to out of memory

Lekshmi Nair created SPARK-32141: Summary: Repartition leads to out of memory Key: SPARK-32141 URL: https://issues.apache.org/jira/browse/SPARK-32141 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-32142) Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite

Hyukjin Kwon created SPARK-32142: Summary: Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite Key: SPARK-32142 URL: https://issues.apache.org/jira/browse

[jira] [Assigned] (SPARK-32142) Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite

[ https://issues.apache.org/jira/browse/SPARK-32142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32142: Assignee: (was: Apache Spark) > Keep the original tests and codes to avoid potential

[jira] [Assigned] (SPARK-32142) Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite

[ https://issues.apache.org/jira/browse/SPARK-32142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32142: Assignee: Apache Spark > Keep the original tests and codes to avoid potential conflicts i

[jira] [Commented] (SPARK-32142) Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite

[ https://issues.apache.org/jira/browse/SPARK-32142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149035#comment-17149035 ] Apache Spark commented on SPARK-32142: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149044#comment-17149044 ] huangtianhua commented on SPARK-32088: -- the test success by the modification https

[jira] [Commented] (SPARK-32136) Spark producing incorrect groupBy results when key is a struct with nullable properties

[ https://issues.apache.org/jira/browse/SPARK-32136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149073#comment-17149073 ] Jason Moore commented on SPARK-32136: - Here is a similar test, and why it's a proble

[jira] [Commented] (SPARK-27780) Shuffle server & client should be versioned to enable smoother upgrade

[ https://issues.apache.org/jira/browse/SPARK-27780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149083#comment-17149083 ] Jason Moore commented on SPARK-27780: - I encounter this on 3.0.0 running with a much

[jira] [Created] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

Lantao Jin created SPARK-32143: -- Summary: Fast fail when the AQE skew join produce too many splits Key: SPARK-32143 URL: https://issues.apache.org/jira/browse/SPARK-32143 Project: Spark Issue Ty

[jira] [Updated] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-32143: --- Description: In handling skewed SortMergeJoin, when matching partitions from the left side and the

[jira] [Commented] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149084#comment-17149084 ] Lantao Jin commented on SPARK-32143: A PR will be submitted soon. > Fast fail when

[jira] [Updated] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-32143: --- Description: In handling skewed SortMergeJoin, when matching partitions from the left side and the

[jira] [Updated] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-32143: --- Description: In handling skewed SortMergeJoin, when matching partitions from the left side and the

  1   2   >