[jira] [Created] (SPARK-3696) Do not override user-defined conf_dir in spark-config.sh

WangTaoTheTonic created SPARK-3696: -- Summary: Do not override user-defined conf_dir in spark-config.sh Key: SPARK-3696 URL: https://issues.apache.org/jira/browse/SPARK-3696 Project: Spark

[jira] [Commented] (SPARK-3696) Do not override user-defined conf_dir in spark-config.sh

[ https://issues.apache.org/jira/browse/SPARK-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148825#comment-14148825 ] Apache Spark commented on SPARK-3696: - User 'WangTaoTheTonic' has created a pull

[jira] [Commented] (SPARK-3688) LogicalPlan can't resolve column correctlly

[ https://issues.apache.org/jira/browse/SPARK-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148855#comment-14148855 ] Apache Spark commented on SPARK-3688: - User 'tianyi' has created a pull request for

[jira] [Commented] (SPARK-3588) Gaussian Mixture Model clustering

[ https://issues.apache.org/jira/browse/SPARK-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148900#comment-14148900 ] Xiangrui Meng commented on SPARK-3588: -- Please follow the instructions at

[jira] [Commented] (SPARK-2377) Create a Python API for Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148992#comment-14148992 ] Kenichi Takagiwa commented on SPARK-2377: - I think it is good to discuss in

[jira] [Created] (SPARK-3698) Case sensitive check in spark sql is incompleted.

Wenchen Fan created SPARK-3698: -- Summary: Case sensitive check in spark sql is incompleted. Key: SPARK-3698 URL: https://issues.apache.org/jira/browse/SPARK-3698 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3698) Case sensitive check in spark sql is incompleted.

[ https://issues.apache.org/jira/browse/SPARK-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149087#comment-14149087 ] Apache Spark commented on SPARK-3698: - User 'cloud-fan' has created a pull request for

[jira] [Commented] (SPARK-3561) Native Hadoop/YARN integration for batch/ETL workloads

[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149148#comment-14149148 ] Oleg Zhurakousky commented on SPARK-3561: - Thank you for the interest guys. We are

[jira] [Resolved] (SPARK-3478) Profile Python tasks stage by stage in worker

[ https://issues.apache.org/jira/browse/SPARK-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3478. --- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2351

[jira] [Closed] (SPARK-3639) Kinesis examples set master as local

[ https://issues.apache.org/jira/browse/SPARK-3639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3639. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee: Aniket

[jira] [Resolved] (SPARK-3614) Filter on minimum occurrences of a term in IDF

[ https://issues.apache.org/jira/browse/SPARK-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3614. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2494

[jira] [Commented] (SPARK-2516) Bootstrapping

[ https://issues.apache.org/jira/browse/SPARK-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149721#comment-14149721 ] Xiangrui Meng commented on SPARK-2516: -- The plan was to implement Bag of Little

[jira] [Updated] (SPARK-2516) Bootstrapping

[ https://issues.apache.org/jira/browse/SPARK-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2516: - Assignee: Yu Ishikawa Bootstrapping - Key: SPARK-2516

[jira] [Resolved] (SPARK-3695) Enable to show host and port in block fetch failure

[ https://issues.apache.org/jira/browse/SPARK-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3695. Resolution: Fixed Fix Version/s: 1.2.0 Enable to show host and port in block fetch

[jira] [Created] (SPARK-3699) sbt console tasks don't clean up SparkContext

William Benton created SPARK-3699: - Summary: sbt console tasks don't clean up SparkContext Key: SPARK-3699 URL: https://issues.apache.org/jira/browse/SPARK-3699 Project: Spark Issue Type:

[jira] [Closed] (SPARK-3476) Yarn ClientBase.validateArgs memory checks wrong

[ https://issues.apache.org/jira/browse/SPARK-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3476. Resolution: Fixed Assignee: Andrew Or Yarn ClientBase.validateArgs memory checks wrong

[jira] [Commented] (SPARK-3699) sbt console tasks don't clean up SparkContext

[ https://issues.apache.org/jira/browse/SPARK-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149795#comment-14149795 ] Apache Spark commented on SPARK-3699: - User 'willb' has created a pull request for

[jira] [Resolved] (SPARK-3531) select null from table would throw a MatchError

[ https://issues.apache.org/jira/browse/SPARK-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3531. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2396

[jira] [Resolved] (SPARK-3393) Align the log4j configuration for Spark SparkSQLCLI

[ https://issues.apache.org/jira/browse/SPARK-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3393. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2263

[jira] [Updated] (SPARK-1547) Add gradient boosting algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1547: - Shepherd: Joseph K. Bradley Add gradient boosting algorithm to MLlib

[jira] [Updated] (SPARK-1547) Add gradient boosting algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1547: - Target Version/s: 1.2.0 Add gradient boosting algorithm to MLlib

[jira] [Commented] (SPARK-3652) upgrade spark sql hive version to 0.13.1

[ https://issues.apache.org/jira/browse/SPARK-3652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149836#comment-14149836 ] Yin Huai commented on SPARK-3652: - [~scwf] Thank you for your PR. Seems it is duplicated

[jira] [Commented] (SPARK-2691) Allow Spark on Mesos to be launched with Docker

[ https://issues.apache.org/jira/browse/SPARK-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149833#comment-14149833 ] Timothy Chen commented on SPARK-2691: - [~tstclair] sounds great! The integration

[jira] [Created] (SPARK-3700) Improve the performance of JSON parser

Yin Huai created SPARK-3700: --- Summary: Improve the performance of JSON parser Key: SPARK-3700 URL: https://issues.apache.org/jira/browse/SPARK-3700 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-3700) Improve the performance of JSON parser

[ https://issues.apache.org/jira/browse/SPARK-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3700: - Assignee: (was: Yin Huai) Improve the performance of JSON parser

[jira] [Updated] (SPARK-3700) Improve the performance of JSON parser

[ https://issues.apache.org/jira/browse/SPARK-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-3700: Fix Version/s: (was: 1.2.0) Improve the performance of JSON parser

[jira] [Updated] (SPARK-3700) Improve the performance of JSON parser

[ https://issues.apache.org/jira/browse/SPARK-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-3700: Fix Version/s: 1.2.0 Improve the performance of JSON parser --

[jira] [Updated] (SPARK-3700) Improve the performance of JSON parser

[ https://issues.apache.org/jira/browse/SPARK-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-3700: Fix Version/s: (was: 1.2.0) Improve the performance of JSON parser

[jira] [Updated] (SPARK-3700) Improve the performance of JSON parser

[ https://issues.apache.org/jira/browse/SPARK-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-3700: Fix Version/s: 1.2.0 Improve the performance of JSON parser --

[jira] [Updated] (SPARK-1547) Add gradient boosting algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-1547: - Description: This task requires adding the gradient boosting algorithm to Spark MLlib.

[jira] [Updated] (SPARK-1547) Add gradient boosting algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-1547: - Description: This task requires adding the gradient boosting algorithm to Spark MLlib.

[jira] [Commented] (SPARK-1547) Add gradient boosting algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149918#comment-14149918 ] Joseph K. Bradley commented on SPARK-1547: -- [~hector.yee] I strongly agree about

[jira] [Updated] (SPARK-3701) Some clean-up work after the refactoring of MLlib's SerDe for PySpark

[ https://issues.apache.org/jira/browse/SPARK-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3701: - Priority: Minor (was: Major) Some clean-up work after the refactoring of MLlib's SerDe for

[jira] [Created] (SPARK-3701) Some clean-up work after the refactoring of MLlib's SerDe for PySpark

Xiangrui Meng created SPARK-3701: Summary: Some clean-up work after the refactoring of MLlib's SerDe for PySpark Key: SPARK-3701 URL: https://issues.apache.org/jira/browse/SPARK-3701 Project: Spark

[jira] [Commented] (SPARK-3701) Some clean-up work after the refactoring of MLlib's SerDe for PySpark

[ https://issues.apache.org/jira/browse/SPARK-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149936#comment-14149936 ] Apache Spark commented on SPARK-3701: - User 'mengxr' has created a pull request for

[jira] [Created] (SPARK-3702) Standardize MLlib classes for learners, models

Joseph K. Bradley created SPARK-3702: Summary: Standardize MLlib classes for learners, models Key: SPARK-3702 URL: https://issues.apache.org/jira/browse/SPARK-3702 Project: Spark Issue

[jira] [Updated] (SPARK-3702) Standardize MLlib classes for learners, models

[ https://issues.apache.org/jira/browse/SPARK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3702: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-1856 Standardize MLlib

[jira] [Commented] (SPARK-3702) Standardize MLlib classes for learners, models

[ https://issues.apache.org/jira/browse/SPARK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149974#comment-14149974 ] Joseph K. Bradley commented on SPARK-3702: -- The design doc is only partly

[jira] [Commented] (SPARK-3702) Standardize MLlib classes for learners, models

[ https://issues.apache.org/jira/browse/SPARK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149980#comment-14149980 ] Joseph K. Bradley commented on SPARK-3702: -- Both JIRAs discuss class hierarchy.

[jira] [Commented] (SPARK-3507) Create RegressionLearner trait and make some currect code implement it

[ https://issues.apache.org/jira/browse/SPARK-3507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149988#comment-14149988 ] Joseph K. Bradley commented on SPARK-3507: -- [~epahomov] Hi, I strongly agree

[jira] [Commented] (SPARK-3702) Standardize MLlib classes for learners, models

[ https://issues.apache.org/jira/browse/SPARK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14149991#comment-14149991 ] Joseph K. Bradley commented on SPARK-3702: -- [SPARK-3251] discusses a subset of

[jira] [Commented] (SPARK-3251) Clarify learning interfaces

[ https://issues.apache.org/jira/browse/SPARK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1415#comment-1415 ] Joseph K. Bradley commented on SPARK-3251: -- I just linked JIRA related to this;

[jira] [Commented] (SPARK-3293) yarn's web show SUCCEEDED when the driver throw a exception in yarn-client

[ https://issues.apache.org/jira/browse/SPARK-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150008#comment-14150008 ] Thomas Graves commented on SPARK-3293: -- I'm going to fix this under SPARK-3627

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150013#comment-14150013 ] Thomas Graves commented on SPARK-3627: -- I'm going to go through all the exit and

[jira] [Assigned] (SPARK-3627) spark on yarn reports success even though job fails

[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-3627: Assignee: Thomas Graves spark on yarn reports success even though job fails

[jira] [Created] (SPARK-3703) Ensemble learning methods

Joseph K. Bradley created SPARK-3703: Summary: Ensemble learning methods Key: SPARK-3703 URL: https://issues.apache.org/jira/browse/SPARK-3703 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-3702) Standardize MLlib classes for learners, models

[ https://issues.apache.org/jira/browse/SPARK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3702: - Assignee: Joseph K. Bradley Standardize MLlib classes for learners, models

[jira] [Created] (SPARK-3704) the types not match adding value form spark row to hive row in SparkSQLOperationManager

wangfei created SPARK-3704: -- Summary: the types not match adding value form spark row to hive row in SparkSQLOperationManager Key: SPARK-3704 URL: https://issues.apache.org/jira/browse/SPARK-3704 Project:

[jira] [Commented] (SPARK-3704) the types not match adding value form spark row to hive row in SparkSQLOperationManager

[ https://issues.apache.org/jira/browse/SPARK-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150281#comment-14150281 ] Apache Spark commented on SPARK-3704: - User 'scwf' has created a pull request for this

[jira] [Created] (SPARK-3705) add case for VoidObjectInspector in inspectorToDataType

wangfei created SPARK-3705: -- Summary: add case for VoidObjectInspector in inspectorToDataType Key: SPARK-3705 URL: https://issues.apache.org/jira/browse/SPARK-3705 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3705) add case for VoidObjectInspector in inspectorToDataType

[ https://issues.apache.org/jira/browse/SPARK-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150313#comment-14150313 ] Apache Spark commented on SPARK-3705: - User 'scwf' has created a pull request for this

[jira] [Reopened] (SPARK-2973) Add a way to show tables without executing a job

[ https://issues.apache.org/jira/browse/SPARK-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson reopened SPARK-2973: --- Reopening this because sql(show tables).take(1) still starts a job. Add a way to show tables

[jira] [Updated] (SPARK-2973) Add a way to show tables without executing a job

[ https://issues.apache.org/jira/browse/SPARK-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson updated SPARK-2973: -- Assignee: Michael Armbrust (was: Cheng Lian) Add a way to show tables without executing a job

[jira] [Created] (SPARK-3706) Cannot run IPython REPL with IPYTHON set to 1 and PYSPARK_PYTHON unset

cocoatomo created SPARK-3706: Summary: Cannot run IPython REPL with IPYTHON set to 1 and PYSPARK_PYTHON unset Key: SPARK-3706 URL: https://issues.apache.org/jira/browse/SPARK-3706 Project: Spark

[jira] [Commented] (SPARK-3706) Cannot run IPython REPL with IPYTHON set to 1 and PYSPARK_PYTHON unset

[ https://issues.apache.org/jira/browse/SPARK-3706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150370#comment-14150370 ] Apache Spark commented on SPARK-3706: - User 'cocoatomo' has created a pull request for

[jira] [Resolved] (SPARK-3543) Write TaskContext in Java and expose it through a static accessor

[ https://issues.apache.org/jira/browse/SPARK-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3543. Resolution: Fixed Fix Version/s: 1.2.0 Write TaskContext in Java and expose it through a

[jira] [Resolved] (SPARK-2895) Support mapPartitionsWithContext in Spark Java API

[ https://issues.apache.org/jira/browse/SPARK-2895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2895. Resolution: Fixed Fix Version/s: 1.2.0 Closing this since we resolved SPARK-3543. Support

[jira] [Commented] (SPARK-3478) Profile Python tasks stage by stage in worker

[ https://issues.apache.org/jira/browse/SPARK-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150393#comment-14150393 ] Apache Spark commented on SPARK-3478: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-3543) Write TaskContext in Java and expose it through a static accessor

[ https://issues.apache.org/jira/browse/SPARK-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150398#comment-14150398 ] Apache Spark commented on SPARK-3543: - User 'rxin' has created a pull request for this

[jira] [Resolved] (SPARK-3675) Allow starting JDBC server on an existing context

[ https://issues.apache.org/jira/browse/SPARK-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3675. Resolution: Fixed Fix Version/s: 1.2.0 Allow starting JDBC server on an existing context