[jira] [Commented] (SPARK-23862) Spark ExpressionEncoder should support java enum type in scala

[ https://issues.apache.org/jira/browse/SPARK-23862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16425015#comment-16425015 ] Apache Spark commented on SPARK-23862: -- User 'fangshil' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23862) Spark ExpressionEncoder should support java enum type in scala

[ https://issues.apache.org/jira/browse/SPARK-23862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23862: Assignee: Apache Spark > Spark ExpressionEncoder should support java enum type in scala >

[jira] [Assigned] (SPARK-23862) Spark ExpressionEncoder should support java enum type in scala

[ https://issues.apache.org/jira/browse/SPARK-23862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23862: Assignee: (was: Apache Spark) > Spark ExpressionEncoder should support java enum type

[jira] [Commented] (SPARK-20114) spark.ml parity for sequential pattern mining - PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-20114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16425006#comment-16425006 ] Apache Spark commented on SPARK-20114: -- User 'WeichenXu123' has created a pull request for this

[jira] [Created] (SPARK-23862) Spark ExpressionEncoder should support java enum type in scala

Fangshi Li created SPARK-23862: -- Summary: Spark ExpressionEncoder should support java enum type in scala Key: SPARK-23862 URL: https://issues.apache.org/jira/browse/SPARK-23862 Project: Spark

[jira] [Commented] (SPARK-19276) FetchFailures can be hidden by user (or sql) exception handling

[ https://issues.apache.org/jira/browse/SPARK-19276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424923#comment-16424923 ] chen xiao commented on SPARK-19276: --- [~imranr], I have created another task about this issue several

[jira] [Commented] (SPARK-16630) Blacklist a node if executors won't launch on it.

[ https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424906#comment-16424906 ] Imran Rashid commented on SPARK-16630: -- Be sure to look at the discussion on this PR about the same

[jira] [Created] (SPARK-23861) Clarify behavior of default window frame boundaries with and without orderBy clause

Li Jin created SPARK-23861: -- Summary: Clarify behavior of default window frame boundaries with and without orderBy clause Key: SPARK-23861 URL: https://issues.apache.org/jira/browse/SPARK-23861 Project:

[jira] [Commented] (SPARK-23809) Active SparkSession should be set by getOrCreate

[ https://issues.apache.org/jira/browse/SPARK-23809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424806#comment-16424806 ] Apache Spark commented on SPARK-23809: -- User 'ericl' has created a pull request for this issue:

[jira] [Updated] (SPARK-23802) PropagateEmptyRelation can leave query plan in unresolved state

[ https://issues.apache.org/jira/browse/SPARK-23802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23802: Fix Version/s: 2.4.0 > PropagateEmptyRelation can leave query plan in unresolved state >

[jira] [Resolved] (SPARK-23802) PropagateEmptyRelation can leave query plan in unresolved state

[ https://issues.apache.org/jira/browse/SPARK-23802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23802. - Resolution: Fixed Assignee: Robert Kruszewski Fix Version/s: 2.3.1 >

[jira] [Resolved] (SPARK-23860) SQLAppStatusListener should handle the cases that an accumulator may be GCed

[ https://issues.apache.org/jira/browse/SPARK-23860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23860. -- Resolution: Duplicate already fixed in SPARK-20652 > SQLAppStatusListener should handle the

[jira] [Resolved] (SPARK-23587) Add interpreted execution for MapObjects expression

[ https://issues.apache.org/jira/browse/SPARK-23587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-23587. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.4.0 >

[jira] [Resolved] (SPARK-22667) Fix model-specific optimization support for ML tuning: Python API

[ https://issues.apache.org/jira/browse/SPARK-22667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-22667. --- Resolution: Duplicate Fix Version/s: 2.3.0 > Fix model-specific optimization

[jira] [Updated] (SPARK-15784) Add Power Iteration Clustering to spark.ml

[ https://issues.apache.org/jira/browse/SPARK-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-15784: Shepherd: Joseph K. Bradley (was: Yanbo Liang) > Add Power Iteration Clustering to spark.ml >

[jira] [Commented] (SPARK-15784) Add Power Iteration Clustering to spark.ml

[ https://issues.apache.org/jira/browse/SPARK-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424609#comment-16424609 ] Yanbo Liang commented on SPARK-15784: - [~josephkb] Please take over this, I'm very busy recently and

[jira] [Commented] (SPARK-20114) spark.ml parity for sequential pattern mining - PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-20114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424605#comment-16424605 ] Joseph K. Bradley commented on SPARK-20114: --- Thanks for the ideas [~yuhaoyan]. Your points

[jira] [Commented] (SPARK-15784) Add Power Iteration Clustering to spark.ml

[ https://issues.apache.org/jira/browse/SPARK-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424592#comment-16424592 ] Joseph K. Bradley commented on SPARK-15784: --- [~yanboliang] Would you like for me to take over

[jira] [Commented] (SPARK-23836) Support returning StructType to the level support in GroupedMap Arrow's "scalar" UDFS (or similar)

[ https://issues.apache.org/jira/browse/SPARK-23836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424567#comment-16424567 ] Bryan Cutler commented on SPARK-23836: -- So does this mean returning a pandas.DataFrame in a scalar

[jira] [Created] (SPARK-23860) SQLAppStatusListener should handle the cases that an accumulator may be GCed

Shixiong Zhu created SPARK-23860: Summary: SQLAppStatusListener should handle the cases that an accumulator may be GCed Key: SPARK-23860 URL: https://issues.apache.org/jira/browse/SPARK-23860

[jira] [Commented] (SPARK-22918) sbt test (spark - local) fail after upgrading to 2.2.1 with: java.security.AccessControlException: access denied org.apache.derby.security.SystemPermission( "engine",

[ https://issues.apache.org/jira/browse/SPARK-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424549#comment-16424549 ] Georg Heiler commented on SPARK-22918: --  +1 same issue > sbt test (spark - local) fail after

[jira] [Updated] (SPARK-23858) Need to apply pyarrow adjustments to complex types with DateType/TimestampType

[ https://issues.apache.org/jira/browse/SPARK-23858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-23858: - Summary: Need to apply pyarrow adjustments to complex types with DateType/TimestampType (was:

[jira] [Updated] (SPARK-23807) Add Hadoop 3 profile with relevant POM fix ups

[ https://issues.apache.org/jira/browse/SPARK-23807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-23807: --- Summary: Add Hadoop 3 profile with relevant POM fix ups (was: Add Hadoop 3 profile with

[jira] [Assigned] (SPARK-23668) Support for imagePullSecrets k8s option

[ https://issues.apache.org/jira/browse/SPARK-23668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan reassigned SPARK-23668: -- Assignee: Andrew Korzhuev > Support for imagePullSecrets k8s option >

[jira] [Updated] (SPARK-23562) RFormula handleInvalid should handle invalid values in non-string columns.

[ https://issues.apache.org/jira/browse/SPARK-23562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23562: -- Description: Currently when handleInvalid is set to 'keep' or 'skip' this only applies

[jira] [Commented] (SPARK-23562) RFormula handleInvalid should handle invalid values in non-string columns.

[ https://issues.apache.org/jira/browse/SPARK-23562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424394#comment-16424394 ] Apache Spark commented on SPARK-23562: -- User 'yogeshg' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23562) RFormula handleInvalid should handle invalid values in non-string columns.

[ https://issues.apache.org/jira/browse/SPARK-23562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23562: Assignee: Apache Spark > RFormula handleInvalid should handle invalid values in

[jira] [Assigned] (SPARK-23562) RFormula handleInvalid should handle invalid values in non-string columns.

[ https://issues.apache.org/jira/browse/SPARK-23562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23562: Assignee: (was: Apache Spark) > RFormula handleInvalid should handle invalid values

[jira] [Assigned] (SPARK-23099) Migrate foreach sink

[ https://issues.apache.org/jira/browse/SPARK-23099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-23099: - Assignee: Jose Torres > Migrate foreach sink > > >

[jira] [Resolved] (SPARK-23099) Migrate foreach sink

[ https://issues.apache.org/jira/browse/SPARK-23099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23099. --- Resolution: Fixed Fix Version/s: 2.4.0 3.0.0 Issue resolved by

[jira] [Created] (SPARK-23859) Initial PR for Instrumentation improvements: UUID and logging levels

Joseph K. Bradley created SPARK-23859: - Summary: Initial PR for Instrumentation improvements: UUID and logging levels Key: SPARK-23859 URL: https://issues.apache.org/jira/browse/SPARK-23859

[jira] [Updated] (SPARK-21187) Complete support for remaining Spark data types in Arrow Converters

[ https://issues.apache.org/jira/browse/SPARK-21187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-21187: - Description: This is to track adding the remaining type support in Arrow Converters. Currently,

[jira] [Commented] (SPARK-23826) TestHiveSparkSession should set default session

[ https://issues.apache.org/jira/browse/SPARK-23826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424356#comment-16424356 ] Apache Spark commented on SPARK-23826: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23828) PySpark StringIndexerModel should have constructor from labels

[ https://issues.apache.org/jira/browse/SPARK-23828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23828: Assignee: Apache Spark > PySpark StringIndexerModel should have constructor from labels >

[jira] [Assigned] (SPARK-23826) TestHiveSparkSession should set default session

[ https://issues.apache.org/jira/browse/SPARK-23826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23826: Assignee: Apache Spark > TestHiveSparkSession should set default session >

[jira] [Assigned] (SPARK-23826) TestHiveSparkSession should set default session

[ https://issues.apache.org/jira/browse/SPARK-23826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23826: Assignee: (was: Apache Spark) > TestHiveSparkSession should set default session >

[jira] [Assigned] (SPARK-23828) PySpark StringIndexerModel should have constructor from labels

[ https://issues.apache.org/jira/browse/SPARK-23828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23828: Assignee: (was: Apache Spark) > PySpark StringIndexerModel should have constructor

[jira] [Commented] (SPARK-23828) PySpark StringIndexerModel should have constructor from labels

[ https://issues.apache.org/jira/browse/SPARK-23828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424357#comment-16424357 ] Apache Spark commented on SPARK-23828: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Created] (SPARK-23858) Need to apply adjustments to complex types with DateType/TimestampType

Bryan Cutler created SPARK-23858: Summary: Need to apply adjustments to complex types with DateType/TimestampType Key: SPARK-23858 URL: https://issues.apache.org/jira/browse/SPARK-23858 Project:

[jira] [Updated] (SPARK-23857) In mesos cluster mode spark submit requires the keytab to be available on the local file system.

[ https://issues.apache.org/jira/browse/SPARK-23857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-23857: Summary: In mesos cluster mode spark submit requires the keytab to be available on

[jira] [Commented] (SPARK-14540) Support Scala 2.12 closures and Java 8 lambdas in ClosureCleaner

[ https://issues.apache.org/jira/browse/SPARK-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424328#comment-16424328 ] Joe Pallas commented on SPARK-14540: The interested reader should know that some more discussion of

[jira] [Comment Edited] (SPARK-23856) Spark jdbc setQueryTimeout option

[ https://issues.apache.org/jira/browse/SPARK-23856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423907#comment-16423907 ] Dmitry Mikhailov edited comment on SPARK-23856 at 4/3/18 4:15 PM: -- Yes,

[jira] [Assigned] (SPARK-23857) In mesos cluster mode keytab Spark submit requires the keytab to be available on the local file system.

[ https://issues.apache.org/jira/browse/SPARK-23857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23857: Assignee: Apache Spark > In mesos cluster mode keytab Spark submit requires the keytab to

[jira] [Commented] (SPARK-23857) In mesos cluster mode keytab Spark submit requires the keytab to be available on the local file system.

[ https://issues.apache.org/jira/browse/SPARK-23857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424144#comment-16424144 ] Apache Spark commented on SPARK-23857: -- User 'skonto' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23857) In mesos cluster mode keytab Spark submit requires the keytab to be available on the local file system.

[ https://issues.apache.org/jira/browse/SPARK-23857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23857: Assignee: (was: Apache Spark) > In mesos cluster mode keytab Spark submit requires

[jira] [Updated] (SPARK-23857) In mesos cluster mode keytab Spark submit requires the keytab to be available on the local file system.

[ https://issues.apache.org/jira/browse/SPARK-23857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-23857: Description: Users could submit their jobs from an external to the cluster host

[jira] [Updated] (SPARK-23857) In mesos cluster mode keytab Spark submit requires the keytab to be available on the local file system.

[ https://issues.apache.org/jira/browse/SPARK-23857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-23857: Description: Users could submit their jobs from an external to the cluster host

[jira] [Updated] (SPARK-23857) In mesos cluster mode keytab Spark submit requires the keytab to be available on the local file system.

[ https://issues.apache.org/jira/browse/SPARK-23857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-23857: Description: Users could submit their jobs from an external to the cluster host

[jira] [Created] (SPARK-23857) In mesos cluster mode keytab Spark submit requires the keytab to be available on the local file system.

Stavros Kontopoulos created SPARK-23857: --- Summary: In mesos cluster mode keytab Spark submit requires the keytab to be available on the local file system. Key: SPARK-23857 URL:

[jira] [Commented] (SPARK-23039) Fix the bug in alter table set location.

[ https://issues.apache.org/jira/browse/SPARK-23039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423996#comment-16423996 ] Thomas Graves commented on SPARK-23039: --- seems to be a dup of  # SPARK-23057 > Fix the bug in

[jira] [Commented] (SPARK-23856) Spark jdbc setQueryTimeout option

[ https://issues.apache.org/jira/browse/SPARK-23856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423907#comment-16423907 ] Dmitry Mikhailov commented on SPARK-23856: -- Yes, i'd like to. Any comments on drawbacks on this 

[jira] [Commented] (SPARK-23856) Spark jdbc setQueryTimeout option

[ https://issues.apache.org/jira/browse/SPARK-23856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423848#comment-16423848 ] Takeshi Yamamuro commented on SPARK-23856: -- It sounds some reasonable to me. It seems there is

[jira] [Created] (SPARK-23856) Spark jdbc setQueryTimeout option

Dmitry Mikhailov created SPARK-23856: Summary: Spark jdbc setQueryTimeout option Key: SPARK-23856 URL: https://issues.apache.org/jira/browse/SPARK-23856 Project: Spark Issue Type: New

[jira] [Created] (SPARK-23855) Performing a Join after a CrossJoin can lead to data corruption

Martin Junghanns created SPARK-23855: Summary: Performing a Join after a CrossJoin can lead to data corruption Key: SPARK-23855 URL: https://issues.apache.org/jira/browse/SPARK-23855 Project:

[jira] [Assigned] (SPARK-23854) Update Guava to 16.0.1

[ https://issues.apache.org/jira/browse/SPARK-23854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23854: Assignee: (was: Apache Spark) > Update Guava to 16.0.1 > -- > >

[jira] [Commented] (SPARK-23854) Update Guava to 16.0.1

[ https://issues.apache.org/jira/browse/SPARK-23854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423657#comment-16423657 ] Apache Spark commented on SPARK-23854: -- User 'Fokko' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23854) Update Guava to 16.0.1

[ https://issues.apache.org/jira/browse/SPARK-23854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23854: Assignee: Apache Spark > Update Guava to 16.0.1 > -- > >

[jira] [Created] (SPARK-23854) Update Guava to 16.0.1

Fokko Driesprong created SPARK-23854: Summary: Update Guava to 16.0.1 Key: SPARK-23854 URL: https://issues.apache.org/jira/browse/SPARK-23854 Project: Spark Issue Type: Improvement