[jira] [Created] (SPARK-22711) _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py

Prateek created SPARK-22711: --- Summary: _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py Key: SPARK-22711 URL: https://issues.apache.org/jira/browse/SPARK-22711

[jira] [Commented] (SPARK-21827) Task fail due to executor exception when enable Sasl Encryption

[ https://issues.apache.org/jira/browse/SPARK-21827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279779#comment-16279779 ] Yishan Jiang commented on SPARK-21827: -- Yes, I am using HDFS. Cores to executor, mostly using

[jira] [Assigned] (SPARK-22710) ConfigBuilder.fallbackConf doesn't trigger onCreate function

[ https://issues.apache.org/jira/browse/SPARK-22710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22710: Assignee: Apache Spark (was: Reynold Xin) > ConfigBuilder.fallbackConf doesn't trigger

[jira] [Commented] (SPARK-22710) ConfigBuilder.fallbackConf doesn't trigger onCreate function

[ https://issues.apache.org/jira/browse/SPARK-22710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279769#comment-16279769 ] Apache Spark commented on SPARK-22710: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22710) ConfigBuilder.fallbackConf doesn't trigger onCreate function

[ https://issues.apache.org/jira/browse/SPARK-22710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22710: Assignee: Reynold Xin (was: Apache Spark) > ConfigBuilder.fallbackConf doesn't trigger

[jira] [Created] (SPARK-22710) ConfigBuilder.fallbackConf doesn't trigger onCreate function

Reynold Xin created SPARK-22710: --- Summary: ConfigBuilder.fallbackConf doesn't trigger onCreate function Key: SPARK-22710 URL: https://issues.apache.org/jira/browse/SPARK-22710 Project: Spark

[jira] [Created] (SPARK-22709) move config related infrastructure from Spark Core to a new module

Wenchen Fan created SPARK-22709: --- Summary: move config related infrastructure from Spark Core to a new module Key: SPARK-22709 URL: https://issues.apache.org/jira/browse/SPARK-22709 Project: Spark

[jira] [Updated] (SPARK-20392) Slow performance when calling fit on ML pipeline for dataset with many columns but few rows

[ https://issues.apache.org/jira/browse/SPARK-20392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-20392: Priority: Major (was: Blocker) > Slow performance when calling fit on ML pipeline for dataset with many

[jira] [Resolved] (SPARK-20392) Slow performance when calling fit on ML pipeline for dataset with many columns but few rows

[ https://issues.apache.org/jira/browse/SPARK-20392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20392. - Resolution: Fixed Fix Version/s: 2.3.0 > Slow performance when calling fit on ML pipeline for

[jira] [Commented] (SPARK-16870) add "spark.sql.broadcastTimeout" into docs/sql-programming-guide.md to help people to how to fix this timeout error when it happenned

[ https://issues.apache.org/jira/browse/SPARK-16870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279679#comment-16279679 ] cathy commented on SPARK-16870: --- Hi Liang Ke, I am using spark-1.6.2, when I ran some jobs, this error

[jira] [Updated] (SPARK-22707) Optimize CrossValidator memory occupation by models in fitting

[ https://issues.apache.org/jira/browse/SPARK-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-22707: --- Description: Via some test I found CrossValidator still exists memory issue, it will still occupy

[jira] [Updated] (SPARK-22707) Optimize CrossValidator memory occupation by models in fitting

[ https://issues.apache.org/jira/browse/SPARK-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-22707: --- Summary: Optimize CrossValidator memory occupation by models in fitting (was: Optimize

[jira] [Commented] (SPARK-22461) Move Spark ML model summaries into a dedicated package

[ https://issues.apache.org/jira/browse/SPARK-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279627#comment-16279627 ] Apache Spark commented on SPARK-22461: -- User 'sethah' has created a pull request for this issue:

[jira] [Updated] (SPARK-22707) Optimize Crossvalidator fitting memory occupation by models

[ https://issues.apache.org/jira/browse/SPARK-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-22707: --- Issue Type: Improvement (was: Bug) > Optimize Crossvalidator fitting memory occupation by models >

[jira] [Created] (SPARK-22708) spark on yarn error but Final app status: SUCCEEDED, exitCode: 0

jinchen created SPARK-22708: --- Summary: spark on yarn error but Final app status: SUCCEEDED, exitCode: 0 Key: SPARK-22708 URL: https://issues.apache.org/jira/browse/SPARK-22708 Project: Spark

[jira] [Commented] (SPARK-22707) Optimize Crossvalidator fitting memory occupation by models

[ https://issues.apache.org/jira/browse/SPARK-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279588#comment-16279588 ] Apache Spark commented on SPARK-22707: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-22707) Optimize Crossvalidator fitting memory occupation by models

[ https://issues.apache.org/jira/browse/SPARK-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22707: Assignee: (was: Apache Spark) > Optimize Crossvalidator fitting memory occupation by

[jira] [Assigned] (SPARK-22707) Optimize Crossvalidator fitting memory occupation by models

[ https://issues.apache.org/jira/browse/SPARK-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22707: Assignee: Apache Spark > Optimize Crossvalidator fitting memory occupation by models >

[jira] [Updated] (SPARK-22707) Optimize Crossvalidator fitting memory occupation by models

[ https://issues.apache.org/jira/browse/SPARK-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-22707: --- Description: Via some test I found CrossValidator still exists memory issue, it will still occupy

[jira] [Created] (SPARK-22707) Optimize Crossvalidator fitting memory occupation by models

Weichen Xu created SPARK-22707: -- Summary: Optimize Crossvalidator fitting memory occupation by models Key: SPARK-22707 URL: https://issues.apache.org/jira/browse/SPARK-22707 Project: Spark

[jira] [Assigned] (SPARK-22686) DROP TABLE IF EXISTS should not show AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-22686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22686: --- Assignee: Dongjoon Hyun > DROP TABLE IF EXISTS should not show AnalysisException >

[jira] [Resolved] (SPARK-22686) DROP TABLE IF EXISTS should not show AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-22686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22686. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.2 Issue resolved by pull

[jira] [Commented] (SPARK-20728) Make ORCFileFormat configurable between sql/hive and sql/core

[ https://issues.apache.org/jira/browse/SPARK-20728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279566#comment-16279566 ] Apache Spark commented on SPARK-20728: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Issue Comment Deleted] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-22126: --- Comment: was deleted (was: [~bago.amirbekian] hmm... directly use java callable looks fine too. We

[jira] [Commented] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279549#comment-16279549 ] Weichen Xu commented on SPARK-22126: [~bago.amirbekian] hmm... directly use java callable looks fine

[jira] [Commented] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279548#comment-16279548 ] Weichen Xu commented on SPARK-22126: [~bago.amirbekian] hmm... directly use java callable looks fine

[jira] [Comment Edited] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279537#comment-16279537 ] Bago Amirbekian edited comment on SPARK-22126 at 12/6/17 2:19 AM: --

[jira] [Commented] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279537#comment-16279537 ] Bago Amirbekian commented on SPARK-22126: - [~WeichenXu123] Sorry I misunderstood, I thought you

[jira] [Commented] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279529#comment-16279529 ] Weichen Xu commented on SPARK-22126: [~bago.amirbekian] Small correction: the callable definition can

[jira] [Comment Edited] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279529#comment-16279529 ] Weichen Xu edited comment on SPARK-22126 at 12/6/17 1:53 AM: -

[jira] [Assigned] (SPARK-20706) Spark-shell not overriding method/variable definition

[ https://issues.apache.org/jira/browse/SPARK-20706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-20706: - Assignee: Mark Petruska > Spark-shell not overriding method/variable definition >

[jira] [Resolved] (SPARK-20706) Spark-shell not overriding method/variable definition

[ https://issues.apache.org/jira/browse/SPARK-20706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20706. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19879

[jira] [Updated] (SPARK-22706) Cannot read Teradata CLOB column type correctly in Spark 2.2.0

[ https://issues.apache.org/jira/browse/SPARK-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22706: -- Priority: Minor (was: Major) On its surface, it looks like it's because the driver's CLOB

[jira] [Commented] (SPARK-22452) DataSourceV2Options should have getInt, getBoolean, etc.

[ https://issues.apache.org/jira/browse/SPARK-22452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279379#comment-16279379 ] Sunitha Kambhampati commented on SPARK-22452: - [~cloud_fan] , I implemented a few methods (

[jira] [Assigned] (SPARK-22452) DataSourceV2Options should have getInt, getBoolean, etc.

[ https://issues.apache.org/jira/browse/SPARK-22452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22452: Assignee: Apache Spark > DataSourceV2Options should have getInt, getBoolean, etc. >

[jira] [Commented] (SPARK-22452) DataSourceV2Options should have getInt, getBoolean, etc.

[ https://issues.apache.org/jira/browse/SPARK-22452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279370#comment-16279370 ] Apache Spark commented on SPARK-22452: -- User 'skambha' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22452) DataSourceV2Options should have getInt, getBoolean, etc.

[ https://issues.apache.org/jira/browse/SPARK-22452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22452: Assignee: (was: Apache Spark) > DataSourceV2Options should have getInt, getBoolean,

[jira] [Created] (SPARK-22706) Cannot read Teradata CLOB column type correctly in Spark 2.2.0

Nannan Yu created SPARK-22706: - Summary: Cannot read Teradata CLOB column type correctly in Spark 2.2.0 Key: SPARK-22706 URL: https://issues.apache.org/jira/browse/SPARK-22706 Project: Spark

[jira] [Resolved] (SPARK-22662) Failed to prune columns after rewriting predicate subquery

[ https://issues.apache.org/jira/browse/SPARK-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22662. - Resolution: Fixed Assignee: Zhenhua Wang Fix Version/s: 2.3.0 > Failed to prune columns

[jira] [Commented] (SPARK-6473) Launcher lib shouldn't try to figure out Scala version when not in dev mode

[ https://issues.apache.org/jira/browse/SPARK-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279355#comment-16279355 ] Peng Cheng commented on SPARK-6473: --- Looks like this issue reappear at some point: getScalaVersion()

[jira] [Comment Edited] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279218#comment-16279218 ] Bago Amirbekian edited comment on SPARK-22126 at 12/5/17 9:58 PM: -- I

[jira] [Comment Edited] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279218#comment-16279218 ] Bago Amirbekian edited comment on SPARK-22126 at 12/5/17 9:53 PM: -- I

[jira] [Updated] (SPARK-22686) DROP TABLE IF EXISTS should not show AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-22686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-22686: -- Summary: DROP TABLE IF EXISTS should not show AnalysisException (was: DROP TABLE IF EXISTS

[jira] [Commented] (SPARK-22126) Fix model-specific optimization support for ML tuning

[ https://issues.apache.org/jira/browse/SPARK-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279218#comment-16279218 ] Bago Amirbekian commented on SPARK-22126: - I started a discussion about potential to this issue

[jira] [Commented] (SPARK-22660) Compile with scala-2.12 and JDK9

[ https://issues.apache.org/jira/browse/SPARK-22660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279208#comment-16279208 ] liyunzhang commented on SPARK-22660: [~srowen]:I have seen the comments in the pr, will fix them

[jira] [Resolved] (SPARK-22701) add ctx.splitExpressionsWithCurrentInputs

[ https://issues.apache.org/jira/browse/SPARK-22701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22701. - Resolution: Fixed Fix Version/s: 2.3.0 > add ctx.splitExpressionsWithCurrentInputs >

[jira] [Assigned] (SPARK-22705) Reduce # of mutable variables in Case, Coalesce, and In

[ https://issues.apache.org/jira/browse/SPARK-22705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22705: Assignee: Apache Spark > Reduce # of mutable variables in Case, Coalesce, and In >

[jira] [Commented] (SPARK-22705) Reduce # of mutable variables in Case, Coalesce, and In

[ https://issues.apache.org/jira/browse/SPARK-22705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278937#comment-16278937 ] Apache Spark commented on SPARK-22705: -- User 'kiszk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22705) Reduce # of mutable variables in Case, Coalesce, and In

[ https://issues.apache.org/jira/browse/SPARK-22705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22705: Assignee: (was: Apache Spark) > Reduce # of mutable variables in Case, Coalesce, and

[jira] [Updated] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-22162: --- Fix Version/s: 2.2.2 > Executors and the driver use inconsistent Job IDs during the new RDD

[jira] [Resolved] (SPARK-22681) Accumulator should only be updated once for each task in result stage

[ https://issues.apache.org/jira/browse/SPARK-22681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22681. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19877

[jira] [Assigned] (SPARK-22681) Accumulator should only be updated once for each task in result stage

[ https://issues.apache.org/jira/browse/SPARK-22681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22681: -- Assignee: Carson Wang > Accumulator should only be updated once for each task in

[jira] [Resolved] (SPARK-22702) Spark sql filter with size function(if exists) leads twice calculation

[ https://issues.apache.org/jira/browse/SPARK-22702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22702. --- Resolution: Invalid This should start on the mailing list. I must say I'm not clear what you're

[jira] [Commented] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278845#comment-16278845 ] Sean Owen commented on SPARK-22683: --- I get it, but you're just finding ways to delay adding executors.

[jira] [Commented] (SPARK-22695) Avoid the generation of useless mutable states by scalaUDF

[ https://issues.apache.org/jira/browse/SPARK-22695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278812#comment-16278812 ] Apache Spark commented on SPARK-22695: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22695) Avoid the generation of useless mutable states by scalaUDF

[ https://issues.apache.org/jira/browse/SPARK-22695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22695: Assignee: Apache Spark > Avoid the generation of useless mutable states by scalaUDF >

[jira] [Assigned] (SPARK-22695) Avoid the generation of useless mutable states by scalaUDF

[ https://issues.apache.org/jira/browse/SPARK-22695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22695: Assignee: (was: Apache Spark) > Avoid the generation of useless mutable states by

[jira] [Assigned] (SPARK-22704) Reduce # of mutable variables in Least and greatest

[ https://issues.apache.org/jira/browse/SPARK-22704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22704: Assignee: (was: Apache Spark) > Reduce # of mutable variables in Least and greatest >

[jira] [Assigned] (SPARK-22704) Reduce # of mutable variables in Least and greatest

[ https://issues.apache.org/jira/browse/SPARK-22704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22704: Assignee: Apache Spark > Reduce # of mutable variables in Least and greatest >

[jira] [Commented] (SPARK-22704) Reduce # of mutable variables in Least and greatest

[ https://issues.apache.org/jira/browse/SPARK-22704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278775#comment-16278775 ] Apache Spark commented on SPARK-22704: -- User 'kiszk' has created a pull request for this issue:

[jira] [Created] (SPARK-22705) Reduce # of mutable variables in Case, Coalesce, and In

Kazuaki Ishizaki created SPARK-22705: Summary: Reduce # of mutable variables in Case, Coalesce, and In Key: SPARK-22705 URL: https://issues.apache.org/jira/browse/SPARK-22705 Project: Spark

[jira] [Created] (SPARK-22704) Reduce # of mutable variables in Least and greatest

Kazuaki Ishizaki created SPARK-22704: Summary: Reduce # of mutable variables in Least and greatest Key: SPARK-22704 URL: https://issues.apache.org/jira/browse/SPARK-22704 Project: Spark

[jira] [Assigned] (SPARK-22703) ColumnarRow should be an immutable view

[ https://issues.apache.org/jira/browse/SPARK-22703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22703: Assignee: Apache Spark (was: Wenchen Fan) > ColumnarRow should be an immutable view >

[jira] [Commented] (SPARK-22703) ColumnarRow should be an immutable view

[ https://issues.apache.org/jira/browse/SPARK-22703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278678#comment-16278678 ] Apache Spark commented on SPARK-22703: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22703) ColumnarRow should be an immutable view

[ https://issues.apache.org/jira/browse/SPARK-22703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22703: Assignee: Wenchen Fan (was: Apache Spark) > ColumnarRow should be an immutable view >

[jira] [Created] (SPARK-22703) ColumnarRow should be an immutable view

Wenchen Fan created SPARK-22703: --- Summary: ColumnarRow should be an immutable view Key: SPARK-22703 URL: https://issues.apache.org/jira/browse/SPARK-22703 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-22262) Failed to recover Spark Structured Streaming job from checkpoint location

[ https://issues.apache.org/jira/browse/SPARK-22262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278629#comment-16278629 ] Alban Hurtaud commented on SPARK-22262: --- Hi, Can we have more info on why is this invalid ? I am

[jira] [Resolved] (SPARK-22526) Document closing of PortableDataInputStream in binaryFiles

[ https://issues.apache.org/jira/browse/SPARK-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22526. --- Resolution: Won't Fix > Document closing of PortableDataInputStream in binaryFiles >

[jira] [Updated] (SPARK-22702) Spark sql filter with size function(if exists) leads twice calculation

[ https://issues.apache.org/jira/browse/SPARK-22702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenfh5 updated SPARK-22702: Description: I occur an issue about spark-sql. When obtaining a Dataset through some logics, I wish to

[jira] [Updated] (SPARK-22702) Spark sql filter with size function(if exists) leads twice calculation

[ https://issues.apache.org/jira/browse/SPARK-22702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenfh5 updated SPARK-22702: Description: I occur an issue about spark-sql. When obtaining a Dataset through some logic, I wish to

[jira] [Created] (SPARK-22702) Spark sql filter with size function(if exists) leads twice calculation

chenfh5 created SPARK-22702: --- Summary: Spark sql filter with size function(if exists) leads twice calculation Key: SPARK-22702 URL: https://issues.apache.org/jira/browse/SPARK-22702 Project: Spark

[jira] [Comment Edited] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278405#comment-16278405 ] Julien Cuquemelle edited comment on SPARK-22683 at 12/5/17 1:09 PM:

[jira] [Commented] (SPARK-22694) Avoid the generation of useless mutable states by regexp functions

[ https://issues.apache.org/jira/browse/SPARK-22694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278508#comment-16278508 ] Apache Spark commented on SPARK-22694: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22694) Avoid the generation of useless mutable states by regexp functions

[ https://issues.apache.org/jira/browse/SPARK-22694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22694: Assignee: (was: Apache Spark) > Avoid the generation of useless mutable states by

[jira] [Assigned] (SPARK-22694) Avoid the generation of useless mutable states by regexp functions

[ https://issues.apache.org/jira/browse/SPARK-22694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22694: Assignee: Apache Spark > Avoid the generation of useless mutable states by regexp

[jira] [Assigned] (SPARK-22693) Avoid the generation of useless mutable states in complexTypeCreator and predicates

[ https://issues.apache.org/jira/browse/SPARK-22693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22693: Assignee: (was: Apache Spark) > Avoid the generation of useless mutable states in

[jira] [Assigned] (SPARK-22693) Avoid the generation of useless mutable states in complexTypeCreator and predicates

[ https://issues.apache.org/jira/browse/SPARK-22693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22693: Assignee: Apache Spark > Avoid the generation of useless mutable states in

[jira] [Commented] (SPARK-22693) Avoid the generation of useless mutable states in complexTypeCreator and predicates

[ https://issues.apache.org/jira/browse/SPARK-22693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278507#comment-16278507 ] Apache Spark commented on SPARK-22693: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20728) Make ORCFileFormat configurable between sql/hive and sql/core

[ https://issues.apache.org/jira/browse/SPARK-20728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20728: --- Assignee: Dongjoon Hyun > Make ORCFileFormat configurable between sql/hive and sql/core >

[jira] [Updated] (SPARK-20728) Make ORCFileFormat configurable between sql/hive and sql/core

[ https://issues.apache.org/jira/browse/SPARK-20728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-20728: Affects Version/s: (was: 2.1.1) 2.3.0 > Make ORCFileFormat configurable

[jira] [Resolved] (SPARK-20728) Make ORCFileFormat configurable between sql/hive and sql/core

[ https://issues.apache.org/jira/browse/SPARK-20728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20728. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19871

[jira] [Resolved] (SPARK-22675) Refactoring PropagateTypes in TypeCoercion

[ https://issues.apache.org/jira/browse/SPARK-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22675. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19874

[jira] [Assigned] (SPARK-22701) add ctx.splitExpressionsWithCurrentInputs

[ https://issues.apache.org/jira/browse/SPARK-22701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22701: Assignee: Wenchen Fan (was: Apache Spark) > add ctx.splitExpressionsWithCurrentInputs >

[jira] [Commented] (SPARK-22701) add ctx.splitExpressionsWithCurrentInputs

[ https://issues.apache.org/jira/browse/SPARK-22701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278475#comment-16278475 ] Apache Spark commented on SPARK-22701: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22701) add ctx.splitExpressionsWithCurrentInputs

[ https://issues.apache.org/jira/browse/SPARK-22701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22701: Assignee: Apache Spark (was: Wenchen Fan) > add ctx.splitExpressionsWithCurrentInputs >

[jira] [Created] (SPARK-22701) add ctx.splitExpressionsWithCurrentInputs

Wenchen Fan created SPARK-22701: --- Summary: add ctx.splitExpressionsWithCurrentInputs Key: SPARK-22701 URL: https://issues.apache.org/jira/browse/SPARK-22701 Project: Spark Issue Type:

[jira] [Commented] (SPARK-22692) Reduce the number of generated mutable states

[ https://issues.apache.org/jira/browse/SPARK-22692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278458#comment-16278458 ] Marco Gaido commented on SPARK-22692: - I felt this was the best way to go in order to split the

[jira] [Assigned] (SPARK-22700) Bucketizer.transform incorrectly drops row containing NaN

[ https://issues.apache.org/jira/browse/SPARK-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22700: Assignee: Apache Spark > Bucketizer.transform incorrectly drops row containing NaN >

[jira] [Assigned] (SPARK-22700) Bucketizer.transform incorrectly drops row containing NaN

[ https://issues.apache.org/jira/browse/SPARK-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22700: Assignee: (was: Apache Spark) > Bucketizer.transform incorrectly drops row containing

[jira] [Commented] (SPARK-22700) Bucketizer.transform incorrectly drops row containing NaN

[ https://issues.apache.org/jira/browse/SPARK-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278438#comment-16278438 ] Apache Spark commented on SPARK-22700: -- User 'zhengruifeng' has created a pull request for this

[jira] [Updated] (SPARK-22700) Bucketizer.transform incorrectly drops row containing NaN

[ https://issues.apache.org/jira/browse/SPARK-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-22700: - Issue Type: Bug (was: Improvement) > Bucketizer.transform incorrectly drops row containing NaN

[jira] [Created] (SPARK-22700) Bucketizer.transform incorrectly drops row containing NaN

zhengruifeng created SPARK-22700: Summary: Bucketizer.transform incorrectly drops row containing NaN Key: SPARK-22700 URL: https://issues.apache.org/jira/browse/SPARK-22700 Project: Spark

[jira] [Assigned] (SPARK-16139) Audit tests for leaked threads

[ https://issues.apache.org/jira/browse/SPARK-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16139: Assignee: (was: Apache Spark) > Audit tests for leaked threads >

[jira] [Assigned] (SPARK-16139) Audit tests for leaked threads

[ https://issues.apache.org/jira/browse/SPARK-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16139: Assignee: Apache Spark > Audit tests for leaked threads > --

[jira] [Commented] (SPARK-16139) Audit tests for leaked threads

[ https://issues.apache.org/jira/browse/SPARK-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278421#comment-16278421 ] Apache Spark commented on SPARK-16139: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Commented] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278405#comment-16278405 ] Julien Cuquemelle commented on SPARK-22683: --- Thanks [~srowen] for your quick feedback, let me

[jira] [Comment Edited] (SPARK-22683) Allow tuning the number of dynamically allocated executors wrt task number

[ https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278405#comment-16278405 ] Julien Cuquemelle edited comment on SPARK-22683 at 12/5/17 11:06 AM: -

[jira] [Commented] (SPARK-22692) Reduce the number of generated mutable states

[ https://issues.apache.org/jira/browse/SPARK-22692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278392#comment-16278392 ] Sean Owen commented on SPARK-22692: --- The sub JIRAs are pretty noisy. Are these really different, or so

[jira] [Created] (SPARK-22699) Avoid the generation of useless mutable states by GenerateSafeProjection

Marco Gaido created SPARK-22699: --- Summary: Avoid the generation of useless mutable states by GenerateSafeProjection Key: SPARK-22699 URL: https://issues.apache.org/jira/browse/SPARK-22699 Project:

[jira] [Created] (SPARK-22698) Avoid the generation of useless mutable states by GenerateUnsafeProjection

Marco Gaido created SPARK-22698: --- Summary: Avoid the generation of useless mutable states by GenerateUnsafeProjection Key: SPARK-22698 URL: https://issues.apache.org/jira/browse/SPARK-22698 Project:

  1   2   >