[jira] [Comment Edited] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-18 Thread Gaurav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331847#comment-16331847 ] Gaurav edited comment on SPARK-18016 at 1/19/18 7:31 AM: - saving DataFrame having

[jira] [Comment Edited] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-18 Thread Gaurav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331847#comment-16331847 ] Gaurav edited comment on SPARK-18016 at 1/19/18 7:29 AM: - saving DataFrame having

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-18 Thread Gaurav (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331847#comment-16331847 ] Gaurav commented on SPARK-18016: saving DataFrame having 4 columns in snappy.parquet format in HDFS

[jira] [Commented] (SPARK-23148) spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces

2018-01-18 Thread jifei_yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331817#comment-16331817 ] jifei_yang commented on SPARK-23148: If you must do this, I think it is best to add escaping it. >

[jira] [Commented] (SPARK-12963) In cluster mode,spark_local_ip will cause driver exception:Service 'Driver' failed after 16 retries!

2018-01-18 Thread Gera Shegalov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331815#comment-16331815 ] Gera Shegalov commented on SPARK-12963: --- We hit the same issue on nodes where a process is not

[jira] [Commented] (SPARK-23154) Document backwards compatibility guarantees for ML persistence

2018-01-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331811#comment-16331811 ] Joseph K. Bradley commented on SPARK-23154: --- CC [~mlnick], [~yanboliang] or others, what do you

[jira] [Commented] (SPARK-12963) In cluster mode,spark_local_ip will cause driver exception:Service 'Driver' failed after 16 retries!

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331799#comment-16331799 ] Apache Spark commented on SPARK-12963: -- User 'gerashegalov' has created a pull request for this

[jira] [Assigned] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23155: Assignee: (was: Apache Spark) > YARN-aggregated executor/driver logs appear

[jira] [Assigned] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23155: Assignee: Apache Spark > YARN-aggregated executor/driver logs appear unavailable when NM

[jira] [Commented] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331776#comment-16331776 ] Apache Spark commented on SPARK-23155: -- User 'gerashegalov' has created a pull request for this

[jira] [Assigned] (SPARK-22808) saveAsTable() should be marked as deprecated

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22808: Assignee: Apache Spark > saveAsTable() should be marked as deprecated >

[jira] [Commented] (SPARK-22808) saveAsTable() should be marked as deprecated

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331773#comment-16331773 ] Apache Spark commented on SPARK-22808: -- User 'brandonJY' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22808) saveAsTable() should be marked as deprecated

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22808: Assignee: (was: Apache Spark) > saveAsTable() should be marked as deprecated >

[jira] [Created] (SPARK-23155) YARN-aggregated executor/driver logs appear unavailable when NM is down

2018-01-18 Thread Gera Shegalov (JIRA)
Gera Shegalov created SPARK-23155: - Summary: YARN-aggregated executor/driver logs appear unavailable when NM is down Key: SPARK-23155 URL: https://issues.apache.org/jira/browse/SPARK-23155 Project:

[jira] [Commented] (SPARK-22977) DataFrameWriter operations do not show details in SQL tab

2018-01-18 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331754#comment-16331754 ] Yuming Wang commented on SPARK-22977: - cc [~cloud_fan] > DataFrameWriter operations do not show

[jira] [Commented] (SPARK-23097) Migrate text socket source

2018-01-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331735#comment-16331735 ] Saisai Shao commented on SPARK-23097: - Thanks, will do. > Migrate text socket source >

[jira] [Commented] (SPARK-23097) Migrate text socket source

2018-01-18 Thread Jose Torres (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331726#comment-16331726 ] Jose Torres commented on SPARK-23097: - Certainly! There's no specific plan right now. I'm working on

[jira] [Commented] (SPARK-23097) Migrate text socket source

2018-01-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331715#comment-16331715 ] Saisai Shao commented on SPARK-23097: - Hi [~joseph.torres], would you mind if I take a shot on this

[jira] [Resolved] (SPARK-23054) Incorrect results of casting UserDefinedType to String

2018-01-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23054. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.3.0 > Incorrect

[jira] [Updated] (SPARK-23154) Document backwards compatibility guarantees for ML persistence

2018-01-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23154: -- Description: We have (as far as I know) maintained backwards compatibility for ML

[jira] [Created] (SPARK-23154) Document backwards compatibility guarantees for ML persistence

2018-01-18 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-23154: - Summary: Document backwards compatibility guarantees for ML persistence Key: SPARK-23154 URL: https://issues.apache.org/jira/browse/SPARK-23154 Project:

[jira] [Assigned] (SPARK-23091) Incorrect unit test for approxQuantile

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23091: Assignee: Apache Spark > Incorrect unit test for approxQuantile >

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2018-01-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331542#comment-16331542 ] Saisai Shao commented on SPARK-23128: - I'm wondering if we need a SPIP for such changes? > A new

[jira] [Assigned] (SPARK-23091) Incorrect unit test for approxQuantile

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23091: Assignee: (was: Apache Spark) > Incorrect unit test for approxQuantile >

[jira] [Commented] (SPARK-23091) Incorrect unit test for approxQuantile

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331541#comment-16331541 ] Apache Spark commented on SPARK-23091: -- User 'srowen' has created a pull request for this issue:

[jira] [Resolved] (SPARK-23142) Add documentation for Continuous Processing

2018-01-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23142. --- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by

[jira] [Assigned] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22962: -- Assignee: Yinan Li > Kubernetes app fails if local files are used >

[jira] [Resolved] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22962. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20320

[jira] [Resolved] (SPARK-23094) Json Readers choose wrong encoding when bad records are present and fail

2018-01-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23094. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20302

[jira] [Assigned] (SPARK-23094) Json Readers choose wrong encoding when bad records are present and fail

2018-01-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23094: Assignee: Burak Yavuz > Json Readers choose wrong encoding when bad records are present

[jira] [Updated] (SPARK-22216) Improving PySpark/Pandas interoperability

2018-01-18 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Jin updated SPARK-22216: --- Description: This is an umbrella ticket tracking the general effort to improve performance and

[jira] [Assigned] (SPARK-23133) Spark options are not passed to the Executor in Docker context

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23133: -- Assignee: Andrew Korzhuev > Spark options are not passed to the Executor in Docker

[jira] [Resolved] (SPARK-23133) Spark options are not passed to the Executor in Docker context

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23133. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20322

[jira] [Created] (SPARK-23153) Support application dependencies in submission client's local file system

2018-01-18 Thread Yinan Li (JIRA)
Yinan Li created SPARK-23153: Summary: Support application dependencies in submission client's local file system Key: SPARK-23153 URL: https://issues.apache.org/jira/browse/SPARK-23153 Project: Spark

[jira] [Commented] (SPARK-23133) Spark options are not passed to the Executor in Docker context

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331241#comment-16331241 ] Apache Spark commented on SPARK-23133: -- User 'foxish' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-19185: -- Assignee: Marcelo Vanzin > ConcurrentModificationExceptions with CachedKafkaConsumers

[jira] [Assigned] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-19185: -- Assignee: (was: Marcelo Vanzin) > ConcurrentModificationExceptions with

[jira] [Assigned] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23152: Assignee: Apache Spark > Invalid guard condition in

[jira] [Assigned] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23152: Assignee: (was: Apache Spark) > Invalid guard condition in

[jira] [Commented] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331194#comment-16331194 ] Apache Spark commented on SPARK-23152: -- User 'tovbinm' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22962: Assignee: (was: Apache Spark) > Kubernetes app fails if local files are used >

[jira] [Assigned] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22962: Assignee: Apache Spark > Kubernetes app fails if local files are used >

[jira] [Commented] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331185#comment-16331185 ] Apache Spark commented on SPARK-22962: -- User 'liyinan926' has created a pull request for this issue:

[jira] [Resolved] (SPARK-23144) Add console sink for continuous queries

2018-01-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23144. --- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends

[jira] [Assigned] (SPARK-22884) ML test for StructuredStreaming: spark.ml.clustering

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22884: Assignee: (was: Apache Spark) > ML test for StructuredStreaming: spark.ml.clustering

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends

[jira] [Commented] (SPARK-22884) ML test for StructuredStreaming: spark.ml.clustering

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331157#comment-16331157 ] Apache Spark commented on SPARK-22884: -- User 'smurakozi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22884) ML test for StructuredStreaming: spark.ml.clustering

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22884: Assignee: Apache Spark > ML test for StructuredStreaming: spark.ml.clustering >

[jira] [Commented] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331153#comment-16331153 ] Anirudh Ramanathan commented on SPARK-22962: I think this isn't super critical for this

[jira] [Resolved] (SPARK-23143) Add Python support for continuous trigger

2018-01-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23143. --- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends

[jira] [Commented] (SPARK-20928) SPIP: Continuous Processing Mode for Structured Streaming

2018-01-18 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331141#comment-16331141 ] Michael Armbrust commented on SPARK-20928: -- There is more work to do so I might leave the

[jira] [Commented] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331140#comment-16331140 ] Marcelo Vanzin commented on SPARK-22962: Yes send a PR. > Kubernetes app fails if local files

[jira] [Commented] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331132#comment-16331132 ] Yinan Li commented on SPARK-22962: -- I agree that before we upstream the staging server, we should fail

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends

[jira] [Updated] (SPARK-23146) Support client mode for Kubernetes cluster backend

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-23146: --- Target Version/s: 2.4.0 > Support client mode for Kubernetes cluster backend >

[jira] [Updated] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22962: --- Affects Version/s: (was: 2.4.0) 2.3.0 > Kubernetes app

[jira] [Updated] (SPARK-23146) Support client mode for Kubernetes cluster backend

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-23146: --- Affects Version/s: (was: 2.4.0) 2.3.0 > Support client

[jira] [Updated] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-22962: --- Affects Version/s: (was: 2.3.0) 2.4.0 > Kubernetes app

[jira] [Commented] (SPARK-22962) Kubernetes app fails if local files are used

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331085#comment-16331085 ] Anirudh Ramanathan commented on SPARK-22962: This is the resource staging server use-case. 

[jira] [Resolved] (SPARK-23016) Spark UI access and documentation

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan resolved SPARK-23016. Resolution: Fixed This is resolved and we've verified it for 2.3.0. > Spark UI

[jira] [Comment Edited] (SPARK-23082) Allow separate node selectors for driver and executors in Kubernetes

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331081#comment-16331081 ] Anirudh Ramanathan edited comment on SPARK-23082 at 1/18/18 7:52 PM: -

[jira] [Commented] (SPARK-23082) Allow separate node selectors for driver and executors in Kubernetes

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331081#comment-16331081 ] Anirudh Ramanathan commented on SPARK-23082: This is an interesting feature request. We had

[jira] [Commented] (SPARK-23133) Spark options are not passed to the Executor in Docker context

2018-01-18 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331074#comment-16331074 ] Anirudh Ramanathan commented on SPARK-23133: Thanks for submitting the PR fixing this. >

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends

[jira] [Commented] (SPARK-20928) SPIP: Continuous Processing Mode for Structured Streaming

2018-01-18 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331045#comment-16331045 ] Thomas Graves commented on SPARK-20928: --- what is status of this, it looks like subtasks are

[jira] [Created] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
Matthew Tovbin created SPARK-23152: -- Summary: Invalid guard condition in org.apache.spark.ml.classification.Classifier Key: SPARK-23152 URL: https://issues.apache.org/jira/browse/SPARK-23152

[jira] [Updated] (SPARK-23152) Invalid guard condition in org.apache.spark.ml.classification.Classifier

2018-01-18 Thread Matthew Tovbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Tovbin updated SPARK-23152: --- Description: When fitting a classifier that extends

[jira] [Assigned] (SPARK-23029) Doc spark.shuffle.file.buffer units are kb when no units specified

2018-01-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23029: - Assignee: Fernando Pereira > Doc spark.shuffle.file.buffer units are kb when no units specified

[jira] [Resolved] (SPARK-23029) Doc spark.shuffle.file.buffer units are kb when no units specified

2018-01-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23029. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20269

[jira] [Issue Comment Deleted] (SPARK-13572) HiveContext reads avro Hive tables incorrectly

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-13572: --- Comment: was deleted (was: Guten Tag, Bonjour, Buongiorno Besten Dank für Ihr E-Mail. Ich

[jira] [Commented] (SPARK-22982) Remove unsafe asynchronous close() call from FileDownloadChannel

2018-01-18 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330943#comment-16330943 ] Andrew Ash commented on SPARK-22982: [~joshrosen] do you have some example stacktraces of what this

[jira] [Assigned] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23147: -- Assignee: Saisai Shao > Stage page will throw exception when there's no complete

[jira] [Resolved] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23147. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20315

[jira] [Updated] (SPARK-23151) Provide a distribution of Spark with Hadoop 3.0

2018-01-18 Thread Louis Burke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Louis Burke updated SPARK-23151: Summary: Provide a distribution of Spark with Hadoop 3.0 (was: Provide a distribution Spark with

[jira] [Created] (SPARK-23151) Provide a distribution Spark with Hadoop 3.0

2018-01-18 Thread Louis Burke (JIRA)
Louis Burke created SPARK-23151: --- Summary: Provide a distribution Spark with Hadoop 3.0 Key: SPARK-23151 URL: https://issues.apache.org/jira/browse/SPARK-23151 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-23150) kafka job failing assertion failed: Beginning offset 5451 is after the ending offset 5435 for topic

2018-01-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23150. --- Resolution: Invalid > kafka job failing assertion failed: Beginning offset 5451 is after the ending

[jira] [Created] (SPARK-23150) kafka job failing assertion failed: Beginning offset 5451 is after the ending offset 5435 for topic

2018-01-18 Thread Balu (JIRA)
Balu created SPARK-23150: Summary: kafka job failing assertion failed: Beginning offset 5451 is after the ending offset 5435 for topic Key: SPARK-23150 URL: https://issues.apache.org/jira/browse/SPARK-23150

[jira] [Assigned] (SPARK-23149) polish ColumnarBatch

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23149: Assignee: Wenchen Fan (was: Apache Spark) > polish ColumnarBatch >

[jira] [Commented] (SPARK-23149) polish ColumnarBatch

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330610#comment-16330610 ] Apache Spark commented on SPARK-23149: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23149) polish ColumnarBatch

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23149: Assignee: Apache Spark (was: Wenchen Fan) > polish ColumnarBatch >

[jira] [Created] (SPARK-23149) polish ColumnarBatch

2018-01-18 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-23149: --- Summary: polish ColumnarBatch Key: SPARK-23149 URL: https://issues.apache.org/jira/browse/SPARK-23149 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-19280) Failed Recovery from checkpoint caused by the multi-threads issue in Spark Streaming scheduler

2018-01-18 Thread Riccardo Vincelli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330558#comment-16330558 ] Riccardo Vincelli edited comment on SPARK-19280 at 1/18/18 2:20 PM:

[jira] [Commented] (SPARK-19280) Failed Recovery from checkpoint caused by the multi-threads issue in Spark Streaming scheduler

2018-01-18 Thread Riccardo Vincelli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330558#comment-16330558 ] Riccardo Vincelli commented on SPARK-19280: --- What is the exception you are getting? For me it

[jira] [Resolved] (SPARK-23141) Support data type string as a returnType for registerJavaFunction.

2018-01-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23141. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20307

[jira] [Assigned] (SPARK-23141) Support data type string as a returnType for registerJavaFunction.

2018-01-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23141: Assignee: Takuya Ueshin > Support data type string as a returnType for

[jira] [Resolved] (SPARK-22036) BigDecimal multiplication sometimes returns null

2018-01-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22036. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20023

[jira] [Assigned] (SPARK-22036) BigDecimal multiplication sometimes returns null

2018-01-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22036: --- Assignee: Marco Gaido > BigDecimal multiplication sometimes returns null >

[jira] [Commented] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330496#comment-16330496 ] Apache Spark commented on SPARK-23147: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23147: Assignee: (was: Apache Spark) > Stage page will throw exception when there's no

[jira] [Assigned] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23147: Assignee: Apache Spark > Stage page will throw exception when there's no complete tasks >

[jira] [Created] (SPARK-23148) spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces

2018-01-18 Thread Bogdan Raducanu (JIRA)
Bogdan Raducanu created SPARK-23148: --- Summary: spark.read.csv with multiline=true gives FileNotFoundException if path contains spaces Key: SPARK-23148 URL: https://issues.apache.org/jira/browse/SPARK-23148

[jira] [Updated] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23147: Description: Current Stage page's task table will throw an exception when there's no complete

[jira] [Created] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-23147: --- Summary: Stage page will throw exception when there's no complete tasks Key: SPARK-23147 URL: https://issues.apache.org/jira/browse/SPARK-23147 Project: Spark

[jira] [Updated] (SPARK-23147) Stage page will throw exception when there's no complete tasks

2018-01-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23147: Attachment: Screen Shot 2018-01-18 at 8.50.08 PM.png > Stage page will throw exception when

  1   2   >