[jira] [Comment Edited] (SPARK-22935) Dataset with Java Beans for java.sql.Date throws CompileException

2018-01-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330207#comment-16330207 ] Liang-Chi Hsieh edited comment on SPARK-22935 at 1/18/18 7:53 AM: -- Can

[jira] [Comment Edited] (SPARK-22935) Dataset with Java Beans for java.sql.Date throws CompileException

2018-01-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330207#comment-16330207 ] Liang-Chi Hsieh edited comment on SPARK-22935 at 1/18/18 7:52 AM: -- Can

[jira] [Commented] (SPARK-22935) Dataset with Java Beans for java.sql.Date throws CompileException

2018-01-17 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330207#comment-16330207 ] Liang-Chi Hsieh commented on SPARK-22935: - Can > Dataset with Java Beans for java.sql.Date

[jira] [Commented] (SPARK-10030) Managed memory leak detected when cache table

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330143#comment-16330143 ] Apache Spark commented on SPARK-10030: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-23143) Add Python support for continuous trigger

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330122#comment-16330122 ] Apache Spark commented on SPARK-23143: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23143) Add Python support for continuous trigger

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23143: Assignee: Tathagata Das (was: Apache Spark) > Add Python support for continuous trigger

[jira] [Assigned] (SPARK-23143) Add Python support for continuous trigger

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23143: Assignee: Apache Spark (was: Tathagata Das) > Add Python support for continuous trigger

[jira] [Resolved] (SPARK-23052) Migrate ConsoleSink to v2

2018-01-17 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23052. --- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by

[jira] [Created] (SPARK-23143) Add Python support for continuous trigger

2018-01-17 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-23143: - Summary: Add Python support for continuous trigger Key: SPARK-23143 URL: https://issues.apache.org/jira/browse/SPARK-23143 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-23143) Add Python support for continuous trigger

2018-01-17 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-23143: -- Issue Type: Improvement (was: Bug) > Add Python support for continuous trigger >

[jira] [Assigned] (SPARK-23142) Add documentation for Continuous Processing

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23142: Assignee: Tathagata Das (was: Apache Spark) > Add documentation for Continuous

[jira] [Assigned] (SPARK-23142) Add documentation for Continuous Processing

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23142: Assignee: Apache Spark (was: Tathagata Das) > Add documentation for Continuous

[jira] [Commented] (SPARK-23142) Add documentation for Continuous Processing

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330104#comment-16330104 ] Apache Spark commented on SPARK-23142: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23141) Support data type string as a returnType for registerJavaFunction.

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23141: Assignee: (was: Apache Spark) > Support data type string as a returnType for

[jira] [Assigned] (SPARK-23141) Support data type string as a returnType for registerJavaFunction.

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23141: Assignee: Apache Spark > Support data type string as a returnType for

[jira] [Commented] (SPARK-23141) Support data type string as a returnType for registerJavaFunction.

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330100#comment-16330100 ] Apache Spark commented on SPARK-23141: -- User 'ueshin' has created a pull request for this issue:

[jira] [Updated] (SPARK-23142) Add documentation for Continuous Processing

2018-01-17 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-23142: -- Issue Type: Improvement (was: Bug) > Add documentation for Continuous Processing >

[jira] [Created] (SPARK-23142) Add documentation for Continuous Processing

2018-01-17 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-23142: - Summary: Add documentation for Continuous Processing Key: SPARK-23142 URL: https://issues.apache.org/jira/browse/SPARK-23142 Project: Spark Issue Type:

[jira] [Created] (SPARK-23141) Support data type string as a returnType for registerJavaFunction.

2018-01-17 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-23141: - Summary: Support data type string as a returnType for registerJavaFunction. Key: SPARK-23141 URL: https://issues.apache.org/jira/browse/SPARK-23141 Project: Spark

[jira] [Comment Edited] (SPARK-23122) Deprecate register* for UDFs in SQLContext and Catalog in PySpark

2018-01-17 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330076#comment-16330076 ] Takuya Ueshin edited comment on SPARK-23122 at 1/18/18 5:57 AM: Issue

[jira] [Resolved] (SPARK-23122) Deprecate register* for UDFs in SQLContext and Catalog in PySpark

2018-01-17 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-23122. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20288

[jira] [Assigned] (SPARK-23122) Deprecate register* for UDFs in SQLContext and Catalog in PySpark

2018-01-17 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-23122: - Assignee: Hyukjin Kwon > Deprecate register* for UDFs in SQLContext and Catalog in

[jira] [Commented] (SPARK-23054) Incorrect results of casting UserDefinedType to String

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330063#comment-16330063 ] Apache Spark commented on SPARK-23054: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-23125) Offset commit failed when spark-streaming batch time is more than kafkaParams session timeout.

2018-01-17 Thread zhaoshijie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16330001#comment-16330001 ] zhaoshijie commented on SPARK-23125: yes, config does not work.The error message suggest "increasing

[jira] [Commented] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329950#comment-16329950 ] Apache Spark commented on SPARK-23140: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23140: Assignee: (was: Apache Spark) > DataSourceV2Strategy is missing in

[jira] [Assigned] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23140: Assignee: Apache Spark > DataSourceV2Strategy is missing in HiveSessionStateBuilder >

[jira] [Updated] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

2018-01-17 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23140: Description: DataSourceV2Strategy is not added into HiveSessionStateBuilder's planner, which will

[jira] [Updated] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

2018-01-17 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23140: Environment: (was: DataSourceV2Strategy is not added into HiveSessionStateBuilder's planner,

[jira] [Created] (SPARK-23140) DataSourceV2Strategy is missing in HiveSessionStateBuilder

2018-01-17 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-23140: --- Summary: DataSourceV2Strategy is missing in HiveSessionStateBuilder Key: SPARK-23140 URL: https://issues.apache.org/jira/browse/SPARK-23140 Project: Spark

[jira] [Resolved] (SPARK-23079) Fix query constraints propagation with aliases

2018-01-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23079. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20278

[jira] [Assigned] (SPARK-23079) Fix query constraints propagation with aliases

2018-01-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23079: --- Assignee: Gengliang Wang > Fix query constraints propagation with aliases >

[jira] [Assigned] (SPARK-23139) Read eventLog file with mixed encodings

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23139: Assignee: (was: Apache Spark) > Read eventLog file with mixed encodings >

[jira] [Assigned] (SPARK-23139) Read eventLog file with mixed encodings

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23139: Assignee: Apache Spark > Read eventLog file with mixed encodings >

[jira] [Commented] (SPARK-23139) Read eventLog file with mixed encodings

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329886#comment-16329886 ] Apache Spark commented on SPARK-23139: -- User 'hellodengfei' has created a pull request for this

[jira] [Created] (SPARK-23139) Read eventLog file with mixed encodings

2018-01-17 Thread DENG FEI (JIRA)
DENG FEI created SPARK-23139: Summary: Read eventLog file with mixed encodings Key: SPARK-23139 URL: https://issues.apache.org/jira/browse/SPARK-23139 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329844#comment-16329844 ] Apache Spark commented on SPARK-23128: -- User 'carsonwang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23128: Assignee: (was: Apache Spark) > A new approach to do adaptive execution in Spark SQL

[jira] [Assigned] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23128: Assignee: Apache Spark > A new approach to do adaptive execution in Spark SQL >

[jira] [Updated] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2018-01-17 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carson Wang updated SPARK-23128: Affects Version/s: (was: 2.2.1) 2.3.0 > A new approach to do adaptive

[jira] [Assigned] (SPARK-22216) Improving PySpark/Pandas interoperability

2018-01-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22216: Assignee: Li Jin > Improving PySpark/Pandas interoperability >

[jira] [Assigned] (SPARK-22216) Improving PySpark/Pandas interoperability

2018-01-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22216: Assignee: (was: Hyukjin Kwon) > Improving PySpark/Pandas interoperability >

[jira] [Assigned] (SPARK-22980) Using pandas_udf when inputs are not Pandas's Series or DataFrame

2018-01-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22980: Assignee: Hyukjin Kwon > Using pandas_udf when inputs are not Pandas's Series or

[jira] [Assigned] (SPARK-22216) Improving PySpark/Pandas interoperability

2018-01-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22216: Assignee: Hyukjin Kwon > Improving PySpark/Pandas interoperability >

[jira] [Resolved] (SPARK-22980) Using pandas_udf when inputs are not Pandas's Series or DataFrame

2018-01-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22980. -- Resolution: Fixed > Using pandas_udf when inputs are not Pandas's Series or DataFrame >

[jira] [Assigned] (SPARK-23094) Json Readers choose wrong encoding when bad records are present and fail

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23094: Assignee: (was: Apache Spark) > Json Readers choose wrong encoding when bad records

[jira] [Assigned] (SPARK-23094) Json Readers choose wrong encoding when bad records are present and fail

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23094: Assignee: Apache Spark > Json Readers choose wrong encoding when bad records are present

[jira] [Assigned] (SPARK-22735) Add VectorSizeHint to ML features documentation

2018-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-22735: - Assignee: Bago Amirbekian > Add VectorSizeHint to ML features documentation >

[jira] [Commented] (SPARK-23094) Json Readers choose wrong encoding when bad records are present and fail

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329803#comment-16329803 ] Apache Spark commented on SPARK-23094: -- User 'brkyvz' has created a pull request for this issue:

[jira] [Commented] (SPARK-23138) Add user guide example for multiclass logistic regression summary

2018-01-17 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329783#comment-16329783 ] Seth Hendrickson commented on SPARK-23138: -- I can submit a PR for this soon. > Add user guide

[jira] [Created] (SPARK-23138) Add user guide example for multiclass logistic regression summary

2018-01-17 Thread Seth Hendrickson (JIRA)
Seth Hendrickson created SPARK-23138: Summary: Add user guide example for multiclass logistic regression summary Key: SPARK-23138 URL: https://issues.apache.org/jira/browse/SPARK-23138 Project:

[jira] [Updated] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2018-01-17 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carson Wang updated SPARK-23128: Description: SPARK-9850 proposed the basic idea of adaptive execution in Spark. In DAGScheduler,

[jira] [Updated] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2018-01-17 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carson Wang updated SPARK-23128: Summary: A new approach to do adaptive execution in Spark SQL (was: Introduce QueryStage to

[jira] [Assigned] (SPARK-21996) Streaming ignores files with spaces in the file names

2018-01-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-21996: Assignee: Xiayun Sun > Streaming ignores files with spaces in the file names >

[jira] [Resolved] (SPARK-23064) Add documentation for stream-stream joins

2018-01-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23064. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20255

[jira] [Resolved] (SPARK-21996) Streaming ignores files with spaces in the file names

2018-01-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21996. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19247

[jira] [Resolved] (SPARK-23119) Fix API annotation in DataSource V2 for streaming

2018-01-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23119. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20286

[jira] [Commented] (SPARK-23137) spark.kubernetes.executor.podNamePrefix is ignored

2018-01-17 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329731#comment-16329731 ] Anirudh Ramanathan commented on SPARK-23137: Looks like consensus is - we can get rid of the

[jira] [Commented] (SPARK-23137) spark.kubernetes.executor.podNamePrefix is ignored

2018-01-17 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329721#comment-16329721 ] Erik Erlandson commented on SPARK-23137: +1, a more general "app prefix" seems more useful >

[jira] [Commented] (SPARK-23137) spark.kubernetes.executor.podNamePrefix is ignored

2018-01-17 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329717#comment-16329717 ] Anirudh Ramanathan commented on SPARK-23137: Good point - I think the option is useful, but

[jira] [Commented] (SPARK-23137) spark.kubernetes.executor.podNamePrefix is ignored

2018-01-17 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329713#comment-16329713 ] Yinan Li commented on SPARK-23137: -- It's actually marked as an \{{internal}} config property. So the fix

[jira] [Created] (SPARK-23137) spark.kubernetes.executor.podNamePrefix is ignored

2018-01-17 Thread Anirudh Ramanathan (JIRA)
Anirudh Ramanathan created SPARK-23137: -- Summary: spark.kubernetes.executor.podNamePrefix is ignored Key: SPARK-23137 URL: https://issues.apache.org/jira/browse/SPARK-23137 Project: Spark

[jira] [Updated] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

2018-01-17 Thread Sameer Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal updated SPARK-23135: --- Target Version/s: 2.3.0 > Accumulators don't show up properly in the Stages page anymore >

[jira] [Resolved] (SPARK-23050) Structured Streaming with S3 file source duplicates data because of eventual consistency.

2018-01-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23050. --- Resolution: Not A Problem The rest of the conversation notwithstanding, earlier comments suggest

[jira] [Commented] (SPARK-23016) Spark UI access and documentation

2018-01-17 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329675#comment-16329675 ] Anirudh Ramanathan commented on SPARK-23016: Works as expected with port forwarding in my

[jira] [Assigned] (SPARK-23104) Document that kubernetes is still "experimental"

2018-01-17 Thread Sameer Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal reassigned SPARK-23104: -- Assignee: Anirudh Ramanathan > Document that kubernetes is still "experimental" >

[jira] [Commented] (SPARK-23104) Document that kubernetes is still "experimental"

2018-01-17 Thread Sameer Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329644#comment-16329644 ] Sameer Agarwal commented on SPARK-23104: Sounds great, thanks! > Document that kubernetes is

[jira] [Assigned] (SPARK-23131) Stackoverflow using ML and Kryo serializer

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23131: Assignee: Apache Spark > Stackoverflow using ML and Kryo serializer >

[jira] [Assigned] (SPARK-23131) Stackoverflow using ML and Kryo serializer

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23131: Assignee: (was: Apache Spark) > Stackoverflow using ML and Kryo serializer >

[jira] [Commented] (SPARK-23131) Stackoverflow using ML and Kryo serializer

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329627#comment-16329627 ] Apache Spark commented on SPARK-23131: -- User 'srowen' has created a pull request for this issue:

[jira] [Resolved] (SPARK-23132) Run ml.image doctests in tests

2018-01-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23132. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20294

[jira] [Commented] (SPARK-23104) Document that kubernetes is still "experimental"

2018-01-17 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329621#comment-16329621 ] Anirudh Ramanathan commented on SPARK-23104: [~sameerag], this is a docs-only change, will

[jira] [Assigned] (SPARK-23132) Run ml.image doctests in tests

2018-01-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23132: Assignee: Hyukjin Kwon > Run ml.image doctests in tests > --

[jira] [Commented] (SPARK-23104) Document that kubernetes is still "experimental"

2018-01-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329618#comment-16329618 ] Marcelo Vanzin commented on SPARK-23104: That should be enough. Since there is no API the user

[jira] [Commented] (SPARK-23104) Document that kubernetes is still "experimental"

2018-01-17 Thread Anirudh Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329608#comment-16329608 ] Anirudh Ramanathan commented on SPARK-23104: Thanks for opening this - is this simply a

[jira] [Assigned] (SPARK-23047) Change MapVector to NullableMapVector in ArrowColumnVector

2018-01-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23047: Assignee: Li Jin > Change MapVector to NullableMapVector in ArrowColumnVector >

[jira] [Resolved] (SPARK-23047) Change MapVector to NullableMapVector in ArrowColumnVector

2018-01-17 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23047. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20239

[jira] [Resolved] (SPARK-23136) Mark packages as experimental

2018-01-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23136. Resolution: Duplicate I actually filed SPARK-23104 yesterday for this. > Mark packages as

[jira] [Commented] (SPARK-23131) Stackoverflow using ML and Kryo serializer

2018-01-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329595#comment-16329595 ] Sean Owen commented on SPARK-23131: --- OK, good to know. A kryo upgrade is overdue, and may still be

[jira] [Created] (SPARK-23136) Mark packages as experimental

2018-01-17 Thread Anirudh Ramanathan (JIRA)
Anirudh Ramanathan created SPARK-23136: -- Summary: Mark packages as experimental Key: SPARK-23136 URL: https://issues.apache.org/jira/browse/SPARK-23136 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-23093) don't modify run id

2018-01-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23093. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20282

[jira] [Assigned] (SPARK-23093) don't modify run id

2018-01-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-23093: Assignee: Jose Torres > don't modify run id > --- > >

[jira] [Updated] (SPARK-22993) checkpointInterval param doc should be clearer

2018-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22993: -- Fix Version/s: 2.3.0 > checkpointInterval param doc should be clearer >

[jira] [Resolved] (SPARK-23033) disable task-level retry for continuous execution

2018-01-17 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23033. --- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by

[jira] [Assigned] (SPARK-23110) ML 2.3 QA: API: Java compatibility, docs

2018-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-23110: - Assignee: Weichen Xu > ML 2.3 QA: API: Java compatibility, docs >

[jira] [Commented] (SPARK-23110) ML 2.3 QA: API: Java compatibility, docs

2018-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329489#comment-16329489 ] Joseph K. Bradley commented on SPARK-23110: --- [~WeichenXu123] said he'd take this task again

[jira] [Commented] (SPARK-23110) ML 2.3 QA: API: Java compatibility, docs

2018-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329487#comment-16329487 ] Joseph K. Bradley commented on SPARK-23110: --- Please check out the old JIRA for this from 2.2

[jira] [Commented] (SPARK-23109) ML 2.3 QA: API: Python API coverage

2018-01-17 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329481#comment-16329481 ] Bryan Cutler commented on SPARK-23109: -- [~josephkb] I can take this, thanks! > ML 2.3 QA: API:

[jira] [Assigned] (SPARK-23114) Spark R 2.3 QA umbrella

2018-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-23114: - Assignee: Felix Cheung > Spark R 2.3 QA umbrella > --- > >

[jira] [Commented] (SPARK-23114) Spark R 2.3 QA umbrella

2018-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329478#comment-16329478 ] Joseph K. Bradley commented on SPARK-23114: --- Thank you! > Spark R 2.3 QA umbrella >

[jira] [Reopened] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

2018-01-17 Thread Sean Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Roberts reopened SPARK-23130: -- > Spark Thrift does not clean-up temporary files (/tmp/*_resources and > /tmp/hive/*.pipeout) >

[jira] [Commented] (SPARK-23130) Spark Thrift does not clean-up temporary files (/tmp/*_resources and /tmp/hive/*.pipeout)

2018-01-17 Thread Sean Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329464#comment-16329464 ] Sean Roberts commented on SPARK-23130: -- Marco - Which JIRA resolves the pipeout issue or

[jira] [Assigned] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23135: Assignee: (was: Apache Spark) > Accumulators don't show up properly in the Stages

[jira] [Commented] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329419#comment-16329419 ] Apache Spark commented on SPARK-23135: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23135: Assignee: Apache Spark > Accumulators don't show up properly in the Stages page anymore >

[jira] [Commented] (SPARK-22976) Worker cleanup can remove running driver directories

2018-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329400#comment-16329400 ] Apache Spark commented on SPARK-22976: -- User 'RussellSpitzer' has created a pull request for this

[jira] [Commented] (SPARK-23135) Accumulators don't show up properly in the Stages page anymore

2018-01-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329398#comment-16329398 ] Marcelo Vanzin commented on SPARK-23135: Nevermind, I was able to get the wrong table after some

[jira] [Commented] (SPARK-23103) LevelDB store not iterating correctly when indexed value has negative value

2018-01-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329380#comment-16329380 ] Marcelo Vanzin commented on SPARK-23103: Given the unit test failure in the PR it might not be as

[jira] [Comment Edited] (SPARK-23103) LevelDB store not iterating correctly when indexed value has negative value

2018-01-17 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329367#comment-16329367 ] Imran Rashid edited comment on SPARK-23103 at 1/17/18 8:07 PM: --- -[~vanzin]

[jira] [Commented] (SPARK-23103) LevelDB store not iterating correctly when indexed value has negative value

2018-01-17 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329367#comment-16329367 ] Imran Rashid commented on SPARK-23103: -- [~vanzin] is this really minor, not a blocker for 2.3? Or

  1   2   >