[jira] [Resolved] (SPARK-12436) If all values of a JSON field is null, JSON's inferSchema should return NullType instead of StringType

2018-06-19 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-12436. --- Resolution: Resolved Resolved by SPARK-23772 (an alternative solution to this JIRA) > If al

[jira] [Created] (SPARK-24609) PySpark doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

2018-06-20 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-24609: - Summary: PySpark doc doesn't explain RandomForestClassifier.featureSubsetStrategy well Key: SPARK-24609 URL: https://issues.apache.org/jira/browse/SPARK-24609 Proje

[jira] [Updated] (SPARK-24609) PySpark/SparkR doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

2018-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24609: -- Summary: PySpark/SparkR doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

[jira] [Updated] (SPARK-24609) PySpark doc doesn't explain RandomForestClassifier.featureSubsetStrategy well

2018-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24609: -- Description: In Scala doc ([https://spark.apache.org/docs/2.3.0/api/scala/index.html#org.apac

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

2018-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Labels: Hydrogen SPIP (was: SPIP) > Accelerator aware task scheduling for Spark > ---

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

2018-06-25 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522661#comment-16522661 ] Xiangrui Meng commented on SPARK-24530: --- [~dongjoon] [~hyukjin.kwon] Could you rep

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

2018-06-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524510#comment-16524510 ] Xiangrui Meng commented on SPARK-24530: --- Confirmed that macOS, python 3, and Sphin

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (using Python 2?)

2018-06-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24530: -- Summary: Sphinx doesn't render autodoc_docstring_signature correctly (using Python 2?) (was:

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

2018-06-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24530: -- Summary: Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pysp

[jira] [Assigned] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

2018-06-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-24530: - Assignee: Hyukjin Kwon > Sphinx doesn't render autodoc_docstring_signature correctly (w

[jira] [Created] (SPARK-24719) ClusteringEvaluator supports integer type labels

2018-07-02 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-24719: - Summary: ClusteringEvaluator supports integer type labels Key: SPARK-24719 URL: https://issues.apache.org/jira/browse/SPARK-24719 Project: Spark Issue Type

[jira] [Updated] (SPARK-24719) ClusteringEvaluator supports integer type labels

2018-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24719: -- Description: ClusterEvaluator should support integer labels because we output integer labels

[jira] [Updated] (SPARK-24719) ClusteringEvaluator supports integer type labels

2018-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24719: -- Description: ClusterEvaluator should support integer labels because we output integer labels

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

2018-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530563#comment-16530563 ] Xiangrui Meng commented on SPARK-24579: --- Addressed the inline comments in the doc.

[jira] [Created] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

2018-07-02 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-24723: - Summary: Discuss necessary info and access in barrier mode + YARN Key: SPARK-24723 URL: https://issues.apache.org/jira/browse/SPARK-24723 Project: Spark Is

[jira] [Created] (SPARK-24724) Discuss necessary info and access in barrier mode + Kubernetes

2018-07-02 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-24724: - Summary: Discuss necessary info and access in barrier mode + Kubernetes Key: SPARK-24724 URL: https://issues.apache.org/jira/browse/SPARK-24724 Project: Spark

[jira] [Updated] (SPARK-24724) Discuss necessary info and access in barrier mode + Kubernetes

2018-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24724: -- Description: In barrier mode, to run hybrid distributed DL training jobs, we need to provide

[jira] [Updated] (SPARK-24725) Discuss necessary info and access in barrier mode + Mesos

2018-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24725: -- Description: In barrier mode, to run hybrid distributed DL training jobs, we need to provide

[jira] [Created] (SPARK-24725) Discuss necessary info and access in barrier mode + Mesos

2018-07-02 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-24725: - Summary: Discuss necessary info and access in barrier mode + Mesos Key: SPARK-24725 URL: https://issues.apache.org/jira/browse/SPARK-24725 Project: Spark I

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

2018-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Shepherd: Xiangrui Meng > Accelerator aware task scheduling for Spark > --

[jira] [Created] (SPARK-24726) Discuss necessary info and access in barrier mode + Standalone

2018-07-02 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-24726: - Summary: Discuss necessary info and access in barrier mode + Standalone Key: SPARK-24726 URL: https://issues.apache.org/jira/browse/SPARK-24726 Project: Spark

[jira] [Updated] (SPARK-24726) Discuss necessary info and access in barrier mode + Standalone

2018-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24726: -- Description: In barrier mode, to run hybrid distributed DL training jobs, we need to provide

[jira] [Commented] (SPARK-24725) Discuss necessary info and access in barrier mode + Mesos

2018-07-02 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530627#comment-16530627 ] Xiangrui Meng commented on SPARK-24725: --- [~dbtsai] Apple Siri is a major user of S

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

2018-07-03 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531737#comment-16531737 ] Xiangrui Meng commented on SPARK-24579: --- [~sethah] [~kiszk] [~rxin] Please request

[jira] [Commented] (SPARK-26410) Support per Pandas UDF configuration

2018-12-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16726155#comment-16726155 ] Xiangrui Meng commented on SPARK-26410: --- On the same cluster, there could be diffe

[jira] [Assigned] (SPARK-25970) Add Instrumentation to PrefixSpan

2018-12-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25970: - Assignee: zhengruifeng > Add Instrumentation to PrefixSpan > --

[jira] [Resolved] (SPARK-25970) Add Instrumentation to PrefixSpan

2018-12-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25970. --- Resolution: Done > Add Instrumentation to PrefixSpan > - > >

[jira] [Commented] (SPARK-24374) SPIP: Support Barrier Execution Mode in Apache Spark

2019-01-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739565#comment-16739565 ] Xiangrui Meng commented on SPARK-24374: --- [~luzengxiang] Did you use Python API or

[jira] [Commented] (SPARK-26410) Support per Pandas UDF configuration

2019-01-25 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752467#comment-16752467 ] Xiangrui Meng commented on SPARK-26410: --- There are several possible solutions to t

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

2019-01-25 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752462#comment-16752462 ] Xiangrui Meng commented on SPARK-24579: --- [~bryanc] If we limit the scope to model

[jira] [Commented] (SPARK-24374) SPIP: Support Barrier Execution Mode in Apache Spark

2019-02-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16766550#comment-16766550 ] Xiangrui Meng commented on SPARK-24374: --- [~luzengxiang] When you create external p

[jira] [Comment Edited] (SPARK-24374) SPIP: Support Barrier Execution Mode in Apache Spark

2019-02-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16766550#comment-16766550 ] Xiangrui Meng edited comment on SPARK-24374 at 2/12/19 10:54 PM: -

[jira] [Assigned] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-24615: - Assignee: Xingbo Jiang > Accelerator-aware task scheduling for Spark >

[jira] [Updated] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Attachment: Accelerator-aware scheduling in Apache Spark 3.0.pdf > Accelerator-aware task sche

[jira] [Updated] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Attachment: SPIP_ Accelerator-aware scheduling.pdf > Accelerator-aware task scheduling for Spa

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778660#comment-16778660 ] Xiangrui Meng commented on SPARK-24615: --- Attached the PDF files. > Accelerator-aw

[jira] [Updated] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

2019-02-27 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Summary: SPIP: Accelerator-aware task scheduling for Spark (was: Accelerator-aware task sched

[jira] [Updated] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

2019-02-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Description: (The JIRA received a major update on 2019/02/28. h2. Background and Motivation

[jira] [Updated] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

2019-02-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Description: (The JIRA received a major update on 2019/02/28.) h2. Background and Motivation

[jira] [Updated] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

2019-02-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Description: (The JIRA received a major update on 2019/02/28. Some comments were based on an

[jira] [Updated] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

2019-02-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Description: (The JIRA received a major update on 2019/02/28. Some comments were based on an

[jira] [Assigned] (SPARK-18924) Improve collect/createDataFrame performance in SparkR

2017-02-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-18924: - Assignee: Xiangrui Meng > Improve collect/createDataFrame performance in SparkR > --

[jira] [Updated] (SPARK-18924) Improve collect/createDataFrame performance in SparkR

2017-02-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-18924: -- Shepherd: (was: Xiangrui Meng) > Improve collect/createDataFrame performance in SparkR >

[jira] [Commented] (SPARK-18924) Improve collect/createDataFrame performance in SparkR

2017-02-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15858963#comment-15858963 ] Xiangrui Meng commented on SPARK-18924: --- I'm going to work on this one. So removed

[jira] [Closed] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

2017-02-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-5226. Resolution: Won't Fix > Add DBSCAN Clustering Algorithm to MLlib > -

[jira] [Commented] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

2017-02-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880786#comment-15880786 ] Xiangrui Meng commented on SPARK-5226: -- I closed this ticket as "Won't Do" due to DBS

[jira] [Created] (SPARK-25376) Scenarios we should handle but missed for barrier execution mode

2018-09-07 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-25376: - Summary: Scenarios we should handle but missed for barrier execution mode Key: SPARK-25376 URL: https://issues.apache.org/jira/browse/SPARK-25376 Project: Spark

[jira] [Updated] (SPARK-25376) Scenarios we should handle but missed for barrier execution mode

2018-09-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25376: -- Description: [~irashid] You mentioned that there are couple scenarios we should handle in bar

[jira] [Updated] (SPARK-25376) Scenarios we should handle but missed for barrier execution mode

2018-09-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25376: -- Description: [~irashid] You mentioned that there are couple scenarios we should handle in bar

[jira] [Updated] (SPARK-25376) Scenarios we should handle but missed in 2.4 for barrier execution mode

2018-09-07 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25376: -- Summary: Scenarios we should handle but missed in 2.4 for barrier execution mode (was: Scenar

[jira] [Created] (SPARK-25378) ArrayData.toArray assume UTF8String

2018-09-07 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-25378: - Summary: ArrayData.toArray assume UTF8String Key: SPARK-25378 URL: https://issues.apache.org/jira/browse/SPARK-25378 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-25378) ArrayData.toArray assume UTF8String

2018-09-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608092#comment-16608092 ] Xiangrui Meng commented on SPARK-25378: --- Seems caused by SPARK-23875. cc: [~viirya

[jira] [Resolved] (SPARK-25345) Deprecate public APIs from ImageSchema

2018-09-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25345. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22349 [https://

[jira] [Updated] (SPARK-25345) Deprecate readImages APIs from ImageSchema

2018-09-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25345: -- Summary: Deprecate readImages APIs from ImageSchema (was: Deprecate public APIs from ImageSch

[jira] [Assigned] (SPARK-25345) Deprecate public APIs from ImageSchema

2018-09-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25345: - Assignee: Weichen Xu > Deprecate public APIs from ImageSchema > ---

[jira] [Created] (SPARK-25382) Remove ImageSchema.readImages in 3.0

2018-09-08 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-25382: - Summary: Remove ImageSchema.readImages in 3.0 Key: SPARK-25382 URL: https://issues.apache.org/jira/browse/SPARK-25382 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-25345) Deprecate readImages APIs from ImageSchema

2018-09-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25345: -- Description: After SPARK-22666, we can deprecate the public APIs in ImageSchema (Scala/Python)

[jira] [Created] (SPARK-25383) Image data source supports sample pushdown

2018-09-08 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-25383: - Summary: Image data source supports sample pushdown Key: SPARK-25383 URL: https://issues.apache.org/jira/browse/SPARK-25383 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-25382) Remove ImageSchema.readImages in 3.0

2018-09-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25382: -- Description: A follow-up task from SPARK-25345. We might need to support sampling (SPARK-25383

[jira] [Commented] (SPARK-25378) ArrayData.toArray assume UTF8String

2018-09-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608165#comment-16608165 ] Xiangrui Meng commented on SPARK-25378: --- This is a breaking change anyway. The err

[jira] [Commented] (SPARK-25378) ArrayData.toArray assume UTF8String

2018-09-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608171#comment-16608171 ] Xiangrui Meng commented on SPARK-25378: --- Btw, thanks for suggesting the right way

[jira] [Commented] (SPARK-25378) ArrayData.toArray assume UTF8String

2018-09-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609467#comment-16609467 ] Xiangrui Meng commented on SPARK-25378: --- I sent a PR to spark-tensorflow-connector

[jira] [Updated] (SPARK-25378) ArrayData.toArray(StringType) assume UTF8String in 2.4

2018-09-10 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25378: -- Summary: ArrayData.toArray(StringType) assume UTF8String in 2.4 (was: ArrayData.toArray assum

[jira] [Commented] (SPARK-25378) ArrayData.toArray(StringType) assume UTF8String in 2.4

2018-09-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612399#comment-16612399 ] Xiangrui Meng commented on SPARK-25378: --- Comments from [~vomjom] at https://github

[jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

2018-09-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618495#comment-16618495 ] Xiangrui Meng commented on SPARK-25321: --- [~WeichenXu123] Could you check whether m

[jira] [Comment Edited] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

2018-09-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16618495#comment-16618495 ] Xiangrui Meng edited comment on SPARK-25321 at 9/18/18 5:21 AM: --

[jira] [Resolved] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25321. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22492 [https://

[jira] [Reopened] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-25321: --- > ML, Graph 2.4 QA: API: New Scala APIs, docs > --- > >

[jira] [Resolved] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25321. --- Resolution: Fixed Issue resolved by pull request 22510 [https://github.com/apache/spark/pull

[jira] [Commented] (SPARK-25320) ML, Graph 2.4 QA: API: Binary incompatible changes

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624115#comment-16624115 ] Xiangrui Meng commented on SPARK-25320: --- We revert the tree Node incompatible chan

[jira] [Assigned] (SPARK-25320) ML, Graph 2.4 QA: API: Binary incompatible changes

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25320: - Assignee: Weichen Xu (was: Bago Amirbekian) > ML, Graph 2.4 QA: API: Binary incompatib

[jira] [Commented] (SPARK-25320) ML, Graph 2.4 QA: API: Binary incompatible changes

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624116#comment-16624116 ] Xiangrui Meng commented on SPARK-25320: --- We will keep "predict()" change because i

[jira] [Resolved] (SPARK-25320) ML, Graph 2.4 QA: API: Binary incompatible changes

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25320. --- Resolution: Fixed > ML, Graph 2.4 QA: API: Binary incompatible changes > ---

[jira] [Assigned] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25321: - Assignee: Weichen Xu (was: Yanbo Liang) > ML, Graph 2.4 QA: API: New Scala APIs, docs

[jira] [Resolved] (SPARK-25324) ML 2.4 QA: API: Java compatibility, docs

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25324. --- Resolution: Done > ML 2.4 QA: API: Java compatibility, docs > --

[jira] [Commented] (SPARK-25324) ML 2.4 QA: API: Java compatibility, docs

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624122#comment-16624122 ] Xiangrui Meng commented on SPARK-25324: --- See discussion in https://issues.apache.o

[jira] [Updated] (SPARK-25323) ML 2.4 QA: API: Python API coverage

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25323: -- Priority: Critical (was: Blocker) > ML 2.4 QA: API: Python API coverage > ---

[jira] [Assigned] (SPARK-25323) ML 2.4 QA: API: Python API coverage

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25323: - Assignee: (was: Bryan Cutler) > ML 2.4 QA: API: Python API coverage > -

[jira] [Commented] (SPARK-25323) ML 2.4 QA: API: Python API coverage

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624123#comment-16624123 ] Xiangrui Meng commented on SPARK-25323: --- [~WeichenXu123] anyone is working on this

[jira] [Assigned] (SPARK-25322) ML, Graph 2.4 QA: API: Experimental, DeveloperApi, final, sealed audit

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25322: - Assignee: (was: Nick Pentreath) > ML, Graph 2.4 QA: API: Experimental, DeveloperApi

[jira] [Assigned] (SPARK-25327) Update MLlib, GraphX websites for 2.4

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25327: - Assignee: (was: Nick Pentreath) > Update MLlib, GraphX websites for 2.4 > -

[jira] [Resolved] (SPARK-25327) Update MLlib, GraphX websites for 2.4

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25327. --- Resolution: Won't Do > Update MLlib, GraphX websites for 2.4 > -

[jira] [Assigned] (SPARK-25325) ML, Graph 2.4 QA: Update user guide for new features & APIs

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25325: - Assignee: (was: Nick Pentreath) > ML, Graph 2.4 QA: Update user guide for new featu

[jira] [Assigned] (SPARK-25326) ML, Graph 2.4 QA: Programming guide update and migration guide

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25326: - Assignee: (was: Nick Pentreath) > ML, Graph 2.4 QA: Programming guide update and mi

[jira] [Commented] (SPARK-25319) Spark MLlib, GraphX 2.4 QA umbrella

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624124#comment-16624124 ] Xiangrui Meng commented on SPARK-25319: --- [~WeichenXu123] Could you check the OPEN

[jira] [Updated] (SPARK-25322) ML, Graph 2.4 QA: API: Experimental, DeveloperApi, final, sealed audit

2018-09-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25322: -- Priority: Critical (was: Blocker) > ML, Graph 2.4 QA: API: Experimental, DeveloperApi, final,

[jira] [Commented] (SPARK-25378) ArrayData.toArray(StringType) assume UTF8String in 2.4

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634335#comment-16634335 ] Xiangrui Meng commented on SPARK-25378: --- I don't think I'm the right person to dec

[jira] [Resolved] (SPARK-25524) Spark datasource for image/libsvm user guide

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25524. --- Resolution: Duplicate > Spark datasource for image/libsvm user guide > -

[jira] [Created] (SPARK-25584) Document libsvm data source in doc site

2018-10-01 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-25584: - Summary: Document libsvm data source in doc site Key: SPARK-25584 URL: https://issues.apache.org/jira/browse/SPARK-25584 Project: Spark Issue Type: Story

[jira] [Commented] (SPARK-25524) Spark datasource for image/libsvm user guide

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634347#comment-16634347 ] Xiangrui Meng commented on SPARK-25524: --- Marked as duplicate and create SPARK-2558

[jira] [Updated] (SPARK-25584) Document libsvm data source in doc site

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25584: -- Description: Currently, we only have Scala/Java API docs for libsvm data source. It would be n

[jira] [Updated] (SPARK-25584) Document libsvm data source in doc site

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25584: -- Component/s: ML > Document libsvm data source in doc site > --

[jira] [Updated] (SPARK-25347) Document image data source in doc site

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25347: -- Component/s: ML > Document image data source in doc site > ---

[jira] [Updated] (SPARK-25326) ML, Graph 2.4 QA: Programming guide update and migration guide

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25326: -- Priority: Major (was: Critical) > ML, Graph 2.4 QA: Programming guide update and migration gu

[jira] [Resolved] (SPARK-25325) ML, Graph 2.4 QA: Update user guide for new features & APIs

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25325. --- Resolution: Won't Do > ML, Graph 2.4 QA: Update user guide for new features & APIs > ---

[jira] [Updated] (SPARK-25323) ML 2.4 QA: API: Python API coverage

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25323: -- Priority: Major (was: Critical) > ML 2.4 QA: API: Python API coverage > -

[jira] [Resolved] (SPARK-25323) ML 2.4 QA: API: Python API coverage

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25323. --- Resolution: Won't Do > ML 2.4 QA: API: Python API coverage > ---

[jira] [Updated] (SPARK-25325) ML, Graph 2.4 QA: Update user guide for new features & APIs

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25325: -- Priority: Major (was: Critical) > ML, Graph 2.4 QA: Update user guide for new features & APIs

[jira] [Resolved] (SPARK-25326) ML, Graph 2.4 QA: Programming guide update and migration guide

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25326. --- Resolution: Won't Do > ML, Graph 2.4 QA: Programming guide update and migration guide >

[jira] [Resolved] (SPARK-25319) Spark MLlib, GraphX 2.4 QA umbrella

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25319. --- Resolution: Done Assignee: Weichen Xu (was: Joseph K. Bradley) > Spark MLlib, GraphX

<    1   2   3   4   5   6   7   8   9   10   >