[GitHub] spark pull request #20475: [SPARK-23256][ML][PYTHON] Add columnSchema method...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/20475


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20475: [SPARK-23256][ML][PYTHON] Add columnSchema method...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/20475#discussion_r165715470
  
--- Diff: python/pyspark/ml/image.py ---
@@ -75,6 +76,23 @@ def ocvTypes(self):
 self._ocvTypes = 
dict(ctx._jvm.org.apache.spark.ml.image.ImageSchema.javaOcvTypes())
 return self._ocvTypes
 
+@property
+def columnSchema(self):
+"""
+Returns the schema for the image column.
+
+:return: a :class:`StructType` for image column,
+``struct``.
+
+.. versionadded:: 2.3.0
--- End diff --

Yea, let me go with 2.4.0.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20475: [SPARK-23256][ML][PYTHON] Add columnSchema method...

[BryanCutler]

Github user BryanCutler commented on a diff in the pull request:

https://github.com/apache/spark/pull/20475#discussion_r165707676
  
--- Diff: python/pyspark/ml/image.py ---
@@ -75,6 +76,23 @@ def ocvTypes(self):
 self._ocvTypes = 
dict(ctx._jvm.org.apache.spark.ml.image.ImageSchema.javaOcvTypes())
 return self._ocvTypes
 
+@property
+def columnSchema(self):
+"""
+Returns the schema for the image column.
+
+:return: a :class:`StructType` for image column,
+``struct``.
+
+.. versionadded:: 2.3.0
--- End diff --

Since this came out of the 2.3 ml QA and it's mostly an improvement to the 
python API, I think maybe 2.4 is best. But it is a new API do maybe it's ok to 
include in 2.3..


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20475: [SPARK-23256][ML][PYTHON] Add columnSchema method...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/20475#discussion_r165369607
  
--- Diff: python/pyspark/ml/image.py ---
@@ -75,6 +76,23 @@ def ocvTypes(self):
 self._ocvTypes = 
dict(ctx._jvm.org.apache.spark.ml.image.ImageSchema.javaOcvTypes())
 return self._ocvTypes
 
+@property
+def columnSchema(self):
+"""
+Returns the schema for the image column.
+
+:return: a :class:`StructType` for image column,
+``struct``.
+
+.. versionadded:: 2.3.0
--- End diff --

I am fine with 2.4.0. Let me know.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20475: [SPARK-23256][ML][PYTHON] Add columnSchema method...

[HyukjinKwon]

GitHub user HyukjinKwon opened a pull request:

https://github.com/apache/spark/pull/20475

[SPARK-23256][ML][PYTHON] Add columnSchema method to PySpark image reader

## What changes were proposed in this pull request?

This PR proposes to add `columnSchema` in Python side too.

```python
>>> from pyspark.ml.image import ImageSchema
>>> ImageSchema.columnSchema.simpleString()

'struct'
```

## How was this patch tested?

Manually tested and unittest was added in `python/pyspark/ml/tests.py`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/HyukjinKwon/spark SPARK-23256

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20475.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20475


commit e180ade8a86c51f7eac1fd63e0febc09f9889f7d
Author: hyukjinkwon 
Date:   2018-02-01T14:14:40Z

Add columnSchema method to PySpark image reader




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org