[spark] branch master updated: [SPARK-27454][ML][SQL] Spark image datasource fail when encounter some illegal images

[meng]

This is an automated email from the ASF dual-hosted git repository.

meng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new d35e81f  [SPARK-27454][ML][SQL] Spark image datasource fail when 
encounter some illegal images
d35e81f is described below

commit d35e81f4bc561598676a508319ec872f7361b069
Author: WeichenXu <weichen...@databricks.com>
AuthorDate: Mon Apr 15 11:55:51 2019 -0700

    [SPARK-27454][ML][SQL] Spark image datasource fail when encounter some 
illegal images
    
    ## What changes were proposed in this pull request?
    
    Fix in Spark image datasource fail when encounter some illegal images.
    
    This related to bugs inside `ImageIO.read` so in spark code I add exception 
handling for it.
    
    ## How was this patch tested?
    
    N/A
    
    Please review http://spark.apache.org/contributing.html before opening a 
pull request.
    
    Closes #24362 from WeichenXu123/fix_image_ds_bug.
    
    Authored-by: WeichenXu <weichen...@databricks.com>
    Signed-off-by: Xiangrui Meng <m...@databricks.com>
---
 mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala 
b/mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
index 0b13eef..a7ddf2f 100644
--- a/mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
+++ b/mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala
@@ -133,7 +133,13 @@ object ImageSchema {
    */
   private[spark] def decode(origin: String, bytes: Array[Byte]): Option[Row] = 
{
 
-    val img = ImageIO.read(new ByteArrayInputStream(bytes))
+    val img = try {
+      ImageIO.read(new ByteArrayInputStream(bytes))
+    } catch {
+      // Catch runtime exception because `ImageIO` may throw unexcepted 
`RuntimeException`.
+      // But do not catch the declared `IOException` (regarded as FileSystem 
failure)
+      case _: RuntimeException => null
+    }
 
     if (img == null) {
       None


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org