HyukjinKwon commented on pull request #34187: URL: https://github.com/apache/spark/pull/34187#issuecomment-935179555
If it fails to read for whatever reason, it will be stored in the invalid image in the row: https://github.com/apache/spark/blob/0494dc90af48ce7da0625485a4dc6917a244d580/mllib/src/main/scala/org/apache/spark/ml/source/image/ImageFileFormat.scala#L87 So, previously, it would fail just in the middle for `javax.imageio.IIOException`. After this change, the image (bytes) will be stored in the returned row for invalid image. The test: https://github.com/apache/spark/blob/master/python/pyspark/ml/tests/test_image.py#L27-L32 actually reads https://github.com/apache/spark/blob/master/data/mllib/images/origin/kittens/not-image.txt that is not an image file (so I think the code should be able to handle `javax.imageio.IIOException` anyway). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
