scravy commented on a change in pull request #31565:
URL: https://github.com/apache/spark/pull/31565#discussion_r576713319
##########
File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
##########
@@ -1072,14 +1072,29 @@ object SparkSubmit extends CommandLineUtils with
Logging {
* Return whether the given primary resource requires running python.
*/
private[deploy] def isPython(res: String): Boolean = {
- res != null && res.endsWith(".py") || res == PYSPARK_SHELL
+ if (res == null) {
+ return false
+ }
+ if (res == PYSPARK_SHELL) {
+ return true
+ }
+ val uri = new java.net.URI(res)
Review comment:
@HyukjinKwon I am handling the possible URI syntax exception now and
also extracted the common code into a separate private function with a comment.
Let me know whether that is more what you had in mind.
Gonna provide a test case later on.
This issue should be labelled SPARK_SUBMIT I think? Or can I add the label
by changing the title? I noticed the square brackets allover the place.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
