Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/23055#discussion_r237188935
--- Diff: python/pyspark/worker.py ---
@@ -22,7 +22,12 @@
import os
import sys
import time
-import resource
+# 'resource' is a Unix specific module.
+has_resource_module = True
+try:
+ import resource
+except ImportError:
+ has_resource_module = False
--- End diff --
I'm -1 too. Remove the check from the Java side. Let Python deal with
whether it can enforce resource limits or not. The JVM should just tell Python
what the limit should be. I don't understand why is that so controversial.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
