Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23055#discussion_r237713255
--- Diff: python/pyspark/worker.py ---
@@ -22,7 +22,12 @@
import os
import sys
import time
-import resource
+# 'resource' is a Unix specific module.
+has_resource_module = True
+try:
+ import resource
+except ImportError:
+ has_resource_module = False
--- End diff --
-1 for adding only Python check from my side. I don't understand why it's
controversial as well. Python can't fail because `resource` only exists in unix
based systems. We only have one exception case Windows which we know in JVM
side.
Configuration control should better be done in JVM side if possible and I
don't think we should allow this feature work differently on Windows and this
should be tested before we document we support it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
