Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/11621#discussion_r55621515
--- Diff: python/pyspark/mllib/common.py ---
@@ -130,20 +130,39 @@ def callMLlibFunc(name, *args):
return callJavaFunc(sc, api, *args)
-class JavaModelWrapper(object):
+class JavaCallable(object):
"""
- Wrapper for the model in JVM
+ Wrapper for an object in JVM to make Java calls
"""
- def __init__(self, java_model):
- self._sc = SparkContext.getOrCreate()
- self._java_model = java_model
+ def __init__(self, sc, java_obj):
+ self._sc = sc
+ self._java_obj = java_obj
def __del__(self):
- self._sc._gateway.detach(self._java_model)
+ self._sc._gateway.detach(self._java_obj)
+
+ @classmethod
+ def fromActiveSparkContext(cls, java_obj):
+ """Create from a currently active context"""
+ sc = SparkContext._active_spark_context
+ return cls(sc, java_obj)
def call(self, name, *a):
--- End diff --
this ends up in the api docs because ML has the `:inherited-members:` flag
on
[here](https://raw.githubusercontent.com/apache/spark/master/python/docs/pyspark.ml.rst)
Is there any way to avoid this besides making these private?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
