Github user catmonkeylee commented on the pull request:
https://github.com/apache/spark/pull/4854#issuecomment-84222343
I don't know how to create a new issue, So I 'm sorry to send this mail to
you. Please Help me.
I run the code on a spark cluster , spark version is 1.3.0
The test code:
===================================
from pyspark import SparkContext, SparkConf
from pyspark.mllib.tree import RandomForest, RandomForestModel
from pyspark.mllib.util import MLUtils
conf = SparkConf().setAppName('LocalTest')
sc = SparkContext(conf=conf)
data = MLUtils.loadLibSVMFile(sc, 'data/mllib/sample_libsvm_data.txt')
print data.count()
(trainingData, testData) = data.randomSplit([0.7, 0.3])
model = RandomForest.trainClassifier(trainingData, numClasses=2,
categoricalFeaturesInfo={},
numTrees=3,
featureSubsetStrategy="auto",
impurity='gini', maxDepth=4,
maxBins=32)
# Evaluate model on test instances and compute test error
predictions = model.predict(testData.map(lambda x: x.features))
labelsAndPredictions = testData.map(lambda lp: lp.label).zip(predictions)
testErr = labelsAndPredictions.filter(lambda (v, p): v != p).count() /
float(testData.count())
print('Test Error = ' + str(testErr))
print('Learned classification forest model:')
print(model.toDebugString())
# Save and load model
_model_path = "/home/s/apps/spark-app/data/myModelPath"
model.save(sc, _model_path)
sameModel = RandomForestModel.load(sc, _model_path)
sc.stop()
===================
run command:
spark-submit --master spark://t0.q.net:7077 --executor-memory 1G
sample_rf.py
======================
Then I get this error :
Traceback (most recent call last):
File "/data1/s/apps/spark-app/app/sample_rf.py", line 25, in <module>
sameModel = RandomForestModel.load(sc, _model_path)
File "/home/s/apps/spark/python/pyspark/mllib/util.py", line 254, in load
java_model = cls._load_java(sc, path)
File "/home/s/apps/spark/python/pyspark/mllib/util.py", line 250, in
_load_java
return java_obj.load(sc._jsc.sc(), path)
File
"/home/s/apps/spark/python/lib/py4j-0.8.2.1-src.zip/py4j/java_gateway.py", line
538, in __call__
File
"/home/s/apps/spark/python/lib/py4j-0.8.2.1-src.zip/py4j/protocol.py", line
300, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling
z:org.apache.spark.mllib.tree.model.RandomForestModel.load.
: java.lang.UnsupportedOperationException: empty collection
at org.apache.spark.rdd.RDD.first(RDD.scala:1191)
at org.apache.spark.mllib.util.Loader$.loadMetadata(modelSaveLoad.scala:125)
at
org.apache.spark.mllib.tree.model.RandomForestModel$.load(treeEnsembleModels.scala:65)
at
org.apache.spark.mllib.tree.model.RandomForestModel.load(treeEnsembleModels.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
at py4j.Gateway.invoke(Gateway.java:259)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:207)
at java.lang.Thread.run(Thread.java:724)
åèªæç iPhone
> å¨ 2015å¹´3æ21æ¥ï¼06:45ï¼jkbradley <[email protected]>
åéï¼
>
> Can you please open up a JIRA to report the bug? Also, what is the app
you were running, and could you provide the code which created and saved the
model originally?
>
> â
> Reply to this email directly or view it on GitHub.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
