Re: Spark MLlib question: load model failed with exception:org.json4s.package$MappingException: Did not find value which can be converted into java.lang.String

2016-08-18 Thread Yanbo Liang 
It looks like you mixed use ALS in spark.ml and spark.mllib package.
You can train the model by either one, meanwhile, you should use the
corresponding save/load functions.
You can not train/save the model by spark.mllib ALS, and then use spark.ml
ALS to load the model. It will throw exceptions.

I saw you use Pipeline to train ALS under spark.ml package. Then you should
use PipelineModel.load to read the model and get corresponding stage in the
pipeline as the ALSModel.

We strongly recommend you to use the spark.ml package which is the primary
API of MLlib. The spark.mllib package is in maintenance mode. So do all
your work under the same APIs.

Thanks
Yanbo

2016-08-17 1:30 GMT-07:00 :

> Hello guys:
>  I have a problem in loading recommend model. I have 2 models, one is
> good(able to get recommend result) and another is not working. I checked
> these 2 models, both are  MatrixFactorizationModel object. But in the
> metadata, one is a PipelineModel and another is a MatrixFactorizationModel.
> Is below exception caused by this?
>
> here are my stack trace:
> Exception in thread "main" org.json4s.package$MappingException: Did not
> find value which can be converted into java.lang.String
> at org.json4s.reflect.package$.fail(package.scala:96)
> at org.json4s.Extraction$.convert(Extraction.scala:554)
> at org.json4s.Extraction$.extract(Extraction.scala:331)
> at org.json4s.Extraction$.extract(Extraction.scala:42)
> at org.json4s.ExtractableJsonAstNode.extract(
> ExtractableJsonAstNode.scala:21)
> at org.apache.spark.mllib.util.Loader$.loadMetadata(
> modelSaveLoad.scala:131)
> at org.apache.spark.mllib.recommendation.
> MatrixFactorizationModel$.load(MatrixFactorizationModel.scala:330)
> at org.brave.spark.ml.RecommandForMultiUsers$.main(
> RecommandForMultiUsers.scala:55)
> at org.brave.spark.ml.RecommandForMultiUsers.main(
> RecommandForMultiUsers.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$
> deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(
> SparkSubmit.scala:181)
> at org.apache.spark.deploy.SparkSubmit$.submit(
> SparkSubmit.scala:206)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.
> scala:121)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>
> The attached files are my codes, FYI.
> RecommandForMultiUsers.scala:55 is :
> val model = MatrixFactorizationModel.load(sc, modelpath)
>
>
> 
>
> ThanksBest regards!
> San.Luo
>
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>

Spark MLlib question: load model failed with exception:org.json4s.package$MappingException: Did not find value which can be converted into java.lang.String

2016-08-17 Thread luohui20001 
Hello guys: I have a problem in loading recommend model. I have 2 models, 
one is good(able to get recommend result) and another is not working. I checked 
these 2 models, both are  MatrixFactorizationModel object. But in the metadata, 
one is a PipelineModel and another is a MatrixFactorizationModel. Is below 
exception caused by this?
here are my stack trace:Exception in thread "main" 
org.json4s.package$MappingException: Did not find value which can be converted 
into java.lang.String
at org.json4s.reflect.package$.fail(package.scala:96)
at org.json4s.Extraction$.convert(Extraction.scala:554)
at org.json4s.Extraction$.extract(Extraction.scala:331)
at org.json4s.Extraction$.extract(Extraction.scala:42)
at 
org.json4s.ExtractableJsonAstNode.extract(ExtractableJsonAstNode.scala:21)
at 
org.apache.spark.mllib.util.Loader$.loadMetadata(modelSaveLoad.scala:131)
at 
org.apache.spark.mllib.recommendation.MatrixFactorizationModel$.load(MatrixFactorizationModel.scala:330)
at 
org.brave.spark.ml.RecommandForMultiUsers$.main(RecommandForMultiUsers.scala:55)
at 
org.brave.spark.ml.RecommandForMultiUsers.main(RecommandForMultiUsers.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at 
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

The attached files are my codes, FYI.
RecommandForMultiUsers.scala:55 is :val model = 
MatrixFactorizationModel.load(sc, modelpath)



 

ThanksBest regards!
San.Luo


codes to generate the bad model.scala
Description: Binary data


codes to generate the good model.scala
Description: Binary data


codes to load the model and generate the recommend result.scala
Description: Binary data

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org