Hi Thushan, Instead, can we introduce a H2O Model object with standard serialization ?
On Wed, Jul 1, 2015 at 9:57 AM, Thushan Ganegedara <[email protected]> wrote: > Hi, > > I got the response from the H-2-O team, it seems that they are not > following the standard serialization procedure. (They are using an > internally generated serialization code). > > Specifically, I've to save all the keys generated by the model (not the > model itself) > > Therefore, I was thinking of including the keys in the MLModel rather than > than the model itself. However, I'm not sure whether that would work > > And then save the keys seperately using a specific H-2-O serializer in the > persist and retrieve methods in the MLModelHandler. > > > > > On Wed, Jul 1, 2015 at 10:35 AM, Thushan Ganegedara <[email protected]> > wrote: > >> Hi all, >> >> I was able to get H-2-O running from WSO2-ML UI. It can take a given file >> and train a deeplearning algorithm and produce the model. >> >> However, I get the following error after the training process. The error >> is due to the inability to deserialize the DeepLearningModel (H-2-O class) >> Though the serialization happens, deserialization produce the following >> error. >> >> I have already posted this on their google group and awaiting a response. >> >> [2015-07-01 09:48:14,490] ERROR >> {org.wso2.carbon.ml.core.impl.MLModelHandler} - >> Failed to build the model [id] 52 >> org.wso2.carbon.ml.core.exceptions.MLModelBuilderException: An error >> occurred wh >> ile building supervised machine learning model: An error occurred while >> building >> stacked autoencoders model: Job aborted due to stage failure: Task 0 in >> stage 1 >> .0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID >> 1, local >> host): java.io.InvalidClassException: hex.deeplearning.DeepLearning; no >> valid co >> nstructor >> at >> java.io.ObjectStreamClass$ExceptionInfo.newInvalidClassException(Obje >> ctStreamClass.java:150) >> at >> java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:768 >> ) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >> 775) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >> 93) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >> >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >> 801) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >> 93) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >> >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >> 801) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >> 93) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >> >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >> 801) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >> 93) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >> >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >> 801) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >> 93) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >> >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >> 801) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >> 93) >> at >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >> >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >> 801) >> at >> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >> at >> java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >> at >> org.apache.spark.serializer.JavaDeserializationStream.readObject(Java >> Serializer.scala:68) >> at >> org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSe >> rializer.scala:94) >> at >> org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:57) >> at org.apache.spark.scheduler.Task.run(Task.scala:64) >> at >> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203) >> >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. >> java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor >> .java:617) >> at java.lang.Thread.run(Thread.java:745) >> >> Driver stacktrace: >> at >> org.wso2.carbon.ml.core.spark.algorithms.DeeplearningModel.buildModel >> (DeeplearningModel.java:94) >> at >> org.wso2.carbon.ml.core.impl.MLModelHandler$ModelBuilder.run(MLModelH >> andler.java:588) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. >> java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor >> .java:617) >> at java.lang.Thread.run(Thread.java:745) >> >> On Tue, Jun 30, 2015 at 2:34 PM, Thushan Ganegedara <[email protected]> >> wrote: >> >>> Thank you very much >>> >>> It works now. >>> >>> On Tue, Jun 30, 2015 at 11:35 AM, Nirmal Fernando <[email protected]> >>> wrote: >>> >>>> Hi Thushan, >>>> >>>> Here you go; >>>> https://github.com/wso2/carbon-ml/commit/959c89e2fbfe04ec4c49290fd68ac08ae8718e6a >>>> >>>> context.getFacts().getDatasetURL() >>>> >>>> On Tue, Jun 30, 2015 at 5:12 AM, Thushan Ganegedara <[email protected]> >>>> wrote: >>>> >>>>> Hello all, >>>>> >>>>> It seems H-2-O takes data as a file (.csv format) But in wso2-ml data >>>>> is passed as JavaRDD<LabeledPoint> to train method. >>>>> >>>>> So clearly there's going to be an inconsistency between deeplearning >>>>> and other algorithms. >>>>> >>>>> I already posted the question in the h-2-o google group. They have >>>>> said it's theoritically possible to pass data to h-2-o in the code itself. >>>>> However, they haven't come out with a specific solution there. So I'm >>>>> waiting on that. >>>>> >>>>> Is there a way to directly access the file within the classifier (or >>>>> maybe pass the file to the classifier), if that option becomes the last >>>>> resort for this task? >>>>> >>>>> Thank you >>>>> >>>>> -- >>>>> Regards, >>>>> >>>>> Thushan Ganegedara >>>>> School of IT >>>>> University of Sydney, Australia >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks & regards, >>>> Nirmal >>>> >>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>> Mobile: +94715779733 >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>>> >>>> >>> >>> >>> -- >>> Regards, >>> >>> Thushan Ganegedara >>> School of IT >>> University of Sydney, Australia >>> >> >> >> >> -- >> Regards, >> >> Thushan Ganegedara >> School of IT >> University of Sydney, Australia >> > > > > -- > Regards, > > Thushan Ganegedara > School of IT > University of Sydney, Australia > -- Thanks & regards, Nirmal Associate Technical Lead - Data Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
