Hello, I'm afraid that won't work. Because we will have to go deep into H-2-O and change their classes to use standard serialization. As I mentioned earlier, they are using a custom serializer for all serialization.
So even if we implement a standard H-2-O serializer, it won't work for the basic classes in H-2-O On Wed, Jul 1, 2015 at 2:31 PM, Nirmal Fernando <[email protected]> wrote: > Hi Thushan, > > Instead, can we introduce a H2O Model object with standard serialization ? > > On Wed, Jul 1, 2015 at 9:57 AM, Thushan Ganegedara <[email protected]> > wrote: > >> Hi, >> >> I got the response from the H-2-O team, it seems that they are not >> following the standard serialization procedure. (They are using an >> internally generated serialization code). >> >> Specifically, I've to save all the keys generated by the model (not the >> model itself) >> >> Therefore, I was thinking of including the keys in the MLModel rather >> than than the model itself. However, I'm not sure whether that would work >> >> And then save the keys seperately using a specific H-2-O serializer in >> the persist and retrieve methods in the MLModelHandler. >> >> >> >> >> On Wed, Jul 1, 2015 at 10:35 AM, Thushan Ganegedara <[email protected]> >> wrote: >> >>> Hi all, >>> >>> I was able to get H-2-O running from WSO2-ML UI. It can take a given >>> file and train a deeplearning algorithm and produce the model. >>> >>> However, I get the following error after the training process. The error >>> is due to the inability to deserialize the DeepLearningModel (H-2-O class) >>> Though the serialization happens, deserialization produce the following >>> error. >>> >>> I have already posted this on their google group and awaiting a response. >>> >>> [2015-07-01 09:48:14,490] ERROR >>> {org.wso2.carbon.ml.core.impl.MLModelHandler} - >>> Failed to build the model [id] 52 >>> org.wso2.carbon.ml.core.exceptions.MLModelBuilderException: An error >>> occurred wh >>> ile building supervised machine learning model: An error occurred while >>> building >>> stacked autoencoders model: Job aborted due to stage failure: Task 0 in >>> stage 1 >>> .0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID >>> 1, local >>> host): java.io.InvalidClassException: hex.deeplearning.DeepLearning; no >>> valid co >>> nstructor >>> at >>> java.io.ObjectStreamClass$ExceptionInfo.newInvalidClassException(Obje >>> ctStreamClass.java:150) >>> at >>> java.io.ObjectStreamClass.checkDeserialize(ObjectStreamClass.java:768 >>> ) >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >>> 775) >>> at >>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>> at >>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >>> 93) >>> at >>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >>> >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >>> 801) >>> at >>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>> at >>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >>> 93) >>> at >>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >>> >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >>> 801) >>> at >>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>> at >>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >>> 93) >>> at >>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >>> >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >>> 801) >>> at >>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>> at >>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >>> 93) >>> at >>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >>> >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >>> 801) >>> at >>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>> at >>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >>> 93) >>> at >>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >>> >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >>> 801) >>> at >>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>> at >>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:19 >>> 93) >>> at >>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) >>> >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1 >>> 801) >>> at >>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) >>> at >>> java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) >>> at >>> org.apache.spark.serializer.JavaDeserializationStream.readObject(Java >>> Serializer.scala:68) >>> at >>> org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSe >>> rializer.scala:94) >>> at >>> org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:57) >>> at org.apache.spark.scheduler.Task.run(Task.scala:64) >>> at >>> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203) >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. >>> java:1142) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor >>> .java:617) >>> at java.lang.Thread.run(Thread.java:745) >>> >>> Driver stacktrace: >>> at >>> org.wso2.carbon.ml.core.spark.algorithms.DeeplearningModel.buildModel >>> (DeeplearningModel.java:94) >>> at >>> org.wso2.carbon.ml.core.impl.MLModelHandler$ModelBuilder.run(MLModelH >>> andler.java:588) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. >>> java:1142) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor >>> .java:617) >>> at java.lang.Thread.run(Thread.java:745) >>> >>> On Tue, Jun 30, 2015 at 2:34 PM, Thushan Ganegedara <[email protected]> >>> wrote: >>> >>>> Thank you very much >>>> >>>> It works now. >>>> >>>> On Tue, Jun 30, 2015 at 11:35 AM, Nirmal Fernando <[email protected]> >>>> wrote: >>>> >>>>> Hi Thushan, >>>>> >>>>> Here you go; >>>>> https://github.com/wso2/carbon-ml/commit/959c89e2fbfe04ec4c49290fd68ac08ae8718e6a >>>>> >>>>> context.getFacts().getDatasetURL() >>>>> >>>>> On Tue, Jun 30, 2015 at 5:12 AM, Thushan Ganegedara <[email protected]> >>>>> wrote: >>>>> >>>>>> Hello all, >>>>>> >>>>>> It seems H-2-O takes data as a file (.csv format) But in wso2-ml data >>>>>> is passed as JavaRDD<LabeledPoint> to train method. >>>>>> >>>>>> So clearly there's going to be an inconsistency between deeplearning >>>>>> and other algorithms. >>>>>> >>>>>> I already posted the question in the h-2-o google group. They have >>>>>> said it's theoritically possible to pass data to h-2-o in the code >>>>>> itself. >>>>>> However, they haven't come out with a specific solution there. So I'm >>>>>> waiting on that. >>>>>> >>>>>> Is there a way to directly access the file within the classifier (or >>>>>> maybe pass the file to the classifier), if that option becomes the last >>>>>> resort for this task? >>>>>> >>>>>> Thank you >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> >>>>>> Thushan Ganegedara >>>>>> School of IT >>>>>> University of Sydney, Australia >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & regards, >>>>> Nirmal >>>>> >>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>> Mobile: +94715779733 >>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> >>>> Thushan Ganegedara >>>> School of IT >>>> University of Sydney, Australia >>>> >>> >>> >>> >>> -- >>> Regards, >>> >>> Thushan Ganegedara >>> School of IT >>> University of Sydney, Australia >>> >> >> >> >> -- >> Regards, >> >> Thushan Ganegedara >> School of IT >> University of Sydney, Australia >> > > > > -- > > Thanks & regards, > Nirmal > > Associate Technical Lead - Data Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > > -- Regards, Thushan Ganegedara School of IT University of Sydney, Australia
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
