Dhruv Sorry I wrote your name wrong! I forgot to explain the classes: KryoUtils is reads and writes Vectors and its features from a buffer of bytes. I still don't know if there is an issue with buffer limits. The second class VectorSerializer is a CustomSerializer child that is called whenever you try to read/write a Mahout Vector.
So that's it. Best regards. Gustavo On Mon, May 30, 2011 at 11:36 PM, Gustavo Enrique Salazar Torres < [email protected]> wrote: > Hi Druv > > I have created issue MAHOUT-715 and attached 2 classes and POM > dependencies. > I hope it helps. > > Best regards > Gustavo > > > On Mon, May 30, 2011 at 7:54 PM, Ted Dunning <[email protected]>wrote: > >> No serialization framework that I know of does checksums as a matter of >> course. >> >> All large scale computing frameworks that I know of do them. The best >> ones >> do the checksum at the client file system layer before even sending >> requests >> to the server. >> >> Bit errors definitely exist, but I saw more errors on disk 10 years ago >> than >> have lately even with the dramatic increases in size. The failure rate >> for >> drives is somewhat less, but still about the same but the transient error >> rate is down dramatically on a per byte stored basis. >> >> >> On Mon, May 30, 2011 at 2:59 PM, Lance Norskog <[email protected]> wrote: >> >> > Do any of these encapsulators do checksums? >> > >> > In light of the recent Google paper about rampant bit errors, and >> > personal experience with high-end disk arrays, it is clear to me that >> > that all stored data should have checksums. File systems, app-level >> > encapsulations, the works. (TCP/IP has checksums, but they are quite >> > weak.) >> > >> > On Mon, May 30, 2011 at 12:02 PM, Ted Dunning <[email protected]> >> > wrote: >> > > Kryo is java specific. >> > > >> > > I would really like to use a portable solution. >> > > >> > > On Mon, May 30, 2011 at 6:02 AM, Grant Ingersoll <[email protected] >> > >wrote: >> > > >> > >> Seems to me like kyro and protostuff (not protobuff) are the big >> winners >> > in >> > >> that one. Avro's competitive on size, but not time. Granted, one >> > should >> > >> never read too much into benchmarks, but... >> > >> >> > > >> > >> > >> > >> > -- >> > Lance Norskog >> > [email protected] >> > >> > > > >
