Re: Size of Java serialized Clojure data structures

2015-08-08 Thread icamts
Hi Alan, I did the same experiment, with the same variable names too :) have a look at the github issue. Finally I discovered prevayler calls reset method on oos every time it writes to it. This is why the overhead is written every time. Klaus, prevayler author, says it is so to prevent leaks an

Re: Size of Java serialized Clojure data structures

2015-08-07 Thread Alan Malloy
You must be doing something wrong, or describing your method badly, because the vector [:a1 1] doesn't take nearly that much space in my experiments. The first object you write to a stream requires quite a bit of overhead, but after that future objects are relatively cheap. Here's an example you

Re: Size of Java serialized Clojure data structures

2015-08-07 Thread icamts
Yes. I suggested nippy. The question is about the size of Java serialized Clojure data structures. Can a two element vector be 1kB in size? Why serialization in my REPL experiment (see the code following the link in my previous mail) produces a 80MB byte buffer while prevayler logs are 1GB? Il

Re: Size of Java serialized Clojure data structures

2015-08-07 Thread Gary Verhaegen
You should probably look at Clojure-specific solutions, starting with EDN (i.e. essentially pr-str), fressian, transit or nippy. Clojure data structures have a lot of properties that can be exploited (we only care about the abstract type and the actual data), so a serializer can make a lot of assu