Peter, Cool let us know if you pursue this and have any numbers to share. It may be the tipping point for going for binary formats like Fressian/MessagePack over JSON representations.
David On Mon, Aug 4, 2014 at 10:26 AM, Peter Schuck <[email protected]> wrote: > That would definitely help, a large percentage of the conversion time is > manually converting strings to utf-8 bytes and back again. Thanks for > pointing that out. > > Peter Schuck > > > On Sun, Aug 3, 2014 at 9:14 AM, David Nolen <[email protected]> wrote: >> >> RE: writing/reading strings I wonder if this won't help >> https://plus.google.com/+AddyOsmani/posts/4GgX9CD6c1X? >> >> David >> >> On Sun, Aug 3, 2014 at 9:51 AM, Peter Schuck <[email protected]> wrote: >> > Fressian's caching strategy is very flexible and powerful but doesn't >> > have a >> > lot of documetnation. I view Fressian's caching as similar to memory >> > references for Clojure values. You can cache any value no matter how >> > large >> > and once the value is cached referencing it again is only a few bytes. >> > Fressian caches symbols, keywords, and tagged value tags by default. >> > >> > What I mean by "significant structural similarity" is a data structure >> > or >> > steam of data structures that have frequent repetitions of compound >> > values. >> > An AST would probably be a canonical example of this. >> > >> > Fressian is about five to ten times as slow as Transit and not really >> > suited >> > for the vast majority of applications where relatively small data >> > structures >> > need to be written and read as quickly as possible. It took significant >> > effort to get Fressian's speed to where it is now and I don't see a way >> > forward to get it to match Transit's speed. The main bottlenecks are >> > reading and writing numbers and strings, hashing value, and determining >> > an >> > value's type. Until those bottlenecks get resolved in future versions >> > of >> > JavaScript piggybacking of of JSON, like Transit, is the way to go for >> > speed. >> > >> > I hope that answers most questions, feel free to ask for more >> > clarification. >> > I learned a lot about how to write fast ClojureScript / JavaScript and >> > plan >> > on writing a blog post about it in the near future. >> > >> > Peter Schuck >> > >> > >> > On Sat, Aug 2, 2014 at 4:23 PM, Alex Miller <[email protected]> wrote: >> >> >> >> I took "significant structural similarity" to primarily mean at least >> >> maps >> >> with similar keys, which Transit caching will cover. >> >> >> >> >> >> On Sat, Aug 2, 2014 at 1:49 PM, David Nolen <[email protected]> >> >> wrote: >> >>> >> >>> In Transit, maps keys, symbols, keywords, and tagged value tags are >> >>> subject to caching. >> >>> >> >>> Fressian's caching strategy is far more flexible from what I >> >>> understand. That said transit-cljs is 20-30X faster than >> >>> cljs.reader/read-string on the benchmarks I've tried across various >> >>> browser and command line JS environments. >> >>> >> >>> David >> >>> >> >>> On Sat, Aug 2, 2014 at 2:26 PM, Sean Grove <[email protected]> wrote: >> >>> > I thought transit's caching only applied to map keys? Pretty unclear >> >>> > on >> >>> > what >> >>> > Fressian's can do >> >>> > >> >>> > >> >>> > On Sat, Aug 2, 2014 at 8:23 AM, Alex Miller <[email protected]> >> >>> > wrote: >> >>> >> >> >>> >> Cool stuff Peter. It would be interesting to compare performance >> >>> >> with >> >>> >> transit-cljs https://github.com/cognitect/transit-cljs. Transit has >> >>> >> the same >> >>> >> caching and extensibility benefits of Fressian but leverages the >> >>> >> very >> >>> >> fast >> >>> >> JavaScript parser capabilities built into the browser, so is likely >> >>> >> faster. >> >>> >> >> >>> >> Alex >> >>> >> >> >>> >> On Friday, August 1, 2014 3:33:58 PM UTC-5, [email protected] >> >>> >> wrote: >> >>> >> > I'm happy to announce a port of Fressian to ClojureScript. >> >>> >> > >> >>> >> > The public api mirrors data.fressian api with a few exceptions. >> >>> >> > Records >> >>> >> > don't have a generic writer but you can easily append the >> >>> >> > handlers >> >>> >> > with the >> >>> >> > write-record function. There is no inheritance lookup for types >> >>> >> > so >> >>> >> > every >> >>> >> > different type will need it's own handler. The tagged helper >> >>> >> > functions >> >>> >> > (tagged-object?, tag, tagged-value) are not included. >> >>> >> > >> >>> >> > I see the use case for Fressian in ClojureScript streaming large >> >>> >> > amounts >> >>> >> > of data that has significant structural similarity. Fressian >> >>> >> > caching >> >>> >> > capabilities allow large values to be represented as a single >> >>> >> > integer in the >> >>> >> > bytestream. >> >>> >> > >> >>> >> > This is my first large ClojureScript library so any suggestions >> >>> >> > for >> >>> >> > improvements are welcome. >> >>> >> > >> >>> >> > Thanks to Bendyworks for letting me develop this during my work >> >>> >> > hours. >> >>> >> > >> >>> >> > Clojars Link: https://clojars.org/longshi >> >>> >> > Repo: https://github.com/spinningtopsofdoom/longshi >> >>> >> > >> >>> >> > Peter Schuck >> >>> >> >> >>> >> -- >> >>> >> Note that posts from new members are moderated - please be patient >> >>> >> with >> >>> >> your first post. >> >>> >> --- >> >>> >> You received this message because you are subscribed to the Google >> >>> >> Groups >> >>> >> "ClojureScript" group. >> >>> >> To unsubscribe from this group and stop receiving emails from it, >> >>> >> send >> >>> >> an >> >>> >> email to [email protected]. >> >>> >> To post to this group, send email to >> >>> >> [email protected]. >> >>> >> Visit this group at http://groups.google.com/group/clojurescript. >> >>> > >> >>> > >> >>> > -- >> >>> > Note that posts from new members are moderated - please be patient >> >>> > with >> >>> > your >> >>> > first post. >> >>> > --- >> >>> > You received this message because you are subscribed to the Google >> >>> > Groups >> >>> > "ClojureScript" group. >> >>> > To unsubscribe from this group and stop receiving emails from it, >> >>> > send >> >>> > an >> >>> > email to [email protected]. >> >>> > To post to this group, send email to [email protected]. >> >>> > Visit this group at http://groups.google.com/group/clojurescript. >> >>> >> >>> -- >> >>> Note that posts from new members are moderated - please be patient >> >>> with >> >>> your first post. >> >>> --- >> >>> You received this message because you are subscribed to a topic in the >> >>> Google Groups "ClojureScript" group. >> >>> To unsubscribe from this topic, visit >> >>> >> >>> https://groups.google.com/d/topic/clojurescript/xhdrGunEXPE/unsubscribe. >> >>> To unsubscribe from this group and all its topics, send an email to >> >>> [email protected]. >> >>> To post to this group, send email to [email protected]. >> >>> Visit this group at http://groups.google.com/group/clojurescript. >> >> >> >> >> >> -- >> >> Note that posts from new members are moderated - please be patient with >> >> your first post. >> >> --- >> >> You received this message because you are subscribed to a topic in the >> >> Google Groups "ClojureScript" group. >> >> To unsubscribe from this topic, visit >> >> >> >> https://groups.google.com/d/topic/clojurescript/xhdrGunEXPE/unsubscribe. >> >> To unsubscribe from this group and all its topics, send an email to >> >> [email protected]. >> >> To post to this group, send email to [email protected]. >> >> Visit this group at http://groups.google.com/group/clojurescript. >> > >> > >> > -- >> > Note that posts from new members are moderated - please be patient with >> > your >> > first post. >> > --- >> > You received this message because you are subscribed to the Google >> > Groups >> > "ClojureScript" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to [email protected]. >> > To post to this group, send email to [email protected]. >> > Visit this group at http://groups.google.com/group/clojurescript. >> >> -- >> Note that posts from new members are moderated - please be patient with >> your first post. >> --- >> You received this message because you are subscribed to a topic in the >> Google Groups "ClojureScript" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/clojurescript/xhdrGunEXPE/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/clojurescript. > > > -- > Note that posts from new members are moderated - please be patient with your > first post. > --- > You received this message because you are subscribed to the Google Groups > "ClojureScript" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/clojurescript. -- Note that posts from new members are moderated - please be patient with your first post. --- You received this message because you are subscribed to the Google Groups "ClojureScript" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/clojurescript.
