Hi Ryan, To answer the big question, all of these data formats are in active use at Cognitect and while I make no promises, I expect them all to be alive and active for the knowable future. Each of them targets a different niche but all of them share the qualities of transmitting extensible typed values.
edn is the best choice for human-readable data. It is however, less efficient to transmit and depends on writing a high-performance parser - this is a high bar in some language environments. edn is most attractive right now to Clojure users b/c of its proximity to Clojure itself. While it has many advantages as an extensible readable literal data format, it's an uphill battle to sell that against other data formats that already have greater mindshare and tooling in other language communities. fressian is the highest performance option - it takes full advantage of a number of compression tricks and has support for arbitrary user-extensible caching. Again, it requires a fair amount of effort to write a fressian library though so it's probably best for JVM-oriented endpoints right now. By seeking greatest performance, fressian also makes tradeoffs that narrow its range of use and interest group. transit is a pragmatic midpoint between these two. It focuses, like fressian, on program-to-program data transmission however, the data can be made readable (like edn) via the json-verbose mode. Like fressian, transit contains caching capabilities but they are more limited and not user-extensible. transit is designed primarily to have the most high-quality implementations per lowest effort - effectively shooting for greater reach than either edn or fressian by lower the bar to implementation. The bar is lowered by reusing high-performance parser for either JSON or messagepack which exist in a large number of languages. Of particular importance is leveraging the very high performance JSON parsers available in JavaScript runtimes, making transit viable as a browser-side endpoint for a fraction of the effort required to write a high performance edn or fressian endpoint. As transit explicitly seeks reach and portability, it is naturally the format with the broadest potential usage. Hopefully that lays out the landscape a bit more. With respect to support, all of these formats are supported by Cognitect. Like everyone else, we're managing priorities across a large number of projects. Filing issues is great, voting on issues is great, reports like this are great. External tooling is welcome (like pretty printers, etc). Because we use these projects inside internal Cognitect projects and products, we do not currently have a community contribution model for most of these as there may be impacts that are not publicly visible. That could change in the future. I am not an expert on everything you mention below but at a glance it looks like on-point feedback and I will raise it with the appropriate people and we can follow back here or on the relevant issues as needed. Thanks, Alex On Sunday, March 15, 2015 at 8:40:21 PM UTC-5, Ryan Schmitt wrote: > > I'm the author of dynamic-object > <https://github.com/rschmitt/dynamic-object>, an open source library that > makes Clojure's data modeling power available to Java programmers. This > includes features like serialization and deserialization. I'll copy this > small usage example from the README to give you a sense of how it works: > > public interface Album extends DynamicObject<Album> { > @Key(":artist") String getArtist(); > @Key(":album") String getAlbum(); > @Key(":tracks") int getTracks(); > @Key(":year") int getYear(); > } > > > String edn = "{:artist \"Meshuggah\", :album \"Chaosphere\", :tracks 8, :year > 1998}";Album album = DynamicObject.deserialize(edn, Album.class); > album.getArtist(); // => "Meshuggah" > > > dynamic-object has always been opinionated about using Edn as the primary > data language on the wire, for a number of reasons. For a long time, I also > thought about adding Fressian support to dynamic-object, and I've recently > done so on an experimental basis. (It looks like this > <https://github.com/rschmitt/dynamic-object/blob/440ecd2f385d1cec80496ba7007bd0755023b19c/src/test/java/com/github/rschmitt/dynamicobject/FressianTest.java>.) > > Some time after I initially released dynamic-object, Transit was also > released, with support for various encodings (JSON, JSON-Verbose, > MessagePack). > > In working (to different extents) with these data languages, I've had some > apprehensions about all of them. > > - There is a lack of tooling available for Edn, such as validators and > pretty-printers. I spent a while looking for an Edn equivalent of python > -mjson.tool and never found one. Clojure's built-in pprint function does > not work out-of-the-box to pretty print arbitrary values, and also appears > to handle some data structures, such as records, incorrectly. (pprint > omits > reader tags when printing records.) pprint's underlying implementation, > cl-format, is extremely powerful and could almost certainly be used to > build a validating Edn pretty-printer, but it would have an unacceptably > long startup time. > - There is a lack of high-quality Edn implementations for different > languages. Because the Edn spec is not very formal or complete, there > seems > to be some uncertainty regarding what constitutes an Edn implementation in > the first place. For instance, clojure.edn parses the Ratio type as a > builtin, even though it is mentioned nowhere in the spec. (Issue > <https://github.com/edn-format/edn/issues/64>.) There are also > oddities such as the recommended C++ implementation > <https://github.com/shaunxcode/edn-cpp> describing itself as > "experimental." > - Fressian's reference Java implementation is almost totally > undocumented. This is a problem, because I'm writing a library that > targets > Java developers; they won't be going through the Clojure bindings (which > are decently documented). Fressian's source code is outstanding, but it's > still not documentation. > - Due to the lack of documentation, it's not clear which parts of > Fressian are actually stable. Stuart Halloway's data.fressian talk > included > some parentheticals about the extension points being subject to change, > which so far they haven't, but that might only be because of the following > point... > - Fressian does not seem to have gotten any attention since the > initial launch. People have submitted GitHub issues > <https://github.com/Datomic/fressian/issues>, including one > surprisingly high-quality bug report, but they have all been ignored. The > JIRA is mostly tumbleweeds. > - The Clojure bindings for Fressian, namely data.fressian, are > essentially incomplete. With the exception of maps, Clojure collection > types do not round trip, and in at least one case (vectors) that is > because > of a blocking issue <http://dev.clojure.org/jira/browse/DFRS-1> in the > underlying Fressian implementation. > - There are no documented best practices for the use of Fressian or > some of its more advanced features like chunking. It is not clear how to > read and write Fressian in a way that facilitates (for instance) ranged > reads from the middle of a resource. It is not clear when checksums should > be used and how they should be validated. It is not clear whether tags > should be namespaced, or how. The only namespaced tag in data.fressian is > for IRecord; none of the other type tags are namespaced. It's not clear > whether this is due to bugwards compatibility. > - Transit is advertised > <https://github.com/cognitect/transit-format#implementations> as a > work-in-progress. This is the main reason I haven't seriously considered > adding Transit support to dynamic-object. > - However, what happens when Transit is stabilized (if that ever > happens)? Since Transit offers a msgpack encoding, will Fressian then be > irrelevant (except for legacy use cases)? There's a FUD aspect here--I > like > Fressian and I want dynamic-object to support it, but I don't want to back > the wrong pony and end up having to support HD-DVD and Betamax for all > time > (so to speak). > - Can these formats be unified? Can Edn and Fressian encodings for > Transit be offered? Would that even accomplish anything? > > I realize that none of these data languages will have the same extent of > support and tooling as JSON or XML, but I want to ensure that > dynamic-object's supported data languages all have attentive stewardship > and bright futures. It's distressing that a lot of the issues with Edn and > Fressian have not gotten much traction. Are these languages still actively > being supported and fostered? If so, how much development activity is > taking place on internal forks? Are any public updates planned for these > languages any time soon? > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
