Hey Kyle, thanks for the Feedback. Appreciate it.
I think you misunderstood the meaning of a "type" in shadow-psql. A "type" is merely the format of how a given value is represented on the wire since the backend needs to understand what we send it. Postgres supports 2 different Wire Formats: Text and Binary. While Text is considered the "default", binary is usually a lot more efficient. pgjdbc for example only supports the text format. I try to be binary first, which works for most types so far. (Numeric is giving me trouble, but I'll eventually figure that out). I allow overwriting the "types" cause not everything I store in postgres is understood by it (EDN, Keywords, ...). By hooking directly into the encode/decode code I can efficiently do the transformation on-the-fly. In my completely unscientific preliminary benchmark I cut the query time from pgjdbc 650ms to shadow-pgsql 200ms and that is for very simple types (50k rows) with no optimizations done yet. I expect the difference to be much larger if you use a timestamp, timestamptz or bytea for example, as the text format for those types carries a bit more overhead. But once everything is stable I will do some real benchmarks. Better performance was not the reason I wrote this, just a pleasant side-effect. As for the amount of work: its pretty much done. Some more exotic features need to be implemented, but those were never available via JDBC anyways (eg. COPY). I think its stable enough that I will begin moving my projects "soon", when I release everything to production I'll probably release a 1.0.0-RC. shadow-pgsql can not be layered on top of JDBC, well technically thats what I did for the last 2 years (https://gist.github.com/thheller/de7ecce0db58130ae6b7) BUT it required some ugly reflection calls since the PGJDBC does not expose all the information I needed. In the end I decided that I'd feel better to rewrite everything from scratch as the documentation is quite good and the protocol is simple. Since the non-Java world does just fine without JDBC, I think we can do too. ;) Also, the "Illusion" JDBC provides that you can just switch databases if you feel like it only holds until you start using postgres-specific features. Not all databases have arrays, hstore or json types. Regards, /thomas On Saturday, August 23, 2014 5:12:30 PM UTC+2, Kyle Cordes wrote: > > On Thursday, August 21, 2014 at 1:00 PM, Thomas Heller wrote: > > Hey Clojure Folk, > > > > I'm close to releasing the first alpha version of > https://github.com/thheller/shadow-pgsql a "native" interface to > PostgreSQL I wrote. > > > > Its an implementation of the native binary protocol without any intent > to ever support JDBC. Mostly because that provides a bunch of features I > never use, but no support for features I wanted. It is mostly Java but I > will probably only use it from Clojure so that is my primary goal going > forward. I think the Java bits are close to stable. > > > > I'm looking for interested beta testers and feedback. I'm bad at writing > docs cause I never know where to start since there are so many features and > differences to JDBC. > > > > > As a user of both Postgres and Clojure, I find this very interesting. It’s > helps with a couple of pain points around JDBC, such the fact that any > nonstandard feature ends up hidden behind a untyped interface passing > strings around. But I also have a couple of bits of feedback that are a > little more skeptical: > > First, the amount of work it will take to get this to a complete enough > state that large projects could safely switch to it, could be substantial. > It makes me wonder if, instead, this could be built as a layer up on top of > the Postgres JDBC driver. This would not be as elegant because it would not > strip out as much unnecessary code, but it may be quite a lot less work. > > Second, it seems to most effectively target people who are both very type > oriented, yet are using Java or Clojure. It seems to me that folks who are > so concerned with types that they would step away from the standard way of > talking to databases generically, might be found over in the community of > people using more rigidly typed languages like Haskell etc. > > Third, although I like the idea of leveraging the features of the tool you > are using (like Postgres), at the same time experiences taught me that, the > more firmly a project seems destined to never switch to a different brand > of database, the more likely some future unexpected opportunity will come > up where that is exactly what is needed. I suppose this is just Murphy’s > Law. > > I don’t want to sound discouraging though, I really like this idea. > > -- > Kyle Cordes > http://kylecordes.com > > > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
