Re: Using a dynamic var for my database connection for implicit connections+transactions
On Monday, August 3, 2015 at 5:19:34 AM UTC-5, Colin Yates wrote: I have heard this approach before, but I have never seen how it works in real life. Interesting. I don't think I've ever worked anywhere that didn't take this kind of approach. Even if we weren't doing it formally. For example, what about 'selects' - where do they happen? They're just part of the overall transaction description. It's [thankfully] been a long time since I worked on a project that worked in raw strings to build SQL. Even working at that level, the SELECT pieces were just a subquery in the main query we were building. The web server(s) would pre-process the incoming request, hand that to the middle-tier layers that did this sort of business logic, those would pass the data description (in this case raw SQL, in others something like XML-RPC) off to the servers that actually have pools of database connections. Even in companies that only needed one virtual host for their tiny little web server and couldn't be bothered to break it into multiple processes, we split this out into multiple logical layers. What about if my updates are not independent (e.g. if the first update works then do the second update otherwise do this completely different thing?). We're working in lisp. This sort of thing is far easier than it is for the poor schmucks who are doing exactly this sort of thing in, say, C# with LINQ or building/parsing/translating XML using python. For simple workflows I can see the elegance. For non-trivial workflows the problem is that _accessing_ the DB and _switching_ on the DB logic tends to be all through the workflow. I'll argue that this sort of isolation gets more important when your system gets more complex. You don't build a modern automatic transmission with sloppier tolerances that were used for a manual 50 years ago. You spend more time engineering the Eiffel Tower than you would a back-yard shed. You don't skimp on O-rings when you're building a space shuttle. I am genuinely interested in the answers to this as yes, the described approach has some great benefits. Let's see. Specific example... This comes from a startup that was bought for IP around 2001. I think it's safe to describe what I remember of their general architecture, since it seems to be extinct. Big picture was warehouses where workers are loading up pallets of goods to be shipped to retail stores. End users wore computers that had headsets and mics. Input was by speech recognition, output through text-to-speech. The wearables didn't have enough horse-power to actually do the speech recognition, so compressed .wav files were transmitted to the server over wifi. We probably should have been using a web interface, but most web apps were written in perl. This was a serious technology shop using sC++/sMFC. We had a ton of different modules/classes that covered different possibilities for any given scenario at various customer sites. They got initialized and wired together by rules defined in an .ini file. (That should have come from a database, but that's a different topic). Most of our database interaction happened when a worker logged in: we cached the various phrases that would be used throughout her shift. And we'd select a list of work. As each work item was completed, the front-end code would notify our database layer. I don't have any idea how that notification worked. It went through some library routine that had a database connection buried far away from any front-end code. Our database was really just a cache for interaction phrases in various languages and the work queues. Periodically, some job would upload our work status reports and download new pending work queues to/from the real database. IIRC, this interaction happened over FTP. We absolutely did not have (or want!) handles to work directly with whatever ancient server that was their authoritative source of all knowledge. Our portion of the system was ~70-80 KLOC. Since it was C++ (and most of that was boilerplate), it was *far* simpler (in terms of business logic) than the 200 KLOC of clojure that we're talking about. Although the big iron side may have pushed it into that realm. You obviously aren't going to be able to predict every possible interaction in the next few minutes. Much less the next 4 hours. But you should be able to predict most of what might happen for any given web request. Life changes when you're working on a highly interactive app like, say, an IDE or a photo editor. At my job today we have issues with multiple users adding and removing the same artifact to a document at something resembling the same time. That problem has a lot to do with the fact that it's a distributed system with at least (I haven't dug into the tier beneath the web server) 4 layers between the end-users and the actual database connections. I'm not sure it would be possible to get rid of more than one of those
Anotating functions for pre-processing
Hi,
I have had good 6 months of fun with Clojure and have big appreciation for
it's way of doing things. Coming from the Java/Spring world however, I
still have this nagging desire to be able to annotate functions and have
some preprocessor pick up these annotations and decorate the code
accordingly. Let me illustrate it:
In Java + Spring
@Transactional
public void someFunction(){...}
the Spring core container has excellent support for preprocessors to
instrument this function with some advice.
-
I wish I could do that in Clojure:
(defn ^:transactional someFunction [...] ...)
and then have somehow means to decorate someFunction (yes, I am aware there
is no container)
I have read some blog posts (about dependency injection in the context of
testing clojure) that discuss *alter-var-root,* but that looks like very
brutal approach.
What would be the advice on that? I am even happy to go with solution that
involves some micro-container spring-like approach.
Cheers
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Using a dynamic var for my database connection for implicit connections+transactions
On Monday, August 3, 2015 at 10:21:09 PM UTC-5, Dmitri wrote: My understanding is that the problem is actually caused by the stateless nature of the functions. You're talking about database interactions. By definition, those are not stateless. The trick is to isolate this statefulness as much as possible. Expanding it just compounds the problem. Since the function accepts the connection as a parameter it's up to the user of the function to ensure that it's passed the correct connection. That depends on what you mean by user of the function. If I'm working on the web server front-end, I shouldn't have any idea about the database connection. I should describe what I want to happen. If I have any concept that multiple databases are involved (I shouldn't, but abstractions leak), then, yes, I have to specify which one I mean. This isn't complicated. It's up to the person writing the database back-end code (also possibly me, if we're talking about a start-up, but then we aren't talking about the 200 KLOC scenario) to turn that description into the side-effects. Every functional solution presented in this thread suffers from this same fundamental problem that the function is not aware of the context it's being run in. That isn't a problem: it's a feature. It doesn't matter what language or programming paradigm you're using in this context. This cuts across problem domains and architectures. The actual database interaction code should be as brain-dead simple (not easy) and bullet-proof as you can possibly make it. And it should be as isolated from the front-end code as you can get away with making it. That isn't a right answer. But it's a good rule of thumb. And you should have very hefty reservations (and very good reasons) about violating it. Respectfully, James -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Metadata
The page http://clojure.org/metadata states that it can also be used by application developers for many purposes, annotating data sources, policy etc. Could anyone elaborate about some real use case scenarios where they need to use this metadata feature ? Regards, -- Eric Le Goff http://fr.linkedin.com/in/elegoff @elegoff https://twitter.com/elegoff -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Using a dynamic var for my database connection for implicit connections+transactions
I think we are talking at cross purposes - I thought you were talking about delaying the mutation until the 'main' logic has finished, whereby the main logic would push the mutation into a queue which were then executed later. Something like queueing up a bunch of commands to be executed later. If you are simply talking about isolating/abstracting database logic (but not the _timeliness_ of when that logic is executed) from the mainline code. The Repository pattern, DataAccess pattern then yes, of course, I think we are all on the same page. The original point was do you/don't you pass in the DB as an explicit arg. I still haven't seen a recommendation for handling protocols whose implementation is persistence agnostic. I also think you might be making a few unwarranted assumptions ;-) - who mentioned building up SQL in raw strings?! for example. James Gatannah writes: On Monday, August 3, 2015 at 5:19:34 AM UTC-5, Colin Yates wrote: I have heard this approach before, but I have never seen how it works in real life. Interesting. I don't think I've ever worked anywhere that didn't take this kind of approach. Even if we weren't doing it formally. For example, what about 'selects' - where do they happen? They're just part of the overall transaction description. It's [thankfully] been a long time since I worked on a project that worked in raw strings to build SQL. Even working at that level, the SELECT pieces were just a subquery in the main query we were building. The web server(s) would pre-process the incoming request, hand that to the middle-tier layers that did this sort of business logic, those would pass the data description (in this case raw SQL, in others something like XML-RPC) off to the servers that actually have pools of database connections. Even in companies that only needed one virtual host for their tiny little web server and couldn't be bothered to break it into multiple processes, we split this out into multiple logical layers. What about if my updates are not independent (e.g. if the first update works then do the second update otherwise do this completely different thing?). We're working in lisp. This sort of thing is far easier than it is for the poor schmucks who are doing exactly this sort of thing in, say, C# with LINQ or building/parsing/translating XML using python. For simple workflows I can see the elegance. For non-trivial workflows the problem is that _accessing_ the DB and _switching_ on the DB logic tends to be all through the workflow. I'll argue that this sort of isolation gets more important when your system gets more complex. You don't build a modern automatic transmission with sloppier tolerances that were used for a manual 50 years ago. You spend more time engineering the Eiffel Tower than you would a back-yard shed. You don't skimp on O-rings when you're building a space shuttle. I am genuinely interested in the answers to this as yes, the described approach has some great benefits. Let's see. Specific example... This comes from a startup that was bought for IP around 2001. I think it's safe to describe what I remember of their general architecture, since it seems to be extinct. Big picture was warehouses where workers are loading up pallets of goods to be shipped to retail stores. End users wore computers that had headsets and mics. Input was by speech recognition, output through text-to-speech. The wearables didn't have enough horse-power to actually do the speech recognition, so compressed .wav files were transmitted to the server over wifi. We probably should have been using a web interface, but most web apps were written in perl. This was a serious technology shop using sC++/sMFC. We had a ton of different modules/classes that covered different possibilities for any given scenario at various customer sites. They got initialized and wired together by rules defined in an .ini file. (That should have come from a database, but that's a different topic). Most of our database interaction happened when a worker logged in: we cached the various phrases that would be used throughout her shift. And we'd select a list of work. As each work item was completed, the front-end code would notify our database layer. I don't have any idea how that notification worked. It went through some library routine that had a database connection buried far away from any front-end code. Our database was really just a cache for interaction phrases in various languages and the work queues. Periodically, some job would upload our work status reports and download new pending work queues to/from the real database. IIRC, this interaction happened over FTP. We absolutely did not have (or want!) handles to work directly with whatever ancient server that was their authoritative source of all knowledge. Our portion of the system was ~70-80 KLOC. Since it was C++ (and most of that was
Re: Using a dynamic var for my database connection for implicit connections+transactions
What I'm talking about is whether it's a better pattern to leave a repetitive and error prone task to the user or encapsulate it in a single place. The whole discussion boils down to following options. The first option is that we keep functions pure and the connection is passed as a parameter by the person writing the code (the user). With this approach the burden is squarely on the user to remember to pass the correct connection to the function. This becomes error prone for things like transactions where the developer has to pass the transaction connection as opposed to the normal database connection. The worst part in this scenario is that the code will run except it won't run transactionally. This is a bug that is difficult to catch as it only surfaces in cases where the transaction should be rolled back. The alternative is to encapsulate the database connection management in the initialization logic in the namespace managing the connection. This way the query functions can be context aware and ensure that the correct connection is used automatically. I simply disagree that leaving repetitive and error prone tasks to the developer as opposed to encapsulating them centrally is a better pattern. I certainly don't see that as a feature. On Wednesday, August 5, 2015 at 8:19:38 AM UTC-4, James Gatannah wrote: On Monday, August 3, 2015 at 10:21:09 PM UTC-5, Dmitri wrote: My understanding is that the problem is actually caused by the stateless nature of the functions. You're talking about database interactions. By definition, those are not stateless. The trick is to isolate this statefulness as much as possible. Expanding it just compounds the problem. Since the function accepts the connection as a parameter it's up to the user of the function to ensure that it's passed the correct connection. That depends on what you mean by user of the function. If I'm working on the web server front-end, I shouldn't have any idea about the database connection. I should describe what I want to happen. If I have any concept that multiple databases are involved (I shouldn't, but abstractions leak), then, yes, I have to specify which one I mean. This isn't complicated. It's up to the person writing the database back-end code (also possibly me, if we're talking about a start-up, but then we aren't talking about the 200 KLOC scenario) to turn that description into the side-effects. Every functional solution presented in this thread suffers from this same fundamental problem that the function is not aware of the context it's being run in. That isn't a problem: it's a feature. It doesn't matter what language or programming paradigm you're using in this context. This cuts across problem domains and architectures. The actual database interaction code should be as brain-dead simple (not easy) and bullet-proof as you can possibly make it. And it should be as isolated from the front-end code as you can get away with making it. That isn't a right answer. But it's a good rule of thumb. And you should have very hefty reservations (and very good reasons) about violating it. Respectfully, James -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Clojure 1.8.0-alpha4
Trying to compile an application using ztellman/vertigo 1.3.0 library. Worked with Clojure 1.7.0, Clojure 1.8.0-alpha4 raises the following exception: An app that worked with vertigo 1.3.0 and Clojure 1.7.0 causes the following exception in the Clojure compiler: java.lang.NoClassDefFoundError: IllegalName: compile__stub.vertigo.bytes.vertigo.bytes/ByteSeq, compiling:(vertigo/bytes.clj:90:1) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6894) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6025) at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6343) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6887) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6025) at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5401) at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3975) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6885) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3769) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6889) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6255) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6887) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler.compile1(Compiler.java:7483) at clojure.lang.Compiler.compile(Compiler.java:7555) at clojure.lang.RT.compile(RT.java:406) at clojure.lang.RT.load(RT.java:451) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5445.invoke(core.clj:5871) at clojure.core$load.invokeStatic(core.clj:5870) at clojure.core$load_one.invokeStatic(core.clj:5671) at clojure.core$load_one.invoke(core.clj) at clojure.core$load_lib$fn__5394.invoke(core.clj:5716) at clojure.core$load_lib.invokeStatic(core.clj:5715) at clojure.core$load_lib.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$load_libs.invokeStatic(core.clj:5753) at clojure.core$load_libs.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$require.invokeStatic(core.clj:5775) at clojure.core$require.doInvoke(core.clj) at clojure.lang.RestFn.invoke(RestFn.java:457) at vertigo.structs$loading__5337__auto3313.invoke(structs.clj:1) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.AFn.applyTo(AFn.java:144) at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3654) at clojure.lang.Compiler.compile1(Compiler.java:7488) at clojure.lang.Compiler.compile1(Compiler.java:7478) at clojure.lang.Compiler.compile(Compiler.java:7555) at clojure.lang.RT.compile(RT.java:406) at clojure.lang.RT.load(RT.java:451) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5445.invoke(core.clj:5871) at clojure.core$load.invokeStatic(core.clj:5870) at clojure.core$load_one.invokeStatic(core.clj:5671) at clojure.core$load_one.invoke(core.clj) at clojure.core$load_lib$fn__5394.invoke(core.clj:5716) at clojure.core$load_lib.invokeStatic(core.clj:5715) at clojure.core$load_lib.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$load_libs.invokeStatic(core.clj:5753) at clojure.core$load_libs.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$require.invokeStatic(core.clj:5775) at clojure.core$require.doInvoke(core.clj) at clojure.lang.RestFn.invoke(RestFn.java:551) at vertigo.core$loading__5337__auto1375.invoke(core.clj:1) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.AFn.applyTo(AFn.java:144) at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3654) at clojure.lang.Compiler.compile1(Compiler.java:7488) at clojure.lang.Compiler.compile1(Compiler.java:7478) at clojure.lang.Compiler.compile(Compiler.java:7555) at clojure.lang.RT.compile(RT.java:406) at clojure.lang.RT.load(RT.java:451) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5445.invoke(core.clj:5871) at clojure.core$load.invokeStatic(core.clj:5870) at clojure.core$load_one.invokeStatic(core.clj:5671) at clojure.core$load_one.invoke(core.clj) at clojure.core$load_lib$fn__5394.invoke(core.clj:5716) at clojure.core$load_lib.invokeStatic(core.clj:5715) at clojure.core$load_lib.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$load_libs.invokeStatic(core.clj:5757) at clojure.core$load_libs.doInvoke(core.clj) at
Re: Metadata
Codox looks for ^no-doc when it is generating documentation to elide docs for that function or namespace. On Thu, 6 Aug 2015 at 12:24 AM Eric Le Goff eleg...@gmail.com wrote: The page http://clojure.org/metadata states that it can also be used by application developers for many purposes, annotating data sources, policy etc. Could anyone elaborate about some real use case scenarios where they need to use this metadata feature ? Regards, -- Eric Le Goff http://fr.linkedin.com/in/elegoff @elegoff https://twitter.com/elegoff -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- -- Daniel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Metadata
Metadata is perhaps most frequently used on vars:
user= (meta #'comp)
{:arglists ([] [f] [f g] [f g fs]),
:doc
Takes a set of functions and returns a fn that is the composition\n of
those fns. The returned fn takes a variable number of args,\n applies the
rightmost of fns to the args, the next\n fn (right-to-left) to the result,
etc.,
:added 1.0,
:static true,
:line 2426,
:column 1,
:file clojure/core.clj,
:name comp,
:ns #object[clojure.lang.Namespace 0x6cc8adff clojure.core]}
And for providing type hints to the compiler:
(.endsWith ^String s .txt)
I've also made use of metadata in Ring to tell the session middleware to
recreate the session with a new key:
(defn handler [request]
{:status 200
:headers {Content-Type text/plain}
:session ^:recreate {:foo bar}
:body baz})
And I've used metadata in Ittyon to denote whether a state transition is
reproducible (i.e. generated by a pure function).
- James
On 5 August 2015 at 13:01, Eric Le Goff eleg...@gmail.com wrote:
The page http://clojure.org/metadata states that it can also be used by
application developers for many purposes, annotating data sources, policy
etc.
Could anyone elaborate about some real use case scenarios where they need
to use this metadata feature ?
Regards,
--
Eric Le Goff
http://fr.linkedin.com/in/elegoff
@elegoff https://twitter.com/elegoff
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an
email to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Anotating functions for pre-processing
I wish I could do that in Clojure:
(defn ^:transactional someFunction [...] ...)
and then have somehow means to decorate someFunction (yes, I am aware there
is no container)
The code you proposed does have an effect on the someFunction var (but not the
function it ends up bound to). That might be enough for you to accomplish what
you’re after.
user= (defn ^:transactional someFunction [x] (prn x))
#'user/some-function
user= (meta #'some-function)
{:ns #Namespace user, :name someFunction, :file ..., :transactional true,
:column 1, :line 1, :arglists ([x])}
—Steve
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Clojure 1.8.0-alpha4
This is a latent bug (now exposed due to other changes in 1.8) in potemkin's deftype+ used by vertigo which was fixed here: https://github.com/ztellman/potemkin/commit/de6b6e8af5ae19adfc21841e029f3f126cfe28a6 I'm not sure what is involved in upgrading your version of potemkin or vertigo to include the fix. On Wednesday, August 5, 2015 at 8:20:14 AM UTC-5, Dragan Djuric wrote: Trying to compile an application using ztellman/vertigo 1.3.0 library. Worked with Clojure 1.7.0, Clojure 1.8.0-alpha4 raises the following exception: An app that worked with vertigo 1.3.0 and Clojure 1.7.0 causes the following exception in the Clojure compiler: java.lang.NoClassDefFoundError: IllegalName: compile__stub.vertigo.bytes.vertigo.bytes/ByteSeq, compiling:(vertigo/bytes.clj:90:1) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6894) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6025) at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6343) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6887) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6025) at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5401) at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3975) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6885) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3769) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6889) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6255) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6887) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler.compile1(Compiler.java:7483) at clojure.lang.Compiler.compile(Compiler.java:7555) at clojure.lang.RT.compile(RT.java:406) at clojure.lang.RT.load(RT.java:451) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5445.invoke(core.clj:5871) at clojure.core$load.invokeStatic(core.clj:5870) at clojure.core$load_one.invokeStatic(core.clj:5671) at clojure.core$load_one.invoke(core.clj) at clojure.core$load_lib$fn__5394.invoke(core.clj:5716) at clojure.core$load_lib.invokeStatic(core.clj:5715) at clojure.core$load_lib.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$load_libs.invokeStatic(core.clj:5753) at clojure.core$load_libs.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$require.invokeStatic(core.clj:5775) at clojure.core$require.doInvoke(core.clj) at clojure.lang.RestFn.invoke(RestFn.java:457) at vertigo.structs$loading__5337__auto3313.invoke(structs.clj:1) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.AFn.applyTo(AFn.java:144) at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3654) at clojure.lang.Compiler.compile1(Compiler.java:7488) at clojure.lang.Compiler.compile1(Compiler.java:7478) at clojure.lang.Compiler.compile(Compiler.java:7555) at clojure.lang.RT.compile(RT.java:406) at clojure.lang.RT.load(RT.java:451) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5445.invoke(core.clj:5871) at clojure.core$load.invokeStatic(core.clj:5870) at clojure.core$load_one.invokeStatic(core.clj:5671) at clojure.core$load_one.invoke(core.clj) at clojure.core$load_lib$fn__5394.invoke(core.clj:5716) at clojure.core$load_lib.invokeStatic(core.clj:5715) at clojure.core$load_lib.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$load_libs.invokeStatic(core.clj:5753) at clojure.core$load_libs.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$require.invokeStatic(core.clj:5775) at clojure.core$require.doInvoke(core.clj) at clojure.lang.RestFn.invoke(RestFn.java:551) at vertigo.core$loading__5337__auto1375.invoke(core.clj:1) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.AFn.applyTo(AFn.java:144) at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3654) at clojure.lang.Compiler.compile1(Compiler.java:7488) at clojure.lang.Compiler.compile1(Compiler.java:7478) at clojure.lang.Compiler.compile(Compiler.java:7555) at clojure.lang.RT.compile(RT.java:406) at clojure.lang.RT.load(RT.java:451) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5445.invoke(core.clj:5871) at
New version of core.async?
Are there plans for a new release of core.async? The last release was in September last year, and there have been some new features and updates added since, including using the latest version of tools.analyzer.jvm. -- -- Daniel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Clojure 1.8.0-alpha4
Adding potemkin and clj-tuple dependencies explicitly solves the problem until they upgrade vertigo. Thanks a lot for the tip! On Wednesday, August 5, 2015 at 4:33:06 PM UTC+2, Alex Miller wrote: This is a latent bug (now exposed due to other changes in 1.8) in potemkin's deftype+ used by vertigo which was fixed here: https://github.com/ztellman/potemkin/commit/de6b6e8af5ae19adfc21841e029f3f126cfe28a6 I'm not sure what is involved in upgrading your version of potemkin or vertigo to include the fix. On Wednesday, August 5, 2015 at 8:20:14 AM UTC-5, Dragan Djuric wrote: Trying to compile an application using ztellman/vertigo 1.3.0 library. Worked with Clojure 1.7.0, Clojure 1.8.0-alpha4 raises the following exception: An app that worked with vertigo 1.3.0 and Clojure 1.7.0 causes the following exception in the Clojure compiler: java.lang.NoClassDefFoundError: IllegalName: compile__stub.vertigo.bytes.vertigo.bytes/ByteSeq, compiling:(vertigo/bytes.clj:90:1) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6894) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6025) at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6343) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6887) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6025) at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5401) at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3975) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6885) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3769) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6889) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6255) at clojure.lang.Compiler.analyzeSeq(Compiler.java:6887) at clojure.lang.Compiler.analyze(Compiler.java:6688) at clojure.lang.Compiler.analyze(Compiler.java:6649) at clojure.lang.Compiler.compile1(Compiler.java:7483) at clojure.lang.Compiler.compile(Compiler.java:7555) at clojure.lang.RT.compile(RT.java:406) at clojure.lang.RT.load(RT.java:451) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5445.invoke(core.clj:5871) at clojure.core$load.invokeStatic(core.clj:5870) at clojure.core$load_one.invokeStatic(core.clj:5671) at clojure.core$load_one.invoke(core.clj) at clojure.core$load_lib$fn__5394.invoke(core.clj:5716) at clojure.core$load_lib.invokeStatic(core.clj:5715) at clojure.core$load_lib.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$load_libs.invokeStatic(core.clj:5753) at clojure.core$load_libs.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$require.invokeStatic(core.clj:5775) at clojure.core$require.doInvoke(core.clj) at clojure.lang.RestFn.invoke(RestFn.java:457) at vertigo.structs$loading__5337__auto3313.invoke(structs.clj:1) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.AFn.applyTo(AFn.java:144) at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3654) at clojure.lang.Compiler.compile1(Compiler.java:7488) at clojure.lang.Compiler.compile1(Compiler.java:7478) at clojure.lang.Compiler.compile(Compiler.java:7555) at clojure.lang.RT.compile(RT.java:406) at clojure.lang.RT.load(RT.java:451) at clojure.lang.RT.load(RT.java:419) at clojure.core$load$fn__5445.invoke(core.clj:5871) at clojure.core$load.invokeStatic(core.clj:5870) at clojure.core$load_one.invokeStatic(core.clj:5671) at clojure.core$load_one.invoke(core.clj) at clojure.core$load_lib$fn__5394.invoke(core.clj:5716) at clojure.core$load_lib.invokeStatic(core.clj:5715) at clojure.core$load_lib.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$load_libs.invokeStatic(core.clj:5753) at clojure.core$load_libs.doInvoke(core.clj) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:635) at clojure.core$require.invokeStatic(core.clj:5775) at clojure.core$require.doInvoke(core.clj) at clojure.lang.RestFn.invoke(RestFn.java:551) at vertigo.core$loading__5337__auto1375.invoke(core.clj:1) at clojure.lang.AFn.applyToHelper(AFn.java:152) at clojure.lang.AFn.applyTo(AFn.java:144) at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3654) at clojure.lang.Compiler.compile1(Compiler.java:7488) at clojure.lang.Compiler.compile1(Compiler.java:7478) at clojure.lang.Compiler.compile(Compiler.java:7555) at
Re: Anotating functions for pre-processing
Thank you Steve
with the help of the robect-hooke library I got close:
(defn ^:test-meta t [a] (println a))
;;copy the metadata to the function object
(def tt (with-meta t (meta #'t)))
(defn adv1 [f a]
(println advising f with meta (meta f))
(f a))
(defn adv2 [f a]
(println hey! f your meta is (meta f)) ;this is going to be nil as
hooke provides it's own wrapper without copying the metadata
(f a))
(add-hook #'tt #'adv1)
(add-hook #'tt #'adv2)
(tt a)
With little extra effort I will get what I want. Truth is the library uses
alter-var-root under the hood. Well, guess that's some of the idiomatic
Clojure way of life I will have to get used to.
On Wednesday, August 5, 2015 at 4:25:55 PM UTC+2, squeegee wrote:
I wish I could do that in Clojure:
(defn ^:transactional someFunction [...] ...)
and then have somehow means to decorate someFunction (yes, I am aware
there is no container)
The code you proposed does have an effect on the someFunction var (but not
the function it ends up bound to). That might be enough for you to
accomplish what you’re after.
user= (defn ^:transactional someFunction [x] (prn x))
#'user/some-function
user= (meta *#'some-function*)
{:ns #Namespace user, :name someFunction, :file ..., *:transactional
true*, :column 1, :line 1, :arglists ([x])}
—Steve
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Using a dynamic var for my database connection for implicit connections+transactions
On 5 August 2015 at 14:03, Dmitri dmitri.sotni...@gmail.com wrote: What I'm talking about is whether it's a better pattern to leave a repetitive and error prone task to the user or encapsulate it in a single place. The whole discussion boils down to following options. The first option is that we keep functions pure and the connection is passed as a parameter by the person writing the code (the user). With this approach the burden is squarely on the user to remember to pass the correct connection to the function. This becomes error prone for things like transactions where the developer has to pass the transaction connection as opposed to the normal database connection. The worst part in this scenario is that the code will run except it won't run transactionally. This is a bug that is difficult to catch as it only surfaces in cases where the transaction should be rolled back. It's worth pointing out that you don't need to use dynamic or global vars to avoid this scenario. You could just remove the original database connection from scope: (defn foobar* [tx] (foo tx) (bar tx)) (defn foobar [db] (sql/with-transaction [tx db] (foobar* tx)) Or shadow the original binding: (defn foobar [db] (sql/with-transaction [db db] (foo db) (bar db))) Ideally you also want a way of testing this behaviour, even with dynamic or global scope. If it's critical to your application that database operations run in a transaction, you should have a way of verifying that. For instance, you might use a protocol to factor out the operations on the database: (defprotocol Database (wrap-transaction [db f]) (foo db) (bar db)) (defn foobar [db] (wrap-transaction db (fn [tx] (foo tx) (bar tx This allows tests to be written to verify that foo and bar are called within wrap-transaction, and to verify our production implementation of the protocol correctly wraps the function f in a SQL transaction. If you're writing something that depends upon behaviour that can't be verified, you're going to run into problems no matter how you structure your application. The alternative is to encapsulate the database connection management in the initialization logic in the namespace managing the connection. This way the query functions can be context aware and ensure that the correct connection is used automatically. Do you mean storing the database connection in a global var, not just a dynamically scoped one? In such a case, how do you run tests without wiping the development database? Do you run the tests in a separate process, or shut down the dev server before running tests, or do you not mind if your development database is cleared by your tests? - James -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Anotating functions for pre-processing
On Wednesday, August 5, 2015 at 4:58:42 AM UTC-7, Georgi Danov wrote: I wish I could do that in Clojure: (defn ^:transactional someFunction [...] ...) How about https://clojure.github.io/java.jdbc/#clojure.java.jdbc/with-db-transaction? These kinds of scope macros are pretty common and have the benefit of being very explicit. Perhaps, though, it's the ability to late-bind such decorators that you're looking for. alter-var-root! is certainly a way to get in to that world. lein test does this kind of thing: https://github.com/technomancy/leiningen/blob/master/src/leiningen/test.clj#L13. I'd be hard-pressed to recommend it though, as I only know about this behavior because it bit me recently. I would encourage you to consider your specific needs and think about a way to address them in a more direct way. Requiring a higher-level construct to set up a function before it works, using alter-var-root! or robert-hooke, is well into spooky action at a distance territory and should be avoided if possible. - Russell -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Recommendations for a schema-based data language for use in Hadoop?
FWIW, We use edn (serialized with nippy [1]) in hadoop it works very well for us: https://github.com/Netflix/PigPen In some places we use maps for the expressiveness and in some we use vectors for more performance. Whatever I lose in raw performance I can trivially throw a few more boxes at, so it makes it a non-issue for us. The flexibility of edn outweighs any performance gains of converting back forth to another format and having to worry about translation errors. -Matt [1] https://github.com/ptaoussanis/nippy On Tuesday, August 4, 2015 at 7:05 PM, Ryan Schmitt wrote: Hi Clojure people, I'm currently working on some problems in the big data space, and I'm more or less starting from scratch with the Hadoop ecosystem. I was looking at ways to work with data in Hadoop, and I realized that (because of how InputFormat splitting works) this is a use case where it's actually pretty important to use a data language with an external schema. This probably means ruling out Edn (for performance and space efficiency reasons) and Fressian (managing the Fressian caching domain seems like it could get complicated), which are my default solutions for everything, so now I'm back to the drawing board. I'd rather not use something braindead like JSON or CSV. It seems like there are a few language-agnostic data languages that are popular in this space, such as: * Thrift * Protobuf * Avro But since the Clojure community has very high standards for data languages, as well as a number of different libraries that run code on Hadoop, I was wondering if anyone could provide a recommendation for a fast, extensible, and well-designed data language to use. (Recommendations of what to avoid are also welcome.) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com (mailto:clojure@googlegroups.com) Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com (mailto:clojure+unsubscr...@googlegroups.com) For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com (mailto:clojure+unsubscr...@googlegroups.com). For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Anotating functions for pre-processing
I would also recommend reconsidering the premise that this is a feature that is needed. Go back to what your design goals and think about implementing them in a slightly different way. Function composition, for example, can do all sorts of wonderful things. At the risk of sounding rude, I've been programming Clojure for about 5-6 years and have never needed a feature like this. So I'm wondering what it is you are building that makes your experience differ from mine. Timothy On Wed, Aug 5, 2015 at 9:58 AM, Russell Mull russell.m...@gmail.com wrote: On Wednesday, August 5, 2015 at 4:58:42 AM UTC-7, Georgi Danov wrote: I wish I could do that in Clojure: (defn ^:transactional someFunction [...] ...) How about https://clojure.github.io/java.jdbc/#clojure.java.jdbc/with-db-transaction? These kinds of scope macros are pretty common and have the benefit of being very explicit. Perhaps, though, it's the ability to late-bind such decorators that you're looking for. alter-var-root! is certainly a way to get in to that world. lein test does this kind of thing: https://github.com/technomancy/leiningen/blob/master/src/leiningen/test.clj#L13. I'd be hard-pressed to recommend it though, as I only know about this behavior because it bit me recently. I would encourage you to consider your specific needs and think about a way to address them in a more direct way. Requiring a higher-level construct to set up a function before it works, using alter-var-root! or robert-hooke, is well into spooky action at a distance territory and should be avoided if possible. - Russell -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- “One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.” (Robert Firth) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Using a dynamic var for my database connection for implicit connections+transactions
I agree that wrapping the functions is a sensible approach. Using
wrap-transaction is precisely how I ended up doing it with the conman yesql
wrapper https://github.com/luminus-framework/conman
The approach I took there is to have the generated functions use the
connection atom, and have with-transaction rebind it to the transactional
connection within its scope. However, the functions also accept an explicit
connection, and with-transaction also provides explicit access to the
transactional connection:
(with-transaction [t-conn conn]
(jdbc/db-set-rollback-only! t-conn)
(create-user!
{:id foo
:first_name Sam
:last_name Smith
:email sam.sm...@example.com})
(get-user {:id foo}))
This approach works for testing, since the connection can be passed
explicitly, but I would argue that in practice it's better to use a
separate test database instead of the dev instance.
On Wednesday, August 5, 2015 at 11:11:19 AM UTC-4, James Reeves wrote:
On 5 August 2015 at 14:03, Dmitri dmitri@gmail.com javascript:
wrote:
What I'm talking about is whether it's a better pattern to leave a
repetitive and error prone task to the user or encapsulate it in a single
place. The whole discussion boils down to following options.
The first option is that we keep functions pure and the connection is
passed as a parameter by the person writing the code (the user). With this
approach the burden is squarely on the user to remember to pass the correct
connection to the function. This becomes error prone for things like
transactions where the developer has to pass the transaction connection as
opposed to the normal database connection. The worst part in this scenario
is that the code will run except it won't run transactionally. This is a
bug that is difficult to catch as it only surfaces in cases where the
transaction should be rolled back.
It's worth pointing out that you don't need to use dynamic or global vars
to avoid this scenario. You could just remove the original database
connection from scope:
(defn foobar* [tx]
(foo tx)
(bar tx))
(defn foobar [db]
(sql/with-transaction [tx db] (foobar* tx))
Or shadow the original binding:
(defn foobar [db]
(sql/with-transaction [db db]
(foo db)
(bar db)))
Ideally you also want a way of testing this behaviour, even with dynamic
or global scope. If it's critical to your application that database
operations run in a transaction, you should have a way of verifying that.
For instance, you might use a protocol to factor out the operations on the
database:
(defprotocol Database
(wrap-transaction [db f])
(foo db)
(bar db))
(defn foobar [db]
(wrap-transaction db
(fn [tx]
(foo tx)
(bar tx
This allows tests to be written to verify that foo and bar are called
within wrap-transaction, and to verify our production implementation of the
protocol correctly wraps the function f in a SQL transaction.
If you're writing something that depends upon behaviour that can't be
verified, you're going to run into problems no matter how you structure
your application.
The alternative is to encapsulate the database connection management in
the initialization logic in the namespace managing the connection. This way
the query functions can be context aware and ensure that the correct
connection is used automatically.
Do you mean storing the database connection in a global var, not just a
dynamically scoped one?
In such a case, how do you run tests without wiping the development
database? Do you run the tests in a separate process, or shut down the dev
server before running tests, or do you not mind if your development
database is cleared by your tests?
- James
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Using a dynamic var for my database connection for implicit connections+transactions
This.
I would repeat with as strong an emphasis as possible the sanity
of using a dedicated test database - how else can you test schema
creation for example...
Dmitri writes:
I agree that wrapping the functions is a sensible approach. Using
wrap-transaction is precisely how I ended up doing it with the conman yesql
wrapper https://github.com/luminus-framework/conman
The approach I took there is to have the generated functions use the
connection atom, and have with-transaction rebind it to the transactional
connection within its scope. However, the functions also accept an explicit
connection, and with-transaction also provides explicit access to the
transactional connection:
(with-transaction [t-conn conn]
(jdbc/db-set-rollback-only! t-conn)
(create-user!
{:id foo
:first_name Sam
:last_name Smith
:email sam.sm...@example.com})
(get-user {:id foo}))
This approach works for testing, since the connection can be passed
explicitly, but I would argue that in practice it's better to use a
separate test database instead of the dev instance.
On Wednesday, August 5, 2015 at 11:11:19 AM UTC-4, James Reeves wrote:
On 5 August 2015 at 14:03, Dmitri dmitri@gmail.com javascript:
wrote:
What I'm talking about is whether it's a better pattern to leave a
repetitive and error prone task to the user or encapsulate it in a single
place. The whole discussion boils down to following options.
The first option is that we keep functions pure and the connection is
passed as a parameter by the person writing the code (the user). With this
approach the burden is squarely on the user to remember to pass the correct
connection to the function. This becomes error prone for things like
transactions where the developer has to pass the transaction connection as
opposed to the normal database connection. The worst part in this scenario
is that the code will run except it won't run transactionally. This is a
bug that is difficult to catch as it only surfaces in cases where the
transaction should be rolled back.
It's worth pointing out that you don't need to use dynamic or global vars
to avoid this scenario. You could just remove the original database
connection from scope:
(defn foobar* [tx]
(foo tx)
(bar tx))
(defn foobar [db]
(sql/with-transaction [tx db] (foobar* tx))
Or shadow the original binding:
(defn foobar [db]
(sql/with-transaction [db db]
(foo db)
(bar db)))
Ideally you also want a way of testing this behaviour, even with dynamic
or global scope. If it's critical to your application that database
operations run in a transaction, you should have a way of verifying that.
For instance, you might use a protocol to factor out the operations on the
database:
(defprotocol Database
(wrap-transaction [db f])
(foo db)
(bar db))
(defn foobar [db]
(wrap-transaction db
(fn [tx]
(foo tx)
(bar tx
This allows tests to be written to verify that foo and bar are called
within wrap-transaction, and to verify our production implementation of the
protocol correctly wraps the function f in a SQL transaction.
If you're writing something that depends upon behaviour that can't be
verified, you're going to run into problems no matter how you structure
your application.
The alternative is to encapsulate the database connection management in
the initialization logic in the namespace managing the connection. This way
the query functions can be context aware and ensure that the correct
connection is used automatically.
Do you mean storing the database connection in a global var, not just a
dynamically scoped one?
In such a case, how do you run tests without wiping the development
database? Do you run the tests in a separate process, or shut down the dev
server before running tests, or do you not mind if your development
database is cleared by your tests?
- James
--
Sent with my mu4e
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Planck 1.0 - Bootstrapped ClojureScript OS X REPL
Mike, thanks for this tool. I want to use it to introduce ClojureScript to some local JavaScript devs, since they would be less likely to install anything that required the JVM. I cannot seem to get `brew install planck` to work, however. It just hangs. It looks like `brew upgrade` hangs indefinitely as well, though, so this may just be my mac's fault. If anyone knows how to fix this, please let me know. On Tuesday, August 4, 2015 at 9:51:31 AM UTC-4, Mike Fikes wrote: Planck 1.3 is now available via Homebrew: http://blog.fikesfarm.com/posts/2015-08-04-pour-a-pint-of-planck.html -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Recommendations for a schema-based data language for use in Hadoop?
Ryan Schmitt rschm...@u.rochester.edu writes: I'm currently working on some problems in the big data space, and I'm more or less starting from scratch with the Hadoop ecosystem. I was looking at ways to work with data in Hadoop, and I realized that (because of how InputFormat splitting works) this is a use case where it's actually pretty important to use a data language with an external schema. At Damballa we extensively use Avro for these sorts of problems. We’ve written a set of Clojure bindings for Avro named “abracad” [1]. Abracad exposes Avro data as native Clojure data (persistent vectors, maps, etc), supports protocol-based de/serialization of custom types, and includes explicit support for defining “EDN-in-Avro” schemas which can include arbitrary Clojure data. We’ve implemented support in the mainline Java Avro project (merged in 1.7.5) for specifying configurable “data models” for MapReduce jobs, which allows Avro MapReduce input to directly produce Clojure data and output to consume Clojure data. And we’ve implemented fairly automatic configuration for such in the Avro dseqs of our “parkour” Clojure-Hadoop/MR integration library [2]. [1] https://github.com/damballa/abracad [2] https://github.com/damballa/parkour -- Marshall Bockrath-Vandegrift llas...@damballa.com Principal Software Engineer, Damballa RD -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ANN] Planck 1.0 - Bootstrapped ClojureScript OS X REPL
Hi Zach, Yeah, it definitely sounds like your Homebrew install has an issue. You can also just download the binary from http://planck.fikesfarm.com http://planck.fikesfarm.com/ (it is a self-contained single-file executable). It works on Mavericks or later. - Mike On Aug 5, 2015, at 1:24 PM, Zach Oakes zsoa...@gmail.com wrote: Mike, thanks for this tool. I want to use it to introduce ClojureScript to some local JavaScript devs, since they would be less likely to install anything that required the JVM. I cannot seem to get `brew install planck` to work, however. It just hangs. It looks like `brew upgrade` hangs indefinitely as well, though, so this may just be my mac's fault. If anyone knows how to fix this, please let me know. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Anotating functions for pre-processing
I wish I could do that in Clojure:
(defn ^:transactional someFunction [...] ...)
and then have somehow means to decorate someFunction (yes, I am aware there
is no container)
The code you proposed does have an effect on the someFunction var (but not the
function it ends up bound to). That might be enough for you to accomplish what
you’re after.
user= (defn ^:transactional someFunction [x] (prn x))
#'user/some-function
user= (meta #'some-function)
{:ns #Namespace user, :name someFunction, :file ..., :transactional true,
:column 1, :line 1, :arglists ([x])}
—Steve
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Using a dynamic var for my database connection for implicit connections+transactions
On 5 August 2015 at 18:04, Dmitri dmitri.sotni...@gmail.com wrote: I agree that wrapping the functions is a sensible approach. Using wrap-transaction is precisely how I ended up doing it with the conman yesql wrapper https://github.com/luminus-framework/conman The approach I took there is to have the generated functions use the connection atom, and have with-transaction rebind it to the transactional connection within its scope. However, the functions also accept an explicit connection, and with-transaction also provides explicit access to the transactional connection So when you're testing, presumably you use a dynamic binding to override the global connection to the test database? - James -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Using a dynamic var for my database connection for implicit connections+transactions
There are a number of options here depending on your workflow. You could
override the dynamic var, create a separate connection for tests and pass
it around explicitly in test code, or get the connection from the
environment. The last option is what I tend to do, the profiles.clj will
contain separate URLs for testing and dev databases:
{:profiles/dev {:env {:database-url
jdbc:postgresql://localhost/myapp_dev?user=db_user_name_herepassword=db_user_password_here}}
:profiles/test {:env {:database-url
jdbc:postgresql://localhost/myapp_test?user=db_user_name_herepassword=db_user_password_here}}}
The tests run in the test profile and so get the test database connection
from the environment.
On Wednesday, August 5, 2015 at 2:34:31 PM UTC-4, James Reeves wrote:
On 5 August 2015 at 18:04, Dmitri dmitri@gmail.com javascript:
wrote:
I agree that wrapping the functions is a sensible approach. Using
wrap-transaction is precisely how I ended up doing it with the conman yesql
wrapper https://github.com/luminus-framework/conman
The approach I took there is to have the generated functions use the
connection atom, and have with-transaction rebind it to the transactional
connection within its scope. However, the functions also accept an explicit
connection, and with-transaction also provides explicit access to the
transactional connection
So when you're testing, presumably you use a dynamic binding to override
the global connection to the test database?
- James
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Clojure, WEB and Enterprise development
Hi! I was using Clojure for a long time. It has been used for private and commercial projects (sniffed by me and hated by others). Now it has been abandoned. It's not giving me any money nor there is no agreement in peers to use it. But... I think that Clojure has a future. Datomic is ready for new Intel mass storage (which is 1000 times faster than current SSD). Clojure as lisp is a head of time for most today programing paradigms. But I'm disappointed by few things: Problem: there is no a bind for the JEE servers - they should be treated as the SQL - you will never reach its maturity Solution: so you should parasite it and make a wrapper around timed services, jms queues, web services, ejb, jpa (with changed semantic fitting to clojure style) My try: I have already build some prototype for building web services in runtime (with build in http server in jdk) Problem: there is still no true web layer - all what you can see is a wrap around Servlet and HTTP protocol concept or own semantic around WWW Solution: you should try to build a layer around GWT or Java Prime Faces, I know that now it is not possible, but you should think what steps are needed to achive that My try: I have build and framework which abstracts from the system domain in GUI and the server side conforms it's protocol (what contexts as tabs, CRUD, contextual operations, elastic permissions - tab context and data context, dialogs for CRUD, filters for listing, tables, dynamic forms modification, validation) all was made first in Clojure and the spec of DSL could produce Swing app, the same spec later could produce GWT app (for known reasons the engine had to be rewritten in Java but spec could be still Clojure data or XML) The GUI engine is so abstract that Web Layer could switch between backend system and provide all the GUI functionality as needed - the limit is that it was ordinary a CRUD app (card management system; football cards management and bans system; CAR system with conjunction with EJBCA; user and persmission managment system; lottery systems - all there where registration system fits) Let the big business promote you potential and left for yourself only the evangelism of the Clojure language... -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: New version of core.async?
This would be mighty useful: http://dev.clojure.org/jira/browse/ASYNC-137 I was planning on moving away from using core.async because I had assumed this issue would not be fixed and released in the forseeable future. Should I have hope? :-) On Thursday, August 6, 2015 at 9:22:56 AM UTC+10, Alex Miller wrote: Yes. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
New version of core.async?
Yes. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Clojure, WEB and Enterprise development
Immutant provides a lot of enterprisy things out of the box. On Wednesday, August 5, 2015 at 5:28:02 PM UTC-4, Olek wrote: Hi! I was using Clojure for a long time. It has been used for private and commercial projects (sniffed by me and hated by others). Now it has been abandoned. It's not giving me any money nor there is no agreement in peers to use it. But... I think that Clojure has a future. Datomic is ready for new Intel mass storage (which is 1000 times faster than current SSD). Clojure as lisp is a head of time for most today programing paradigms. But I'm disappointed by few things: Problem: there is no a bind for the JEE servers - they should be treated as the SQL - you will never reach its maturity Solution: so you should parasite it and make a wrapper around timed services, jms queues, web services, ejb, jpa (with changed semantic fitting to clojure style) My try: I have already build some prototype for building web services in runtime (with build in http server in jdk) Problem: there is still no true web layer - all what you can see is a wrap around Servlet and HTTP protocol concept or own semantic around WWW Solution: you should try to build a layer around GWT or Java Prime Faces, I know that now it is not possible, but you should think what steps are needed to achive that My try: I have build and framework which abstracts from the system domain in GUI and the server side conforms it's protocol (what contexts as tabs, CRUD, contextual operations, elastic permissions - tab context and data context, dialogs for CRUD, filters for listing, tables, dynamic forms modification, validation) all was made first in Clojure and the spec of DSL could produce Swing app, the same spec later could produce GWT app (for known reasons the engine had to be rewritten in Java but spec could be still Clojure data or XML) The GUI engine is so abstract that Web Layer could switch between backend system and provide all the GUI functionality as needed - the limit is that it was ordinary a CRUD app (card management system; football cards management and bans system; CAR system with conjunction with EJBCA; user and persmission managment system; lottery systems - all there where registration system fits) Let the big business promote you potential and left for yourself only the evangelism of the Clojure language... -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
ANN: ClojureScript 1.7.48
ClojureScript, the Clojure compiler that emits JavaScript source code. README and source code: https://github.com/clojure/clojurescript Leiningen dependency information: [org.clojure/clojurescript 1.7.48] This release updates the Google Closure Compiler and Library dependencies to the latest. It also addresses a REPL regression, adds a goog-define helper macro for goog.define interop, and includes various other small fixes, changes and enhancements. As always feedback welcome! ## 1.7.48 ## Enhancements * provide goog-define macro to support proper use of goog.define * CLJS-1177: A compiler support for non-Closure transforms (JSX, etc) * CLJS-1296: browser REPL should queue prints before connection then flush after connection * add :dump-core compiler option for cljs.js config * CLJS-1386: Symbols should be added to the constants table ## Changes * Bump Closure Compiler dependency * Bump Closure Library dependency ## Fixes * CLJS-1392: cljs.repl/source regression * CLJS-1391: Error when building for target :nodejs * CLJS-1388: Stacktrace element handling for :output-dir w/o file/line/column * CLJS-1311: Improve error reporting when converting JavaScript modules * CLJS-1387: support local Closure libs that conform to classpath -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Clojure, WEB and Enterprise development
On 5 August 2015 at 22:28, Olek aleksander.nas...@gmail.com wrote: Problem: there is no a bind for the JEE servers - they should be treated as the SQL - you will never reach its maturity Solution: so you should parasite it and make a wrapper around timed services, jms queues, web services, ejb, jpa (with changed semantic fitting to clojure style) I'm not aware of any efforts to fully wrap the JEE stack, and you're the first person I'm aware of that considers this a significant issue. Perhaps it's less of a problem than you think? Problem: there is still no true web layer - all what you can see is a wrap around Servlet and HTTP protocol concept or own semantic around WWW Solution: you should try to build a layer around GWT or Java Prime Faces, I know that now it is not possible, but you should think what steps are needed to achive that You have an unusual definition of what constitutes a true web layer. Trying to wrap a functional layer around a heavily OOP library rarely works out well. It's very difficult, perhaps impossible, to present a clean immutable interface to something that is inherently mutable underneath. The direction Clojure is currently going in is to wrap more functional frameworks, like Facebook's React. Om or Reagent are two of the most popular libraries built on React, and David Nolen has some interesting ideas on how the client and server should communicate in future. - James -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Recommendations for a schema-based data language for use in Hadoop?
I suggest using prismatic`s schema library, and generating kryo serializers for your schematized records at compile time. These serializations can be very compact by leveraging the schemas, and kryo is very fast. I've been having success with this approach on Apache Spark. If you aren't married to using hadoop, and you want performance, l suggest you investigate spark as well. I'm planning to extract this automatic schema-based kryo serializer macro junk and release a lib ... when I get around to it. I'd be glad to share the code if you want. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
