Which version of Clojure are you using? Does clojure-hadoop or Schema include AOT-compiled versions of other libraries and/or core namespaces?
If the answers are 1.7.0 Alpha 5 and "yes" then you've run into the same problem I and a few others did: the previously undefined behavior of loading both AOT and JIT versions of the same code now has a defined behavior (preferring AOT) - and you get this exception. Sean On Jan 28, 2015, at 11:39 PM, Jason Wolfe <ja...@w01fe.com> wrote: > First off, I apologize in advance for not having a reduced test case, and > express my sincere gratitude in advance for any assistance. I've been > tearing my hair out for a day or so and not making headway, and figured > someone here might recognize some keywords and have a pointer in the right > direction. (I'm admittedly pretty green when it comes to class loading, and > have largely exhausted my google fu). > > Problem: > > I'm submitting a hadoop job using clojure-hadoop. All is well with a simple > job, but once I require something that transitively depends on Schema, I end > up with: > > clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: > No implementation of method: :walker of protocol: #'schema.core/Schema found > for class: clojure.core$long, compiling:(crane/config.clj:33:4) > > It works fine when run in-process with hadoop-mapreduce-client-jobclient, but > not with bin/hadoop -jar. This stunk of a classloader issue, and after > digging in it seems that there are multiple versions of clojure.core$long > floating around. The version on which the protocol is extended is not the > same class for the fn that the symbol 'long resolves to in client code. > > Context: > > clojure-hadoop is AOT-compiled, and after being loaded by hadoop it > dynamically loads the target namespace (not AOT-compiled, nor any other of > the code in question) using > https://github.com/alexott/clojure-hadoop/blob/master/src/clojure_hadoop/load.clj#L3 > > From here, schema is transitively required, and then client namespaces > attempt to use the Schema protocol to generate validators, and when the > schema 'long is used (which resolves to the fn with class clojure.core$long), > it fails to find the appropriate method. > > After repeated head-bashing, I've determined that there are (at least two) > versions of the clojure.core$long class floating around -- the one used to > extend the protocol, which stems from a DynamicClassLoader, and the one that > 'long resolves to in client code, which stems from a URLClassLoader. The > URLClassLoader is the loader of the current thread and Compiler, but not > @(clojure.lang.Compiler/LOADER). > > Attempts: > > I've tried wrapping the clojure-hadoop loading code with > .setContextClassLoader on some obvious candidates and binding > *use-context-classloader* around the code doing the loading, with no avail. > I've tried changing the schema code to reference the class in different ways > (class (resolve 'long)), (class 'long), etc and that hasn't made a > difference. I've checked and the clojure-hadoop jar doesn't contain any > .class files for clojure, schema, or other offending code. > > Plea: > > I suspect there's something obvious I'm missing. (In retrospect it seems > like the design of Schema may be suboptimal in light of this, but if possible > I'd like to figure out a workaround without changing that substantially). > Thanks in advance for your help -- any and all pointers are welcome. > > -Jason > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
