Re: Help sought on issue with AOT, hadoop, classloaders, and consistency of Clojure fn classes

2015-02-02 Thread Marshall Bockrath-Vandegrift
On Friday, January 30, 2015 at 5:00:31 PM UTC-5, Jason Wolfe wrote: Thanks for the recommendation. For now we're looking for a simple low-level interface to MR, but we're also keeping an eye on parkour and pigpen for more complex tasks down the road. Can you explain why I might prefer

Re: Help sought on issue with AOT, hadoop, classloaders, and consistency of Clojure fn classes

2015-02-02 Thread Jason Wolfe
On Mon, Feb 2, 2015 at 7:12 AM, Marshall Bockrath-Vandegrift llas...@gmail.com wrote: On Friday, January 30, 2015 at 5:00:31 PM UTC-5, Jason Wolfe wrote: Thanks for the recommendation. For now we're looking for a simple low-level interface to MR, but we're also keeping an eye on parkour and

Re: Help sought on issue with AOT, hadoop, classloaders, and consistency of Clojure fn classes

2015-01-30 Thread Marshall Bockrath-Vandegrift
Not a solution to your immediate problem, but if this is for new development (not an existing mass of clojure-hadoop code), I'd suggest looking at Parkour instead. As the main Parkour developer I'm obviously biased, but Parkour exists in part because the compilation model used by

Re: Help sought on issue with AOT, hadoop, classloaders, and consistency of Clojure fn classes

2015-01-30 Thread Jason Wolfe
On Friday, January 30, 2015 at 7:27:12 AM UTC-8, Marshall Bockrath-Vandegrift wrote: Not a solution to your immediate problem, but if this is for new development (not an existing mass of clojure-hadoop code), I'd suggest looking at Parkour instead. As the main Parkour developer I'm

Re: Help sought on issue with AOT, hadoop, classloaders, and consistency of Clojure fn classes

2015-01-29 Thread Sean Corfield
Which version of Clojure are you using? Does clojure-hadoop or Schema include AOT-compiled versions of other libraries and/or core namespaces? If the answers are 1.7.0 Alpha 5 and yes then you've run into the same problem I and a few others did: the previously undefined behavior of loading

Re: Help sought on issue with AOT, hadoop, classloaders, and consistency of Clojure fn classes

2015-01-29 Thread Jason Wolfe
On Thursday, January 29, 2015 at 9:45:25 AM UTC-8, Sean Corfield wrote: Which version of Clojure are you using? 1.6.0, both for AOT and at runtime. Does clojure-hadoop or Schema include AOT-compiled versions of other libraries and/or core namespaces? No, as far as I know the only AOT

Re: Help sought on issue with AOT, hadoop, classloaders, and consistency of Clojure fn classes

2015-01-29 Thread shlomivaknin
Hey, Just to be sure, are you loading the full uberjar to hadoop? On Wednesday, January 28, 2015 at 11:39:54 PM UTC-8, Jason Wolfe wrote: First off, I apologize in advance for not having a reduced test case, and express my sincere gratitude in advance for any assistance. I've been

Re: Help sought on issue with AOT, hadoop, classloaders, and consistency of Clojure fn classes

2015-01-29 Thread Jason Wolfe
Thanks for the help everyone. After some more fiddling, with two changes I can hack things to work but it's not pretty. First, this change to schema: https://github.com/Prismatic/schema/commit/e835d447ea2525471a40d65768b5e6435a469fc7 Second, if I just (require 'top-level-ns) things still

Re: Help sought on issue with AOT, hadoop, classloaders, and consistency of Clojure fn classes

2015-01-29 Thread Jason Wolfe
On Thursday, January 29, 2015 at 4:18:15 PM UTC-8, shlomi...@gmail.com wrote: Hey, Just to be sure, are you loading the full uberjar to hadoop? Yes. The issue isn't too few classes being found, it's too many :). If I start an nrepl server inside the job and futz around with loading

Help sought on issue with AOT, hadoop, classloaders, and consistency of Clojure fn classes

2015-01-28 Thread Jason Wolfe
First off, I apologize in advance for not having a reduced test case, and express my sincere gratitude in advance for any assistance. I've been tearing my hair out for a day or so and not making headway, and figured someone here might recognize some keywords and have a pointer in the right