That's exactly the tool. Thanks a lot :)
On Sun, May 25, 2014 at 10:07 PM, Atamert Ölçgen <mu...@muhuk.com> wrote: > If I'm not mistaken, PigPen allows you to execute queries locally, without > a hadoop cluster etc. Does that solve your problem? > > > https://github.com/Netflix/PigPen/wiki/Design-and-Features#testing-local-execution-and-debugging > > > > On Mon, May 26, 2014 at 3:26 AM, Hesen Peng <hesen.p...@gmail.com> wrote: > >> Hi everybody, >> >> I have been doing some random machine learning doodling and have always >> been haunted with this problem. >> >> When I just wanna set out and try if my methodology would work, I would >> program in Clojure aiming at a small dataset available within one single >> host memory (or even just doodle R or Python. I know what's blasphemy). >> >> When I wanna implement the results, I would usually have to re-write the >> whole process in Cascalog just in order to apply them in Hadoop data. I >> know I can still just start up to write Hadoop functions. But that's a bit >> overshoot since I have to run simulation study to verify the performance, >> correctness etc before even going into production development. Besides, >> having hadoop in memory to sit between my program and the computer just >> slows down computation. >> >> I've day-dreamed a lot some smart ways to do prototyping and, with the >> flip of a switch, my functions just become easily applicable to Hadoop >> data. Wondering if there's any ways to do that? >> >> I actually asked around and some one who I admire a lot suggested writing >> a Hadoop emulator-like wrapper for my input data. Wondering if there's an >> easier way? I would appreciate any input. Thanks. >> >> Hesen >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+unsubscr...@googlegroups.com. >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Kind Regards, > Atamert Ölçgen > > -+- > --+ > +++ > > www.muhuk.com > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "Clojure" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/clojure/pw-xPg4D-DA/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Hesen Peng 彭河森 -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
