Hi Keith, thanks sincerely for the advice and relating your experiences.
regards Peter Cogan On 21 May 2012, at 18:57, Keith Wiley wrote: > Hubris aside (mine I mean), I'm probably not qualified to be the last word on > this with regard to advising you on your project's overall direction, so > hopefully others will chime in on this discussion...but that said, when I > started with hadoop a few years ago and our project exceed the capabilities > of direct-Java hadoop interfacing, we briefly entertained the notion of pipes > and learned (to the best of my recollection) that pipes was officially losing > its support within the hadoop community (it may have even been depricated > although I don't remember for sure) and that streaming was being officially > advocated as the replacement (unless I am grossly exaggerating the advice I > received at the time). In fact, the primary reason I abandoned pipes wasn't > so much that the technology itself wouldn't work, but that there was far less > community support for it than for streaming, so whenever I got stuck I > couldn't really ask for help (or I could ask but I would be inquiring a much > smaller pool of potential responders, thus decreasing the scope of any > response). > > At any rate, I have been quite satisfied with streaming. Gotchas have > included coding from a scratch an entire partially abstract C++ class family > to offer a streaming interface to communicate between hadoop and underlying > native processing, to say nothing of coding up binary/ascii conversion in a > variety of formats to shuffle data across the i/o streams (for reasons I > can't recall now, the binary streaming interface wasn't a good option for us > at the time). > > Still, all's well. > > Best of luck. > > On May 21, 2012, at 10:26 , Peter Cogan wrote: > >> >> Hi Keith, >> >> Yeah - maybe that was my first mistake. Since I haven't had much luck >> getting pipes working I've started looking at streaming. I am still 'early >> in my hadoop efforts' so now is the time for big course corrections >> >> thanks >> Peter >> >> On 21 May 2012, at 18:11, Keith Wiley wrote: >> >>> Mind if I ask why you prefer pipes to streaming? I abandoned pipes early >>> in my hadoop efforts and went all engines full on streaming. >>> >>> On May 21, 2012, at 04:07 , Peter Cogan wrote: >>> >>>> Hi, >>>> >>>> I am trying to install pipes on os x following the instructions here > > > ________________________________________________________________________________ > Keith Wiley kwi...@keithwiley.com keithwiley.com > music.keithwiley.com > > "The easy confidence with which I know another man's religion is folly teaches > me to suspect that my own is also." > -- Mark Twain > ________________________________________________________________________________ >
