Thanks for finding this issue, Allen. This is somewhat of a tangent, but I'm curious who still uses hadoop-pipes. Certainly development has more or less stopped on it. I think the last 5 years of commits on it have been things like updating version numbers, fixing missing #includes, moving to CMake... and before that, moving Hadoop's build system to Maven.
There's a lot of questionable things in pipes, including the fact that there are no unit tests, and it unconditionally builds with gcc -O0. I think many people moved to using Hadoop streaming instead of Hadoop pipes, since it works with other programming languages better. Should we retire this component at some point, or does it still fill a role? best, Colin On Tue, Oct 27, 2015 at 7:44 PM, Allen Wittenauer <a...@altiscale.com> wrote: > > Today I discovered that an old, old code change (circa 2012) caused > certain maven modules to be skipped during the precommit testing. That code > had been carried forward through all the rewrites, bug fixes, etc, over the > years likely because it seemed the correct thing to do. It is clearly not, > since at least hadoop-pipes was getting ignored. I have not evaluated the > impact of the bug on other parts of the Hadoop code base. > > I’ve got sitting in the Hadoop beta test branch of Yetus what I think > is a potential fix to that bug. Initial testing shows all systems are go. > > Additionally: > > * After several weeks, Yetus being used for Hadoop Common has been > more or as stable as trunk’s, but testing significantly more parts. > * People are still reporting bugs on trunk’s test-patch that have > been fixed for months in Yetus. > * People are still confused as to which version is running where, > despite the email thread just a few days ago. (*exasperated sigh here*) > > So I’m going to turn on Yetus for *ALL* Hadoop precommit jobs later > tonight. (Given how backed up Jenkins is at the moment, there is plenty of > time. haha) Anyway, if you see “Powered by Yetus” in the Hadoop QA posts, > you’ve got Yetus. If you don’t see it, it ran on trunk’s test-patch. > > (… and now begin the threads on everyone freaking out and/or not > reading the above …)
