Conceptually, we should test almost everything on both local engine and map reduce engine. I usually found that 1) sometimes output from local is different from map reduce engine 2) some problems only exist in one of them. As Mathieu suggested, we should build a testing utility to help us test both backends (or more in the future) without duplicating the code. I don't have any solution for this yet. Anyone has a good idea?
Pi On Mon, Apr 21, 2008 at 6:32 PM, Mathieu Poumeyrol <[EMAIL PROTECTED]> wrote: > All, > > During my — not finished yet — struggle with the "using" clause for the > order operator, I found out that the tests for the order clause where only > run in MapReduce mode, not in local mode. I proposed a patch to fix this > test, but Pi rightly suggest that we look for a more global approach to run > the tests systematically in both modes. Indeed, I think I have read > somewhere that the new (and very exciting) "illustrate" operator is only > tested and only work in local mode — for now. > > We could have an abstract test class responsible for PigContext setup and > teardown (local mode or MiniCluster based). All (or most of) the unit tests > would inherit from it. The ant script could run the test suite twice passing > some property to the abstract class to define which PigContext version > should be used. Some tests may still want to "skip" one of the two modes, > but at least running test in two modes would become the default and natural > behavior. > > What do you think ? > > -- > Mathieu Poumeyrol — IDM
