Just verify numFiles may not be sufficient. I will see if I can enhance the verification code to handle both. Thanks. -Chun
On Mon, Oct 26, 2015 at 10:01 AM, Jacques Nadeau <[email protected]> wrote: > My current fix is to remove the specific lists. (A test which is flaky > isn't very valuable.) You can see my current branch at [1]. You can look at > my most recent commit to see the changes I made to the expected results > files to get the tests to report the correct result. > > If you want to do file name verification (as opposed to simply numFiles > verification), it seems like you should be doing a structured verification > of the json plan rather than trying to do something with the text plan. > > [1] https://github.com/dremio/drill-test-framework > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Mon, Oct 26, 2015 at 9:36 AM, Chun Chang <[email protected]> wrote: > > > Jacques, > > > > Thanks for the heads up. And what is your proposed fix? I think we can > > easily enhance the test framework to handle file ordering cases. This > way, > > we don't need to modify any existing tests. > > > > Thanks, > > Chun > > > > On Sat, Oct 24, 2015 at 9:11 PM, Jacques Nadeau <[email protected]> > > wrote: > > > > > A large number of tests associated with MD-185 are unintentionally > > brittle. > > > Many of these tests reference more than one file. The tests are > planning > > > tests and thus are prone to failure if the file ordering isn't the same > > as > > > what was used for test generation. > > > > > > In the case of these types of tests, people should be cautious of > adding > > > expected results that include multiple files in a certain order. I'm > > > working on a patch fix for these specific tests but wanted to let > people > > > know to be cautious of these issues in the future. > > > > > > thanks! > > > > > > -- > > > Jacques Nadeau > > > CTO and Co-Founder, Dremio > > > > > >
