On Tue, Nov 30, 2010 at 4:47 AM, Shai Erera <[email protected]> wrote: > > Like you said, the rest of the tests just increase the test running time. >
I'm not completely sure about this: do we always switch over our tests to do the equivalent checks both against the new API and the old API when we make API changes? There could be bugs in our 'backwards handling' that are actually logic bugs that the new tests dont detect. So I'm a little concerned about only running "pure" simplistic API tests in backwards. On the other hand, I'm really worried about what Shai brings up here: we are doing some refactoring of the tests system and there is more shared code at the moment: similar to MockRAMDirectory. Because we worry about preventing things like index corruption, its my opinion we need things like MockRAMDirectory, and they should be able to break all the rules/etc (use pkg-private APIS) if we can prevent bugs. Just look at our trunk or 3.x tests and imagine them as backwards tests... these sort of "utilities" like RandomIndexWriter will be more fragile to internal/experimental/pkg-private changes, but as mentioned above i think these are good to have in backwards tests. So, I think at the moment I'm leaning towards the idea of going back towards a checkout that we can modify, in combination with us all soliciting more reviews / longer time for any backports to stable branches that require backwards tests modifications? I understand Uwe's point too, its dangerous to modify the code and seems to defeat the purpose of backwards, but I think this is going to be a more serious problem after releasing 3.1! --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
