Hi I'd like to discuss the semantics of our API and how backwards tests relate to it. First, I'd like to confirm my understanding - currently it relates to 3x, but it will apply to 4x after 4.0 will be released:
Public/Protected -- this API is 'public' and we should maintain back-compat, in the form of jar drop-in. That is, we cannot rename or modify it, w/o deprecating first (I leave *exceptions* deliberately outside the discussion). Package-private -- this is not public API and while users can use it if they declare their classes under the relevant package, they should not expect jar drop-in support. Public @internal -- this is public API following Java language, however not public to the users. We need to make this API public so that Lucene can access it, but it's used for internal purposes only. Users can still use it, however cannot expect jar drop-in support. Public @experimental -- this API is intended to be made 'public' one day, however we're still working on it, and even though it's checked-in or even released, it may change unexpectedly. Not sure we want to say that jar drop-in support cannot be expected, though according to the definition we are allowed to change it ... so perhaps it's like @internal, only w/ the intention to make it public one day. Both @internal and @experimental tags should be removed if they do not apply anymore. Now comes the question about backwards tests -- our tests touch all the API types above, however they are not resilient to changes in 3 out of 4 of them. In the past this wasn't a problem - the backwards layer had both src/java and src/test, tests we compiled against src/java and then executed against core.jar. This allowed changing the source code of the "non public" API and make the same changes to backwards/src/java, and tests would still run. This had a disadvantage too - it was 'easier' to break back-compat on the first API type (the *true* public API) because you could still change bw/src/java and be done w/ it. Today though bw tests are compiled against the previous release source. But if you make changes to the non public API, they break while they shouldn't. So the question is what can we do about the backwards tests so that we can still make allowed changes to the API w/o them breaking? * We can say that unit tests should not test package-private / @internal / @experimental classes, but I don't believe in it. * We can re-introduce bw/src/java and ask all committers to make careful changes to it. If we're careful, we won't introduce any *true* public API break. The second is the more realistic solution IMO, but since this was the situation in the past and changed to how it is today, I don't know if it's acceptable. Whatever we do though, we cannot have backwards tests dictate what is public API and what isn't, because bw tests are compiled following Java semantics, that have nothing to do w/ Lucene's 'public' notion. Shai
