Vyacheslav, Thank you for answers!
>> I'm not sure what is a problem here? At the moment it's a little bit hard to understand the impact of such a change because the number of test suites is small. Let's Imagine you have X compatibility test suites where X is a relatively big number, let's say 20. A new version of Ignite has released and a developer should update compatibility tests to run it against the new version. A developer should manually find all of these 20 test suites and in each of these suites and add a new version to test it (new method). This is a long process and there could be developer mistakes, some of the test suites can be missed and nobody will know about it before real compatibility problem appears. We already faced with the situation when after new version release (2.5-2.6) no compatibility tests were modified to support new version because nobody remembers about it and there is no such process after release. Adding new pom is a very simple step and can be done by any human who even may not be familiar with the compatibility module and may not know anything which compatibility tests are used. If we include adding pom file to post-release process all existing tests will automatically detect new version and next TC run will immediately show to us results. If there are some compatibility problems during development new version TC run will show it. At the current approach, compatibility tests are out of focus and we may not see potential issues. >> This is not about usability, it's about the framework's internals At the current approach, a developer should take care of dependency management for the old versions. E.g. if you look at IgnitePKIndexesMigrationToUnwrapPkTest you can see "magic" H2 dependency version. I still don't understand why this version was explicitly added and not just inherited from ignite-indexing. If we delegate all dependency management to Maven itself it will simplify tests development. Tests will look much simpler. >> Please, describe the issue in more details? Suppose you have a closure that invokes some deprecated method which will be removed in next release. After method removal, such tests will be broken. At the current framework, I can't see a resolution of that problem. In a new framework, you can keep the old version of closure, mark it with special annotation and place to separate module with old Ignite version dependency. This closure will be compiled using an old version of Ignite and the user can still run it on the old version. вт, 19 февр. 2019 г. в 21:44, Vyacheslav Daradur <daradu...@gmail.com>: > Hi, Pavel! > > First of all, I'd like to clarify that the Compatibility Testing > Framework was designed to work with a cluster of multi-version nodes. > The main idea is to run a test to verify backward compatibility or do > some kind of rolling upgrades. > > It's not about persistence compatibility, but actually, it is used for > this now. > > >> 1) It's uncomfortable to add and support new versions of Ignite product. > > I'm not sure what is a problem here? > As I understand you suggest to do similar things - adding new pom.xml > when new a version is released. Pay attention that not all tests > should be tested on all versions, some test may want to test specific > versions each other. > > >> 2) Manual maven dependency through java api. > > This is not about usability, it's about the framework's internals. If > you have issues let's fix it and improve approach if needed. > > >> 3) It doesn't cover the case when some code which works on the current > version will not work on older versions due to compile/runtime > incompatibility. > > Please, describe the issue in more details? > > On Tue, Feb 19, 2019 at 7:55 PM Pavel Kovalenko <jokse...@gmail.com> > wrote: > > > > Anton, > > > > >> What about the current compatibility framework? > > Current compatibility framework will be removed after final adjusting to > > new framework. > > > > >> Could you please share examples for each feature you mentioned? > > You can see example in PR e.g. file > > modules/compatibility/ignite-versions/2.1.0/pom.xml > > < > https://github.com/apache/ignite/pull/5974/files#diff-3c44ef223c31de9ff1e10bd7699cbcdc > > > > > > I think such representing of Ignite product version is more cleaner than > > existing approach with Java classes with dependencies and manual > dependecy > > management. > > > > >> Anyway. I don't like the idea to implement something new instead of > > improving the existing. > > If you have concrete proposals how to improve current framework please > > share it. > > > > > > вт, 19 февр. 2019 г. в 19:11, Anton Vinogradov <a...@apache.org>: > > > > > +5,327 −59 > > > What about the current compatibility framework? > > > I see no removal or updates. > > > > > > >> Each new version is represented by a single pom > > > Sound not good. > > > Could you please share examples for each feature you mentioned? > > > > > > Anyway. I don't like the idea to implement something new instead of > > > improving the existing. > > > > > > On Tue, Feb 19, 2019 at 6:48 PM Pavel Kovalenko <jokse...@gmail.com> > > > wrote: > > > > > > > Igniters, > > > > > > > > I would like to start a discussion about replacement existing > > > > persistence compatibility test framework with the newer version. > > > > The main purpose of that action is simplifying compatibility tests > > > > development and support. > > > > > > > > The current version of the test framework has 3 disadvantages: > > > > 1) It's uncomfortable to add and support new versions of Ignite > product. > > > > When a new version has released a developer must manually add new > test > > > > methods to all existing test suites to support the new product > version. > > > > 2) Manual maven dependency through java api. > > > > If a new version of Ignite product has some specific dependency > version > > > > (like H2) a developer should take care about and manually cover this > case > > > > using java api. > > > > 3) It doesn't cover the case when some code which works on the > current > > > > version will not work on older versions due to compile/runtime > > > > incompatibility. > > > > > > > > A new version of the framework that is under development right now > [1] > > > > doesn't have such problems. > > > > Here is a list of key features and possibilities: > > > > > > > > 1) One codebase - multiple versions support. > > > > The key feature of the new framework is significant simplifying > adding > > > and > > > > supporting the new versions of Ignite product. > > > > Each new version is represented by a single pom file that contains > Ignite > > > > dependencies of specific version (core, indexing, etc.). All > > > subdependecies > > > > like H2 are covered by Maven automatically. > > > > To add a new version for compatibility a developer just need to add > a new > > > > pom file with a new version of Ignite. All existing tests will > > > > automatically detect the new version and will run tests against this > > > > version without additional changes. This feature covers p. 1 and 2 > of the > > > > existing framework. > > > > 2) Unified API to access and run code on old and new versions of > Ignite. > > > > Each of Ignite instance is represented by remote JVM with a single > point > > > of > > > > interaction - running abstract closures. Each closure is just a class > > > which > > > > implement a "runner interface". Each closure object is serialized to > a > > > file > > > > through Xstream library and executed on remote Ignite JVM. This > approach > > > > gives unified access to interact with both old and new versions of > Ignite > > > > and simplifies overall tests development. > > > > 3) Ignite versions support for closures. Sometimes a closure > couldn't be > > > > run on newer or older Ignite version due to compile/runtime > > > incompatibility > > > > of that code. To resolve this problem a special annotation has > > > introduced. > > > > This annotation named as "@Since" contains minimal possible Ignite > > > product > > > > version where annotated closure can be compiled and run. Such > closures > > > are > > > > processed on Ignite versions compile phase and this makes it > possible to > > > > use one closure for multiple Ignite versions without additional code > > > > changes. Closures and versioning resolve p. 3 of the existing > framework. > > > > > > > > [1] - https://issues.apache.org/jira/browse/IGNITE-11133 > > > > > > > > I would like to hear any opinions about the new framework. > > > > > > > > > > > -- > Best Regards, Vyacheslav D. >
