I think that's a very good point Radim. But on the other hand I think we may treat it as an implementation detail. The TCK must run without Infinispan server.
I also believe it would be beneficial to define as least 2-3 levels of support, e.g. basic (only basic operations, no consistent hash support), advanced (all features). This way we could validate some more exotic clients such as Go (https://github.com/ugol/infinispan-go). And in my opinion, a final version of this document go to infinispan design repository: https://github.com/infinispan/infinispan-designs Apart from that, that's huge effort and really great job! On Tue, Apr 11, 2017 at 5:05 PM Radim Vansa <rva...@redhat.com> wrote: > Since these tests use real server(s), many of them test not only the > client behaviour (generating correct commands according to the > protocol), but server, too. While this is practical (we need to test > server somehow, too), there's nothing all the tests across languages > will have physically in common and all comparison is prone to human error. > > If we want to test various implementations of the client, maybe it would > make sense to give the clients a fake server that will have just a > scenario of expected commands to receive and pre-defined responses. We > could use audit log to generate such scenario based on the actual Java > tests. > > But then we'd have to test the actual behaviour on server, and we'd need > a way to issue the commands. > > Just my 2c > > Radim > > On 04/11/2017 02:33 PM, Martin Gencur wrote: > > Hello all, > > we have been working on https://issues.jboss.org/browse/ISPN-7120. > > > > Anna has finished the first step from the JIRA - collecting information > > about tests in the Java HotRod client test suite (including server > > integration tests) and it is now prepared for wider review. > > > > She created a spreadsheet [1]. The spread sheet includes for each Java > > test its name, the suggested target package in the TCK, whether to > > include it in the TCK or not, and some other notes. The suggested > > package also poses grouping for the tests (e.g. tck.query, tck.near, > > tck.xsite, ...) > > > > Let me add that right now the goal is not to create a true TCK [2]. The > > goal is to make sure that all implementations of the HotRod protocol > > have sufficient test coverage and possibly the same server side of the > > client-server test (including the server version and configuration). > > > > What are the next step? > > > > * Please review the list (at least a quick look) and see if some of the > > tests which are NOT suggested for the TCK should be added or vice versa. > > * I suppose the next step would then be to check other implementations > > (C#, C++, NodeJS, ..) and identify tests which are missing there (there > > will surely be some). > > * Gradually implement the missing tests in the other implementations > > Note: Here we should ensure that the server is configured in the same > > way for all implementations. One way to achieve this (thanks Anna for > > suggestion!) is to have a shell/batch scripts for CLI which would be > > executed before the tests. This can probably be done for all impls. and > > both UNIX/WINDOWS. I also realize that my PR for ISPN [3] becomes > > useless because it uses Creaper (Java) and we need a language-neutral > > solution for configuring the server. > > > > Some other notes: > > * there are some duplicated tests in hotrod-client and server > > integration test suites, in this case it probably makes sense to only > > include in the TCK the server integration test > > * tests from the hotrod-client module which are supposed to be part of > > the TCK should be copied to the server integration test suite one day > > (possibly later) > > > > Please let us know what you think. > > > > Thanks, > > Martin > > > > > > [1] > > > https://docs.google.com/spreadsheets/d/1bZBBi5m4oLL4lBTZhdRbIC_EA0giQNDZWzFNPWrF5G4/edit#gid=0 > > [2] https://en.wikipedia.org/wiki/Technology_Compatibility_Kit > > [3] https://github.com/infinispan/infinispan/pull/5012 > > _______________________________________________ > > infinispan-dev mailing list > > infinispan-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > -- > Radim Vansa <rva...@redhat.com> > JBoss Performance Team > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > -- SEBASTIAN ŁASKAWIEC INFINISPAN DEVELOPER Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig>
_______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
