Hi, > for direction on how to test SMW. I don't know a thing about the automated > testing, so I'd appreciate guidance on how to get started.
Let me try to elaborate a bit on this topic. Our automated testing approach (as explained in the cited document you linked) contains two varying approaches. The first one is to use PHPUnit directly, which is to test an individual class for its consistency with the interface or the contract it is expected to fulfill. While this approach is the preferred method of adding automated scripts it can be difficult and sometimes impossible (in terms of time/effort vs. result) to reach a conclusive implementation that would test the boundaries of a class (this is especially the case for most legacy code we have in SMW). > That said, there's not much said about how to manually test SMW. When you > want testing on a new branch of SMW, do you have a checklist of basic > functions that should be tested? The second approach is a bit more user friendly and is based on a JSON scripted test that generates assertions based on the content described by a test file [0]. The advantage of this method is that those tests are to a certain degree closer to an input a user would made by comparing the output that is expected from either a parser or a result printer. For example, if you pick [1] it should be fairly obvious what is tested and further allowing a "normal" user to copy the test case (the data not the file) into a wiki with the expectation to replicate the same results as specified by the test. It is not exactly a checklist but those test follow a certain structure and were created because of an issue a user had or they seemed to cover a general use case. Those tests in [0] do use a real DB connection (are not mocked and more resources intensive as the whole MW setup is used) and are executed on each PR against different DB settings (mysql, sqlite, postgres, SPARQLStore etc.) which gives a good indication about the supported platforms. The more we have of such scripted integration tests the closer is our predication that the system behaves as expected. Of course, some things can't be tested which has been proven in the past by issues that appear in tandem with MediaWiki. [2] is a typical case that no matter how much you test it is unlikely to be caught by any test (due to the constraints the environmental setting dictates). PS : We don't do browser tests as that is quite tedious to setup and run without employing to much infrastructure. Maybe running against a webdriver [3] could be an alternative for testing browser output. [0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/tree/master/tests/phpunit/Integration/ByJsonScript/Fixtures [1] https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/tests/phpunit/Integration/ByJsonScript/Fixtures/p-0204%20%5B%23870%2Cen%5D%20set%20parser%20with%20error%20output.json [2] https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1201#issuecomment-147389206 [3] https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/201 Cheers On 10/28/15, Jeroen De Dauw <jeroended...@gmail.com> wrote: > Hey Daren, > > What exactly are you trying to achieve? Do you want to verify SMW runs > correctly on your system? Do you want to contribute to the project by > testing? Something else? > > We do not have a list of scenarios to manually test, as this is not a > maintainable approach. Instead we have several thousand scripted tests > which we can run automatically. If you want to contribute to the project, > manually testing is not the way to go. With some exceptions, for which > which we have a testers group. You can join this group and be notified when > one of the developers wants some manual testing to be done, which will > include a description of what to do. If you want to join this group, please > send me your GitHub username. > > Cheers > > -- > Jeroen De Dauw - http://www.bn2vs.com > Software craftsmanship advocate > Developer at Wikimedia Germany > ~=[,,_,,]:3 > ------------------------------------------------------------------------------ _______________________________________________ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
