Thanks Jeroen and James. I'll read through this several times and see if I can figure out how to do some PHPUnit tests. I'm actually already added to the SMW Testers team, but I have to admit I haven't really helped since I haven't (yet) learned how to use PHPUnit
While I do want to contribute back to SMW directly, I'm currently focused on a project with James Montalvo where we can install a SMW wiki farm with VE, Elasticsearch, and all the other bells and whistles with a few simple commands on a barebones CentOS install. Each time we add to or modify these scripts, we have been running manual tests. We will definitely learn how to use PHPUnit to add automated testing to our project. Wish we could join you in Barcelona! Daren On Wed, Oct 28, 2015 at 8:00 AM, James HK <jamesin.hongkon...@gmail.com> wrote: > Hi, > > > for direction on how to test SMW. I don't know a thing about the > automated > > testing, so I'd appreciate guidance on how to get started. > > Let me try to elaborate a bit on this topic. Our automated testing > approach (as explained in the cited document you linked) contains two > varying approaches. > > The first one is to use PHPUnit directly, which is to test an > individual class for its consistency with the interface or the > contract it is expected to fulfill. While this approach is the > preferred method of adding automated scripts it can be difficult and > sometimes impossible (in terms of time/effort vs. result) to reach a > conclusive implementation that would test the boundaries of a class > (this is especially the case for most legacy code we have in SMW). > > > That said, there's not much said about how to manually test SMW. When you > > want testing on a new branch of SMW, do you have a checklist of basic > > functions that should be tested? > > The second approach is a bit more user friendly and is based on a JSON > scripted test that generates assertions based on the content described > by a test file [0]. The advantage of this method is that those tests > are to a certain degree closer to an input a user would made by > comparing the output that is expected from either a parser or a result > printer. > > For example, if you pick [1] it should be fairly obvious what is > tested and further allowing a "normal" user to copy the test case (the > data not the file) into a wiki with the expectation to replicate the > same results as specified by the test. > > It is not exactly a checklist but those test follow a certain > structure and were created because of an issue a user had or they > seemed to cover a general use case. > > Those tests in [0] do use a real DB connection (are not mocked and > more resources intensive as the whole MW setup is used) and are > executed on each PR against different DB settings (mysql, sqlite, > postgres, SPARQLStore etc.) which gives a good indication about the > supported platforms. > > The more we have of such scripted integration tests the closer is our > predication that the system behaves as expected. > > Of course, some things can't be tested which has been proven in the > past by issues that appear in tandem with MediaWiki. [2] is a typical > case that no matter how much you test it is unlikely to be caught by > any test (due to the constraints the environmental setting dictates). > > PS : We don't do browser tests as that is quite tedious to setup and > run without employing to much infrastructure. Maybe running against a > webdriver [3] could be an alternative for testing browser output. > > [0] > https://github.com/SemanticMediaWiki/SemanticMediaWiki/tree/master/tests/phpunit/Integration/ByJsonScript/Fixtures > > [1] > https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/tests/phpunit/Integration/ByJsonScript/Fixtures/p-0204%20%5B%23870%2Cen%5D%20set%20parser%20with%20error%20output.json > > [2] > https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1201#issuecomment-147389206 > > [3] https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/201 > > Cheers > > On 10/28/15, Jeroen De Dauw <jeroended...@gmail.com> wrote: > > Hey Daren, > > > > What exactly are you trying to achieve? Do you want to verify SMW runs > > correctly on your system? Do you want to contribute to the project by > > testing? Something else? > > > > We do not have a list of scenarios to manually test, as this is not a > > maintainable approach. Instead we have several thousand scripted tests > > which we can run automatically. If you want to contribute to the project, > > manually testing is not the way to go. With some exceptions, for which > > which we have a testers group. You can join this group and be notified > when > > one of the developers wants some manual testing to be done, which will > > include a description of what to do. If you want to join this group, > please > > send me your GitHub username. > > > > Cheers > > > > -- > > Jeroen De Dauw - http://www.bn2vs.com > > Software craftsmanship advocate > > Developer at Wikimedia Germany > > ~=[,,_,,]:3 > > > -- __________________ http://enterprisemediawiki.org http://mixcloud.com/darenwelsh http://www.beatportfolio.com
------------------------------------------------------------------------------
_______________________________________________ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
