[Wikitech-l] Selenium testing framework - Fix available in Revision 67575?
Hi, I am following up on a thread from May 27th where there is a bug in run Selenium Tests. I was wondering if the fix for Run Selenium Tests is available in Revision 67575 found at http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/PagedTiffHandler/tests/ http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/PagedTiffHandler/tests/%20. In the meantime I will keep all the local configuration data in the RunSeleniumTests as instructed.* *I was about to copy and paste the php tests in Selenium IDE to see what would happen and if I could run from there. Michelle Knight Message: 6 Date: Thu, 27 May 2010 17:47:48 + (UTC) From: Dan Nessett dness...@yahoo.com Subject: Re: [Wikitech-l] Selenium testing framework- Firefox browsers compatible To: wikitech-l@lists.wikimedia.org Message-ID: htmb83$ru...@dough.gmane.org Content-Type: text/plain; charset=UTF-8 On Wed, 26 May 2010 17:11:33 -0700, Michelle Knight wrote: Hi Dan, There is a list of browsers compatible with Selenium (See http://seleniumhq.org/about/http://seleniumhq.org/about/platforms.html#browsers platforms.html#browsershttp://seleniumhq.org/about/platforms.html#browsers). The page states that Selenium works with Firefox 2+ when a Linux OS is used (I think Ubuntu would fall under this category ). I am using Firefox 3.5.9 on Ubuntu 9.10 . I have been finishing another project (my grandfather visited me in Oregon from Ohio) and have not played with the at the Selenium Framework since May 14th. I will let you know if I see the error messages. Michelle Knight Message: 5 Date: Tue, 18 May 2010 17:44:03 + (UTC) From: Dan Nessett dness...@yahoo.com Subject: Re: [Wikitech-l] Selenium testing framework To: wikitech-l@lists.wikimedia.org Message-ID: hsujl3$v7...@dough.gmane.org Content-Type: text/plain; charset=UTF-8 On Tue, 18 May 2010 19:27:38 +0200, Markus Glaser wrote: Hi Dan, I had these error messages once when I used Firefox 3.6 for testing. Until recently, Selenium did not support this browser. Apparently now they do, but I did not have a chance to test this yet. So the solution for me was to point Selenium to a Firefox 3.5. Cheers, Markus My OS is Ubuntu 8.04. The version of Firefox is 3.0.19. Since Ubuntu automatically updates versions of its software, I assume this is the most up-to-date. Is there a list of browser versions compatible with selenium? *Thanks for the pointer to the list, Michelle. As it turned out there was bug in RunSeleniumTests that accessed global data before LocalSeleniumSettings was included. Markus has fixed this problem and is testing it before checking it in to the repository. Before this fix is available, you should put all your local configuration data in RunSeleniumTests. Regards, Dan* -- -- Dan Nessett Regards, Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework- Firefox browsers compatible
On Wed, 26 May 2010 17:11:33 -0700, Michelle Knight wrote: Hi Dan, There is a list of browsers compatible with Selenium (See http://seleniumhq.org/about/platforms.html#browsers ). The page states that Selenium works with Firefox 2+ when a Linux OS is used (I think Ubuntu would fall under this category ). I am using Firefox 3.5.9 on Ubuntu 9.10 . I have been finishing another project (my grandfather visited me in Oregon from Ohio) and have not played with the at the Selenium Framework since May 14th. I will let you know if I see the error messages. Michelle Knight Message: 5 Date: Tue, 18 May 2010 17:44:03 + (UTC) From: Dan Nessett dness...@yahoo.com Subject: Re: [Wikitech-l] Selenium testing framework To: wikitech-l@lists.wikimedia.org Message-ID: hsujl3$v7...@dough.gmane.org Content-Type: text/plain; charset=UTF-8 On Tue, 18 May 2010 19:27:38 +0200, Markus Glaser wrote: Hi Dan, I had these error messages once when I used Firefox 3.6 for testing. Until recently, Selenium did not support this browser. Apparently now they do, but I did not have a chance to test this yet. So the solution for me was to point Selenium to a Firefox 3.5. Cheers, Markus My OS is Ubuntu 8.04. The version of Firefox is 3.0.19. Since Ubuntu automatically updates versions of its software, I assume this is the most up-to-date. Is there a list of browser versions compatible with selenium? Thanks for the pointer to the list, Michelle. As it turned out there was bug in RunSeleniumTests that accessed global data before LocalSeleniumSettings was included. Markus has fixed this problem and is testing it before checking it in to the repository. Before this fix is available, you should put all your local configuration data in RunSeleniumTests. Regards, Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework- Firefox browsers compatible
Hoi, Selenium is not compatible with Ubuntu.. Thanks, GerardM On 27 May 2010 02:11, Michelle Knight mknight113...@gmail.com wrote: Hi Dan, There is a list of browsers compatible with Selenium (See http://seleniumhq.org/about/platforms.html#browsers ). The page states that Selenium works with Firefox 2+ when a Linux OS is used (I think Ubuntu would fall under this category ). I am using Firefox 3.5.9 on Ubuntu 9.10 . I have been finishing another project (my grandfather visited me in Oregon from Ohio) and have not played with the at the Selenium Framework since May 14th. I will let you know if I see the error messages. Michelle Knight Message: 5 Date: Tue, 18 May 2010 17:44:03 + (UTC) From: Dan Nessett dness...@yahoo.com Subject: Re: [Wikitech-l] Selenium testing framework To: wikitech-l@lists.wikimedia.org Message-ID: hsujl3$v7...@dough.gmane.org Content-Type: text/plain; charset=UTF-8 On Tue, 18 May 2010 19:27:38 +0200, Markus Glaser wrote: Hi Dan, I had these error messages once when I used Firefox 3.6 for testing. Until recently, Selenium did not support this browser. Apparently now they do, but I did not have a chance to test this yet. So the solution for me was to point Selenium to a Firefox 3.5. Cheers, Markus My OS is Ubuntu 8.04. The version of Firefox is 3.0.19. Since Ubuntu automatically updates versions of its software, I assume this is the most up-to-date. Is there a list of browser versions compatible with selenium? ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework- Firefox browsers compatible
Gerard Meijssen schrieb: Hoi, Selenium is not compatible with Ubuntu.. Thanks, GerardM works fine for me -- daniel ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework- Firefox browsers compatible
Hoi, I read some more about this.. It turns out that even though I asked for an update, the software did not update. I just upgraded from release 1.02 to 1.07 and now it works. The documentation states that from 1.05 updates are pushed. Thanks, GerardM On 27 May 2010 21:21, Daniel Kinzler dan...@brightbyte.de wrote: Gerard Meijssen schrieb: Hoi, Selenium is not compatible with Ubuntu.. Thanks, GerardM works fine for me -- daniel ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework- Firefox browsers compatible
Hi Dan, There is a list of browsers compatible with Selenium (See http://seleniumhq.org/about/platforms.html#browsers ). The page states that Selenium works with Firefox 2+ when a Linux OS is used (I think Ubuntu would fall under this category ). I am using Firefox 3.5.9 on Ubuntu 9.10 . I have been finishing another project (my grandfather visited me in Oregon from Ohio) and have not played with the at the Selenium Framework since May 14th. I will let you know if I see the error messages. Michelle Knight Message: 5 Date: Tue, 18 May 2010 17:44:03 + (UTC) From: Dan Nessett dness...@yahoo.com Subject: Re: [Wikitech-l] Selenium testing framework To: wikitech-l@lists.wikimedia.org Message-ID: hsujl3$v7...@dough.gmane.org Content-Type: text/plain; charset=UTF-8 On Tue, 18 May 2010 19:27:38 +0200, Markus Glaser wrote: Hi Dan, I had these error messages once when I used Firefox 3.6 for testing. Until recently, Selenium did not support this browser. Apparently now they do, but I did not have a chance to test this yet. So the solution for me was to point Selenium to a Firefox 3.5. Cheers, Markus My OS is Ubuntu 8.04. The version of Firefox is 3.0.19. Since Ubuntu automatically updates versions of its software, I assume this is the most up-to-date. Is there a list of browser versions compatible with selenium? ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework: troubleshooting tips
Hi Dan and Markus I have added some troubleshooting tips, based on notes I took during the Friday May 14 meeting, to the Selenium Framework page: http://www.mediawiki.org/wiki/SeleniumFramework#Working_example I think it has the tip about port . My intent was to add information for problem solving and notes as I was using Selenium. If this is something that needs to be updated, then let me know. Also let me know if I need to reference the person who suggested the tip on the page (to give appropriate credit). I will be copying the php framework into Selenium IDE and giving it a try. Michelle Knight --- Hi Markus, Despite my initial problems getting the Selenium Framework to run, I think it is a great start. Now that I have the PagedTiffHandler working, here is some feedback on the current framework: + When I svn up ../tests (or any ancestor directory), the local changes + I make to RunSeleniumTests cause a local conflict error. Eventually, many of the configuration changes I made should appear in LocalSeleniumSettings, but it isn't clear that is possible for all of them. For example, I change the commented out set_include_path to include my local PHP/PEAR directory. Can this be set in LocalSeleniumSettings? Another difference is the include_once() for each test suite. Is it possible to move these into LocalSeleniumSettings? + It appears there is no way to tell RunSeleniumTests to use a selenium server port other than . It would be useful to have a -port parameter on RunSeleniumServer for this. For example, if there are multiple developers working on the same machine, they probably need to use different selenium servers differentiated by different port numbers. I don't mind working on both of these issues, but since you are the original architect of the framework, it is probably best for you to comment on them first and perhaps suggest what you consider to be the best approach to their resolution. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Sun, 02 May 2010 02:20:06 +0200, Markus Glaser wrote: Hi everybody, at the Wkimedia Developers' Workshop, I introduced a Selenium testing framework for MediaWiki. Since it has now been promoted to maintenance/tests, I have provided some initial information it on http://www.mediawiki.org/wiki/SeleniumFramework . I would be very happy about comments and ideas for further improvement. Also, if you intend to use the framework for your tests, please let me know. I will be happy to assist. Regards, Markus Glaser __ Social Web Technologien Leiter Softwareentwicklung Hallo Welt! - Medienwerkstatt GmbH __ Untere Bachgasse 15 93047 Regensburg Tel. +49 (0) 941 - 56 95 94 - 92 www.hallowelt.bizhttp://www.hallowelt.biz/ gla...@hallowelt.bizmailto:gla...@hallowelt.biz Sitz: Regensburg Handelsregister: HRB 10467 E.USt.Nr.: DE 253050833 Geschäftsführer: Anja Ebersbach, Markus Glaser, Dr. Richard Heigl, Radovan Kubani __ Hi Markus, Despite my initial problems getting the Selenium Framework to run, I think it is a great start. Now that I have the PagedTiffHandler working, here is some feedback on the current framework: + When I svn up ../tests (or any ancestor directory), the local changes I make to RunSeleniumTests cause a local conflict error. Eventually, many of the configuration changes I made should appear in LocalSeleniumSettings, but it isn't clear that is possible for all of them. For example, I change the commented out set_include_path to include my local PHP/PEAR directory. Can this be set in LocalSeleniumSettings? Another difference is the include_once() for each test suite. Is it possible to move these into LocalSeleniumSettings? + It appears there is no way to tell RunSeleniumTests to use a selenium server port other than . It would be useful to have a -port parameter on RunSeleniumServer for this. For example, if there are multiple developers working on the same machine, they probably need to use different selenium servers differentiated by different port numbers. I don't mind working on both of these issues, but since you are the original architect of the framework, it is probably best for you to comment on them first and perhaps suggest what you consider to be the best approach to their resolution. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
Hi Dan, thanks for your feedback, I am very happy that you finally managed to run the tests. I also appreciate your suggestions. Adding a port parameter is surely useful. As to including the tests in LocalSeleniumSettings.php, I will have a look into that. I assume that we have to change the order standards are set in RunSeleniumTests.php, since the included tests depend on the PEAR libraries, which in turn depend on the PEAR path set above. In the end, I would like to include the tests by a command line parameter, thus making the framework more flexible. However, I can see that hardcoding the tests in the settings file will save some typing work. If you want to make the changes yourself, please feel free to do so. I am planning to provide an update of the framework with a working example by tomorrow evening (about this time), and I will include your suggestions as well if you have not done so already :) Regards, Markus -Ursprüngliche Nachricht- Von: wikitech-l-boun...@lists.wikimedia.org [mailto:wikitech-l-boun...@lists.wikimedia.org] Im Auftrag von Dan Nessett Gesendet: Montag, 24. Mai 2010 20:08 An: wikitech-l@lists.wikimedia.org Betreff: Re: [Wikitech-l] Selenium testing framework On Sun, 02 May 2010 02:20:06 +0200, Markus Glaser wrote: Hi everybody, at the Wkimedia Developers' Workshop, I introduced a Selenium testing framework for MediaWiki. Since it has now been promoted to maintenance/tests, I have provided some initial information it on http://www.mediawiki.org/wiki/SeleniumFramework . I would be very happy about comments and ideas for further improvement. Also, if you intend to use the framework for your tests, please let me know. I will be happy to assist. Regards, Markus Glaser __ Social Web Technologien Leiter Softwareentwicklung Hallo Welt! - Medienwerkstatt GmbH __ Untere Bachgasse 15 93047 Regensburg Tel. +49 (0) 941 - 56 95 94 - 92 www.hallowelt.bizhttp://www.hallowelt.biz/ gla...@hallowelt.bizmailto:gla...@hallowelt.biz Sitz: Regensburg Handelsregister: HRB 10467 E.USt.Nr.: DE 253050833 Geschäftsführer: Anja Ebersbach, Markus Glaser, Dr. Richard Heigl, Radovan Kubani __ Hi Markus, Despite my initial problems getting the Selenium Framework to run, I think it is a great start. Now that I have the PagedTiffHandler working, here is some feedback on the current framework: + When I svn up ../tests (or any ancestor directory), the local changes + I make to RunSeleniumTests cause a local conflict error. Eventually, many of the configuration changes I made should appear in LocalSeleniumSettings, but it isn't clear that is possible for all of them. For example, I change the commented out set_include_path to include my local PHP/PEAR directory. Can this be set in LocalSeleniumSettings? Another difference is the include_once() for each test suite. Is it possible to move these into LocalSeleniumSettings? + It appears there is no way to tell RunSeleniumTests to use a selenium server port other than . It would be useful to have a -port parameter on RunSeleniumServer for this. For example, if there are multiple developers working on the same machine, they probably need to use different selenium servers differentiated by different port numbers. I don't mind working on both of these issues, but since you are the original architect of the framework, it is probably best for you to comment on them first and perhaps suggest what you consider to be the best approach to their resolution. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Mon, 17 May 2010 20:16:35 +, Dan Nessett wrote: On Mon, 17 May 2010 19:11:21 +, Dan Nessett wrote: During the meeting last Friday, someone (I sorry, I don't remember who) mentioned he had created a test that runs with the currently checked in selenium code. Is that test code available somewhere (it doesn't appear to be in the current revision)? I found the answer. On the SeleniumFramework page is a pointer to a worked example (see: http://www.mediawiki.org/wiki/ SeleniumFramework#Working_example). The instructions for getting the tests to work aren't totally transparent. The test file you include is: ../phase3/extensions/PagedTiffHandler/selenium/ PagedTiffHandler_tests.php (Not: ../phase3/extensions/PagedTiffHandler/tests/ PagedTiffHandlerTest.php) Also, the instructions in SOURCES.txt specify getting all of the test images from: http://www.libtiff.org/images.html But, when accessing the URL supplied on that page for the images (ftp:// ftp.remotesensing.org/pub/libtiff/pics-3.6.1.tar.gz) a FILE NOT FOUND error is returned. There is a new version of the pics file in ..libtiff, but they do not contain the correct images. The correct URL is: ftp:// ftp.remotesensing.org/pub/libtiff/old/pics-3.6.1.tar.gz. However, this tar file does not include the images required by the PagedTiffHandler tests. I thought I would apprise readers of this thread that I have the PagedTiffHandler example working. The problem turned out to be a bug in the Framework that meant LocalSeleniumSettings wasn't being read at the correct point in the startup sequence. According to Markus, he has fixed this bug and is now testing it. I presume he will let us know when the fix is committed. If you want to run the example before the fix is available, just make all your configuration settings to RunSeleniumTests.php. Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Tue, 18 May 2010 01:04:01 +0200, Markus Glaser wrote: Hi Dan, the test fails at checking the prerequisites. It tries to load the image page and looks for a specific div element which is not present if the image was not uploaded correctly (id=filetoc). This might have changed across the versions of MediaWiki. Did you install the PagedTiffHandler extension? It depends on ImageMagick, so it might have rejected the upload. Although then it should have produced an error message ;) So the other question is, which MediaWiki version do you run the tests on? Regards, Markus Hi Markus, Some further information. I originally uploaded Multipage.tiff before the extension was installed. I thought this might be the problem, so I uploaded it again after the extension was available. However, this did not solve the problem. Also, I am getting the following error from the selenium-server: 08:41:10.344 INFO - Got result: ERROR Server Exception: sessionId led to start new browser session: org.openqa.selenium.server.browserlaunchers.InvalidBrowserExecutableException: The specified path to the browser executable is invalid. doesn't exist; perhaps this session was already stopped? on session led to start new browser session: org.openqa.selenium.server.browserlaunchers.InvalidBrowserExecutableException: The specified path to the browser executable is invalid. 08:41:10.347 INFO - Command request: testComplete[, ] on session led to start new browser session: org.openqa.selenium.server.browserlaunchers.InvalidBrowserExecutableException: The specified path to the browser executable is invalid. 08:41:10.347 INFO - Got result: OK on session led to start new browser session: org.openqa.selenium.server.browserlaunchers.InvalidBrowserExecutableException: The specified path to the browser executable is invalid. I am using a LocalSeleniumSettings.php with the following parameters: // Hostname of selenium server $wgSeleniumTestsSeleniumHost = 'http://localhost'; // URL of the wiki to be tested. $wgSeleniumTestsWikiUrl = 'http://localhost'; // Wiki login. Used by Selenium to log onto the wiki $wgSeleniumTestsWikiUser = 'Wikiadmin'; $wgSeleniumTestsWikiPassword = 'Wikiadminpw'; // Use the *chrome handler in order to be able to test file uploads $wgSeleniumTestsBrowsers['firefox'] = '*firefox /usr/bin/firefox'; $wgSeleniumTestsBrowsers['ff-chrome'] = '*chrome /usr/bin/firefox'; // Actually, use this browser $wgSeleniumTestsUseBrowser = 'ff-chrome'; Regards, -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
Hi Dan, I had these error messages once when I used Firefox 3.6 for testing. Until recently, Selenium did not support this browser. Apparently now they do, but I did not have a chance to test this yet. So the solution for me was to point Selenium to a Firefox 3.5. Cheers, Markus ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Tue, 18 May 2010 19:27:38 +0200, Markus Glaser wrote: Hi Dan, I had these error messages once when I used Firefox 3.6 for testing. Until recently, Selenium did not support this browser. Apparently now they do, but I did not have a chance to test this yet. So the solution for me was to point Selenium to a Firefox 3.5. Cheers, Markus My OS is Ubuntu 8.04. The version of Firefox is 3.0.19. Since Ubuntu automatically updates versions of its software, I assume this is the most up-to-date. Is there a list of browser versions compatible with selenium? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Tue, 18 May 2010 19:27:38 +0200, Markus Glaser wrote: Hi Dan, I had these error messages once when I used Firefox 3.6 for testing. Until recently, Selenium did not support this browser. Apparently now they do, but I did not have a chance to test this yet. So the solution for me was to point Selenium to a Firefox 3.5. Cheers, Markus Hi Markus, I thought it might be best to move this discussion off-line for a bit until we get the problems sorted out and then post the solution(s) to wikitech-l. This thread is getting fairly long and is getting into fairly complex issues. I tried emailing you at the address shown in your posts, but the email was returned as undeliverable. My email address is dness...@yahoo.com. If you think taking the issue off-line while we sort it out is a good thing to do, then email me from some address that I can use and I will update you on the status of my attempts to get PagedTiffHandler_tests.php to work. As a teaser, it appears there is a problem with the sequence of processing vis-a-vis LocalSettings and LocalSeleniumSettings Cheers, Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
During the meeting last Friday, someone (I sorry, I don't remember who) mentioned he had created a test that runs with the currently checked in selenium code. Is that test code available somewhere (it doesn't appear to be in the current revision)? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
During the meeting last Friday, someone (I sorry, I don't remember who) mentioned he had created a test that runs with the currently checked in selenium code. Is that test code available somewhere (it doesn't appear to be in the current revision)? Markus Glaser has Selenium tests checked in with his PagedTiffHandler extension that run properly with the framework. See: http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/PagedTiffHandler/ selenium/ V/r, Ryan Lane ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Mon, 17 May 2010 19:11:21 +, Dan Nessett wrote: During the meeting last Friday, someone (I sorry, I don't remember who) mentioned he had created a test that runs with the currently checked in selenium code. Is that test code available somewhere (it doesn't appear to be in the current revision)? I found the answer. On the SeleniumFramework page is a pointer to a worked example (see: http://www.mediawiki.org/wiki/ SeleniumFramework#Working_example). The instructions for getting the tests to work aren't totally transparent. The test file you include is: ../phase3/extensions/PagedTiffHandler/selenium/PagedTiffHandler_tests.php (Not: ../phase3/extensions/PagedTiffHandler/tests/ PagedTiffHandlerTest.php) Also, the instructions in SOURCES.txt specify getting all of the test images from: http://www.libtiff.org/images.html But, when accessing the URL supplied on that page for the images (ftp:// ftp.remotesensing.org/pub/libtiff/pics-3.6.1.tar.gz) a FILE NOT FOUND error is returned. There is a new version of the pics file in ..libtiff, but they do not contain the correct images. The correct URL is: ftp:// ftp.remotesensing.org/pub/libtiff/old/pics-3.6.1.tar.gz. However, this tar file does not include the images required by the PagedTiffHandler tests. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Mon, 17 May 2010 22:54:38 +0200, Markus Glaser wrote: Hi Dan, will provide a working example with no need to include any extensions in the course of this week. In the meantime, you might want to make sure that $wgSeleniumTiffTestUploads = false; in PagedTiffHandler_tests.php. Then, the test will not try to upload any of the pictures from libtiff. In order for the tests to succeed, you need to upload Multipage.tiff into the wiki. If there are any images missing, please let me know and I will send them to you. Actually, I didn't want to check in a third-party archive into the svn because of copyright considerations. The images seem to be public domain, but to me, it was not totally clear, whether they are. Are there any policies regarding this case? I assume, when there are more tests, especially with file uploads, the issue might arise again. Cheers, Markus Thanks Markus, $wgSeleniumTiffTestUploads does indeed equal false. I was failing on the upload of Multipage.tiff until I added 'tiff' to $wgFileExtensions. Now I am failing because allChecksOk is false. It appears this happens on line 32 of PagedTiffHandler_tests.php on the statement: if ($source != 'filetoc') $this-allChecksOk = false; I'm not an image expert, so I don't know why this is happening. Regards, Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
Hi Dan, the test fails at checking the prerequisites. It tries to load the image page and looks for a specific div element which is not present if the image was not uploaded correctly (id=filetoc). This might have changed across the versions of MediaWiki. Did you install the PagedTiffHandler extension? It depends on ImageMagick, so it might have rejected the upload. Although then it should have produced an error message ;) So the other question is, which MediaWiki version do you run the tests on? Regards, Markus ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Tue, 18 May 2010 01:04:01 +0200, Markus Glaser wrote: Hi Dan, the test fails at checking the prerequisites. It tries to load the image page and looks for a specific div element which is not present if the image was not uploaded correctly (id=filetoc). This might have changed across the versions of MediaWiki. Did you install the PagedTiffHandler extension? It depends on ImageMagick, so it might have rejected the upload. Although then it should have produced an error message ;) So the other question is, which MediaWiki version do you run the tests on? Regards, Markus Hi Markus, I am running on the latest version in trunk (1.17alpha r66296). There was no error when I uploaded the image. All of the extended details seem correct. I installed the extension. I don't have either exiv2 or vips installed, but according to the installation instructions these are optional. Here are the configuration values I used: # PagedTiffHandler extension require_once($IP/extensions/PagedTiffHandler/PagedTiffHandler.php); $wgTiffIdentifyRejectMessages = array( '/^identify: Compression algorithm does not support random access/', '/^identify: Old-style LZW codes, convert file/', '/^identify: Sorry, requested compression method is not configured/', '/^identify: ThunderDecode: Not enough data at scanline/', '/^identify: .+?: Read error on strip/', '/^identify: .+?: Can not read TIFF directory/', '/^identify: Not a TIFF/', ); $wgTiffIdentifyBypassMessages = array( '/^identify: .*TIFFReadDirectory/', '/^identify: .+?: unknown field with tag .+? encountered/' ); $wgImageMagickIdentifyCommand = '/usr/bin/identify'; $wgTiffUseExiv = false; $wgTiffUseVips = false; // Maximum number of embedded files in tiff image $wgTiffMaxEmbedFiles = 1; // Maximum resolution of embedded images (product of width x height pixels) $wgTiffMaxEmbedFileResolution = 2560; // max. Resolution 1600 x 1600 pixels // Maximum size of meta data $wgTiffMaxMetaSize = 67108864; // 64kB // TTL of Cacheentries for Errors $wgTiffErrorCacheTTL = 84600; Is there some way to use the wiki to look for the file property that is causing the problem? Regards, Dan -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
One of the URLs supplied by Ryan during the recent phone conference doesn't work. Specifically: http:// grid.tesla.usability.wikimedia.org:. I get the error: HTTP ERROR: 404 NOT_FOUND RequestURI=/ ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On Sat, May 15, 2010 at 12:12 PM, Dan Nessett dness...@yahoo.com wrote: One of the URLs supplied by Ryan during the recent phone conference doesn't work. Specifically: http:// grid.tesla.usability.wikimedia.org:. I get the error: HTTP ERROR: 404 NOT_FOUND RequestURI=/ That URL is for use in selenese scripts, not via a browser. http://grid.tesla.usability.wikimedia.org:/console in a browser will return the environments currently available for the selenese scripts. Note that some of those environments aren't currently functional as I work out bugs. Respectfully, Ryan Lane ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
Hi Jyothis, Thanks for the tip. I'll look into PushToTest and see whether it will be useful to us. -p Jyothis Edathoot wrote: If we are considering automated testing platforms, pushTotesthttp://www.pushtotest.com/index.phpis a good platform to consider. They come highly recommended by Tibco and is in use in many large scale corporations (including the one I work at). They have both community and enterprise editions around, which are fully functional. In a nutshell, it can integrate many, like selenium, test4gen, soapui etc etc. The tool can take a unit test script in selenium and turn it into functional, load and service monitoring tests in very easy steps. I was a part of the initial evaluation team at my work place and found the tool to be quite useful. We had an opportunity to work with their core team and they were quite helpful as well. This works well with RIA, Cloud etc too. The way we plan to use it is to have the developers create the selenium unit test at the development stage so that we can re-use the same script for the following stages (functional test and regression testing) of development. Just my two cents. Regards, Jyothis. http://www.Jyothis.net http://ml.wikipedia.org/wiki/User:Jyothis http://meta.wikimedia.org/wiki/User:Jyothis I am the first customer of http://www.netdotnet.com woods are lovely dark and deep, but i have promises to keep and miles to go before i sleep and lines to go before I press sleep completion date = (start date + ((estimated effort x 3.1415926) / resources) + ((total coffee breaks x 0.25) / 24)) + Effort in meetings On Sat, May 1, 2010 at 10:34 PM, Conrad Irwin conrad.ir...@googlemail.comwrote: On 05/02/2010 01:20 AM, Markus Glaser wrote: Hi everybody, at the Wkimedia Developers' Workshop, I introduced a Selenium testing framework for MediaWiki. Since it has now been promoted to maintenance/tests, I have provided some initial information it on http://www.mediawiki.org/wiki/SeleniumFramework . I would be very happy about comments and ideas for further improvement. Also, if you intend to use the framework for your tests, please let me know. I will be happy to assist. Regards, Markus Glaser It's a good leap in the right direction, a few comments: I spent about half an hour trying to get PagedDiffHandler tests to run, (though some uploads still seem to fail by uploading successfully): * Please distribute all the images needed with the extension, I followed the link in SOURCES.txt, and found that the link on that page was broken (though it wasn't hard to find a fixed link). * Don't hardcode \\ as the directory separator - windows is not here! * Use parser tests instead of selenium where appropriate (this will save you lots of typing and a bit of test execution time) * Make the tests pass by default :p. Comments about the framework itself: * Don't hardcode the firefox path - selenium's guess is better than yours. * Should always shutdown selenium properly (I was often left with myriads of firefoxes) * !Always run against a clean database! (it is unrealistic to expect people to write perfect tests, and particularly while developing the tests having an automatically clean database is essential). * Get configuration from the main LocalSettings.php and use a mechanism like parser tests for registering selenium test files. (making it as easy as possible to run tests is a good idea). waffle How to deal with configuration (and by extension databases) for selenium tests... It is a tricky question, and to solve it properly requires a bit of effort. I think you need to send a selenium token with each request (like you suggested, though it'd be nicer to use a cookie rather than a url parameter in my opinion). Either you could do a log in and get a session id, or you could simply update the cookie on every request with: (selenium_test_name, timestamp, start_of_test, HASH(selenium_test_name, timestamp, shared_secret)) Providing mediawiki is in selenium mode, and the timestamp is recent, the selenium_test_name matches a registered selenium test and the hash is correct, then we can be reasonably sure this is a valid selenium request (I assume no-one steals the secret, much as you assume no-one reads wikisysop's password out of the file - besides you can always update it, and it's only a problem if selenium mode is enabled). If timestamp == start_of_test then you can clean the test databases/ upload directories (or do that on log in). You can then also load any test-specific configuration from the selenium_test_name test on every request (and change cache-prefixes and upload directories to avoid collisions). If you allow for one database (or pseudo database with table prefixes, like parser test) per selenium_test_name then you can run different tests in parallel; otherwise you could generate a new prefix based on the start_of_test and the
Re: [Wikitech-l] Selenium testing framework
Aryeh Gregor wrote: On Sat, May 1, 2010 at 10:34 PM, Conrad Irwin conrad.ir...@googlemail.com wrote: * Don't hardcode \\ as the directory separator - windows is not here! In particular, always hardcode '/' for path separators -- it's easier to read due to lack of escaping, and works on Windows just as well as '\\' as far as I know. For most things, it does work, there is a caveat for shelling out to CMD.EXE though: http://msdn.microsoft.com/en-us/library/77859s1t%28v=VS.71%29.aspx -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Selenium testing framework
Hi everybody, at the Wkimedia Developers' Workshop, I introduced a Selenium testing framework for MediaWiki. Since it has now been promoted to maintenance/tests, I have provided some initial information it on http://www.mediawiki.org/wiki/SeleniumFramework . I would be very happy about comments and ideas for further improvement. Also, if you intend to use the framework for your tests, please let me know. I will be happy to assist. Regards, Markus Glaser __ Social Web Technologien Leiter Softwareentwicklung Hallo Welt! - Medienwerkstatt GmbH __ Untere Bachgasse 15 93047 Regensburg Tel. +49 (0) 941 - 56 95 94 - 92 www.hallowelt.bizhttp://www.hallowelt.biz/ gla...@hallowelt.bizmailto:gla...@hallowelt.biz Sitz: Regensburg Handelsregister: HRB 10467 E.USt.Nr.: DE 253050833 Geschäftsführer: Anja Ebersbach, Markus Glaser, Dr. Richard Heigl, Radovan Kubani __ ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
On 05/02/2010 01:20 AM, Markus Glaser wrote: Hi everybody, at the Wkimedia Developers' Workshop, I introduced a Selenium testing framework for MediaWiki. Since it has now been promoted to maintenance/tests, I have provided some initial information it on http://www.mediawiki.org/wiki/SeleniumFramework . I would be very happy about comments and ideas for further improvement. Also, if you intend to use the framework for your tests, please let me know. I will be happy to assist. Regards, Markus Glaser It's a good leap in the right direction, a few comments: I spent about half an hour trying to get PagedDiffHandler tests to run, (though some uploads still seem to fail by uploading successfully): * Please distribute all the images needed with the extension, I followed the link in SOURCES.txt, and found that the link on that page was broken (though it wasn't hard to find a fixed link). * Don't hardcode \\ as the directory separator - windows is not here! * Use parser tests instead of selenium where appropriate (this will save you lots of typing and a bit of test execution time) * Make the tests pass by default :p. Comments about the framework itself: * Don't hardcode the firefox path - selenium's guess is better than yours. * Should always shutdown selenium properly (I was often left with myriads of firefoxes) * !Always run against a clean database! (it is unrealistic to expect people to write perfect tests, and particularly while developing the tests having an automatically clean database is essential). * Get configuration from the main LocalSettings.php and use a mechanism like parser tests for registering selenium test files. (making it as easy as possible to run tests is a good idea). waffle How to deal with configuration (and by extension databases) for selenium tests... It is a tricky question, and to solve it properly requires a bit of effort. I think you need to send a selenium token with each request (like you suggested, though it'd be nicer to use a cookie rather than a url parameter in my opinion). Either you could do a log in and get a session id, or you could simply update the cookie on every request with: (selenium_test_name, timestamp, start_of_test, HASH(selenium_test_name, timestamp, shared_secret)) Providing mediawiki is in selenium mode, and the timestamp is recent, the selenium_test_name matches a registered selenium test and the hash is correct, then we can be reasonably sure this is a valid selenium request (I assume no-one steals the secret, much as you assume no-one reads wikisysop's password out of the file - besides you can always update it, and it's only a problem if selenium mode is enabled). If timestamp == start_of_test then you can clean the test databases/ upload directories (or do that on log in). You can then also load any test-specific configuration from the selenium_test_name test on every request (and change cache-prefixes and upload directories to avoid collisions). If you allow for one database (or pseudo database with table prefixes, like parser test) per selenium_test_name then you can run different tests in parallel; otherwise you could generate a new prefix based on the start_of_test and the selenium_test_name - but you'd quickly need a way to delete the databases completely once you'd finished./waffle Thanks for your great work so far! Conrad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium testing framework
If we are considering automated testing platforms, pushTotesthttp://www.pushtotest.com/index.phpis a good platform to consider. They come highly recommended by Tibco and is in use in many large scale corporations (including the one I work at). They have both community and enterprise editions around, which are fully functional. In a nutshell, it can integrate many, like selenium, test4gen, soapui etc etc. The tool can take a unit test script in selenium and turn it into functional, load and service monitoring tests in very easy steps. I was a part of the initial evaluation team at my work place and found the tool to be quite useful. We had an opportunity to work with their core team and they were quite helpful as well. This works well with RIA, Cloud etc too. The way we plan to use it is to have the developers create the selenium unit test at the development stage so that we can re-use the same script for the following stages (functional test and regression testing) of development. Just my two cents. Regards, Jyothis. http://www.Jyothis.net http://ml.wikipedia.org/wiki/User:Jyothis http://meta.wikimedia.org/wiki/User:Jyothis I am the first customer of http://www.netdotnet.com woods are lovely dark and deep, but i have promises to keep and miles to go before i sleep and lines to go before I press sleep completion date = (start date + ((estimated effort x 3.1415926) / resources) + ((total coffee breaks x 0.25) / 24)) + Effort in meetings On Sat, May 1, 2010 at 10:34 PM, Conrad Irwin conrad.ir...@googlemail.comwrote: On 05/02/2010 01:20 AM, Markus Glaser wrote: Hi everybody, at the Wkimedia Developers' Workshop, I introduced a Selenium testing framework for MediaWiki. Since it has now been promoted to maintenance/tests, I have provided some initial information it on http://www.mediawiki.org/wiki/SeleniumFramework . I would be very happy about comments and ideas for further improvement. Also, if you intend to use the framework for your tests, please let me know. I will be happy to assist. Regards, Markus Glaser It's a good leap in the right direction, a few comments: I spent about half an hour trying to get PagedDiffHandler tests to run, (though some uploads still seem to fail by uploading successfully): * Please distribute all the images needed with the extension, I followed the link in SOURCES.txt, and found that the link on that page was broken (though it wasn't hard to find a fixed link). * Don't hardcode \\ as the directory separator - windows is not here! * Use parser tests instead of selenium where appropriate (this will save you lots of typing and a bit of test execution time) * Make the tests pass by default :p. Comments about the framework itself: * Don't hardcode the firefox path - selenium's guess is better than yours. * Should always shutdown selenium properly (I was often left with myriads of firefoxes) * !Always run against a clean database! (it is unrealistic to expect people to write perfect tests, and particularly while developing the tests having an automatically clean database is essential). * Get configuration from the main LocalSettings.php and use a mechanism like parser tests for registering selenium test files. (making it as easy as possible to run tests is a good idea). waffle How to deal with configuration (and by extension databases) for selenium tests... It is a tricky question, and to solve it properly requires a bit of effort. I think you need to send a selenium token with each request (like you suggested, though it'd be nicer to use a cookie rather than a url parameter in my opinion). Either you could do a log in and get a session id, or you could simply update the cookie on every request with: (selenium_test_name, timestamp, start_of_test, HASH(selenium_test_name, timestamp, shared_secret)) Providing mediawiki is in selenium mode, and the timestamp is recent, the selenium_test_name matches a registered selenium test and the hash is correct, then we can be reasonably sure this is a valid selenium request (I assume no-one steals the secret, much as you assume no-one reads wikisysop's password out of the file - besides you can always update it, and it's only a problem if selenium mode is enabled). If timestamp == start_of_test then you can clean the test databases/ upload directories (or do that on log in). You can then also load any test-specific configuration from the selenium_test_name test on every request (and change cache-prefixes and upload directories to avoid collisions). If you allow for one database (or pseudo database with table prefixes, like parser test) per selenium_test_name then you can run different tests in parallel; otherwise you could generate a new prefix based on the start_of_test and the selenium_test_name - but you'd quickly need a way to delete the databases completely once you'd finished./waffle Thanks for your great work so far! Conrad
Re: [Wikitech-l] Selenium testing framework
On Sat, May 1, 2010 at 10:34 PM, Conrad Irwin conrad.ir...@googlemail.com wrote: * Don't hardcode \\ as the directory separator - windows is not here! In particular, always hardcode '/' for path separators -- it's easier to read due to lack of escaping, and works on Windows just as well as '\\' as far as I know. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
