Re: [Wikitech-l] Selenium Framework - standard directory for selenium tests
Hi, in SMW we have a maintenance folder just like MW does. Would it be possible to have the tests in there, i.e. SemanticMediaWiki/maintenance/tests/selenium (up to now, I have put them in there)? Regards Benedikt -- Karlsruhe Institute of Technology (KIT) Institute of Applied Informatics and Formal Description Methods (AIFB) Benedikt Kämpgen Research Associate Kaiserstraße 12 Building 11.40 76131 Karlsruhe, Germany Phone: +49 721 608-7946 Fax: +49 721 608-6580 Email: benedikt.kaemp...@kit.edu Web: http://www.kit.edu/ KIT - University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association -Original Message- From: wikitech-l-boun...@lists.wikimedia.org [mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of Markus Glaser Sent: Monday, September 13, 2010 10:28 PM To: pdha...@wikimedia.org; Wikimedia developers Subject: Re: [Wikitech-l] Selenium Framework - standard directory for selenium tests Hi, I agree. Tests from extensions should follow the pattern Extensions/EXTENSION/tests/selenium whereas test for the core should be placed into maintenance/tests/selenium Any objections? Regards, Markus -Ursprüngliche Nachricht- Von: wikitech-l-boun...@lists.wikimedia.org [mailto:wikitech-l-boun...@lists.wikimedia.org] Im Auftrag von Priyanka Dhanda Gesendet: Montag, 13. September 2010 21:28 An: wikitech-l@lists.wikimedia.org Betreff: Re: [Wikitech-l] Selenium Framework - standard directory for selenium tests Hi Dan, I believe we decided that selenium tests should go under: Extension/tests/selenium/ -p On 09/13/2010 12:04 PM, Dan Nessett wrote: Are there any standards for where to put selenium tests? Right now the Simple Selenium test is in phase3/maintenance/tests/selenium and the PagedTiffHandler selenium tests are in PagedTiffHandler/selenium. This suggests a convention of putting extension selenium test files in a sub- directory of the top-level directory named 'selenium'. Is that an official convention? -- Priyanka Dhanda Code Maintenance Engineer Wikimedia Foundation http://wikimediafoundation.org San Francisco, CA ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] UsabilityInitiative extension changes
2010/9/17 Trevor Parscal tpars...@wikimedia.org: There is at least one other extension which depends on UsabilityInitiative that I know of (SimpleSurvey), and there may be others; these need to be detached from UsabilityInitiative.php before November (when we roughly plan to deploy 1.17 last I heard). Please do not detach SimpleSurvey from UsabilityInitiative.php just yet. We intend to deploy changes to SimpleSurvey for the article rating rollout next week, so I'd like it to remain in sync with 1.16wmf4 as much as possible until then. Once SimpleSurvey has been deployed and seems to be running happily, by all means go bonkers with it. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Selenium Framework - standard directory for selenium tests
Hi Benedikt, technically speaking, it should not be a problem, since the path to the test suites can be given relative to mw root. This holds for the current solution (via autoloader) as well as for the upcoming version with ini-files. Remains the issue of standardization. It seems reasonable to me for large extensions to mirror the mw directory structure. So I suggest this for standard for extensions: * Extensions that do mirror the mw directory structure should place the tests similar to mw: extensions/EXTENSION/maintenance/tests/selenium * Extensions that don't should place the tests in extensions/EXTENSION/tests/selenium Regards, Markus Markus Glaser Social Web Technologien Leiter Softwareentwicklung Hallo Welt! - Medienwerkstatt GmbH __ Untere Bachgasse 15 93047 Regensburg Tel. +49 (0) 941 - 56 95 94 92 Fax. +49 (0) 941 - 50 27 58 13 www.hallowelt.biz gla...@hallowelt.biz Sitz: Regensburg Handelsregister: HRB 10467 E.USt.Nr.: DE 253050833 Geschäftsführer: Anja Ebersbach, Markus Glaser, Dr. Richard Heigl, Radovan Kubani -Ursprüngliche Nachricht- Von: wikitech-l-boun...@lists.wikimedia.org [mailto:wikitech-l-boun...@lists.wikimedia.org] Im Auftrag von Benedikt Kaempgen Gesendet: Freitag, 17. September 2010 08:05 An: Wikimedia developers; pdha...@wikimedia.org Betreff: Re: [Wikitech-l] Selenium Framework - standard directory for selenium tests Hi, in SMW we have a maintenance folder just like MW does. Would it be possible to have the tests in there, i.e. SemanticMediaWiki/maintenance/tests/selenium (up to now, I have put them in there)? Regards Benedikt -- Karlsruhe Institute of Technology (KIT) Institute of Applied Informatics and Formal Description Methods (AIFB) Benedikt Kämpgen Research Associate Kaiserstraße 12 Building 11.40 76131 Karlsruhe, Germany Phone: +49 721 608-7946 Fax: +49 721 608-6580 Email: benedikt.kaemp...@kit.edu Web: http://www.kit.edu/ KIT - University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association -Original Message- From: wikitech-l-boun...@lists.wikimedia.org [mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of Markus Glaser Sent: Monday, September 13, 2010 10:28 PM To: pdha...@wikimedia.org; Wikimedia developers Subject: Re: [Wikitech-l] Selenium Framework - standard directory for selenium tests Hi, I agree. Tests from extensions should follow the pattern Extensions/EXTENSION/tests/selenium whereas test for the core should be placed into maintenance/tests/selenium Any objections? Regards, Markus -Ursprüngliche Nachricht- Von: wikitech-l-boun...@lists.wikimedia.org [mailto:wikitech-l-boun...@lists.wikimedia.org] Im Auftrag von Priyanka Dhanda Gesendet: Montag, 13. September 2010 21:28 An: wikitech-l@lists.wikimedia.org Betreff: Re: [Wikitech-l] Selenium Framework - standard directory for selenium tests Hi Dan, I believe we decided that selenium tests should go under: Extension/tests/selenium/ -p On 09/13/2010 12:04 PM, Dan Nessett wrote: Are there any standards for where to put selenium tests? Right now the Simple Selenium test is in phase3/maintenance/tests/selenium and the PagedTiffHandler selenium tests are in PagedTiffHandler/selenium. This suggests a convention of putting extension selenium test files in a sub- directory of the top-level directory named 'selenium'. Is that an official convention? -- Priyanka Dhanda Code Maintenance Engineer Wikimedia Foundation http://wikimediafoundation.org San Francisco, CA ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] list of things to do for image dumps
On September 10, emijrp wrote: Hi Lars, are you going to upload more logs to Internet Archive? No, I can't. I have not downloaded more recent logs. I only uploaded what was on my disk, because I needed to free some space. Domas website only shows the last 3 (?) months. I think that there are many of these files at Toolserver, but we must preserve this raw data in another secure (for posterity) place. Must? Says who? That sounds like a naive opinion. If you have an interest, you can do the job. Otherwise they will get lost. In the future, maybe this should be a task for the paid staff, but so far it has not been. -- Lars Aronsson (l...@aronsson.se) Aronsson Datateknik - http://aronsson.se ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Accidental 302 hijack by interwiki index: Google v Wikimedia bug
On Thu, Sep 16, 2010 at 4:09 PM, Andrew Cates andrew.ca...@soschildren.org wrote: Sorry about this but I think it is a reasonable request for help.. Google seems to be delivering wikipedia URLs for pages with a 302 interwiki link eg http://www.google.co.uk/search?q=schools+wikipediaie=utf-8oe=utf-8aq=tclient=firefox-arlz=1R1GGGL_en-GB___GB347 gets en.wikipedia.org/wiki/SchoolsWP%3Aindex%3Ahome as the URL with http://schools-wikipedia.org/ as content As far as I can see interwiki redirects are no longer used by Wikimedia software which just delivers straight links to target off the wikipedia pages. So nothing links through the 302 but Google returns it anyway. Anyway could we therefore disable the temporary redirects to fix this problem, or not serve them to Google or if there is some reason why the redirects have to stay, show Google perm redirects instead (it is not as if we will want the redirects to point at anywhere else) This looks like a Google bug. I don't think we should just disable 302s to interwikis -- you shouldn't have to run MediaWiki (and know Wikipedia's interwiki link table) to be able to reliably link to things in the same way you can link to them on Wikipedia. Right now third-party software can do stuff like a href=http://en.wikipedia.org/wiki/$1; and replace $1 by user input, and it will work basically like [[$1]] typed on Wikipedia, and that's good. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] cite/ref error
Hi, We are creating an off-line version of the wikipedia, but we continue to have problems getting some citation items to render correctly. When using the cite template and citing a paper or journal and the URL tag is missing, you get an error in the citation list at the bottom with error Missing operand for . (see example below from the article Cyprus). ^ Drayton, Penny (January 1993). [Expression error: Missing operand for Aphrodite's island]. Wood water 2 (41). Cited by: Trubshaw, Bob (February 1993). The Black Stone – the Omphalos of the Goddess. Mercian Mysteries (14). http://www.indigogroup.co.uk/edge/blstone.htm. Retrieved 2006-11-12. In Cyprus is another highly venerated We haven't been able to find exactly where the error occurs since there are several layers of templates. Does anybody know if this was just a template problem briefly or if there is way to track it down. We are using the 6/22/2010 dump. I did look at the change history of several templates but I haven't found any that were fixed between that dump date and the current online version. Thanks, Brent ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] UsabilityInitiative extension changes
On 9/17/10 4:14 AM, Roan Kattouw wrote: 2010/9/17 Trevor Parscaltpars...@wikimedia.org: There is at least one other extension which depends on UsabilityInitiative that I know of (SimpleSurvey), and there may be others; these need to be detached from UsabilityInitiative.php before November (when we roughly plan to deploy 1.17 last I heard). Please do not detach SimpleSurvey from UsabilityInitiative.php just yet. We intend to deploy changes to SimpleSurvey for the article rating rollout next week, so I'd like it to remain in sync with 1.16wmf4 as much as possible until then. Once SimpleSurvey has been deployed and seems to be running happily, by all means go bonkers with it. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l No worries, I'm not in a rush on that one. - Trevor ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] using parserTests code for selenium test framework
I have been tasked to evaluate whether we can use the parserTests db code for the selenium framework. I just looked it over and have serious reservations. I would appreciate any comments on the following analysis. The environment for selenium tests is different than that for parserTests. It is envisioned that multiple concurrent tests could run using the same MW code base. Consequently, each test run must: + Use a db that if written to will not destroy other test wiki information. + Switch in a new images and math directory so any writes do not interfere with other tests. + Maintain the integrity of the cache. Note that tests would *never* run on a production wiki (it may be possible to do so if they do no writes, but safety considerations suggest they should always run on a test data, not production data). In fact production wikis should always retain the setting $wgEnableSelenium = false, to ensure selenium test are disabled. Given this background, consider the following (and feel free to comment on it): parserTests temporary table code: A fixed set of tables are specified in the code. parserTests creates temporary tables with the same name, but using a different static prefix. These tables are used for the parserTests run. Problems using this approach for selenium tests: + Selenium tests on extensions may require use of extension specific tables, the names of which cannot be elaborated in the code. + Concurrent test runs of parserTests are not supported, since the temporary tables have fixed names and therefore concurrent writes to them by parallel test runs would cause interference. + Clean up from aborted runs requires dropping fossil tables. But, if a previous run tested an extension with extension-specific tables, there is no way for a test of some other functionality to figure out which tables to drop. For these reasons, I don't think we can reuse the parserTests code. However, I am open to arguments to the contrary. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
Dan Nessett wrote: Given this background, consider the following (and feel free to comment on it): parserTests temporary table code: A fixed set of tables are specified in the code. parserTests creates temporary tables with the same name, but using a different static prefix. These tables are used for the parserTests run. Problems using this approach for selenium tests: + Selenium tests on extensions may require use of extension specific tables, the names of which cannot be elaborated in the code. The extensions could list their table names. No problem there. + Concurrent test runs of parserTests are not supported, since the temporary tables have fixed names and therefore concurrent writes to them by parallel test runs would cause interference. So it gets changed to a random name with a fixed prefix... What concerns me is + Clean up from aborted runs requires dropping fossil tables. But, if a previous run tested an extension with extension-specific tables, there is no way for a test of some other functionality to figure out which tables to drop. Run a script dropping all tables with a fixed prefix (the shared part of the tests) when you have no tests running. For these reasons, I don't think we can reuse the parserTests code. However, I am open to arguments to the contrary. There may be other issues with that code, and using a separate db would be preferable if you have enough permissions, but this doesn't seem like real problems. What concerns me is that Oracle is using (r58669) a different prefix for the parsertests table. If it has some restriction on [medium-large] table names, there may not be possible to run the tests there using the long table names that we could produce. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Fri, 17 Sep 2010 18:40:53 +, Dan Nessett wrote: I have been tasked to evaluate whether we can use the parserTests db code for the selenium framework. I just looked it over and have serious reservations. I would appreciate any comments on the following analysis. The environment for selenium tests is different than that for parserTests. It is envisioned that multiple concurrent tests could run using the same MW code base. Consequently, each test run must: + Use a db that if written to will not destroy other test wiki information. + Switch in a new images and math directory so any writes do not interfere with other tests. + Maintain the integrity of the cache. Note that tests would *never* run on a production wiki (it may be possible to do so if they do no writes, but safety considerations suggest they should always run on a test data, not production data). In fact production wikis should always retain the setting $wgEnableSelenium = false, to ensure selenium test are disabled. Given this background, consider the following (and feel free to comment on it): parserTests temporary table code: A fixed set of tables are specified in the code. parserTests creates temporary tables with the same name, but using a different static prefix. These tables are used for the parserTests run. Problems using this approach for selenium tests: + Selenium tests on extensions may require use of extension specific tables, the names of which cannot be elaborated in the code. + Concurrent test runs of parserTests are not supported, since the temporary tables have fixed names and therefore concurrent writes to them by parallel test runs would cause interference. + Clean up from aborted runs requires dropping fossil tables. But, if a previous run tested an extension with extension-specific tables, there is no way for a test of some other functionality to figure out which tables to drop. For these reasons, I don't think we can reuse the parserTests code. However, I am open to arguments to the contrary. After reflection, here are some other problems. + Some tests assume the existence of data in the db. For example, the PagedTiffHandler tests assume the image Multipage.tiff is already loaded. However, this requires an entry in the image table. You could modify the test to clone the existing image table, but that means you have problems with: + Some tests assume certain data is *not* in the db. PagedTiffHandler has tests that upload images. These cannot already be in the images table. So, you can't simply clone the images table. All of this suggests to me that a better strategy is: + When the test run begins, clone a db associated with the test suite. + Switch the wiki to use this db and return a cookie or some other state information that identifies this test run configuration. + When the test suite runs, each wiki access supplies this state so the wiki code can switch in the correct db. + Cleanup of test runs requires removing the cloned db. + To handled aborted runs, there needs to be a mechanism to time out cloned dbs and the state associated with the test run. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Fri, 17 Sep 2010 21:05:12 +0200, Platonides wrote: Dan Nessett wrote: Given this background, consider the following (and feel free to comment on it): parserTests temporary table code: A fixed set of tables are specified in the code. parserTests creates temporary tables with the same name, but using a different static prefix. These tables are used for the parserTests run. Problems using this approach for selenium tests: + Selenium tests on extensions may require use of extension specific tables, the names of which cannot be elaborated in the code. The extensions could list their table names. No problem there. + Concurrent test runs of parserTests are not supported, since the temporary tables have fixed names and therefore concurrent writes to them by parallel test runs would cause interference. So it gets changed to a random name with a fixed prefix... What concerns me is + Clean up from aborted runs requires dropping fossil tables. But, if a previous run tested an extension with extension-specific tables, there is no way for a test of some other functionality to figure out which tables to drop. Run a script dropping all tables with a fixed prefix (the shared part of the tests) when you have no tests running. For these reasons, I don't think we can reuse the parserTests code. However, I am open to arguments to the contrary. There may be other issues with that code, and using a separate db would be preferable if you have enough permissions, but this doesn't seem like real problems. What concerns me is that Oracle is using (r58669) a different prefix for the parsertests table. If it has some restriction on [medium-large] table names, there may not be possible to run the tests there using the long table names that we could produce. The strategy you suggest is reasonable. But, I think it requires significant changes to the parserTests code. The question then is: is it simpler to modify this code or just write something new? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] UsabilityInitiative extension changes
Trevor Parscal tpars...@wikimedia.org wrote in message news:4c92b55a.7090...@wikimedia.org... ... before November (when we roughly plan to deploy 1.17 last I heard). I'll hold you to that... 8-P --HM ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
Dan Nessett wrote: After reflection, here are some other problems. + Some tests assume the existence of data in the db. For example, the PagedTiffHandler tests assume the image Multipage.tiff is already loaded. However, this requires an entry in the image table. You could modify the test to clone the existing image table, but that means you have problems with: + Some tests assume certain data is *not* in the db. PagedTiffHandler has tests that upload images. These cannot already be in the images table. So, you can't simply clone the images table. Interesting. I haven't seen PagedTiffHandler tests. What normal parsertest do is to upload existing images and add the needed articles to the empty tables. Previously, the image tables entries were added directly by SQL. I changed it in r70917 to use recordUpload2() instead. All of this suggests to me that a better strategy is: + When the test run begins, clone a db associated with the test suite. Having another database would be the optimal solution, but it's not always possible. OTOH MaxSem replied to my concern in r58669: Oracle table names are limited to 32 characters. Mysql limit is of 64 characters* which gives more margin. Our longest table is msg_resource_links with 18 characters. If we choose a prefix like mwtest_ we could use another underscore plus 6 random digits for identifying the instance and still be Oracle compliant. * http://dev.mysql.com/doc/refman/5.1/en/identifiers.html + Switch the wiki to use this db and return a cookie or some other state information that identifies this test run configuration. I think you mean for remote petitions, not just for internal queries, where do you expect to store that data? ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] using parserTests code for selenium test framework
On Sat, 18 Sep 2010 00:53:04 +0200, Platonides wrote: + Switch the wiki to use this db and return a cookie or some other state information that identifies this test run configuration. I think you mean for remote petitions, not just for internal queries, where do you expect to store that data? Not sure what you mean by remote petitions. Selenium requests always come through the web portal from the selenium server. So, no internal queries are involved. Where to store the data is an open question, one that requires consultation with others. However, here are some thoughts: + The data must be persistent. If the wiki crashes for some reason, there may be cloned dbs and test-specific copies of images and images/math hanging around. (Depending how we handle the cache information, there may also be fossil cache data). This requires cleanup after a wiki crash. + It would be possible to store the data in a file or in a master db table. Which is best (or if something else is better) is a subject for discussion. We may be able to use the mechanisms in the code that supports access to different language versions of a wiki (e.g., Wikipedia) using the same code. I am not familiar with these mechanisms, so this approach requires help from someone who is. -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Keeping record of imported licensed text
On Fri, 10 Sep 2010 23:11:27 +, Dan Nessett wrote: We are currently attempting to refactor some specific modifications to the standard MW code we use (1.13.2) into an extension so we can upgrade to a more recent maintained version. One modification we have keeps a flag in the revisions table specifying that article text was imported from WP. This flag generates an attribution statement at the bottom of the article that acknowledges the import. I don't want to start a discussion about the various legal issues surrounding text licensing. However, assuming we must acknowledge use of licensed text, a legitimate technical issue is how to associate state with an article in a way that records the import of licensed text. I bring this up here because I assume we are not the only site that faces this issue. Some of our users want to encode the attribution information in a template. The problem with this approach is anyone can come along and remove it. That would mean the organization legally responsible for the site would entrust the integrity of site content to any arbitrary author. We may go this route, but for the sake of this discussion I assume such a strategy is not viable. So, the remainder of this post assumes we need to keep such licensing state in the db. After asking around, one suggestion was to keep the licensing state in the page_props table. This seems very reasonable and I would be interested in comments by this community on the idea. Of course, there has to be a way to get this state set, but it seems likely that could be achieved using an extension triggered when an article is edited. Since this post is already getting long, let me close by asking whether support for associating licensing information with articles might be useful to a large number of sites. If so, the perhaps it belongs in the core. One thing I haven't seen so far (probably because it doesn't belong on Wikitech) is a discussion of the policy requirements. In open source software development, you have to carry forward licenses even if you substantially change the code content. The only way around this is a clean room implementation (e.g., how BSD Unix got around ATT's original licensing for Unix). Is this also true for textual content? If so, then once you import such content into an article you are obliged to carry forward any licensing conditions on that import on for all subsequent revisions. Where is the proper place to discuss these kinds of questions? -- -- Dan Nessett ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
