Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
On Fri, Feb 22, 2013 at 7:06 PM, Matthew Flaschen mflasc...@wikimedia.orgwrote: On 02/22/2013 09:38 PM, Chad wrote: So, I've seen this site tossed around quite a bit recently, and I'm curious: is there any plan to start integrating this jenkins and our other jenkins? Depends on what you mean by integrate. Right now the sweet spot for browser tests shown at https://wmf.ci.cloudbees.com/ is to track the deployment schedule over individual code commits and to target integrated institutional test environments like test2wiki and beta cluster, while https://integration.mediawiki.org/ci/ mostly targets unit-type tests run on the Jenkins host itself. There are a lot of builds there right now already. In the longer term we want to have browser tests targeting more specialized test environments and more granular code commits. There are lots of ways that Jenkins instances can share data, so when that sort of activity comes along we'll figure out the details at that time. More importantly: is there any chance to get the results of these sorts of tests in Gerrit? I think it's great that we're expanding test coverage, but without feedback on people's patches they're usually unaware that they're breaking things. As of today browser test status changes are being reported to #wikimedia-dev by a bot named wmf-jenkins-bot, e.g.: (09:30:18 AM) wmf-jenkins-bot-: Project _debug-irc build #17: SUCCESS in 90 ms: https://wmf.ci.cloudbees.com/job/_debug-irc/17/ Integration with Gerrit as well as Jenkins is certainly feasible, and as the information provided by these tests becomes more closely tied to the code itself rather than the environments in which the code is deployed, we can put that integration in place as it becomes valuable. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
On Fri, Feb 22, 2013 at 6:38 PM, Chad innocentkil...@gmail.com wrote: So, I've seen this site tossed around quite a bit recently, and I'm curious: is there any plan to start integrating this jenkins and our other jenkins? Plans exist, but I am pretty sure there are no deadlines. More importantly: is there any chance to get the results of these sorts of tests in Gerrit? I am not a Jenkins ninja, so I am probably not the right person to answer. Željko ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
(anonymous) wrote: But right now, I don't sense a huge amount of friction between the WMF's needs and the non-WMF MediaWiki-using community. The most that can be said is that the WMF is focused on its sites and doesn't make third party use a priority. This doesn't stop support for other databases, though: Oracle, MS SQL, PostgreSQL, SQLite, or even my recent changes to separate out DB schema changes in MySQL. Yes, unfortunately schema changes and other DB related changesets tend to only get applied to MySQL/SQLite, and the other DBs tend to get ignored or lag behind by a few months. That's about my only gripe, that, and that setting jenkins up for these other backends was never completed. JFTR: AFAICS Jenkins is still not set up for MySQL :-) (cf. https://bugzilla.wikimedia.org/35912). Tim ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
On 02/22/2013 03:42 AM, Thorsten Glaser wrote: I am a bit unhappy that instead of a database, MySQL is used/preferred, but (after the last few bugfixes), PostgreSQL works, so I’m set. Please do not hesitate to file any bugs for things that don't work for you in PG. And if they aren't getting resolved quickly enough, please ping me. I expect us (as in, my employer) to not follow every single MW release quickly, and Debian probably won’t either (most‐ ly for lack of manpower, I guess). And this is the exact reason that I initiated LTS support for 1.19. We'll make releases every 6 months, but you can be assured that we'll support 1.19 for a while. With my Debian Developer hat on, I don’t sense much in that area of complaints either. I installed the package last night on http://home.nichework.com/ -- dns may not be propagated yet -- and was disappointed that you didn't use the CLI installer to set up a wiki using debconf. There were a couple of other nits, but I think that overall it is a great thing. Thanks, Mark -- http://hexmode.com/ There is no path to peace. Peace is the path. -- Mahatma Gandhi, Non-Violence in Peace and War ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
On Thu, Feb 21, 2013 at 2:01 PM, Tyler Romeo tylerro...@gmail.com wrote: - Where is QA? I mean, I know somewhere somebody is probably doing some sort of testing, but having worked as a QA engineer I haven't seen anything in MW that would resemble proper and traditional testing (excluding the unit testing). Where's the list of test cases that need to be performed for each release? How can one make new test cases and add them? etc. Maybe this already exists, but if it does it's definitely not documented well enough. Disclaimer: I am one of the QA people. We are testing all the time, but there are just 3-4 of us, as far as I know. We are looking for help. As far as I know, there will be Write your first Test Scenario in plain English event[1] on the week of March 11, if you would like to help. Feel free to add features/scenarios to the backlog[2] in the meantime. Let me know if you need help with that. (Test results for our browser automation project are available[3] to everybody, by the way). As an example, Siebrand provided a few features and scenarios[4] today and Chris and I have automated them[5][6]. (I have just noticed that the tests that we worked on today all failed because we make a mistake. It will be fixed probably on Monday.) Željko -- [1] http://www.mediawiki.org/wiki/QA/Weekly_goals [2] http://www.mediawiki.org/wiki/QA/Browser_testing/Test_backlog [3] https://wmf.ci.cloudbees.com/ [4] http://etherpad.wikimedia.org/i18n-qa [5] https://github.com/wikimedia/qa-browsertests/blob/master/features/accept_language.feature [6] https://github.com/wikimedia/qa-browsertests/blob/master/features/universal_language_selector.feature ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
On Fri, Feb 22, 2013 at 9:32 PM, Željko Filipin zfili...@wikimedia.org wrote: Feel free to add features/scenarios to the backlog[2] in the meantime. Let me know if you need help with that. (Test results for our browser automation project are available[3] to everybody, by the way). [snip] [3] https://wmf.ci.cloudbees.com/ So, I've seen this site tossed around quite a bit recently, and I'm curious: is there any plan to start integrating this jenkins and our other jenkins? More importantly: is there any chance to get the results of these sorts of tests in Gerrit? I think it's great that we're expanding test coverage, but without feedback on people's patches they're usually unaware that they're breaking things. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
On 02/22/2013 09:38 PM, Chad wrote: So, I've seen this site tossed around quite a bit recently, and I'm curious: is there any plan to start integrating this jenkins and our other jenkins? More importantly: is there any chance to get the results of these sorts of tests in Gerrit? I think it's great that we're expanding test coverage, but without feedback on people's patches they're usually unaware that they're breaking things. I agree. I think our goal should be to have all the tests (QUnit, Cucumber, PHPUnit (generally already happens for this one)) result in Jenkins votes on Gerrit. Matt Flaschen ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
(Adding a couple of mailing lists so others can weigh in. Changing subject so those added aren't completely lost.) On 02/21/2013 11:55 AM, Quim Gil wrote: Ok, just a question as humble 3rd party MediaWiki user and technical volunteer coordinator at the WMF: is there a possibility to consider having a regular free software release process? master/unstable --- (testing releases?) --- stable releases ... I think the current process is ok-ish in the short term: non-WMF contributors are getting +2 and 3rd parties are getting tarballs. As you say, I think the current process is Ok(ish) for now. We need to get others in the MediaWiki ecosystem involved in core before this becomes something we really need to do. It would be great to have developers from other significant MediaWiki sites (like Referata, Wikia, Citizendium, etc) become more involved and start introducing features or hooks that they use into core or making the extensions available. Of course, some of those developers have already been involved. But right now, I don't sense a huge amount of friction between the WMF's needs and the non-WMF MediaWiki-using community. The most that can be said is that the WMF is focused on its sites and doesn't make third party use a priority. This doesn't stop support for other databases, though: Oracle, MS SQL, PostgreSQL, SQLite, or even my recent changes to separate out DB schema changes in MySQL. That said, I'm very interested in this conversation. As MZ will remind you, I did advocate for the formation of the MediaWiki Foundation. Mark. -- http://hexmode.com/ There is no path to peace. Peace is the path. -- Mahatma Gandhi, Non-Violence in Peace and War ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
On 2013-02-21 1:40 PM, Mark A. Hershberger m...@everybody.org wrote: (Adding a couple of mailing lists so others can weigh in. Changing subject so those added aren't completely lost.) On 02/21/2013 11:55 AM, Quim Gil wrote: Ok, just a question as humble 3rd party MediaWiki user and technical volunteer coordinator at the WMF: is there a possibility to consider having a regular free software release process? master/unstable --- (testing releases?) --- stable releases ... I think the current process is ok-ish in the short term: non-WMF contributors are getting +2 and 3rd parties are getting tarballs. As you say, I think the current process is Ok(ish) for now. We need to get others in the MediaWiki ecosystem involved in core before this becomes something we really need to do. It would be great to have developers from other significant MediaWiki sites (like Referata, Wikia, Citizendium, etc) become more involved and start introducing features or hooks that they use into core or making the extensions available. Of course, some of those developers have already been involved. But right now, I don't sense a huge amount of friction between the WMF's needs and the non-WMF MediaWiki-using community. The most that can be said is that the WMF is focused on its sites and doesn't make third party use a priority. This doesn't stop support for other databases, though: Oracle, MS SQL, PostgreSQL, SQLite, or even my recent changes to separate out DB schema changes in MySQL. That said, I'm very interested in this conversation. As MZ will remind you, I did advocate for the formation of the MediaWiki Foundation. Mark. -- http://hexmode.com/ There is no path to peace. Peace is the path. -- Mahatma Gandhi, Non-Violence in Peace and War ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l Having wikipedia use unstable versions helps to catch many bugs. Not only are wikimedians great testers (they (ab)use the software in insane ways), by in large bugs encountered by wikimedians get reported effectively. Thus the use of unstablish releases on wikimedia allows for much more stable core releases. -bawolff ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
On 02/21/2013 12:50 PM, Brian Wolff wrote: Thus the use of unstablish releases on wikimedia allows for much more stable core releases. Thanks for pointing this out. I meant to say this, too. I guess the question I want other MediaWiki users to answer is: Are there any concerns that mitigate this benefit? Mark. -- http://hexmode.com/ There is no path to peace. Peace is the path. -- Mahatma Gandhi, Non-Violence in Peace and War ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
On Thu, Feb 21, 2013 at 11:39 AM, Mark A. Hershberger m...@everybody.org wrote: But right now, I don't sense a huge amount of friction between the WMF's needs and the non-WMF MediaWiki-using community. The most that can be said is that the WMF is focused on its sites and doesn't make third party use a priority. This doesn't stop support for other databases, though: Oracle, MS SQL, PostgreSQL, SQLite, or even my recent changes to separate out DB schema changes in MySQL. Yes, unfortunately schema changes and other DB related changesets tend to only get applied to MySQL/SQLite, and the other DBs tend to get ignored or lag behind by a few months. That's about my only gripe, that, and that setting jenkins up for these other backends was never completed. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
I'd like to see MediaWiki gain a more stable release process as well. I think some of the primary things that we're lacking are: - Where is QA? I mean, I know somewhere somebody is probably doing some sort of testing, but having worked as a QA engineer I haven't seen anything in MW that would resemble proper and traditional testing (excluding the unit testing). Where's the list of test cases that need to be performed for each release? How can one make new test cases and add them? etc. Maybe this already exists, but if it does it's definitely not documented well enough. - Stable sets of expected release features. In most companies I've worked in, every single bug upon being reported is immediately scheduled for a release, even if it just means deferring it to an unknown Future Release. But doing a quick search in Bugzilla shows MW has 5000+ bugs that are not scheduled, some of which are even high priority bugs. I've probably said this before, but it'd be good if consumers knew beforehand what they may expect in the next MW release (even if it's only tentative) so that they can debate whether or not to prepare for an upgrade. I've been told before that that's what the release notes are for, but that's not the point. Release notes currently only include stuff that's already done. - Faster review process. This is something that's not at all easy, and I know many are aware, but it takes a while to get reviews. I mean, there are people like me who get notifications for every new change in gerrit, and thus I'll see everything even if I wasn't added as a reviewer, but not everybody does that, which leaves the question of how to get your stuff reviewed and who to go to. There's a list of MW.org that has some people, but I've found it's usually not helpful until you're more involved and actually know who those people are. Other than those process issues, there are a few feature issues that IMO I think are holding people back: - As said, DB support, especially for high-use systems like Postgres and MSSQL. - Enterprise platforms. What if I want to deploy MW onto AWS or VMWare? Many companies have pre-packaged systems for this. For example, at the company I'm working at now, deploying their product to AWS is as easy as copying the ID number into the web GUI and clicking deploy. Also, is there any tracking on HipHop support? - Non-PHP. This is probably far off in the future, but eventually it'd be nice to be able to setup MW without having to deal with PHP at all, i.e., have a configuration file in YAML or something. Even PHP frameworks like Symfony have abstracted out the PHP, and that's a case where you're actually developing *in* PHP. :P *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com On Thu, Feb 21, 2013 at 1:01 PM, OQ overlo...@gmail.com wrote: On Thu, Feb 21, 2013 at 11:39 AM, Mark A. Hershberger m...@everybody.org wrote: But right now, I don't sense a huge amount of friction between the WMF's needs and the non-WMF MediaWiki-using community. The most that can be said is that the WMF is focused on its sites and doesn't make third party use a priority. This doesn't stop support for other databases, though: Oracle, MS SQL, PostgreSQL, SQLite, or even my recent changes to separate out DB schema changes in MySQL. Yes, unfortunately schema changes and other DB related changesets tend to only get applied to MySQL/SQLite, and the other DBs tend to get ignored or lag behind by a few months. That's about my only gripe, that, and that setting jenkins up for these other backends was never completed. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Do we need to change the MW release process to better involve the non-WMF community?
On Thu, Feb 21, 2013 at 2:01 PM, Tyler Romeo tylerro...@gmail.com wrote: I'd like to see MediaWiki gain a more stable release process as well. I think some of the primary things that we're lacking are: - Where is QA? I mean, I know somewhere somebody is probably doing some sort of testing, but having worked as a QA engineer I haven't seen anything in MW that would resemble proper and traditional testing (excluding the unit testing). Where's the list of test cases that need to be performed for each release? How can one make new test cases and add them? etc. Maybe this already exists, but if it does it's definitely not documented well enough. Two answers, possibly oversimplified: First, supporting Mediawiki for 3rd parties is not a priority for WMF in recent times, so QA efforts have been focused elsewhere. Second, volunteer QA testing in general *is* a priority for WMF right now, so a volunteer QA effort to test Mediawiki releases would be a likely candidate for WMF support. This sort of project would fall naturally into the effort we're calling Features Testing, and we're looking to support that sort of project by way of a Group http://www.mediawiki.org/wiki/Groups/Proposals/Features_testing. -Chris ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
