Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-04-01 Thread Christopher Jones
On 3/31/12 2:38 PM, Alexey Shein wrote: 1 апреля 2012 г. 0:27 пользователь Johannes Schlüter johan...@schlueters.de написал: On Sat, 2012-03-31 at 13:21 +0500, Alexey Shein wrote: By constantly publishing newsletter with failed / xfail bugs you're telling them That's our current problems.

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-04-01 Thread Ferenc Kovacs
On Sun, Apr 1, 2012 at 9:39 AM, Christopher Jones christopher.jo...@oracle.com wrote: On 3/31/12 2:38 PM, Alexey Shein wrote: 1 апреля 2012 г. 0:27 пользователь Johannes Schlüter johan...@schlueters.de написал: On Sat, 2012-03-31 at 13:21 +0500, Alexey Shein wrote: By constantly

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-04-01 Thread Alexey Shein
1 апреля 2012 г. 2:38 пользователь Alexey Shein con...@gmail.com написал: 1 апреля 2012 г. 0:27 пользователь Johannes Schlüter johan...@schlueters.de написал: On Sat, 2012-03-31 at 13:21 +0500, Alexey Shein wrote: By constantly publishing newsletter with failed / xfail bugs you're telling

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Alexey Shein
30 марта 2012 г. 22:16 пользователь Christopher Jones christopher.jo...@oracle.com написал: On 3/29/12 3:00 PM, Alexey Shein wrote: Hi, internals! I've got a suggestion about refactoring our tests suite. I'd like to remove XFAIL institution and mark all failing tests just as FAIL. XFAIL

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Rasmus Lerdorf
On 03/30/2012 11:25 PM, Alexey Shein wrote: Hmm, that's different. You get a notification if there's some change on that bug (new comment/state changed/patch etc.). If bug didn't change for years, you won't get any notifications - it's more likely you forget about it. That's not true. There

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Stas Malyshev
Hi! 2) Keep devs' attention on known failures - XFAIL doesn't solve that. You remember about them when you run tests and if you want make attention at them. Which devs you are referring to? Why you assume their attention needs help? What I propose is a single *daily* newsletter saying Hey,

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Stas Malyshev
Hi! That's not true. There is a weekly reminder email if you have outstanding open bugs assigned to you. Although I haven't seen one for a little while, so we may finally have given up on that since it was completely ineffective. Actually, this one I'd like to keep - though I'd prefer

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Alexey Shein
31 марта 2012 г. 12:50 пользователь Stas Malyshev smalys...@sugarcrm.com написал: Hi! 2) Keep devs' attention on known failures - XFAIL doesn't solve that. You remember about them when you run tests and if you want make attention at them. Which devs you are referring to? Why you assume

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Alexey Shein
31 марта 2012 г. 12:34 пользователь Rasmus Lerdorf ras...@lerdorf.com написал: On 03/30/2012 11:25 PM, Alexey Shein wrote: Hmm, that's different. You get a notification if there's some change on that bug (new comment/state changed/patch etc.). If bug didn't change for years, you won't get any

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Rasmus Lerdorf
On 03/31/2012 01:21 AM, Alexey Shein wrote: 31 марта 2012 г. 12:50 пользователь Stas Malyshev smalys...@sugarcrm.com написал: Hi! 2) Keep devs' attention on known failures - XFAIL doesn't solve that. You remember about them when you run tests and if you want make attention at them. Which

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Yasuo Ohgaki
2012/3/31 Stas Malyshev smalys...@sugarcrm.com: Hi! That's not true. There is a weekly reminder email if you have outstanding open bugs assigned to you. Although I haven't seen one for a little while, so we may finally have given up on that since it was completely ineffective. Actually,

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Pierre Joye
hi, On Sat, Mar 31, 2012 at 11:56 AM, Yasuo Ohgaki yohg...@ohgaki.net wrote: +1 for monthly. It is the case already and I get them regularly. Cheers, -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org -- PHP Internals - PHP Runtime Development Mailing List To

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Johannes Schlüter
On Fri, 2012-03-30 at 10:16 -0700, Christopher Jones wrote: The XFAIL mechanism reflects the reality of open source that not all bugs are fixed. I wonder what that has to do with open source ... besides maybe TeX there's no non-trivial bug free software. johannes -- PHP Internals - PHP

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Johannes Schlüter
On Sat, 2012-03-31 at 13:27 +0500, Alexey Shein wrote: Ok, we have a weekly reminder to bug maintainers (that maybe not working). Those are working. At least for me :-) There was some trouble with mails for individual changes, but recently I got an you have been assigned mail, too. That's a

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Johannes Schlüter
On Sat, 2012-03-31 at 13:21 +0500, Alexey Shein wrote: By constantly publishing newsletter with failed / xfail bugs you're telling them That's our current problems. Maybe you could help us with them. This way we could convert that discussing energy into some good patches. While many people

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-31 Thread Alexey Shein
1 апреля 2012 г. 0:27 пользователь Johannes Schlüter johan...@schlueters.de написал: On Sat, 2012-03-31 at 13:21 +0500, Alexey Shein wrote: By constantly publishing newsletter with failed / xfail bugs you're telling them That's our current problems. Maybe you could help us with them. This way

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-30 Thread Dirk Haun
Stas Malyshev wrote: And if we had rule of no failing tests, we'd have no releases for years now, because nobody is fixing those tests and bugs behind them. I was about to suggest that maybe PHP should have a rule: no release with failing tests. What's the point of a test that fails (or

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-30 Thread Stas Malyshev
Hi! I was about to suggest that maybe PHP should have a rule: no release with failing tests. In current situation, this rule would be a bit shorter: no release. What's the point of a test that fails (or XFAILs)? Either something is broken - then it should be fixed. Or the test makes no

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-30 Thread Pierre Joye
hi, On Fri, Mar 30, 2012 at 8:02 AM, Dirk Haun d...@haun-online.de wrote: What's the point of a test that fails (or XFAILs)? Either something is broken - then it should be fixed. Or the test makes no sense - then it should be removed. See the archive about the reasons, it has been

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-30 Thread Alexey Shein
30 марта 2012 г. 5:55 пользователь Stas Malyshev smalys...@sugarcrm.com написал: Hi! The difference started from 5.3.9 release when we start to pay *much more* attention to tests. You now can cleanly see failing tests, it's not that huge list, so it's big difference. Yes, and removing

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-30 Thread Christopher Jones
On 3/29/12 3:00 PM, Alexey Shein wrote: Hi, internals! I've got a suggestion about refactoring our tests suite. I'd like to remove XFAIL institution and mark all failing tests just as FAIL. XFAIL has a problem that it hides attention from failing tests depending on not yet fixed bugs (most

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-30 Thread Christopher Jones
On 3/30/12 4:20 AM, Alexey Shein wrote: That's not a noise. See p.1 above. If we don't setup *constant* notifications, people won't feel pressure. We do get constant notification of bugs assigned to us. I don't believe it has any impact on the fix rate. We need a balance between carrot

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-29 Thread Stas Malyshev
Hi! I've got a suggestion about refactoring our tests suite. I'd like to remove XFAIL institution and mark all failing tests just as FAIL. XFAIL has a problem that it hides attention from failing tests depending on not yet fixed bugs (most important), not yet implemented features (less

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-29 Thread Ferenc Kovacs
Please note that we were in that position and we moved from there. So to move back, we need some argument about what's different this time from the place we were a year ago. could you elaborate on this part? where were we a year ago? XFAILs serve now as a pain-killers, we've got about

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-29 Thread Alexey Shein
30 марта 2012 г. 3:19 пользователь Stas Malyshev smalys...@sugarcrm.com написал: Hi! I've got a suggestion about refactoring our tests suite. I'd like to remove XFAIL institution and mark all failing tests just as FAIL. XFAIL has a problem that it hides attention from failing tests depending

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-29 Thread Stas Malyshev
Hi! could you elaborate on this part? where were we a year ago? We had many failing tests that now XFAILs classified as regular FAILs. yeah, but as we did see, the current approach makes it very easy to hide even the not so small issues (for example the bunch of date related XFAILS which

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-29 Thread Stas Malyshev
Hi! The difference started from 5.3.9 release when we start to pay *much more* attention to tests. You now can cleanly see failing tests, it's not that huge list, so it's big difference. Yes, and removing XFAILs would kill that advantage. The main idea I'm trying to say is that it's

Re: [PHP-DEV] Change all XFAIL tests to FAIL

2012-03-29 Thread Yasuo Ohgaki
Hi, As a distribution maintainer, I would like to distinct failed test that is ok and failed test that is not ok. I think release versions should not have test that fails. How about add Dev and Release mode for tests with Dev mode as default? -- Yasuo Ohgaki yohg...@ohgaki.net -- PHP