Re: [PHP-DEV] Tests repository

2010-03-17 Thread Geoffrey Sneddon
On 12 Mar 2010, at 12:37, Jani Taskinen wrote: On 03/12/2010 12:29 PM, Hannes Magnusson wrote: On Fri, Mar 12, 2010 at 11:18, Jani Taskinen wrote: Having tests in multiple branches is PITA. Hasn't anyone considered that the best way would be to move all tests into their own repository (di

Re: [PHP-DEV] Tests repository

2010-03-17 Thread Geoffrey Sneddon
On 12 Mar 2010, at 20:23, Pierre Joye wrote: On Fri, Mar 12, 2010 at 8:10 PM, Alexey Zakhlestin wrote: On 12.03.2010, at 22:06, Pierre Joye wrote: Many tests fail because they are written for a given platform, or even worst (from a portability point of view), only for a given configurat

Re: [PHP-DEV] Tests repository

2010-03-17 Thread Geoffrey Sneddon
On 12 Mar 2010, at 23:33, Jani Taskinen wrote: 13.3.2010 0:18, Stanislav Malyshev wrote: Hi! There are going to be some technical challenges. Some (maybe a lot) of test will need updates or rewriting. run-tests.php may need more improvements than what is already planned. Knowing this, I

Re: [PHP-DEV] Tests repository

2010-03-17 Thread Derick Rethans
On Fri, 12 Mar 2010, Daniel Convissor wrote: > On Fri, Mar 12, 2010 at 06:53:49PM -0500, Eric Stewart wrote: > > > > If I'm given the choice of those differences being in one file or in three > > different files (5.2, 5.3, 6.0)? I choose one file. Even if that files is > > more complex. It's stil

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Rafael Dohms
On Fri, Mar 12, 2010 at 9:52 PM, Pierre Joye wrote > On Sat, Mar 13, 2010 at 12:27 AM, Stanislav Malyshev wrote: >> Hi! >> It is possible, but that means instead of 2 trees of files you'd have these >> trees distributed inside test files in a myriad of small if()s: >> if(version == X) { test thi

Re: [PHP-DEV] Tests repository (run-tests.php)

2010-03-12 Thread Hannes Magnusson
On Fri, Mar 12, 2010 at 22:18, Eric Stewart wrote: > On Fri, Mar 12, 2010 at 6:37 AM, Jani Taskinen wrote: > >> On 03/12/2010 12:29 PM, Hannes Magnusson wrote: >> >>> On Fri, Mar 12, 2010 at 11:18, Jani Taskinen >>>  wrote: >>> Having tests in multiple branches is PITA. Hasn't anyone conside

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Daniel Convissor
On Fri, Mar 12, 2010 at 06:53:49PM -0500, Eric Stewart wrote: > > If I'm given the choice of those differences being in one file or in three > different files (5.2, 5.3, 6.0)? I choose one file. Even if that files is > more complex. It's still one file. One commit. One place to look for > problems

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Pierre Joye
On Sat, Mar 13, 2010 at 12:27 AM, Stanislav Malyshev wrote: > Hi! > >> It is always possible to write a test for both versions (except >> namespace). However it requires a radical change on how we test >> things. >> Instead of testing the output of given script, we will have to add >> logics in a

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Eric Stewart
On Fri, Mar 12, 2010 at 6:27 PM, Stanislav Malyshev wrote: > Hi! > > > It is always possible to write a test for both versions (except >> namespace). However it requires a radical change on how we test >> things. >> Instead of testing the output of given script, we will have to add >> logics in

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Stanislav Malyshev
Hi! It is always possible to write a test for both versions (except namespace). However it requires a radical change on how we test things. Instead of testing the output of given script, we will have to add logics in a test, something similar to the classic xUnit frameworks. For example, we will

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Pierre Joye
On Sat, Mar 13, 2010 at 12:04 AM, Eric Stewart wrote: > I think specific examples (ie: a link to a 5.2 test and the corresponding > 5.3 test) which are currently impossible to merge would be most beneficial > at this point. > > I'll search for some this evening. If anyone knows of one or two off

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Stanislav Malyshev
Hi! I think specific examples (ie: a link to a 5.2 test and the corresponding 5.3 test) which are currently impossible to merge would be most beneficial at this point. Do a diff between two test dirs (you can use engine tests, php main tests, etc.), get a list of different files and diff them

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Stanislav Malyshev
Hi! What tests are you really talking about here? I thought we have regression tests in there which test that stuff does not change between Yes, we have those. But we also have other tests, which are different between versions. should not be any need for any updates given the tests aren't

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Eric Stewart
On Fri, Mar 12, 2010 at 5:39 PM, Pierre Joye wrote: > On Fri, Mar 12, 2010 at 11:33 PM, Jani Taskinen > wrote: > > 13.3.2010 0:18, Stanislav Malyshev wrote: > >> > >> Hi! > >> > >>> There are going to be some technical challenges. Some (maybe a lot) of > >>> test > >>> will need updates or rewri

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Pierre Joye
On Fri, Mar 12, 2010 at 11:33 PM, Jani Taskinen wrote: > 13.3.2010 0:18, Stanislav Malyshev wrote: >> >> Hi! >> >>> There are going to be some technical challenges. Some (maybe a lot) of >>> test >>> will need updates or rewriting. run-tests.php may need more improvements >>> than what is already

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Jani Taskinen
13.3.2010 0:18, Stanislav Malyshev wrote: Hi! There are going to be some technical challenges. Some (maybe a lot) of test will need updates or rewriting. run-tests.php may need more improvements than what is already planned. Knowing this, I would still rather update run-tests.php and fix the te

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Stanislav Malyshev
Hi! There are going to be some technical challenges. Some (maybe a lot) of test will need updates or rewriting. run-tests.php may need more improvements than what is already planned. Knowing this, I would still rather update run-tests.php and fix the tests, then continue to applying tests to dif

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Eric Stewart
On Fri, Mar 12, 2010 at 4:55 PM, Stanislav Malyshev wrote: > Hi! > > > That's easy to say - but how exactly you're going to test >>> functionality of, say. ext/intl without testing the underlying ICU >>> library? >>> >> >> Well, here's the way I see it: >> >> Extensions (including ext/intl) decl

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Stanislav Malyshev
Hi! That's easy to say - but how exactly you're going to test functionality of, say. ext/intl without testing the underlying ICU library? Well, here's the way I see it: Extensions (including ext/intl) declare their API and tests are made againt this API. No more and no less. It's all sounds

Re: [PHP-DEV] Tests repository (run-tests.php)

2010-03-12 Thread Eric Stewart
On Fri, Mar 12, 2010 at 6:37 AM, Jani Taskinen wrote: > On 03/12/2010 12:29 PM, Hannes Magnusson wrote: > >> On Fri, Mar 12, 2010 at 11:18, Jani Taskinen >> wrote: >> >>> Having tests in multiple branches is PITA. Hasn't anyone considered that >>> the >>> best way would be to move all tests into

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Pierre Joye
On Fri, Mar 12, 2010 at 8:39 PM, Alexey Zakhlestin wrote: > > On 12.03.2010, at 22:23, Pierre Joye wrote: > >> On Fri, Mar 12, 2010 at 8:10 PM, Alexey Zakhlestin >> wrote: >>> >>> On 12.03.2010, at 22:06, Pierre Joye wrote: >>> Many tests fail because they are written for a given platform,

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Alexey Zakhlestin
On 12.03.2010, at 22:23, Pierre Joye wrote: > On Fri, Mar 12, 2010 at 8:10 PM, Alexey Zakhlestin wrote: >> >> On 12.03.2010, at 22:06, Pierre Joye wrote: >> >>> Many tests fail because they are written for a given platform, or even >>> worst (from a portability point of view), only for a given

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Alexey Zakhlestin
On 12.03.2010, at 22:20, Stanislav Malyshev wrote: > Hi! > >> Well, these tests should just be removed/rewritten. >> Php-tests should test php, not libraries > > That's easy to say - but how exactly you're going to test functionality of, > say. ext/intl without testing the underlying ICU libra

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Pierre Joye
On Fri, Mar 12, 2010 at 8:10 PM, Alexey Zakhlestin wrote: > > On 12.03.2010, at 22:06, Pierre Joye wrote: > >> Many tests fail because they are written for a given platform, or even >> worst (from a portability point of view), only for a given >> configuration (library version, system version,etc.

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Stanislav Malyshev
Hi! Well, these tests should just be removed/rewritten. Php-tests should test php, not libraries That's easy to say - but how exactly you're going to test functionality of, say. ext/intl without testing the underlying ICU library? -- Stanislav Malyshev, Zend Software Architect s...@zend.com

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Alexey Zakhlestin
On 12.03.2010, at 22:06, Pierre Joye wrote: > Many tests fail because they are written for a given platform, or even > worst (from a portability point of view), only for a given > configuration (library version, system version,etc.). And that's not > about windows vs other, tests can work on a De

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Pierre Joye
On Fri, Mar 12, 2010 at 7:14 PM, Jani Taskinen wrote: > On 03/12/2010 08:08 PM, Stanislav Malyshev wrote: >> >> Hi! >> >>> Having tests in multiple branches is PITA. Hasn't anyone considered that >>> the best way would be to move all tests into their own repository >>> (directory..whatever :) in S

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Stanislav Malyshev
Hi! That's why we'd need to add some section to select the minimum version required to run the test. How minimum version would help? If 5.2 requires one test, 5.3 another and trunk another - you either have to have 3 tests or declare that only thing you're testing is trunk (which is obviousl

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Jani Taskinen
On 03/12/2010 04:15 PM, Ulf Wendel wrote: For a transition period there's likely to be more work and the number of test failures is likely to go up. That is nothing to really worry about as long as you manage to educate users that it is not a quality defect of PHP as such but a temporary matter o

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Jani Taskinen
On 03/12/2010 08:08 PM, Stanislav Malyshev wrote: Hi! Having tests in multiple branches is PITA. Hasn't anyone considered that the best way would be to move all tests into their own repository (directory..whatever :) in SVN..? Considering they are supposed to be Yes, but: some tests are versi

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Stanislav Malyshev
Hi! Having tests in multiple branches is PITA. Hasn't anyone considered that the best way would be to move all tests into their own repository (directory..whatever :) in SVN..? Considering they are supposed to be Yes, but: some tests are version-dependant, some are not. And since we have this

Re: [PHP-DEV] Tests repository (was: Re: [PHP-DEV] PHP 6)

2010-03-12 Thread Gwynne Raskind
On Mar 12, 2010, at 5:33 AM, Pierre Joye wrote: >> Having tests in multiple branches is PITA. Hasn't anyone considered that the >> best way would be to move all tests into their own repository >> (directory..whatever :) in SVN..? Considering they are supposed to be used >> for testing against regre

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Ulf Wendel
Jani Taskinen schrieb: What you risk is that not each and every test is prepared for being run with every version - although, maybe, in theory it should be. This is It should not be theory for regression tests? If new release does not pass the old tests but the old versions still do, then it's

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Jani Taskinen
On 03/12/2010 01:48 PM, Ulf Wendel wrote: Jani Taskinen schrieb: Having tests in multiple branches is PITA. Hasn't anyone considered that the best way would be to move all tests into their own repository (directory..whatever :) in SVN..? Considering they are supposed to be used for testing again

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Jordi Boggiano
On 12.03.2010 12:37, Jani Taskinen wrote: >> Some tests however are not supposed to work in earlier releases, so we >> need to either add a new >> ==SKIP-VERSION== >> 5.2, 5.1, 5.0 > > Perhaps something like required min version is better. Imo you need both >V.E.R, signature.asc Description: Op

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Ulf Wendel
Jani Taskinen schrieb: Having tests in multiple branches is PITA. Hasn't anyone considered that the best way would be to move all tests into their own repository (directory..whatever :) in SVN..? Considering they are supposed to be used for testing against regressions and BC breaks, they should

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Jani Taskinen
On 03/12/2010 12:29 PM, Hannes Magnusson wrote: On Fri, Mar 12, 2010 at 11:18, Jani Taskinen wrote: Having tests in multiple branches is PITA. Hasn't anyone considered that the best way would be to move all tests into their own repository (directory..whatever :) in SVN..? Considering they are s

Re: [PHP-DEV] Tests repository

2010-03-12 Thread Lester Caine
Hannes Magnusson wrote: On Fri, Mar 12, 2010 at 11:18, Jani Taskinen wrote: Having tests in multiple branches is PITA. Hasn't anyone considered that the best way would be to move all tests into their own repository (directory..whatever :) in SVN..? Considering they are supposed to be used for t

Re: [PHP-DEV] Tests repository (was: Re: [PHP-DEV] PHP 6)

2010-03-12 Thread Pierre Joye
On Fri, Mar 12, 2010 at 11:18 AM, Jani Taskinen wrote: > Having tests in multiple branches is PITA. Hasn't anyone considered that the > best way would be to move all tests into their own repository > (directory..whatever :) in SVN..? Considering they are supposed to be used > for testing against r

Re: [PHP-DEV] Tests repository (was: Re: [PHP-DEV] PHP 6)

2010-03-12 Thread Hannes Magnusson
On Fri, Mar 12, 2010 at 11:18, Jani Taskinen wrote: > Having tests in multiple branches is PITA. Hasn't anyone considered that the > best way would be to move all tests into their own repository > (directory..whatever :) in SVN..? Considering they are supposed to be used > for testing against regr