Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Wed, Mar 6, 2013 at 4:45 PM, Lennart Regebro rege...@gmail.com wrote: Perhaps we can solve this outside distutils-sig so that distutils-sig can concentrate on the harder problems? It's a distutils-sig problem because you need a way to publish any new testing related metadata, and because we're planning to evolve a hooks system to cover the different steps in the build process in a decoupled manner. Run the tests will be just another hook, but we're not up to dealing with that yet (the only hook that will be in metadata 2.0 is the post-install hook that will bring the wheel format up to the point of being a near-total replacement for ./setup.py install, and even that isn't written up formally yet - it's just a post in a thread on distutils-sig). You could, as Daniel suggested, work on defining a PEP 426 extension as a prototype concept, but it won't help you much until PEP 426 support is widespread, and by then we'll probably be looking at the meta-build system more broadly and figuring out the full set of desired hooks (including test invocation). Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Tue, Mar 5, 2013 at 5:02 PM, Lennart Regebro rege...@gmail.com wrote: I don't care much what that mechanism is, but I think the easiest way to get there is to tell people to extend distutils with a test command (or use Distribute) and perhaps add such a command in 3.4 that will do the unittest discover thingy. I remember looking into zope.testrunner hooking into that mechanism as well, but I don't remember what the outcome was. There is no easy way forward at this point in time. There just isn't. If people want to dispute that claim, please feel free to solve all the other problems distutils-sig is trying to tackle, so we can pay attention to this one. We'll get to this eventually - there are just several other more important things ahead of it in the queue for packaging and distribution infrastructure enhancements (and python-dev is not the group that will solve them). Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Mar 4, 2013, at 11:13 PM, Robert Collins robe...@robertcollins.net wrote: In principle maybe. Need to talk with the trial developers, nose developers, py.test developers etc - to get consensus on a number of internal API friction points. Some of trial's lessons might be also useful for the stdlib going forward, given the hope of doing some event-loop stuff in the core. But, I feel like this might be too much to cover at the language summit; there could be a test frameworks summit of its own, of about equivalent time and scope, and we'd still have a lot to discuss. Is there a unit testing SIG someone from Twisted ought to be a member of, to represent Trial, and to get consensus on these points going forward? -glyph ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Tue, Mar 5, 2013 at 10:02 AM, Glyph gl...@twistedmatrix.com wrote: On Mar 4, 2013, at 11:13 PM, Robert Collins robe...@robertcollins.net wrote: In principle maybe. Need to talk with the trial developers, nose developers, py.test developers etc - to get consensus on a number of internal API friction points. Some of trial's lessons might be also useful for the stdlib going forward, given the hope of doing some event-loop stuff in the core. But, I feel like this might be too much to cover at the language summit; there could be a test frameworks summit of its own, of about equivalent time and scope, and we'd still have a lot to discuss. Is there a unit testing SIG someone from Twisted ought to be a member of, to represent Trial, and to get consensus on these points going forward? The testing-in-python list is pretty much where most test tool authors hang out, see http://lists.idyll.org/listinfo/testing-in-python Also, maybe related, i am heading the tox effort which many people use to have a frontend for their testing process, see http://tox.testrun.org -- it has a somewhat different focus in that it sets up virtualenv and install test specific dependencies. However, positional arguments (often used to select tests) can be configured to be passed on to the test runner of choice. I was considering extending tox to directly support nose, pytest, unittest and trial drivers and offer a unified (minimal) command line API. Am open to collaboration on that. cheers, holger -glyph ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/holger.krekel%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On 5 Mar 2013, at 07:19, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 4 Mar 2013 15:47:37 -0800 Eli Bendersky eli...@gmail.com wrote: On Mon, Mar 4, 2013 at 1:28 PM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 4 Mar 2013 13:26:57 -0800 Eli Bendersky eli...@gmail.com wrote: [Splitting into a separate thread] Do we really need to overthink something that requires a trivial alias to set up for one's own convenience? Picking a Python version (as Barry mentions) is just one of the problems. What's wrong with: alias rupytests='python3 -m unittest discover alias runpytests2='python2 -m unittest discover ? Don't get me wrong, I love the discover option and agree that it should be the recommended way to go - but isn't this largely a documentation issue? I would personally call it a typing issue :-) python -m unittest discover is just too long. Command-line options for advanced capabilities can get long, yes. The whole point is that discovery is not advanced capability, it's pretty basic by today's standards. So it should actually be the default behaviour (like it is with nose). For Python 3.3 onwards python -m unittest does run test discovery by default. However if you want to provide parameters you still need the discover subcommand to disambiguate from the other command line options. So I agree - a shorthand command would be an improvement. Michael Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On 5 Mar 2013, at 09:02, Glyph gl...@twistedmatrix.com wrote: On Mar 4, 2013, at 11:13 PM, Robert Collins robe...@robertcollins.net wrote: In principle maybe. Need to talk with the trial developers, nose developers, py.test developers etc - to get consensus on a number of internal API friction points. Some of trial's lessons might be also useful for the stdlib going forward, given the hope of doing some event-loop stuff in the core. But, I feel like this might be too much to cover at the language summit; there could be a test frameworks summit of its own, of about equivalent time and scope, and we'd still have a lot to discuss. Is there a unit testing SIG someone from Twisted ought to be a member of, to represent Trial, and to get consensus on these points going forward? The testing-on-python mailing list is probably the best place (and if doesn't have that status already I'd be keen to elevate it to official sig for Python testing issues status). http://lists.idyll.org/listinfo/testing-in-python Like the massively distributed testing use case, I'd be very happy for the standard library testing capabilities to better support this use case - but I wouldn't like to design their apis around that as the sole use case. :-) Michael -glyph ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Mar 05, 2013, at 02:11 AM, Donald Stufft wrote: Doesn't setuptools/distribute already have a setup.py test command? That seems like the easiest way forward? Yes, and in theory it can make `python setup.py test` work well. But there are lots of little details (such as API differences for ensuring that doctests run, additional tests discovery, etc.) that make this often not work so well in practice. Some of that is social and some of it is technical. I still claim that including test suite information in a package's metadata would be a win, but maybe that's just too much to hope for right now. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Tue, Mar 5, 2013 at 2:49 PM, Barry Warsaw ba...@python.org wrote: On Mar 05, 2013, at 02:11 AM, Donald Stufft wrote: Doesn't setuptools/distribute already have a setup.py test command? That seems like the easiest way forward? Yes, and in theory it can make `python setup.py test` work well. But there are lots of little details (such as API differences for ensuring that doctests run, additional tests discovery, etc.) that make this often not work so well in practice. Some of that is social and some of it is technical. I still claim that including test suite information in a package's metadata would be a win, but maybe that's just too much to hope for right now. It would be a win, but parsing the metadata is just not what happens right now, let alone writing anything about which and where the modules are defined in the sdist. We can barely install packages by using the dependency metadata from PKG-INFO; pip always re-generates it from setup.py egg_info. Your testing metadata prototype would only have to write two lines to the metadata instead of one a-la: Extension: flufl; flufl/test_suite: nose.collector; document the extension; write some tool to actually parse the metadata and invoke the tests; it may become a core feature in the next version, or having a monolithic specification may become less important. Thanks, Daniel Holth ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On 6 Mar 2013 05:51, Barry Warsaw ba...@python.org wrote: On Mar 05, 2013, at 02:11 AM, Donald Stufft wrote: Doesn't setuptools/distribute already have a setup.py test command? That seems like the easiest way forward? Yes, and in theory it can make `python setup.py test` work well. But there are lots of little details (such as API differences for ensuring that doctests run, additional tests discovery, etc.) that make this often not work so well in practice. Some of that is social and some of it is technical. I still claim that including test suite information in a package's metadata would be a win, but maybe that's just too much to hope for right now. I think it's the right answer, too, but PEP 426 is already huge, so metadata 2.1 is likely the earliest we will try to tackle the problem. Cheers, Nick. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Tue, Mar 5, 2013 at 8:11 AM, Donald Stufft donald.stu...@gmail.com wrote: I don't care much what that mechanism is, but I think the easiest way to get there is to tell people to extend distutils with a test command (or use Distribute) and perhaps add such a command in 3.4 that will do the unittest discover thingy. I remember looking into zope.testrunner hooking into that mechanism as well, but I don't remember what the outcome was. Doesn't setuptools/distribute already have a setup.py test command? Yes, but distutils do not. That seems like the easiest way forward? Yup. Although I can understand people if they want something that is independent of packaging/distribution. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Tue, Mar 5, 2013 at 8:13 AM, Robert Collins robe...@robertcollins.net wrote: On 5 March 2013 20:02, Lennart Regebro rege...@gmail.com wrote: What's needed here is not a tool that can run all unittests in existence, but an official way for automated tools to run tests, with the ability for any test and test framework to hook into that, so that you can run any test suite automatically from an automated tool. The, once that mechanism has been identified/implemented, we need to tell everybody to do this. I think the command line is the right place to do that - declare as metadata the command line to run a packages tests. Yeah, that's good and simple solution. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Tue, Mar 5, 2013 at 9:25 AM, Nick Coghlan ncogh...@gmail.com wrote: On Tue, Mar 5, 2013 at 5:02 PM, Lennart Regebro rege...@gmail.com wrote: I don't care much what that mechanism is, but I think the easiest way to get there is to tell people to extend distutils with a test command (or use Distribute) and perhaps add such a command in 3.4 that will do the unittest discover thingy. I remember looking into zope.testrunner hooking into that mechanism as well, but I don't remember what the outcome was. There is no easy way forward at this point in time. There just isn't. If people want to dispute that claim, please feel free to solve all the other problems distutils-sig is trying to tackle, so we can pay attention to this one. I have to admit that of all the packaging problems out there, this is one of the easiest ones. ;-) That said, it's not easy. We'll get to this eventually - there are just several other more important things ahead of it in the queue for packaging and distribution infrastructure enhancements (and python-dev is not the group that will solve them). To be honest I'm not sure distutils-sig is the right place for this. It's really only a packaging problem because Setuptools has a test command. :-) Perhaps we can solve this outside distutils-sig so that distutils-sig can concentrate on the harder problems? //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
[Splitting into a separate thread] Do we really need to overthink something that requires a trivial alias to set up for one's own convenience? Picking a Python version (as Barry mentions) is just one of the problems. What's wrong with: alias rupytests='python3 -m unittest discover alias runpytests2='python2 -m unittest discover ? Don't get me wrong, I love the discover option and agree that it should be the recommended way to go - but isn't this largely a documentation issue? Eli On Mon, Mar 4, 2013 at 11:14 AM, Barry Warsaw ba...@python.org wrote: On Mar 04, 2013, at 07:41 PM, Antoine Pitrou wrote: $ python -m unittest discover $ python setup.py test $ python setup.py nosetests $ python -m nose test $ nosetests-X.Y Besides having a multitude of choices, there's almost no way to automatically discover (e.g. by metadata inspection or some such) how to invoke the tests. You're often lucky if there's a README.test and it's still accurate. I hope we can have a pytest utility that does the right thing in 3.4 :-) Typing python -m unittest discover is too cumbersome. Where is this work being done (e.g. is there a PEP)? One thing to keep in mind is how to invoke this on a system with multiple versions of Python available. For example, in Debian, a decision was recently made to drop all the nosetests-X.Y scripts from /usr/bin[1]. This makes sense when you think about having at least two major versions of Python (2.x and 3.x) and maybe up to four (2.6, 2.7, 3.2, 3.3), *plus* debug versions of each. Add to that, we don't actually know at package build time which versions of Python you might have installed on your system. A suggestion was made to provide a main entry point so that `pythonX.Y -m nose` would work, which makes sense to me and was adopted by the nose-devs[2]. So while a top level `pytest` command may make sense, it also might not ;). While PEP 426 has a way to declare test dependencies (a good thing), it seems to have no way to declare how to actually run the tests. Cheers, -Barry [1] Start of thread: http://comments.gmane.org/gmane.linux.debian.devel.python/8572 [2] https://github.com/nose-devs/nose/issues/634 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/eliben%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Mon, 4 Mar 2013 13:26:57 -0800 Eli Bendersky eli...@gmail.com wrote: [Splitting into a separate thread] Do we really need to overthink something that requires a trivial alias to set up for one's own convenience? Picking a Python version (as Barry mentions) is just one of the problems. What's wrong with: alias rupytests='python3 -m unittest discover alias runpytests2='python2 -m unittest discover ? Don't get me wrong, I love the discover option and agree that it should be the recommended way to go - but isn't this largely a documentation issue? I would personally call it a typing issue :-) python -m unittest discover is just too long. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On 5 March 2013 10:26, Eli Bendersky eli...@gmail.com wrote: [Splitting into a separate thread] Do we really need to overthink something that requires a trivial alias to set up for one's own convenience? The big thing is automated tools, not developers. When distributors want to redistribute packages they want to be sure they work. Running the tests is a pretty good signal for that, but having every package slightly different adds to the work they need to do. Being able to do 'setup.py test' consistently, everywhere - that would be great. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Mar 05, 2013, at 11:01 AM, Robert Collins wrote: The big thing is automated tools, not developers. Exactly. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Mon, Mar 4, 2013 at 1:28 PM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 4 Mar 2013 13:26:57 -0800 Eli Bendersky eli...@gmail.com wrote: [Splitting into a separate thread] Do we really need to overthink something that requires a trivial alias to set up for one's own convenience? Picking a Python version (as Barry mentions) is just one of the problems. What's wrong with: alias rupytests='python3 -m unittest discover alias runpytests2='python2 -m unittest discover ? Don't get me wrong, I love the discover option and agree that it should be the recommended way to go - but isn't this largely a documentation issue? I would personally call it a typing issue :-) python -m unittest discover is just too long. Command-line options for advanced capabilities can get long, yes. It's not a reason to add an extra layer, which is extra complexity, IMHO. The user is free to create his own shortcuts if this is too much typing. Moreover, many projects already have a way to run all tests from their root directory. As a case in point, we also have the useful: $ python -m SimpleHTTPServer So why not create a new pyserve command to reduce the amount of typing? Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Mon, Mar 4, 2013 at 2:14 PM, Barry Warsaw ba...@python.org wrote: On Mar 05, 2013, at 11:01 AM, Robert Collins wrote: The big thing is automated tools, not developers. Exactly. I don't understand. Is python -m unittest discover too much typing for automatic tools? If anything, it's much more portable across Python versions since any new coommand/script won't be added before 3.4, while the longer version works in 2.7 and 3.2+ Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On 5 March 2013 12:49, Eli Bendersky eli...@gmail.com wrote: On Mon, Mar 4, 2013 at 2:14 PM, Barry Warsaw ba...@python.org wrote: On Mar 05, 2013, at 11:01 AM, Robert Collins wrote: The big thing is automated tools, not developers. Exactly. I don't understand. Is python -m unittest discover too much typing for automatic tools? If anything, it's much more portable across Python versions since any new coommand/script won't be added before 3.4, while the longer version works in 2.7 and 3.2+ It isn't about length. It is about knowing that *that* is what to type (and btw that exact command cannot run twisted's tests, among many other projects tests). Perhaps we are talking about different things. A top level script to run tests is interesting, but orthogonal to the thing Barry was asking for. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Mon, Mar 4, 2013 at 4:09 PM, Robert Collins robe...@robertcollins.netwrote: On 5 March 2013 12:49, Eli Bendersky eli...@gmail.com wrote: On Mon, Mar 4, 2013 at 2:14 PM, Barry Warsaw ba...@python.org wrote: On Mar 05, 2013, at 11:01 AM, Robert Collins wrote: The big thing is automated tools, not developers. Exactly. I don't understand. Is python -m unittest discover too much typing for automatic tools? If anything, it's much more portable across Python versions since any new coommand/script won't be added before 3.4, while the longer version works in 2.7 and 3.2+ It isn't about length. It is about knowing that *that* is what to type (and btw that exact command cannot run twisted's tests, among many other projects tests). Perhaps we are talking about different things. A top level script to run tests is interesting, but orthogonal to the thing Barry was asking for. Perhaps :-) I'm specifically referring to a new top-level script that will run all unittests in discovery mode from the current directory, as a shortcut to python -m unittest discover. ISTM this is at leas in part what was discussed, and my email was in this context. Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On 5 March 2013 13:35, Eli Bendersky eli...@gmail.com wrote: Perhaps :-) I'm specifically referring to a new top-level script that will run all unittests in discovery mode from the current directory, as a shortcut to python -m unittest discover. ISTM this is at leas in part what was discussed, and my email was in this context. So that is interesting, but its not sufficient to meet the automation need Barry is calling out, unless all test suites can be run by 'python -m unittest discover' with no additional parameters [and a pretty large subset cannot]. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Mar 05, 2013, at 01:09 PM, Robert Collins wrote: It isn't about length. It is about knowing that *that* is what to type (and btw that exact command cannot run twisted's tests, among many other projects tests). Perhaps we are talking about different things. A top level script to run tests is interesting, but orthogonal to the thing Barry was asking for. Right, two different things. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Tue, Mar 5, 2013 at 1:41 AM, Robert Collins robe...@robertcollins.net wrote: So that is interesting, but its not sufficient to meet the automation need Barry is calling out, unless all test suites can be run by 'python -m unittest discover' with no additional parameters [and a pretty large subset cannot]. But can they be changed so they are? That's gotta be the important bit. What's needed here is not a tool that can run all unittests in existence, but an official way for automated tools to run tests, with the ability for any test and test framework to hook into that, so that you can run any test suite automatically from an automated tool. The, once that mechanism has been identified/implemented, we need to tell everybody to do this. I don't care much what that mechanism is, but I think the easiest way to get there is to tell people to extend distutils with a test command (or use Distribute) and perhaps add such a command in 3.4 that will do the unittest discover thingy. I remember looking into zope.testrunner hooking into that mechanism as well, but I don't remember what the outcome was. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Tuesday, March 5, 2013 at 2:02 AM, Lennart Regebro wrote: On Tue, Mar 5, 2013 at 1:41 AM, Robert Collins robe...@robertcollins.net (mailto:robe...@robertcollins.net) wrote: So that is interesting, but its not sufficient to meet the automation need Barry is calling out, unless all test suites can be run by 'python -m unittest discover' with no additional parameters [and a pretty large subset cannot]. But can they be changed so they are? That's gotta be the important bit. What's needed here is not a tool that can run all unittests in existence, but an official way for automated tools to run tests, with the ability for any test and test framework to hook into that, so that you can run any test suite automatically from an automated tool. The, once that mechanism has been identified/implemented, we need to tell everybody to do this. I don't care much what that mechanism is, but I think the easiest way to get there is to tell people to extend distutils with a test command (or use Distribute) and perhaps add such a command in 3.4 that will do the unittest discover thingy. I remember looking into zope.testrunner hooking into that mechanism as well, but I don't remember what the outcome was. Doesn't setuptools/distribute already have a setup.py test command? That seems like the easiest way forward? //Lennart ___ Python-Dev mailing list Python-Dev@python.org (mailto:Python-Dev@python.org) http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On 5 March 2013 20:02, Lennart Regebro rege...@gmail.com wrote: On Tue, Mar 5, 2013 at 1:41 AM, Robert Collins robe...@robertcollins.net wrote: So that is interesting, but its not sufficient to meet the automation need Barry is calling out, unless all test suites can be run by 'python -m unittest discover' with no additional parameters [and a pretty large subset cannot]. But can they be changed so they are? That's gotta be the important bit. In principle maybe. Need to talk with the trial developers, nose developers, py.test developers etc - to get consensus on a number of internal API friction points. What's needed here is not a tool that can run all unittests in existence, but an official way for automated tools to run tests, with the ability for any test and test framework to hook into that, so that you can run any test suite automatically from an automated tool. The, once that mechanism has been identified/implemented, we need to tell everybody to do this. I think the command line is the right place to do that - declare as metadata the command line to run a packages tests. I don't care much what that mechanism is, but I think the easiest way to get there is to tell people to extend distutils with a test command (or use Distribute) and perhaps add such a command in 3.4 that will do the unittest discover thingy. I remember looking into zope.testrunner hooking into that mechanism as well, but I don't remember what the outcome was. Agreed. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)
On Mon, 4 Mar 2013 15:47:37 -0800 Eli Bendersky eli...@gmail.com wrote: On Mon, Mar 4, 2013 at 1:28 PM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 4 Mar 2013 13:26:57 -0800 Eli Bendersky eli...@gmail.com wrote: [Splitting into a separate thread] Do we really need to overthink something that requires a trivial alias to set up for one's own convenience? Picking a Python version (as Barry mentions) is just one of the problems. What's wrong with: alias rupytests='python3 -m unittest discover alias runpytests2='python2 -m unittest discover ? Don't get me wrong, I love the discover option and agree that it should be the recommended way to go - but isn't this largely a documentation issue? I would personally call it a typing issue :-) python -m unittest discover is just too long. Command-line options for advanced capabilities can get long, yes. The whole point is that discovery is not advanced capability, it's pretty basic by today's standards. So it should actually be the default behaviour (like it is with nose). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
