Re: [Python-Dev] OpenBSD buildbot has many failures
On 01.04.15 07:52, Davin Potts wrote: I am personally interested in seeing all tests pass on OpenBSD and am willing to put forth effort to help that be so. I would be happy to be added to any issues that get opened against OpenBSD. That said, I have concerns about the nature of when and how these failures came about — specifically I worry that other devs have committed the changes which prompted these failures yet they did not pay attention nor take responsibility when it happened. Having monitored certain buildbots for a while to see how the community behaves and devs fail to react when a failure is triggered by a commit, I think we should do much better in taking individual responsibility for prompting these failures. http://bugs.python.org/issue?%40columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignee%2Cstatus%2Ctype&%40filter=status&%40search_text=openbsd&submit=search&status=1 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OpenBSD buildbot has many failures
Le mercredi 1 avril 2015, Davin Potts a écrit : > > I am personally interested in seeing all tests pass on OpenBSD and am > willing to put forth effort to help that be so. Great! > I would be happy to be added to any issues that get opened against > OpenBSD. You can do a search in bugs.python.org. If I recall recorrectly, there are only a few open issues. > That said, I have concerns about the nature of when and how these failures > came about — specifically I worry that other devs have committed the > changes which prompted these failures yet they did not pay attention nor > take responsibility when it happened. That's the purpose of my previous email. If the buildbot is always red, nobody will check it anymore. I'm quite sure that the current OpenBSD 5.5 was always red since its setup. Sometimes, I try to fix some isssues. Fixing OpenBSD issues usually means install OpenBSD at home because most issues are specific to OpenBSD. Even for LibreSSL, OpenBSD behaves differently (on the version number) than FreeBSD. Having monitored certain buildbots for a while to see how the community > behaves and devs fail to react when a failure is triggered by a commit, I > think we should do much better in taking individual responsibility for > prompting these failures. When I started to work on Python, I was surprised to not get emails from buildbots. There are different issues. Dome buildbots are always red. Threre are still many sporadic issues: temporary network failures seen as bugs, threads issues. Multiprocessing issues. Recently, I saw some rare asyncio issues. So we reduce the number of these issues before spaming developers. Currently, there is an IRC bot on #python-dev which notify when buildbot color changes. Or sometiles I chceck the huge waterfall page. By the way, it became difficult to browse this page because there are too many offline buildbots. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] MemoryError and other bugs on AMD64 OpenIndiana 3.x
Hi, 2015-03-29 21:03 GMT+02:00 Jesus Cea : > I have contacted the machine manager and he has said to me that 16GB > were taken in the "/tmp" directory by buildbot user. He has deleted them > and everything should be fine now. Lets see. > > Anyway if buildbot is leaving garbage behind we should take care of it. > Doing a daily cron for manual cleaning up seems a bit hacky :). Usually, created files are all removed. Serhiy even added a test recently for that in regrtest. The problem is that Python may keep some created files on test failures. > I would be interested, too, in getting an email when one of my buildbots > is red for more than, lets say, 4 days in a row. An email per day would > be fine. There are still MemoryError: http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/9543/steps/test/logs/stdio http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/9542/steps/test/logs/stdio http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/9541/steps/test/logs/stdio etc. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] http.client: Determining Content-Length
On 31 March 2015 at 17:55, Demian Brecht wrote: > Hi all, > > I'm not sure whether this should be python-list or here, but given it's a > premature code review for http.client, I figured I'd post here first. > > I'm in the process of adding support for chunked transfer encoding to > http.client (issue #12319). One of the bits of functionality that I'm working > on in is ironing out some of the kinks out in determining the content length > of the request bodies. Given the number of data types allowed as bodies it > does get a little messy, so I wanted to get some feedback here and see if > anyone can shoot holes through it prior to updating the current patch. The > tests are passing, but there may be use cases not accounted for in the new > implementation. > > https://gist.github.com/demianbrecht/f94be5a51e32bb9c81e1 > > The above is intended to replace _set_content_length (current state of the > patch can be seen here: > http://bugs.python.org/review/12319/diff/14331/Lib/http/client.py). There is > a comprehensive list of data types currently supported can be found here: > http://bugs.python.org/issue23740. Comments and feedback are much appreciated. Nothing problematic leaps out at me here, but only if I look at your proposal in the context of the full patch. Your full patch does seem to have an edge case: for HTTP/1.1, if I pass a generator and no content-length you'll blow up by indexing into it. That's probably fine (your only option in this case would be to frame this request by half-closing the TCP stream, which I seem to recall you ruling out elsewhere), but you might want to wrap the exception in something more helpful: not sure. Cory ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OpenBSD buildbot has many failures
On 01/04/2015 08:32, Victor Stinner wrote: > When I started to work on Python, I was surprised to not get emails from > buildbots. > > Currently, there is an IRC bot on #python-dev which notify when buildbot > color changes. Or sometiles I chceck the huge waterfall page. By the > way, it became difficult to browse this page because there are too many > offline buildbots. On the back of Victor's recent emails re buildbots, I've knocked something up which can be scheduled to email the status of some or all buildbots: https://github.com/tjguk/buildbotinfo It's rough-and-ready but it does work. If it's useful to anyone, feel free to use / clone / fork / whatever. By all means send PRs or raise Issues but I've already overrun the little time I'd given myself to get this working so I'm not sure when I'll get to them. There are two modules: the underlying buildbot.py which lightly wraps the XML-RPC interface; and the buildbotinfo.py which uses it to generate some readable output according to some selection criteria. See the project README for some details: https://github.com/tjguk/buildbotinfo/blob/master/README.rst TJG ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OpenBSD buildbot has many failures
Hi, 2015-04-01 12:47 GMT+02:00 Tim Golden : > On the back of Victor's recent emails re buildbots, I've knocked > something up which can be scheduled to email the status of some or all > buildbots: > > https://github.com/tjguk/buildbotinfo Are you aware of this previous project? https://code.google.com/p/bbreport/ I also wrote two very simple scripts to download and parse buildbot output: https://bitbucket.org/haypo/misc/src/5929cc110f0352cecb384adceae3647f26fa693e/python/buildbot_download.py?at=default https://bitbucket.org/haypo/misc/src/5929cc110f0352cecb384adceae3647f26fa693e/python/buildbot_parse.py?at=default I'm more interested on the parser part. My first goal was to compute a summary in one line: success, timeout, fatal error, etc. The next step would be to parse test failures to be able to ignore tests known to fail, to only detect regressions. It would be better than the current "red" / "green" status. For example, I want to always ignore MemoryError just because I'm unable to fix buildbots which are known to regulary fail with MemoryError (ex: OpenIndiana). I used Jenkins which uses JUnit reports. It's much more powerful because it computes statistics on tests to check which tests fail randomly, which tests are new failures, etc. I don't think that it would be complex to generate a JUnit report (or another easy to parse report) in regrtest. But XML is maybe not the best format to handle timeout and fatal errors :-) Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OpenBSD buildbot has many failures
On 01/04/2015 13:13, Victor Stinner wrote: > Hi, > > 2015-04-01 12:47 GMT+02:00 Tim Golden : >> On the back of Victor's recent emails re buildbots, I've knocked >> something up which can be scheduled to email the status of some or all >> buildbots: >> >> https://github.com/tjguk/buildbotinfo > > Are you aware of this previous project? > https://code.google.com/p/bbreport/ Yes -- I originally forked it with a view to extending it the way I wanted, but I was rewriting so much that in the end I started afresh and ended up with just my own code. (The way you do...) [... snip useful ideas about more finely-tuned results ...] I'd also thought of various directions things could go. TBH, though, I knew if I started to get ambitious I'd end up with something which tried to do everything and didn't do anything. I might go somewhere with the thing in the future, but for now it does what it does and if it helps someone -- including me -- at the simplest level, well that's a good thing. TJG ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] AF_IUCV support for sockets
Hi, I have opened an enhancement request (23830) with a suggested path that adds support for the AF_IUCV protocol family: IUCV is a hypervisor mediated communications method for z/VM guest virtual machines. Linux on z Systems (aka s390x) has supported this via the use of AF_IUCV sockets for many years (added to kernel Feb 2007). This suggested patch adds support to Python 2.7.9 for this socket type. I have built Python on both s390x and x86_64: both build cleanly and the test added to test_socket.py runs cleanly on s390x and is skipped on other platforms. -- components: Library (Lib) files: af_iucv.patch keywords: patch messages: 239743 nosy: neale priority: normal severity: normal status: open title: Add AF_IUCV support to sockets type: enhancement versions: Python 2.7 Added file: http://bugs.python.org/file38765/af_iucv.patch I would like help in improving the patch and making it a candidate for acceptance into the code base. Neale ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Check deques against common sequence tests (except for slicing).
On Wed, Apr 1, 2015 at 10:11 AM, raymond.hettinger wrote: > https://hg.python.org/cpython/rev/393189326adb > changeset: 95350:393189326adb > user:Raymond Hettinger > date:Wed Apr 01 08:11:09 2015 -0700 > summary: > Check deques against common sequence tests (except for slicing). > > files: > Lib/test/test_deque.py | 16 > 1 files changed, 16 insertions(+), 0 deletions(-) > > > diff --git a/Lib/test/test_deque.py b/Lib/test/test_deque.py > --- a/Lib/test/test_deque.py > +++ b/Lib/test/test_deque.py > @@ -843,6 +843,21 @@ > # SF bug #1486663 -- this used to erroneously raise a TypeError > SubclassWithKwargs(newarg=1) > > +class TestSequence(seq_tests.CommonTest): > +type2test = deque > + > +def test_getitem(self): > +# For now, bypass tests that require slicing > +pass > + > +def test_getslice(self): > +# For now, bypass tests that require slicing > +pass > + > +def test_subscript(self): > +# For now, bypass tests that require slicing > +pass Instead of making these empty passing tests, it's better to set them to 'None' so that unittest doesn't run them (and thus doesn't report success when it hasn't actually tested something). -- Zach ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 487 vs 422 (dynamic class decoration)
I recently got an inquiry from some of my users about porting some of my libraries to Python 3 that make use of the Python 2 __metaclass__ facility. While checking up on the status of PEP 422 today, I found out about its recently proposed replacement, PEP 487. While PEP 487 is a generally fine PEP, it actually *rules out* the specific use case that I wanted PEP 422 for in the first place: dynamic addition of callbacks or decorators for use at class creation time without requiring explicit inheritance or metaclass participation. (So that e.g. method decorators can access the enclosing class at class definition time.) As discussed previously prior to the creation of PEP 422, it is not possible to port certain features of my libraries to work on Python 3 without some form of that ability, and the only thing that I know of that could even *potentially* provide that ability outside of PEP 422 is monkeypatching __build_class__ (which might not even work). That is, the very thing that PEP 422 was created to avoid the need for. ;-) One possible alteration would be to replace __init_subclass__ with some sort of __init_class__ invoked on the class that provides it, not just subclasses. That would allow the kind of dynamic decoration that PEP 422 allows. However, this approach was rather specifically ruled out in earlier consideration of PEP 422, so Another alternative would be to have the default __init_subclass__ look at a class-level __decorators__ attribute, as originally discussed for PEP 422. That would solve *my* problem, but feels too much like adding more than One Way To Do It. So... honestly, I'm not sure where to go from here. Is there any chance that this is going to be changed, or revert to the PEP 422 approach, or... something? If so, what Python version will the "something" be in? Or is this use case just going to be a dead parrot in Python 3, period? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 487 vs 422 (dynamic class decoration)
On 2 April 2015 at 07:35, PJ Eby wrote: > I recently got an inquiry from some of my users about porting some of > my libraries to Python 3 that make use of the Python 2 __metaclass__ > facility. While checking up on the status of PEP 422 today, I found > out about its recently proposed replacement, PEP 487. > > While PEP 487 is a generally fine PEP, it actually *rules out* the > specific use case that I wanted PEP 422 for in the first place: > dynamic addition of callbacks or decorators for use at class creation > time without requiring explicit inheritance or metaclass > participation. (So that e.g. method decorators can access the > enclosing class at class definition time.) How hard is the requirement against relying on a mixin class or class decorator to request the defining class aware method decorator support? Is the main concern with the fact that failing to apply the right decorator/mixin at the class level becomes a potentially silent failure where the class aware method decorators aren't invoked properly? > So... honestly, I'm not sure where to go from here. Is there any > chance that this is going to be changed, or revert to the PEP 422 > approach, or... something? If so, what Python version will the > "something" be in? Or is this use case just going to be a dead parrot > in Python 3, period? My preference at this point would definitely be to introduce a mixin class into the affected libraries and frameworks with an appropriate PEP 487 style __init_subclass__ that was a noop in Python 2 (which would rely on metaclass injection instead), but implemented the necessary "defining class aware" method decorator support in Python 3. The question of dynamically injecting additional base classes from the class body to allow the use of certain method decorators to imply specific class level behaviour could then be addressed as a separate proposal (e.g. making the case for an "__append_mixins__" attribute), rather than being linked directly to the question of how we going about defining inherited creation time behaviour without needing a custom metaclass. Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 487 vs 422 (dynamic class decoration)
On Wed, Apr 1, 2015 at 10:39 PM, Nick Coghlan wrote: > On 2 April 2015 at 07:35, PJ Eby wrote: >> I recently got an inquiry from some of my users about porting some of >> my libraries to Python 3 that make use of the Python 2 __metaclass__ >> facility. While checking up on the status of PEP 422 today, I found >> out about its recently proposed replacement, PEP 487. >> >> While PEP 487 is a generally fine PEP, it actually *rules out* the >> specific use case that I wanted PEP 422 for in the first place: >> dynamic addition of callbacks or decorators for use at class creation >> time without requiring explicit inheritance or metaclass >> participation. (So that e.g. method decorators can access the >> enclosing class at class definition time.) > > How hard is the requirement against relying on a mixin class or class > decorator to request the defining class aware method decorator > support? Is the main concern with the fact that failing to apply the > right decorator/mixin at the class level becomes a potentially silent > failure where the class aware method decorators aren't invoked > properly? The concern is twofold: it breaks proper information hiding/DRY, *and* it fails silently. It should not be necessary for clients of package A1 (that uses a decorator built using package B2) to mixin a metaclass or decorator from package C3 (because B2 implemented its decorators using C3), just for package A1's decorator to work properly in the *client package's class*. (And then, of course, this all silently breaks if you forget, and the breakage might happen at the A1, B2, or C3 level.) Without a way to hook into the class creation process, there is no way to verify correctness and prevent the error from passing silently. (OTOH, if there *is* a way to hook into the creation process, the problem is solved: there's no need to mix anything in anyway, because the hook can do whatever the mixin was supposed to do.) The only way PEP 487 could be a solution is if the default `object.__init_subclass__` supported one of the earlier __decorators__ or __autodecorate__ proposals, or if the PEP were for an `__init_class__` that operated on the defining class, instead of operating only on subclasses. (I need to hook the creation of a class that's *being defined*, not the definition of its future subclasses.) > My preference at this point would definitely be to introduce a mixin > class into the affected libraries and frameworks with an appropriate > PEP 487 style __init_subclass__ that was a noop in Python 2 (which > would rely on metaclass injection instead), but implemented the > necessary "defining class aware" method decorator support in Python 3. If this were suitable for the use case, I'd have done it already. DecoratorTools has had a mixin that provides a __class_init__ feature since 2007, which could be ported to Python 3 in a straighforward manner as a third-party module. (It's just a mixin that provides a metaclass; under 3.x it could probably just be a plain metaclass with no mixin.) > The question of dynamically injecting additional base classes from the > class body to allow the use of certain method decorators to imply > specific class level behaviour could then be addressed as a separate > proposal (e.g. making the case for an "__append_mixins__" attribute), > rather than being linked directly to the question of how we going > about defining inherited creation time behaviour without needing a > custom metaclass. Then maybe we should do that first, since PEP 487 doesn't do anything you can't *already* do with a mixin, all the way back to Python 2.2. IOW, there's no need to modify the core just to have *that* feature, since if you control the base class you can already do what PEP 487 does in essentially every version of Python, ever. If that's all PEP 487 is going to do, it should just be a PyPI package on a stdlib-inclusion track, not a change to core Python. It's not actually adding back any of the dynamicness (dynamicity? hookability?) that PEP 3115 took away. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
