[ANN] Courses in Colorado: Introduction to Python and Python for Scientists and Engineers - one day left for early bird rate
Python Course in Golden, CO, USA **There is only one day left to take advantage of the early bird rate.** Introduction to Python and Python for Scientists and Engineers -- June 3 - 4, 2011 Introduction to Python June 5, 2011 Python for Scientists and Engineers Both courses can be booked individually or together. Venue: Colorado School of Mines, Golden, CO (20 minutes west of Denver) Trainer: Mike Müller Target Audience --- The introductory course is designed for people with basic programming background. Since it is a general introduction to Python it is suitable for everybody interested in Python. The scientist's course assumes a working knowledge of Python. You will be fine if you take the two-day introduction before hand. The topics are of general interest for scientists and engineers. Even though some examples come from the groundwater modeling domain, they are easy to understand for people without prior knowledge in this field. About the Trainer - Mike Müller, has been teaching Python since 2004. He is the founder of Python Academy and regularly gives open and in-house Python courses as well as tutorials at PyCon US, OSCON, EuroSciPy and PyCon Asia-Pacific. More Information and Course Registration http://igwmc.mines.edu/short-course/intro_python.html -- Mike mmuel...@python-academy.de -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: Trying to understand html.parser.HTMLParser
Andrew Berg, 19.05.2011 02:39: On 2011.05.18 03:30 AM, Stefan Behnel wrote: Well, it pretty clearly states that on the PyPI page, but I also added it to the project home page now. lxml 2.3 works with any CPython version from 2.3 to 3.2. Thank you. I never would've looked at PyPI for info on a project that has its own site. You should, especially for standardised information like this. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: python logging
You are right that behavior isn't documented and might be a bug. You could
report it.
Bye
El 19 de mayo de 2011 00:42, Ian Kelly ian.g.ke...@gmail.com escribió:
2011/5/18 Ian Kelly ian.g.ke...@gmail.com:
Ah, that's it. I was using Python 2.5. Using 2.7 I get the same
result that you do.
Still, it's a surprising change that doesn't seem to be documented as
such. I'm not sure whether it's a regression or an intentional
change.
I was wrong, it's more complicated than that.
Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit
(Intel)] on win32
Type help, copyright, credits or license for more information.
import logging
logging.getLogger('log').warning('test')
No handlers could be found for logger log
logging.warning('test')
WARNING:root:test
logging.getLogger('log').warning('test')
WARNING:log:test
Apparently, getLogger() is unconfigured by default, but if you just
use the root logger once, then they magically get configured.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to select Python web frameworks and which one is the best framework ?
On Thu, May 19, 2011 at 9:23 AM, VGNU Linux vgnuli...@gmail.com wrote: On Tue, May 17, 2011 at 1:20 PM, VGNU Linux vgnuli...@gmail.com wrote: Hi all, I am confused on which web framework to select for developing a small data driven web application. Application will have features generally found in now-a-days web application like security, database connectivity, authentication etc. I found few web frameworks over the net like django, zope 2/3, pylons, turbogears, web2py, grok etc. etc. I just want to know that how developers/managers/organizations select a framework which best suits their needs ? what are the parameters for selection ? also which is the best and widely used web framework for python ? I apologize if this is a repeated question. Please guys help as I am novice to web development. Search the list archives. This topic has been discussed many many times. -- regards, kushal -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Thu, 19 May 2011 06:21:08 +0100, Hans Georg Schaathun wrote: : Are you talking about the Mayfair classical cipher here? I am talking about the system used in public transport cards like Oyster and Octopus. I am not sure how classical it is, or whether mayfair/mayfare referred to the system or just a cipher. I think Geremy is talking about the Playfair cipher: http://en.wikipedia.org/wiki/Playfair_cipher Any way, it was broken, and it took years. You don't know that. All you know is that it took years for people to realise that it had been broken, when a security researcher publicly announced the MIFARE cipher had been broken. If criminals had broken the cipher, they would have had no incentive to publicize the fact, and the companies running Oyster and similar ticketing schemes would have no incentive to admit they were broken. Far from it: all the incentives are against disclosure. So it's possible that Oyster cards have been counterfeited for years without anyone but the counterfitters, and possibly the Oyster card people themselves, knowing. The real barrier to cracking Oyster cards is not that the source code is unavailable, but that the intersection of the set of those who know how to break encryption, and the set of those who want to break Oyster cards, is relatively small. I don't know how long it took to break the encryption, but I'd guess that it was probably a few days of effort by somebody skilled in the art. http://www.usenix.org/events/sec08/tech/full_papers/nohl/nohl_html/index.html -- Steven -- http://mail.python.org/mailman/listinfo/python-list
pypi mirror
Hi there. I would like to make a local mirror of some packages that are on pypi. What options do You recommend ? I am leaning towards z3c.pypimirror because it was kind of first on my google search results. Regards Sławek -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On 19 May 2011 08:47:28 GMT, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: : The real barrier to cracking Oyster cards is not that the source code is : unavailable, but that the intersection of the set of those who know how : to break encryption, and the set of those who want to break Oyster cards, : is relatively small. I don't know how long it took to break the encryption, : but I'd guess that it was probably a few days of effort by somebody : skilled in the art. : : http://www.usenix.org/events/sec08/tech/full_papers/nohl/nohl_html/index.html In that paper, more than one art seem to have been applied. An open design would have eliminated the need for image analysis and reduced the requirement on hardware/electronics skills. Hence, the obfuscation has made that intersection you talk about smaller, and increased the cost of mounting the attack. As the system was broken anyway, it is hardly a victory for obfuscation, but that's beside the point. The work of that paper is almost certainly more than just «a few days of effort». There are simply to many technical issues to tackle, and they must be tackled one by one. The cost of mounting the attack is to figure out what it takes to do it, before spend the resources barking up the wrong tree. For each successful attack, there probably is a number of failed ones. Thanks for the reference. BTW. That's not the only attack on MIFARE. I cannot remember the details of the other. -- :-- Hans Georg -- http://mail.python.org/mailman/listinfo/python-list
Re: smtplib is broken when using TLS
Please file a ticket on: http://bugs.python.org/ Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/5/17 nirinA raseliarison nirina.raseliari...@gmail.com: i think this has the same origin as the ftplib test failure. Python 3.2.1rc1 (default, May 17 2011, 22:01:34) [GCC 4.6.0] on linux2 Type copyright, credits or license() for more information. RESTART send: 'ehlo [127.0.0.1]\r\n' reply: b'250-mx.google.com at your service, [41.188.13.184]\r\n' reply: b'250-SIZE 35882577\r\n' reply: b'250-8BITMIME\r\n' reply: b'250-STARTTLS\r\n' reply: b'250 ENHANCEDSTATUSCODES\r\n' reply: retcode (250); Msg: b'mx.google.com at your service, [41.188.13.184]\nSIZE 35882577\n8BITMIME\nSTARTTLS\nENHANCEDSTATUSCODES' send: 'STARTTLS\r\n' reply: b'220 2.0.0 Ready to start TLS\r\n' reply: retcode (220); Msg: b'2.0.0 Ready to start TLS' Traceback (most recent call last): File /root/template_mail_text.py, line 49, in module server.starttls() File /usr/local/lib/python3.2/smtplib.py, line 649, in starttls self.sock = ssl.wrap_socket(self.sock, keyfile, certfile) File /usr/local/lib/python3.2/ssl.py, line 509, in wrap_socket ciphers=ciphers) File /usr/local/lib/python3.2/ssl.py, line 266, in __init__ raise x File /usr/local/lib/python3.2/ssl.py, line 262, in __init__ self.do_handshake() File /usr/local/lib/python3.2/ssl.py, line 441, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [Errno 1] _ssl.c:392: error:140943FC:SSL routines:SSL3_READ_BYTES:sslv3 alert bad record mac -- nirinA -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: python logging
On May 18, 11:42 pm, Ian Kelly ian.g.ke...@gmail.com wrote:
I was wrong, it's more complicated than that.
logging.getLogger('log').warning('test')
No handlers could be found for logger loglogging.warning('test')
WARNING:root:test
logging.getLogger('log').warning('test')
WARNING:log:test
Apparently, getLogger() is unconfigured by default, but if you just
use the root logger once, then they magically get configured.
The difference is that you called the module-level convenience
function -
logging.warning('test')
The module-level convenience functions call basicConfig(), which
configures a console handler on the root logger if no handlers are
present there. This is documented at
http://docs.python.org/library/logging.html#logging.log
(see para starting PLEASE NOTE:)
and
http://docs.python.org/howto/logging.html#advanced-logging-tutorial
(search for If you call the functions)
This is not a behaviour change - it's been like this since logging
appeared in Python, see
http://hg.python.org/cpython/annotate/f72b1f8684a2/Lib/logging/__init__.py#l1145
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: test_argparse.py FAILED (failures=6)
There's no point in posting this here. Use the bug tracker: http://bugs.python.org/ --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ 2011/5/17 nirinA raseliarison nirina.raseliari...@gmail.com: == FAIL: test_failures_many_groups_listargs (__main__.TestFileTypeW) -- Traceback (most recent call last): File Lib/test/test_argparse.py, line 216, in wrapper test_func(self) File Lib/test/test_argparse.py, line 235, in test_failures raises(ArgumentParserError, parser.parse_args, args) AssertionError: ArgumentParserError not raised by parse_args == FAIL: test_failures_many_groups_sysargs (__main__.TestFileTypeW) -- Traceback (most recent call last): File Lib/test/test_argparse.py, line 216, in wrapper test_func(self) File Lib/test/test_argparse.py, line 235, in test_failures raises(ArgumentParserError, parser.parse_args, args) AssertionError: ArgumentParserError not raised by parse_args == FAIL: test_failures_no_groups_listargs (__main__.TestFileTypeW) -- Traceback (most recent call last): File Lib/test/test_argparse.py, line 216, in wrapper test_func(self) File Lib/test/test_argparse.py, line 235, in test_failures raises(ArgumentParserError, parser.parse_args, args) AssertionError: ArgumentParserError not raised by parse_args == FAIL: test_failures_no_groups_sysargs (__main__.TestFileTypeW) -- Traceback (most recent call last): File Lib/test/test_argparse.py, line 216, in wrapper test_func(self) File Lib/test/test_argparse.py, line 235, in test_failures raises(ArgumentParserError, parser.parse_args, args) AssertionError: ArgumentParserError not raised by parse_args == FAIL: test_failures_one_group_listargs (__main__.TestFileTypeW) -- Traceback (most recent call last): File Lib/test/test_argparse.py, line 216, in wrapper test_func(self) File Lib/test/test_argparse.py, line 235, in test_failures raises(ArgumentParserError, parser.parse_args, args) AssertionError: ArgumentParserError not raised by parse_args == FAIL: test_failures_one_group_sysargs (__main__.TestFileTypeW) -- Traceback (most recent call last): File Lib/test/test_argparse.py, line 216, in wrapper test_func(self) File Lib/test/test_argparse.py, line 235, in test_failures raises(ArgumentParserError, parser.parse_args, args) AssertionError: ArgumentParserError not raised by parse_args -- Ran 1599 tests in 6.298s FAILED (failures=6) Traceback (most recent call last): File Lib/test/test_argparse.py, line 4681, in module test_main() File Lib/test/test_argparse.py, line 4673, in test_main support.run_unittest(__name__) File /mnt/sda14/pack_build/Python-3.2.1rc1/Lib/test/support.py, line 1184, in run_unittest _run_suite(suite) File /mnt/sda14/pack_build/Python-3.2.1rc1/Lib/test/support.py, line 1167, in _run_suite raise TestFailed(err) test.support.TestFailed: multiple errors occurred -- nirinA -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: connect SIGINT to custom interrupt handler
On Wed, 18 May 2011 07:16:40 -0700, Jean-Paul Calderone wrote: Setting SA_RESTART on SIGINT is probably the right thing to do. It's not totally clear to me from the messages in this thread if you managed to get that approach working. He didn't; select() isn't SA_RESTART-able. Unfortunately, the author of select.select() decided to treat EINTR as an error. It isn't; it's a third way, neither success nor failure, similar to EAGAIN. Normally, you would treat EINTR from select() in the same way as a timeout. While it's possible to work around this, the interface encourages getting it wrong. -- http://mail.python.org/mailman/listinfo/python-list
pexpect: TIMEOUT no longer clears child.before
The behaviour of pexpect has changed between version 2.1 and 2.3. In version 2.1, the following code would result in child.before being cleared -: child.expect(pexpect.TIMEOUT,1) In version 2.3, this is no longer the case. No matter how many times the above code is run, child.before continues to hold the output from previous commands. It is important to be able to clear the contents of child.before between each command. What is the correct way to do this in version 2.3? Adrian. -- http://mail.python.org/mailman/listinfo/python-list
Re: pypi mirror
Hi Sławek, You could also do a local or private egg repository. I documented how I did this for my internal projects here: http://www.sourceweaver.com/posts/private-python-egg-repository I hadn't come across z3c.pymirror before. All the best, Oisin On 19 May 2011 10:12, Slafs slaf...@gmail.com wrote: Hi there. I would like to make a local mirror of some packages that are on pypi. What options do You recommend ? I am leaning towards z3c.pypimirror because it was kind of first on my google search results. Regards Sławek -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
python2+3
[changing subject, seems a good idea...] On May 19, 2:13 am, Terry Reedy tjre...@udel.edu wrote: On 5/18/2011 9:42 AM, lkcl wrote: he's got a good point, terry. breaking backwards-compatibility was a completely mad and incomprehensible decision. I see that I should take everything you (or Harris) say with a big grain of salt;-). *lol* - i realise it's a hell of a lot of work to get a python interpreter to support two different grammars and syntaxes: you can appreciate that i'm in a better position than most to understand that [i've also done a port of python to gnu-win32 so am familiar with the codebase] now that the code for both has been written, tested and proven, the task of adding one t't'othr is a leetle less of a challenging task. there is one teeny tiny annoying challenge: namespace clashes in c, of functions and data structures between the two codebases. however, this isn't as bad as it first seems. it's only the public face (Python.h) which has to remain the same, and that can be taken care of by having data structures that are identical internally and externally (macros to the rescue) where the internal name is Python2_ and the external name Python_xxx. where it becomes less of a maintenance burden is when you say ok, that's it: definitely end-of-the-road for all 2.N development, period. has anyone considered the idea of literally creating a Python2/ subdirectory in the python3 codebase, literally just dropping the entire python2.N code directly into it, renaming all functions and data structures, adding a --2-compatible switch to the python3 argc/ v and seeing what happens? no interoperability, no maintenance, no compatibility - just support for python 2 directly in the python3 binary. l. -- http://mail.python.org/mailman/listinfo/python-list
Gambit REPL app for iPhone/iPod touch/iPad
A version of the Gambit Scheme system for iPhone/iPod touch/iPad is now available on the Apple App Store: http://itunes.apple.com/us/app/gambit-repl/id434534076?mt=8ls=1 Gambit REPL is a complete version of Gambit (http:// dynamo.iro.umontreal.ca/~gambit) including the interpreter, debugger and built-in library (but not the compiler). The standard REPL is provided and also a script editor. Scripts can be saved, attached to function keys, and the main function of the application can be redefined to customize the application to your needs. Applications can be debugged remotely by telneting to the device from a desktop machine. The application is sold for $0.99 as a demonstration that you can distribute on the App Store paid applications written in Scheme and containing an interpreter, something that wasn't clear with previous versions of the App Store developer agreement. A version of Gambit REPL for Android is not planned (by me). The Gambit Scheme system compiles fine on Android (for example see https://github.com/seoushi/gambit-android-example), but the user interface layer would have to be changed and tested, and I don't own an Android device. If someone wants to try porting Gambit REPL to Android let me know and I can help. The sources of Gambit REPL for iOS are in the examples/iOS subdirectory of the Gambit source distribution. Marc -- http://mail.python.org/mailman/listinfo/python-list
Problem with multiprocessing
Hi all,
I'm a bit struggling to understand a KeyError raised by the multiprocessing
library.
My idea is pretty simple. I want to create a server that will spawn a number of
workers that will share the same socket and handle requests independently. The
goal is to build a 3-tier structure where all requests are handled via an http
server and then dispatched to nodes sitting in a cluster and from nodes to
workers via the multiprocessing managers...
There is one public server, one node per machine and x number of workers on
each machine depending on the number of cores... I know I can use a more
sophisticated library, but for such a simple task (I'm just prototyping here) I
would just use the multiprocessing library... Is this possible or I should
explore directly other solutions ? I feel I'm very close to have something
working here ...
The problem with the code below is that if I run the server as
`python server.py 1` , that is, using only one process, it works as expected.
However if I spawn two processes (`python server.py 2`) listening for
connections, I get a nasty error :
$python client.py ping
Traceback (most recent call last):
File client.py, line 24, in module
sys.exit(main(sys.argv))
File client.py, line 21, in main
print m.solver(args[1])._getvalue()
File /usr/lib/python2.6/multiprocessing/managers.py, line 637, in temp
authkey=self._authkey, exposed=exp
File /usr/lib/python2.6/multiprocessing/managers.py, line 894, in
AutoProxy
incref=incref)
File /usr/lib/python2.6/multiprocessing/managers.py, line 700, in
__init__
self._incref()
File /usr/lib/python2.6/multiprocessing/managers.py, line 750, in
_incref
dispatch(conn, None, 'incref', (self._id,))
File /usr/lib/python2.6/multiprocessing/managers.py, line 79, in
dispatch
raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---
Traceback (most recent call last):
File /usr/lib/python2.6/multiprocessing/managers.py, line 181, in
handle_request
result = func(c, *args, **kwds)
File /usr/lib/python2.6/multiprocessing/managers.py, line 402, in incref
self.id_to_refcount[ident] += 1
KeyError: '7fb51084c518'
---
My understanding is that all processes share the same socket (from the
Manager). When a client wants to connect, a new connection is created and
server independently by that process. If you look at the server trace (using
logging), it actually receives the connection, handles it, but fails to
communicate back to the client.
Can anybody shed some light for me and maybe propose a solution ?
thanks
pietro
Server :
import sys
from multiprocessing.managers import BaseManager, BaseProxy, Process
def baz(aa) :
l = []
for i in range(3) :
l.append(aa)
return l
class SolverManager(BaseManager): pass
class MyProxy(BaseProxy): pass
manager = SolverManager(address=('127.0.0.1', 5), authkey='mpm')
manager.register('solver', callable=baz, proxytype=MyProxy)
def serve_forever(server):
try :
server.serve_forever()
except KeyboardInterrupt:
pass
def runpool(n):
server = manager.get_server()
workers = []
for i in range(int(n)):
Process(target=serve_forever, args=(server,)).start()
if __name__ == '__main__':
runpool(sys.argv[1])
Client :
import sys
from multiprocessing.managers import BaseManager, BaseProxy
import multiprocessing, logging
class SolverManager(BaseManager): pass
class MyProxy(BaseProxy): pass
def main(args) :
SolverManager.register('solver')
m = SolverManager(address=('127.0.0.1', 5), authkey='mpm')
m.connect()
print m.solver(args[1])._getvalue()
if __name__ == '__main__':
sys.exit(main(sys.argv))
also tried on stack overflow and the python list, but I didn't manage to come up
with a working solution yet...
--
http://en.wikipedia.org/wiki/Posting_style
--
http://mail.python.org/mailman/listinfo/python-list
Re: pyjamas 0.8alpha1 release
On May 18, 11:02 pm, Terry Reedy tjre...@udel.edu wrote: On 5/18/2011 5:24 AM, lkcl wrote: There seem to be two somewhat separate requirement issues: the interpreter binary and the language version. yes. [with the startling possibility of compiling the entire pyjs compiler into javascript and executing under e.g. spidermonkey's /usr/ bin/js turning that into just the one requirement - but that's another story] a) at the moment ahttp://python.org2.N interpreter is required to actually run the translator. if you usehttp://python.org2.5 or 2.6 you do not need to use the --internal-ast option. if you use 2.4, 2.7 or above, you will need to use --internal-ast because we're heavily reliant on the internal c-based compile module [without the --internal-ast option enabled]. I presume '--internal-ast' is a 'compile the interpreter' option. mmm... sort of. what it does is, in translator.py (the pyjs python- to-javascript compiler) is this: if options.internal_ast: import pyjamas.pgen.compile as compile # python version of compile module else: import compile # standard python compile module that internal compile module contains a complete independent grammar.txt file, and is basically lib2to3 re-ported *back* to being interoperable with, and a complete replacement for, the standard python compile module. including the ast module. we haven't *quite* got round to doing a compile the interpreter yet - it's half way there. the work that daniel kluev is doing, for his gsoc2011 project, will be a complete compile the interpreter. by the time he's done, we'll even have eval() and exec() builtins... inside a web browser. Since I have never compilied Python (or anything else for perhaps 15 years), I do not understand completely. Am I correct to guess that the PSF Windows binaries for 2.7 were not compiled with the flag, and will not work? If so, Windows users should, I would think, use the latest 2.6.6 binaries. ahh... i'm reading this as requiring http://python.org c code to be compiled with different compilation flags. this isn't anything to do with compiling c-code: pyjs the translator is and always will be a pure vanilla python application, requiring absolutely no modifications to the python interpreter whatsoever. the --internal-ast option is an option which is passed to the pyjs application (which is a pure and vanilla python application). here's an example, shows the inter-relationships: /usr/bin/python pyjsbuild.py --internal-ast Hello.py python.exe pyjsbuild.py --internal-ast Hello.py * /usr/bin/python or python.exe - python2.N. * pyjsbuild.py - the pyjs python-to-javascript translator * --internal-ast - an option to pyjsbuild to tell it to use a version of compiler.ast which is written in pure python [and supports the 2.6 grammar] * Hello.py - the pyjamas application, written in python, being translated to javascript, for execution in a web browser. b) the actual pyjs interpreter grammar (AST) was 2.5 but has mostly been updated to 2.6. actual python syntax / features are therefore mostly 2.5, with someone having pointed out that slice has different return results it's hard to say exactly which is best to be picked, 2.5 or 2.6. nobody's needed slice until recently, so it's not an issue that's ever come up before. If I understand this, the safe thing to do is to stick with 2.5 syntax and omit exotic 3.x features put into 2.6 for eventual porting to 3.x. yes. that would be the easiest path. however, we have a teeny problem: pyjamas desktop (pyjd). that's where you get to *really* execute the python code, exactly the same code that you munged through the pyjs compiler. in the pyjd case, what is going on is that it is *python* that has access to the features of the web browser engine (DOM functions, HTML5 etc.) and that is... ho hum, each engine is itself a massive project, and has to be a python c module (except for the MSHTML engine, which is COM). so for pyjd, we *have* to support whatever the latest easiest python 2.N is. that in turn dictates that we have to keep up-to-date with the python 2.N syntax/grammar, and, consequently, it dictates logically that the pyjs compiler, which *must* be interoperable with pyjd, must also support the latest python 2.N syntax/grammar. bummer, huh? :) fortunately, nobody on the pyjamas mailing list has really noticed that there _are_ any differences (!) between 2.5, 6 and 7 - except for daniel kluev, the gsoc2011 student, who is using pyjs with python-pyv8 to actually create a version of a python interpreter... by translating everything to javascript! the reason why they haven't really noticed is because the use-case for pyjamas UI code is much much smaller (due to web browsers being a restricted execution environment - see ongoing discussion below). the thing is - it's worth reiterating: you just... really don't need as much python interoperability for
Re: Problem with multiprocessing
Share as little as possible between your various processes - shared, mutable
state is a parallelism tragedy.
If you can avoid sharing an entire dictionary, do so. It'd probably be
better to dedicate one process to updating your dictionary, and then using a
multiprocessing.Queue to pass delta records from your workers to your
dictionary management process.
Also, I'm inclined to doubt it's going to work well to have multiple
processes doing I/O on the same socket - you'd probably best have a process
that does all the I/O on the socket, and then, again, have one or more
multprocessing.Queue's that pass I/O results/requests around.
On Thu, May 19, 2011 at 6:10 AM, Pietro Abate
pietro.ab...@pps.jussieu.frwrote:
Hi all,
I'm a bit struggling to understand a KeyError raised by the multiprocessing
library.
My idea is pretty simple. I want to create a server that will spawn a
number of
workers that will share the same socket and handle requests independently.
The
goal is to build a 3-tier structure where all requests are handled via an
http
server and then dispatched to nodes sitting in a cluster and from nodes to
workers via the multiprocessing managers...
There is one public server, one node per machine and x number of workers on
each machine depending on the number of cores... I know I can use a more
sophisticated library, but for such a simple task (I'm just prototyping
here) I
would just use the multiprocessing library... Is this possible or I should
explore directly other solutions ? I feel I'm very close to have something
working here ...
The problem with the code below is that if I run the server as
`python server.py 1` , that is, using only one process, it works as
expected.
However if I spawn two processes (`python server.py 2`) listening for
connections, I get a nasty error :
$python client.py ping
Traceback (most recent call last):
File client.py, line 24, in module
sys.exit(main(sys.argv))
File client.py, line 21, in main
print m.solver(args[1])._getvalue()
File /usr/lib/python2.6/multiprocessing/managers.py, line 637, in
temp
authkey=self._authkey, exposed=exp
File /usr/lib/python2.6/multiprocessing/managers.py, line 894, in
AutoProxy
incref=incref)
File /usr/lib/python2.6/multiprocessing/managers.py, line 700, in
__init__
self._incref()
File /usr/lib/python2.6/multiprocessing/managers.py, line 750, in
_incref
dispatch(conn, None, 'incref', (self._id,))
File /usr/lib/python2.6/multiprocessing/managers.py, line 79, in
dispatch
raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---
Traceback (most recent call last):
File /usr/lib/python2.6/multiprocessing/managers.py, line 181, in
handle_request
result = func(c, *args, **kwds)
File /usr/lib/python2.6/multiprocessing/managers.py, line 402, in
incref
self.id_to_refcount[ident] += 1
KeyError: '7fb51084c518'
---
My understanding is that all processes share the same socket (from the
Manager). When a client wants to connect, a new connection is created and
server independently by that process. If you look at the server trace
(using
logging), it actually receives the connection, handles it, but fails to
communicate back to the client.
Can anybody shed some light for me and maybe propose a solution ?
thanks
pietro
Server :
import sys
from multiprocessing.managers import BaseManager, BaseProxy, Process
def baz(aa) :
l = []
for i in range(3) :
l.append(aa)
return l
class SolverManager(BaseManager): pass
class MyProxy(BaseProxy): pass
manager = SolverManager(address=('127.0.0.1', 5), authkey='mpm')
manager.register('solver', callable=baz, proxytype=MyProxy)
def serve_forever(server):
try :
server.serve_forever()
except KeyboardInterrupt:
pass
def runpool(n):
server = manager.get_server()
workers = []
for i in range(int(n)):
Process(target=serve_forever, args=(server,)).start()
if __name__ == '__main__':
runpool(sys.argv[1])
Client :
import sys
from multiprocessing.managers import BaseManager, BaseProxy
import multiprocessing, logging
class SolverManager(BaseManager): pass
class MyProxy(BaseProxy): pass
def main(args) :
SolverManager.register('solver')
m = SolverManager(address=('127.0.0.1', 5), authkey='mpm')
m.connect()
print m.solver(args[1])._getvalue()
if __name__ == '__main__':
sys.exit(main(sys.argv))
also tried on stack overflow and the python list, but I didn't manage to
come up
with a working
how to get PID from subprocess library
hello, I'm using to launch a program by subprocess.getstatusoutput. I'd like to know whether I can get the program ID, in order to avoid another launch. For clarity sake, I'm calling aria2 (the download manager for linux) and I wouldn't like to call one more instance of it. So what will I use to find the PID of the launched program? -- goto /dev/null -- http://mail.python.org/mailman/listinfo/python-list
Re: how to get PID from subprocess library
The best module for doing such things is subprocess. And the Popen object has a pid attribute (http://docs.python.org/library/subprocess.html#subprocess.Popen.pid) -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Wed, May 18, 2011 at 10:21 PM, Hans Georg Schaathun h...@schaathun.net wrote: On Wed, 18 May 2011 14:34:46 -0700, geremy condra debat...@gmail.com wrote: : Systems can be designed that are absolutely secure under reasonable : assumptions. The fact that it has assumptions does not make your : statement true. : (...) : I can't tell if you're trying to play word games with the distinction : between system and module or if you're just saying that you aren't : sure what FIPS actually certifies. Could you please clarify? The distinction between system and module is rather significant. If you only consider modules, you have bounded your problem and drastically limited the complexity. Ah, the 'word games' option. I'm not going to spend a lot of time arguing this one: HSMs are clearly the domain of systems research, are referred to in both technical and nontechnical documents as 'keystone systems', and the FIPS standard under which they are certified specifically calls them systems more times than I care to count. They are, to the people who make and use them, systems, and your attempt at redefinition won't change that. : Are you talking about the Mayfair classical cipher here? I am talking about the system used in public transport cards like Oyster and Octopus. I am not sure how classical it is, or whether mayfair/mayfare referred to the system or just a cipher. Any way, it was broken, and it took years. Ah, MIFARE. That's a different story, and no, I don't believe they would have been broken sooner if the specs were released. The importance (and difficulty) of securing devices like smartcards wasn't really recognized until much later, and certainly people with a foot in both worlds were very rare for a long time. Also remember that DES (with its 56-bit keys) was recertified just a few months before MIFARE (with its 48-bit keys) was first released- it was a different world. : The entire field of formal modeling and verification has grown around : solving this problem. My new favorite in the field is formal models : and techniques for analyzing security protocols, but there are other : works discussing OS kernel verification (which has gotten a lot of : attention lately) and tons of academic literature. Google (scholar) is : the place to go. Sure, but now you are considering modules, rather than systems again. It is when these reliable components are put together to form systems that people fail (empirically). Let me get this straight: your argument is that operating *systems* aren't systems? : If you can't say with confidence that something meets minimum security : standards, the answer is not to try to say it meets high security : standards. So what? The levels of assurance have nothing to do with standards. The levels of assurance refer to the /confidence/ you can have that the standards are met. The increasing levels of assurance don't just signify that you've checked for problems- it certifies that you don't have them, at least insofar as that level of testing is able to find. Insisting that this doesn't, or shouldn't, translate into tighter security doesn't make much sense. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Wed, May 18, 2011 at 7:54 PM, harrismh777 harrismh...@charter.net wrote: Littlefield, Tyler wrote: snip Four resources that you will what to look into, in no particular order: Erickson, Jon, Hacking: The Art of Exploitation, 2nd ed, San Francisco: No Starch Press, 2008. This would be a very good choice. It's a bit light on details, but makes up for it by being exceptionally well-written and very accessible. Anonymous, Maximum Linux Security: A Hacker's Guide to Protecting Your Linux Server and Workstation, Indianapolis: Sams Publishing, 2000. (check for other editions) (this volume is a good read, even for other platforms, but is geared specifically to Linux) This is a good volume, but very dated. I'd probably pass on it. Graves, Kimberly, CEH Certified Ethical Hacker: Study Guide, Indianapolis: Wiley Publishing, 2010. Briefly glancing over the TOC, this actually looks surprisingly good. CEH itself is a joke among black hats, but if this gets down to the nitty-gritty of actually performing the attacks it covers it sounds like a buy. Seitz, Justin, Gray Hat Python: Python Programming for Hackers and Reverse Engineers, San Francisco: No Starch Press, 2009. I'd skip this one, as it isn't really focused on what you want. The web application hacker's handbook is probably more along the lines of what you need, if you're going for a book. There's also an older volume called 'counter hack' that gives a good overview of some of the ways that attacks proceed. Another recommend I'm surprised hasn't popped up already: 'security power tools' is a good way to get your foot in the door. It has a practical, no-nonsense approach and is split into self-contained chapters so you don't waste too much of your time on tools that aren't relevant to you. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Thu, 19 May 2011 10:23:47 -0700, geremy condra debat...@gmail.com wrote: : Let me get this straight: your argument is that operating *systems* : aren't systems? You referred to the kernel and not the system. The complexities of the two are hardly comparable. There probably are different uses of system; in computer security literature¹ it often refers, not only to a product (hardware/software) an actual installation and configuration of that product in a specific context. /I/ did not redefine it. Speaking of reasonable assumptions, one necessary assumption which is particularly dodgy is that whoever deploys and configures it understands all the assumptions and do not break them through ignorance. Is your concern with security purely from a developer's viewpoint, so that you don't have to worry about the context in which it will be deployed? : So what? The levels of assurance have nothing to do with standards. : The levels of assurance refer to the /confidence/ you can have that : the standards are met. : : The increasing levels of assurance don't just signify that you've : checked for problems- it certifies that you don't have them, at least : insofar as that level of testing is able to find. Insisting that this : doesn't, or shouldn't, translate into tighter security doesn't make : much sense. Tighter sure, but the security requirements and the requirement on testing and/or validation are orthogonal scales. The higher levels of assurance are based on formal methods while the lower ones are based primarily on testing. I read your initial comment to imply that if you cannot get satisfactory assurance using the lower levels, you won't get any at the higher levels. That does not make any sense. Obviously, if you were implying that no system passes the lower levels, then of course they won't pass the higher levels, but then, if that's the case, we would all know that we cannot even design /seemingly/ secure systems. And nobody has suggested that so far. ¹ e.g. Dieter Gollmann for just one ref off the top of my head. -- :-- Hans Georg -- http://mail.python.org/mailman/listinfo/python-list
Windows Registry Keys
Hi I have installed a new version of Python27 in a new directory. I want to get this info into the registry so, when I install Numpy, it will use my new Python TIA -Mathew -- http://mail.python.org/mailman/listinfo/python-list
os.access giving incorrect results on Windows
Hi All, Please help me in solving this issue. I want to check the write permissions on a directory on windows from my python script. I tried to use os.access(dirpath, os.W_OK) to check whether the user has write access or not, but it gives me incorrect result. It always gives me False even if the user has write permission. Interestingly this function works just fine on Linux platforms. There is another way to check the write permissions by creating a temporary file inside the directory then removing it. Catch the exception if create file is not allowed to decide the iswritable value. But this is changing the last modification time (timestamp) of the directory which is a critical issue for our application. Please suggest a solution. Thanks Ayaskant- -- http://mail.python.org/mailman/listinfo/python-list
Re: os.access giving incorrect results on Windows
On 19/05/2011 20:37, Ayaskanta Swain wrote: Please help me in solving this issue. I want to check the write permissions on a directory on windows from my python script. I tried to use *os.access(dirpath, os.W_OK)*to check whether the user has write access or not, but it gives me incorrect result. It always gives me False even if the user has write permission. Interestingly this function works just fine on Linux platforms. This is basically issue2528 [1]. The problem is that, although Windows (and Python) expose a version of os.access to match the Posix function, the meaning is so far removed on Windows as to be useless. Really what you need to do is use the AccessCheck API, which is a little bit tortuous, but is intended for this purpose. You can look at the code in my patch for that issue to get an idea of how to do it. Does that help? TJG [1] http://bugs.python.org/issue2528 -- http://mail.python.org/mailman/listinfo/python-list
Re: os.access giving incorrect results on Windows
On 2011.05.19 02:43 PM, Tim Golden wrote: This is basically issue2528 [1]. The problem is that, although Windows (and Python) expose a version of os.access to match the Posix function, the meaning is so far removed on Windows as to be useless. Does this affect just os.W_OK and directories or all of os.access()? In any case, this information should really be reflected in the docs. -- http://mail.python.org/mailman/listinfo/python-list
Re: os.access giving incorrect results on Windows
On 19/05/2011 20:56, Andrew Berg wrote: On 2011.05.19 02:43 PM, Tim Golden wrote: This is basically issue2528 [1]. The problem is that, although Windows (and Python) expose a version of os.access to match the Posix function, the meaning is so far removed on Windows as to be useless. Does this affect just os.W_OK and directories or all of os.access()? In any case, this information should really be reflected in the docs. The current code is very naive: * A R_OK check always succeeds if the file's attributes can be read at all * A W_OK check fails if the file has its DOS read-only attribute set * A W_OK check always succeeds for a directory (because read-only means something else for directories). Would you care to propose some wording for the docs? I'm quite happy to commit if we can come to an agreement. TJG -- http://mail.python.org/mailman/listinfo/python-list
Re: Windows Registry Keys
On Thu, 19 May 2011 21:03:34 +0300, Mathew myea...@jpl.nasa.gov wrote: I have installed a new version of Python27 in a new directory. I want to get this info into the registry so, when I install Numpy, it will use my new Python If I understand correctly (you have multiple Python 2.7 installations and what to make this new one the current), you should change this key: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Python\PythonCore\2.7\InstallPath (ignore Wow6432Node if you are not on 64-bit Windows) That'd would probably be enough for the Numpy installer. I assume you are aware about the changes that may be required to the PATH and file associations. -- http://mail.python.org/mailman/listinfo/python-list
application level monitoring for python
Hi is there any open source library for python that can allow application level monitoring ? For example,application can send per request level/ aggregated monitoring events and some remote server dump it and show in the monitoring graph in real time ? What's best way of doing that ? -- http://mail.python.org/mailman/listinfo/python-list
Re: os.access giving incorrect results on Windows
On 2011.05.19 03:08 PM, Tim Golden wrote: * A R_OK check always succeeds if the file's attributes can be read at all So is this the same as F_OK then, or does it return false if the user isn't allowed to read permissions? * A W_OK check fails if the file has its DOS read-only attribute set DOS attribute? * A W_OK check always succeeds for a directory (because read-only means something else for directories). Would you care to propose some wording for the docs? I'm quite happy to commit if we can come to an agreement. I'm a beginner when it comes to Python, but I could give it a shot. A big red warning box explaining how the code under Windows doesn't use ACLs under the os.access() entry (above the notes) seems appropriate. A warning box under os.W_OK saying something like Under Windows, access() will always indicate that a directory is writable. would also fit. You know more about this than I do. I'm running Windows 7 right now and I have a Python 3.2 interpreter window open if you want me to test/confirm something. :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
On 2011-05-18, Hans Georg Schaathun h...@schaathun.net wrote: Now Mac OS X has maintained the folder concept of older mac generations, and Windows has cloned it. They do not want the user to understand recursive data structures, and therefore, naturally, avoid the word. You imply they want to keep their users ignorant of these structures, as if to keep something valuable from them. Wouldn't it be more honest, more to the point and much simpler to state they don't NEED the user to understand recursive - or indeed any other - data structures? And that the user doesn't NEED to understand or know about them, just to use them? After all they are users. They use their system for fun, learning or work. Even a very competent or advanced use of a tool (computer, car, mobile phone, fridge, TV, radio, toilet) in no way implies an understanding of it's inner workings. Nor the need, nor the desire. PS: Isn't this thread much ado about nothing? :-) It starts with the misconception (or should I say confusion?) between performing a recursive job and using a recursive tool to do it. And then it blazes off in these huge discusions about semantics to define a definition of an abstraction of a alleady theoretical problem. Glorious, just frelling glorious.:-) We have an expression for that. But I'll avoid using it, since it has the word 'masturbation' in it... And PPS: the P(P)S's don't specifically refer to your posting. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to understand html.parser.HTMLParser
On 2011.05.16 02:26 AM, Karim wrote:
Use regular expression for bad HTLM or beautifulSoup (google it), below
a exemple to extract all html links:
Actually, using regex wasn't so bad:
import re
import urllib.request
url = 'http://x264.nl/x264/?dir=./64bit/8bit_depth'
page = str(urllib.request.urlopen(url).read(), encoding='utf-8') #
urlopen() returns a bytes object, need to get a normal string
rev_re = re.compile('revision[0-9][0-9][0-9][0-9]')
num_re = re.compile('[0-9][0-9][0-9][0-9]')
rev = rev_re.findall(str(page))[0] # only need the first item since
the first listing is the latest revision
num = num_re.findall(rev)[0] # findall() always returns a list
print(num)
prints out the revision number - 1995. 'revision1995' might be useful,
so I saved that to rev.
This actually works pretty well for consistently formatted lists. I
suppose I went about this the wrong way - I thought I needed to parse
the HTML to get the links and do simple regexes on those, but I can just
do simple regexes on the entire HTML document.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Trying to understand html.parser.HTMLParser
On 05/19/2011 11:35 PM, Andrew Berg wrote:
On 2011.05.16 02:26 AM, Karim wrote:
Use regular expression for bad HTLM or beautifulSoup (google it), below
a exemple to extract all html links:
Actually, using regex wasn't so bad:
import re
import urllib.request
url = 'http://x264.nl/x264/?dir=./64bit/8bit_depth'
page = str(urllib.request.urlopen(url).read(), encoding='utf-8') #
urlopen() returns a bytes object, need to get a normal string
rev_re = re.compile('revision[0-9][0-9][0-9][0-9]')
num_re = re.compile('[0-9][0-9][0-9][0-9]')
rev = rev_re.findall(str(page))[0] # only need the first item since
the first listing is the latest revision
num = num_re.findall(rev)[0] # findall() always returns a list
print(num)
prints out the revision number - 1995. 'revision1995' might be useful,
so I saved that to rev.
This actually works pretty well for consistently formatted lists. I
suppose I went about this the wrong way - I thought I needed to parse
the HTML to get the links and do simple regexes on those, but I can just
do simple regexes on the entire HTML document.
Great for you!
Use what works well and easy to code, always the simpler is the better.
For complicate search link to avoid using too complex and bugs prone regex
you can derived the code I gave on HTMLParser with max comparison.
Anyway you get the choice which is cool, not be stuck on only one solution.
Cheers
Karim
--
http://mail.python.org/mailman/listinfo/python-list
Re: Trying to understand html.parser.HTMLParser
Andrew Berg wrote: ElementTree doesn't seem to have been updated in a long time, so I'll assume it won't work with Python 3. I don't know how to use it, but you'll find ElementTree as xml.etree in Python 3. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: python2+3
On Thu, 19 May 2011 05:42:29 -0700, lkcl wrote: has anyone considered the idea of literally creating a Python2/ subdirectory in the python3 codebase, literally just dropping the entire python2.N code directly into it, renaming all functions and data structures, adding a --2-compatible switch to the python3 argc/ v and seeing what happens? no interoperability, no maintenance, no compatibility - just support for python 2 directly in the python3 binary. No maintenance? The code won't maintain itself, people using it won't stop reporting bugs. Are you suggesting that the CPython developers should just grit their teeth and refuse to maintain or support code that is part of the official release? Who is your suggested user-base? Those who want support for 2.7 to continue won't be happy with an unmaintained, unsupported compatibility layer. Those who are happy to keep on using obsolete software won't need this compatibility layer, they can just keep using the Python 2.7 they already have. Or 2.5, or 1.5 if they prefer. (I have 1.5 on my machine, installed from source, and it works perfectly well.) And what of the others? Jython, IronPython, PyPy, Stackless ... once they make the move to Python 3 compatibility, are they expected to implement this compatibility layer as well? I dare say that once the original developers stop supporting Python 2.7 in three or seven(?) years, there will be a good niche for some commercial distribution like ActiveState to continue support. Or one of the many critics who insist that Python 3 is a mistake can put their money where their mouth is and continue support themselves. After all, Python is open source. Somebody (you?) could even fork the code base and implement your suggested compatibility layer, or backport Python 3 features, be really daring and create a 2/3 hybrid. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Faster Recursive Fibonacci Numbers
On Tue, 17 May 2011 10:02:21 -0700, geremy condra wrote: or O(1): φ = (1 + sqrt(5)) / 2 def fib(n): numerator = (φ**n) - (1 - φ)**n denominator = sqrt(5) return round(numerator/denominator) I'd just like to point out that, strictly speaking, it's only O(1) if you assume that exponentiation is O(1). Computer scientists often like to make this simplifying assumption, and it might even be true for floats, but for long ints and any numeric data types with unlimited precision, it won't be. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Faster Recursive Fibonacci Numbers
On Fri, May 20, 2011 at 8:47 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Tue, 17 May 2011 10:02:21 -0700, geremy condra wrote: or O(1): φ = (1 + sqrt(5)) / 2 numerator = (φ**n) - (1 - φ)**n I'd just like to point out that, strictly speaking, it's only O(1) if you assume that exponentiation is O(1). Computer scientists often like to make this simplifying assumption, and it might even be true for floats, but for long ints and any numeric data types with unlimited precision, it won't be. Python doesn't have arbitrary precision non-integers, does it? So this is going to be done with floats. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Faster Recursive Fibonacci Numbers
On Thu, May 19, 2011 at 3:47 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Tue, 17 May 2011 10:02:21 -0700, geremy condra wrote: or O(1): φ = (1 + sqrt(5)) / 2 def fib(n): numerator = (φ**n) - (1 - φ)**n denominator = sqrt(5) return round(numerator/denominator) I'd just like to point out that, strictly speaking, it's only O(1) if you assume that exponentiation is O(1). Computer scientists often like to make this simplifying assumption, and it might even be true for floats, but for long ints and any numeric data types with unlimited precision, it won't be. Great point, and something I should have paid attention to. Thanks. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
Pascal J. Bourguignon p...@informatimago.com writes: t...@sevak.isi.edu (Thomas A. Russ) writes: This will only work if there is a backpointer to the parent. No, you don't need backpointers; some cases have been mentionned in the other answer, but in general: (defun parent (tree node) (if (member node (children tree)) tree (some (lambda (child) (parent child node)) (children tree Yes, the question wasn't about time complexity. :-p Um, this is a recursive function. Inside PARENT, there is another call to PARENT. -- Thomas A. Russ, USC/Information Sciences Institute -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
Hans Georg Schaathun h...@schaathun.net writes: [Followup-To: header set to comp.lang.python.] Ignored, since I don't follow that group. On Wed, 18 May 2011 20:20:01 +0200, Raymond Wiker raw@RAWMBP-2.local wrote: : I don't think anybody mentioned *binary* trees. The context was : directory traversal, in which case you would have nodes with an : arbitrary (almost) number of children. If we are being specific, then directory trees do have parent pointers. My point was really that «standard tree representations» is not a well-defined concept, and having parent pointers is as standard as not having them. I suppose that I just assumed the standard mathematical definition of a tree as a directed, acyclic graph. It seems that in the context of computability that one would tend toward using the mathematical definitions. Certainly in a complex graph with labeled links or trees with backpointers, you would have an alternate data structure that you could follow. So, to be more precise, then: For directed, acyclic graphs without backpointers, you cannot write an iterative tree walker without simulating a stack. The larger point is that there are algorithms that are inherently recursive and for which there is no natural iterative algorithm. -- Thomas A. Russ, USC/Information Sciences Institute -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
t...@sevak.isi.edu (Thomas A. Russ) writes:
Pascal J. Bourguignon p...@informatimago.com writes:
t...@sevak.isi.edu (Thomas A. Russ) writes:
This will only work if there is a backpointer to the parent.
No, you don't need backpointers; some cases have been mentionned in the
other answer, but in general:
(defun parent (tree node)
(if (member node (children tree))
tree
(some (lambda (child) (parent child node)) (children tree
Yes, the question wasn't about time complexity.
:-p
Um, this is a recursive function. Inside PARENT, there is another call
to PARENT.
Feel free to derecursive it.
--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.
--
http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Thu, May 19, 2011 at 11:23 AM, Hans Georg Schaathun h...@schaathun.net wrote: On Thu, 19 May 2011 10:23:47 -0700, geremy condra debat...@gmail.com wrote: : Let me get this straight: your argument is that operating *systems* : aren't systems? You referred to the kernel and not the system. The complexities of the two are hardly comparable. I don't know about that. Among the many verified microkernels, at least two projects have formally verified both their kernel and their toolchain, and one of them claims they've verified everything in their TCB and are headed towards verified POSIX compliance in 2012. That would seem to be a fairly large system (and definitely a complete OS) to me. Another (seL4) says they've formally verified security of a complete system that includes a userspace and the ability to run other OSes in fully isolated containers, which also seems to be quite complete. Finally, there's one from Microsoft research that claims similar properties but which apparently isn't interested in compatibility, which I'm not sure how to interpret in terms of usefulness and size. In any event, higher level systems- like electronic voting mechanisms and automotive sensor networks- have also been verified, which seems to run counter to your original point. Also, not sure if it's open to the general public but if you're interested in this kind of thing and live near seattle, I think there's actually going to be a talk on verifying a POSIX userspace implementation here tomorrow. TL;DR version: large systems have indeed been verified for their security properties. There probably are different uses of system; in computer security literature¹ it often refers, not only to a product (hardware/software) an actual installation and configuration of that product in a specific context. /I/ did not redefine it. You chose a word with a many meanings, used it to make a very broad statement which is only a little bit true, and then pretended that you had the One True Definition in your pocket. I don't think that's legitimate, but whatever; let's just say that we meant different things by the word and drop it. Speaking of reasonable assumptions, one necessary assumption which is particularly dodgy is that whoever deploys and configures it understands all the assumptions and do not break them through ignorance. Yup. Nothing is safe from idiots. Is your concern with security purely from a developer's viewpoint, so that you don't have to worry about the context in which it will be deployed? My viewpoint is that of an attacker, since that's more or less my job. I read your initial comment to imply that if you cannot get satisfactory assurance using the lower levels, you won't get any at the higher levels. That does not make any sense. Well, this is kind of like my point. My point was that you really don't get anything at the lower levels, and that they should fix that (which is far more useful to a normal consumer) rather than trying to talk about formal verification and similar tools, which are only going to be used on a tiny fraction of products. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Fri, May 20, 2011 at 10:56 AM, geremy condra debat...@gmail.com wrote: Speaking of reasonable assumptions, one necessary assumption which is particularly dodgy is that whoever deploys and configures it understands all the assumptions and do not break them through ignorance. Yup. Nothing is safe from idiots. Which means that the assumption really is that you are evaluating a system, not a bald piece of code. I don't consider that an assumption. When you're writing code that you will yourself deploy, you take full responsibility; when you let other people deploy it, they have to take ultimate responsibility (although they will legitimately expect you to provide an install script and/or instructions). There are idiots in this world. Have you met them? Met them? I listen to you every week! -- The Goon Show, and so absolutely true Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Inheriting Object
Hi Guys, I have been wondering for a while now as to why some classes inherit Object? And what does it really do for the class? Can anyone shed some light on this? Regards, Nav -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Thu, May 19, 2011 at 6:33 PM, Chris Angelico ros...@gmail.com wrote: On Fri, May 20, 2011 at 10:56 AM, geremy condra debat...@gmail.com wrote: Speaking of reasonable assumptions, one necessary assumption which is particularly dodgy is that whoever deploys and configures it understands all the assumptions and do not break them through ignorance. Yup. Nothing is safe from idiots. I actually think I need to take this statement back. The more I think about it, the less convinced I am that it's correct- I can at least conceive of violable systems which cannot be misconfigured. So, sorry about that. Which means that the assumption really is that you are evaluating a system, not a bald piece of code. I don't consider that an assumption. When you're writing code that you will yourself deploy, you take full responsibility; when you let other people deploy it, they have to take ultimate responsibility (although they will legitimately expect you to provide an install script and/or instructions). Sure, although I would personally still call it an assumption. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Fri, May 20, 2011 at 12:30 PM, geremy condra debat...@gmail.com wrote: On Fri, May 20, 2011 at 10:56 AM, geremy condra debat...@gmail.com wrote: Yup. Nothing is safe from idiots. I actually think I need to take this statement back. The more I think about it, the less convinced I am that it's correct- I can at least conceive of violable systems which cannot be misconfigured. So, sorry about that. If it is, then you're not deploying it, you're just pushing buttons and acting like a user. I still stand by the view that the one with the root password is the one responsible for the computer's security; and if you have the root filesystem password, there's no way that something can be made unmisconfigurable. (You CAN, however, make something that's out-of-the-box secure, so someone just does a 'sudo apt-get install yoursystem' and it's specced up nicely. This is a Good Thing.) Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Inheriting Object
On 20/05/2011 03:13, Navkirat Singh wrote: Hi Guys, I have been wondering for a while now as to why some classes inherit Object? And what does it really do for the class? Can anyone shed some light on this? Read section 3.3 New-style and classic classes in the Python docs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Inheriting Object
Hi Nav: Here is the long why. http://www.python.org/download/releases/2.2.3/descrintro/ I guess for most programs, there is no big difference, but if you use some special features that might be different. Say, could use super when using type() instead of class(), also, when using multiple inheritance, you can't multiply inherit from different built-in types. Some new features such as property() is not supported in type either. BRs William On 01/-9/-28163 03:59 AM, Navkirat Singh wrote: Hi Guys, I have been wondering for a while now as to why some classes inherit Object? And what does it really do for the class? Can anyone shed some light on this? Regards, Nav -- http://mail.python.org/mailman/listinfo/python-list
Re: Inheriting Object
Hi Nav: Here is the long why. http://www.python.org/download/releases/2.2.3/descrintro/ I guess for most programs, there is no big difference. But in term of some new added features in python, you might not be able to work. Say, you could not use super in type, also you can't multiply inherit from different built-in types, what's more you could not use some builtin feature such as property. BRs William On 01/-9/-28163 03:59 AM, Navkirat Singh wrote: Hi Guys, I have been wondering for a while now as to why some classes inherit Object? And what does it really do for the class? Can anyone shed some light on this? Regards, Nav -- http://mail.python.org/mailman/listinfo/python-list
Re: Inheriting Object
Hi Nav: Here is the long why. http://www.python.org/download/releases/2.2.3/descrintro/ I guess for most programs, there is no big difference. But in term of some new added features in python, you might not be able to work. Say, you could not use super in type, also you can't multiply inherit from different built-in types, what's more you could not use some builtin feature such as property. BRs William On 01/-9/-28163 03:59 AM, Navkirat Singh wrote: Hi Guys, I have been wondering for a while now as to why some classes inherit Object? And what does it really do for the class? Can anyone shed some light on this? Regards, Nav -- http://mail.python.org/mailman/listinfo/python-list
Re: Inheriting Object
On Fri, May 20, 2011 at 8:25 AM, MRAB pyt...@mrabarnett.plus.com wrote: On 20/05/2011 03:13, Navkirat Singh wrote: Hi Guys, I have been wondering for a while now as to why some classes inherit Object? And what does it really do for the class? Can anyone shed some light on this? Read section 3.3 New-style and classic classes in the Python docs. -- http://mail.python.org/mailman/listinfo/python-list Thanks Guys...I will look deeper into this. I thought I read somewhere that it was required in older python releases, but in newer releases it is not. I might be wrong though. Nav -- http://mail.python.org/mailman/listinfo/python-list
Re: turn monitor off and on
On Mon, May 16, 2011 at 7:29 PM, Gabriel Genellina gagsl-...@yahoo.com.arwrote: Your script worked fine for me, 2.6 and XP also. Perhaps your monitor device driver is buggy or does not implement the required functionality. Mine is from Philips. I'm actually using windows 7. Maybe its the difference in OS? Anyway, yes, the monitor turns back on if the keys are pressed. I've gone with a pygame/autoit hack which doesn't turn the monitor off (just black) and then removes the black screen (no offence). Thanks again for the helps -- http://mail.python.org/mailman/listinfo/python-list
starting a separate thread in maya
Hi, I'm using python2.5 in maya 2009 x64 (in linux). I have a script running and somewhere in the script, I want to start another python script that might not return and i don't want the main maya python script to wait for it to finish, even more, after the second script started, I'd like the main script to continue processing. I've tried using threading and maya.utils.executeInMainThread and os.popen and os.spawn, none seem to work this way. Is it not possible to do this in python2.5? Thanks again for any help. Cheers -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Thu, 19 May 2011 17:56:12 -0700, geremy condra debat...@gmail.com wrote: : TL;DR version: large systems have indeed been verified for their : security properties. : (...) : Yup. Nothing is safe from idiots. The difficult part is mapping those properties to actual requirements and threat models. Formal methods do not help on that step. It takes more than a non-idiot to avoid misunderstandings on the interface betweeen professions. Either way, the assumption that your system will not be handled by idiots is only reasonable if you yourself is the only user. -- :-- Hans Georg -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
On Thu, 19 May 2011 23:21:30 +0200, Rikishi42 skunkwo...@rikishi42.net wrote: : On 2011-05-18, Hans Georg Schaathun h...@schaathun.net wrote: : Now Mac OS X has maintained the folder concept of older mac generations, : and Windows has cloned it. They do not want the user to understand : recursive data structures, and therefore, naturally, avoid the word. : : You imply they want to keep their users ignorant of these structures, as if : to keep something valuable from them. Wouldn't it be more honest, more to : the point and much simpler to state they don't NEED the user to understand : recursive - or indeed any other - data structures? And that the user doesn't : NEED to understand or know about them, just to use them? Admittedly, my wording had unintended implictions. Mac OS X /targets/ users who do not need to understand the underlying structure. However, the system also has users who do. : After all they are users. They use their system for fun, learning or work. : Even a very competent or advanced use of a tool (computer, car, mobile phone, : fridge, TV, radio, toilet) in no way implies an understanding of it's inner : workings. Nor the need, nor the desire. For a general purpose computer, that is simply not true in general. : PS: Isn't this thread much ado about nothing? :-) Most threads are. : It starts with the misconception (or should I say confusion?) between : performing a recursive job and using a recursive tool to do it. And then it : blazes off in these huge discusions about semantics to define a definition : of an abstraction of a alleady theoretical problem. And explaining the source of the misconception and the varying use would be irrelevant? : And PPS: the P(P)S's don't specifically refer to your posting. Thanks :-) -- :-- Hans Georg -- http://mail.python.org/mailman/listinfo/python-list
Recursion in Computer Science
There have been a number of unrelated discussions regarding recursion on this list. I believe that recursion occurs in a wider spread of areas than is usually recognised. Heres a list of some such areas. Please note I am using recursion in a broad and somewhat fuzzy sense. Narrow specific definitions would lead to conclusions like: -- Prolog has no functions or procedures so it has no recursion -- Since recursion is equivalent to stack + iteration therefore Fortran supports recursion. Heres (an initial approx to) such a list: --- recursive functions recursive data -- eg linked lists, trees nesting self reference -- Y combinator well founded induction structural induction bootstrapping language to describe language -- syntax - yacc in yacc language to describe language -- semantics - lisp in lisp -- metacircularity Goedel's theorem - meta-mathematics - ordinary math undecidability - universal TM - Turing machine as ordinary computer reentrancy [An OS-like program fails to be reentrant for the same reason that Fortan-like language fails to support recursion -- Non use of stack] virtualization in OS Introspection in modern languages Models to metamodels corecursion - laziness, infinite datastructures - semantics of generators/iterators in python Von Neuman machine - code is data -- therefore quondam hardware becomes software - data can be code -- viruses -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
On May 20, 2:21 am, Rikishi42 skunkwo...@rikishi42.net wrote: On 2011-05-18, Hans Georg Schaathun h...@schaathun.net wrote: Now Mac OS X has maintained the folder concept of older mac generations, and Windows has cloned it. They do not want the user to understand recursive data structures, and therefore, naturally, avoid the word. You imply they want to keep their users ignorant of these structures, as if to keep something valuable from them. Wouldn't it be more honest, more to the point and much simpler to state they don't NEED the user to understand recursive - or indeed any other - data structures? And that the user doesn't NEED to understand or know about them, just to use them? After all they are users. They use their system for fun, learning or work. Even a very competent or advanced use of a tool (computer, car, mobile phone, fridge, TV, radio, toilet) in no way implies an understanding of it's inner workings. Nor the need, nor the desire. PS: Isn't this thread much ado about nothing? :-) It starts with the misconception (or should I say confusion?) between performing a recursive job and using a recursive tool to do it. And then it blazes off in these huge discusions about semantics to define a definition of an abstraction of a alleady theoretical problem. Glorious, just frelling glorious. :-) We have an expression for that. But I'll avoid using it, since it has the word 'masturbation' in it... And PPS: the P(P)S's don't specifically refer to your posting. -- When in doubt, use brute force. -- Ken Thompson Well... I was rethinking my earlier argument with Ian Kelly about the similarity/differences between recursion in theory and in CS. On rethinking my position I come to the conclusion that I am arguing like an chimp -- and therefore not making my real point which is that that recursion is more widespread in computer science than is recognised. This is discussed separately here http://groups.google.com/group/comp.lang.python/browse_thread/thread/212e6477262125e9# [I agree with you Xah that recursion is a technical word that should not be foisted onto lay users.] -- http://mail.python.org/mailman/listinfo/python-list
Re: Recursion in Computer Science
On Fri, May 20, 2011 at 3:05 PM, rusi rustompm...@gmail.com wrote: - data can be code -- viruses It's not JUST viruses. There's plenty of legitimate reasons for your data to actually be code... that's how compilers work! :) Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Recursion in Computer Science
On May 20, 10:18 am, Chris Angelico ros...@gmail.com wrote: On Fri, May 20, 2011 at 3:05 PM, rusi rustompm...@gmail.com wrote: - data can be code -- viruses It's not JUST viruses. There's plenty of legitimate reasons for your data to actually be code... that's how compilers work! :) Chris Angelico Yes sure Thanks. An exhaustive list would be much longer (and I got tired of typing) -- http://mail.python.org/mailman/listinfo/python-list
validating a class against an ABC at definition time
Thinking about class APIs and validating a class against an API. The abc
module provides the tools to do some of this. One thing I realized, that I
hadn't noticed before, is that the abstractness of a class is measured when
instances of the class are created. This happens in object.__new__
(pyobject.c). Validating thus when a class is defined would not be as
helpful, since there is no guarantee that the class is not meant to be
abstract as well.
However, I have found that it is sometimes nice to validate a class at
definition time. This is particularly true for a class that does not
inherit from the abstract base class (but registers instead).
Taking cues from abc.py and pyobject.c, here is a stab at a class decorator
that validates a class against another.
def validate(abc):
if not isinstance(abc, type):
raise TypeError(Can only validate against classes)
def decorator(cls):
if not __debug__:
return cls
if not isinstance(cls, type):
raise TypeError(Can only validate classes)
abstracts = set()
for name in getattr(abc, __abstractmethods__, set()):
value = getattr(cls, name, None)
if not value:
abstracts.add(name)
elif getattr(value, __isabstractmethod__, False):
abstracts.add(name)
if abstracts:
sorted_methods = sorted(abstracts)
joined = , .join(sorted_methods)
msg = Class {} does not implement abstract methods {} of class
{}
raise TypeError(msg.format(cls.__name__, joined, abc.__name__))
return cls
return decorator
Stack this with the ABCMeta.register method and you can ensure that your
class is compliant with the ABC at the time you register it on that ABC.
Does anyone find this irrelevant or superfluous? I know that it would be a
good idea to stay on top of your class's implementation of an ABC's abstract
methods. However, this seems like a good way of doing that
programmatically.
Does anyone know a better way to do ABC validation at definition time?
Thanks.
-eric
--
http://mail.python.org/mailman/listinfo/python-list
hash values and equality
Several folk have said that objects that compare equal must hash equal,
and the docs also state this
http://docs.python.org/dev/reference/datamodel.html#object.__hash__
I'm hoping somebody can tell me what horrible thing will happen if this
isn't the case? Here's a toy example of a class I'm thinking of writing
that will compare equal with int's, but hash differently:
-- class Wierd():
... def __init__(self, value):
... self.value = value
... def __eq__(self, other):
... return self.value == other
... def __hash__(self):
... return hash((self.value + 13) ** 3)
...
-- one = Wierd(1)
-- two = Wierd(2)
-- three = Wierd(3)
-- one
Wierd object at 0x00BFE710
-- one == 1
True
-- one == 2
False
-- two == 2
True
-- three == 3
True
-- d = dict()
-- d[one] = '1'
-- d[two] = '2'
-- d[three] = '3'
-- d
{Wierd object at 0x00BFE710: '1',
Wierd object at 0x00BFE870: '3',
Wierd object at 0x00BFE830: '2'}
-- d[1] = '1.0'
-- d[2] = '2.0'
-- d[3] = '3.0'
-- d
{Wierd object at 0x00BFE870: '3',
1: '1.0',
2: '2.0',
3: '3.0',
Wierd object at 0x00BFE830: '2',
Wierd object at 0x00BFE710: '1'}
-- d[2]
'2.0'
-- d[two]
'2'
All information greatly appreciated!
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
[issue7969] shutil.copytree error handling non-standard and partially broken
Ameya Lokare lokare.am...@gmail.com added the comment: I've attached a proposed doc patch. I figured I'd move the Error argument format in the description of shutil.copytree, since the format is different for other functions (shutil.move, for example). -- keywords: +patch nosy: +lokare.ameya Added file: http://bugs.python.org/file22026/shutil_copytree_doc.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7969 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4934] tp_del and tp_version_tag undocumented
Changes by Martin von Gagern martin.vgag...@gmx.net: -- nosy: +gagern ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4934 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12068] test_logging failure in test_rollover
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: It's not the same error, but there's no point opening another issue for it. The earlier error occurred more frequently, and was specifically due to some changes I made, whereas this error hasn't occurred before and appears to be due to some sort of network glitch. On the same buildbot, later revisions have run without errors, with no changes to test_logging: 162ed9841f147f37d2077a1848eb9aff130b71fb 05836e84e584d3c529e74fba4b99da95df17ef24 c45e92bd4d81713603b7271cc7f61b7457296563 I'll mark as pending and keep an eye on it, but if the failure doesn't recur in the next week or so I'll close the issue again. -- resolution: fixed - status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12068 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11969] Can't launch multiproccessing.Process on methods
Ram Rachum cool...@cool-rr.com added the comment: Test attached. -- Added file: http://bugs.python.org/file22027/test.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11969 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1746656] IPv6 Interface naming/indexing functions
STINNER Victor victor.stin...@haypocalc.com added the comment: Here's a patch: - those functions now accept and return str, not bytes arrays You use UTF-8 encoding: +Is, ni[i].if_index, ni[i].if_name); +if (!PyArg_ParseTuple(args, s:if_nametoindex, ifname)) You should also use the FS encoding with surrogateescape error handler: - parse arguments using O format with PyUnicode_FSConverter: it gives you a PyBytes object, then use PyBytes_AS_STRING() and PyBytes_GET_SIZE() - use PyUnicode_DecodeFSDefault() to decode a byte string -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1746656 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1746656] IPv6 Interface naming/indexing functions
STINNER Victor victor.stin...@haypocalc.com added the comment:
Example with a non-ASCII interface name:
$ sudo tunctl -u haypo -t unicodeé
Set 'unicodeé' persistent and owned by uid 1000
$ sudo ifconfig -a|grep unicode|hexdump -C
75 6e 69 63 6f 64 65 c3 a9 20 4c 69 6e 6b 20 65 |unicode.. Link e|
0010 6e 63 61 70 3a 45 74 68 65 72 6e 65 74 20 20 48 |ncap:Ethernet H|
0020 57 61 64 64 72 20 64 36 3a 30 38 3a 31 63 3a 65 |Waddr d6:08:1c:e|
0030 30 3a 33 33 3a 30 36 20 20 0a|0:33:06 .|
003a
So in my setup (UTF-8 locale encoding), U+00E9 is encoded as {0xc3, 0xa9}
(UTF-8).
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1746656
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12107] TCP listening sockets created without FD_CLOEXEC flag
New submission from Christophe Devriese christophe.devri...@gmail.com: The specific issue this is creating is that a malicious user could use this socket in a subprocess which is started from a library (ie. I'm using a .so, which calls fork/exec). A second failure mode is starting a daemon from withing, say, a django application. Djano opens a TCP listening socket, then starts up a daemon to provide some sort of service in the background. The daemon keeps running and inherits the socket. Now you restart the django app. It refuses to start ! Why ? Because the socket was inherited, the listening socket isn't actually closed, and this results in the socket being stuck in CLOSE_WAIT as long as the daemon is running. It seems to me that it is almost never the case that you'd want a TCP listening socket to be preserved across exec, and not setting this flag should thus be considered a bug for 2 reasons : 1) it results in accidental disclosure of information that shouldn't be exposed in certain cases. 2) it can result in denial of service Solution : update SocketServer.py : in the class TCPServer add the following 2 lines in __init__ after self.socket = socket( ...: flags = fcntl.fcntl(self.socket, fcntl.F_GETFD) fcntl.fcntl(self.socket, fcntl.F_SETFD, flags | fcntl.FD_CLOEXEC) -- messages: 136273 nosy: Christophe.Devriese priority: normal severity: normal status: open title: TCP listening sockets created without FD_CLOEXEC flag type: security ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12107 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12107] TCP listening sockets created without FD_CLOEXEC flag
Changes by Nadeem Vawda nadeem.va...@gmail.com: -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12107 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12108] test_packaging monkeypatches httplib
Changes by Antoine Pitrou pit...@free.fr: -- nosy: pitrou priority: normal severity: normal status: open title: test_packaging monkeypatches httplib ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12108 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12109] test_packaging monkeypatches httplib
New submission from Antoine Pitrou pit...@free.fr: == ERROR: test_https (test.test_urllib2_localnet.TestUrlopen) -- Traceback (most recent call last): File D:\Buildslave\3.x.moore-windows\build\lib\test\test_urllib2_localnet.py, line 457, in test_https data = self.urlopen(https://localhost:%s/bizarre; % handler.port) File D:\Buildslave\3.x.moore-windows\build\lib\test\test_urllib2_localnet.py, line 364, in urlopen f = urllib.request.urlopen(url, data, **kwargs) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 138, in urlopen return opener.open(url, data, timeout) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 369, in open response = self._open(req, data) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 387, in _open '_open', req) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 347, in _call_chain result = func(*args) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 1179, in https_open context=self._context, check_hostname=self._check_hostname) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 1116, in do_open h = http_class(host, timeout=req.timeout, **http_conn_args) TypeError: https_conn_wrapper() got an unexpected keyword argument 'check_hostname' == ERROR: test_https_with_cafile (test.test_urllib2_localnet.TestUrlopen) -- Traceback (most recent call last): File D:\Buildslave\3.x.moore-windows\build\lib\test\test_urllib2_localnet.py, line 465, in test_https_with_cafile cafile=CERT_localhost) File D:\Buildslave\3.x.moore-windows\build\lib\test\test_urllib2_localnet.py, line 364, in urlopen f = urllib.request.urlopen(url, data, **kwargs) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 138, in urlopen return opener.open(url, data, timeout) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 369, in open response = self._open(req, data) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 387, in _open '_open', req) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 347, in _call_chain result = func(*args) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 1179, in https_open context=self._context, check_hostname=self._check_hostname) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 1116, in do_open h = http_class(host, timeout=req.timeout, **http_conn_args) TypeError: https_conn_wrapper() got an unexpected keyword argument 'check_hostname' -- assignee: tarek components: Distutils2, Tests messages: 136274 nosy: alexis, eric.araujo, pitrou, tarek priority: normal severity: normal stage: needs patch status: open title: test_packaging monkeypatches httplib versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12109 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12110] test_packaging monkeypatches httplib
New submission from Antoine Pitrou pit...@free.fr: == ERROR: test_https (test.test_urllib2_localnet.TestUrlopen) -- Traceback (most recent call last): File D:\Buildslave\3.x.moore-windows\build\lib\test\test_urllib2_localnet.py, line 457, in test_https data = self.urlopen(https://localhost:%s/bizarre; % handler.port) File D:\Buildslave\3.x.moore-windows\build\lib\test\test_urllib2_localnet.py, line 364, in urlopen f = urllib.request.urlopen(url, data, **kwargs) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 138, in urlopen return opener.open(url, data, timeout) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 369, in open response = self._open(req, data) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 387, in _open '_open', req) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 347, in _call_chain result = func(*args) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 1179, in https_open context=self._context, check_hostname=self._check_hostname) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 1116, in do_open h = http_class(host, timeout=req.timeout, **http_conn_args) TypeError: https_conn_wrapper() got an unexpected keyword argument 'check_hostname' == ERROR: test_https_with_cafile (test.test_urllib2_localnet.TestUrlopen) -- Traceback (most recent call last): File D:\Buildslave\3.x.moore-windows\build\lib\test\test_urllib2_localnet.py, line 465, in test_https_with_cafile cafile=CERT_localhost) File D:\Buildslave\3.x.moore-windows\build\lib\test\test_urllib2_localnet.py, line 364, in urlopen f = urllib.request.urlopen(url, data, **kwargs) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 138, in urlopen return opener.open(url, data, timeout) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 369, in open response = self._open(req, data) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 387, in _open '_open', req) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 347, in _call_chain result = func(*args) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 1179, in https_open context=self._context, check_hostname=self._check_hostname) File D:\Buildslave\3.x.moore-windows\build\lib\urllib\request.py, line 1116, in do_open h = http_class(host, timeout=req.timeout, **http_conn_args) TypeError: https_conn_wrapper() got an unexpected keyword argument 'check_hostname' -- assignee: tarek components: Distutils2, Tests messages: 136275 nosy: alexis, eric.araujo, pitrou, tarek priority: normal severity: normal stage: needs patch status: open title: test_packaging monkeypatches httplib versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12110 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12111] email's use of __setitem__ is highly counterintuitive
New submission from Toni Mueller tonimuel...@users.sourceforge.net: email's usage of __setitem__ is highly counterintuitive to the point of being dangerous. The documented behaviour is (quote): __setitem__(name, val) Add a header to the message with field name name and value val. The field is appended to the end of the message’s existing fields. Note that this does not overwrite or delete any existing header with the same name. If you want to ensure that the new header is the only one present in the message with field name name, delete the field first, e.g.: ... (taken from http://docs.python.org/library/email.message.html ) The use case of *appending* a header of the same type (eg. a Received: header) should be performed by the add_header() method, or an extend_header() method, or something similar, and not by abusing the __setitem__ method. The current behaviour imho deviates extremely from the behaviour of similar libraries in all other programming languages that I'm aware of, and from the standard dict functionality, too. It makes it much too easy to have duplicate headers, esp., duplicate To: headers, resulting in mailbombing and information leakage. For the potential damage, this property of the library is highly under-advertised. A side effect appears to be that trying to have your message headers set up in a unique fashion, probably the most frequent use case, one has to make sure to use each operator only once, or decorate everything with a del msg[myheader], as the operation is not idempotent. -- messages: 136276 nosy: tonimueller priority: normal severity: normal status: open title: email's use of __setitem__ is highly counterintuitive type: behavior versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12111 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12110] test_packaging monkeypatches httplib
Changes by Antoine Pitrou pit...@free.fr: -- resolution: - duplicate status: open - closed superseder: - test_packaging monkeypatches httplib ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12110 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12108] test_packaging monkeypatches httplib
Changes by Antoine Pitrou pit...@free.fr: -- resolution: - invalid status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12108 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12109] test_packaging monkeypatches httplib
Antoine Pitrou pit...@free.fr added the comment: $ grep -r https_conn_wrapper * Lib/packaging/tests/test_command_upload_docs.py:146:def https_conn_wrapper(*args): Lib/packaging/tests/test_command_upload_docs.py:152: upload_docs_mod.http.client.HTTPSConnection = https_conn_wrapper -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12109 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12109] test_packaging monkeypatches httplib
Roundup Robot devnull@devnull added the comment: New changeset 56aa0aca7186 by Tarek Ziade in branch 'default': Issue #12109 fixing typo in packaging's test_command_upload_docs http://hg.python.org/cpython/rev/56aa0aca7186 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12109 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12111] email's use of __setitem__ is highly counterintuitive
R. David Murray rdmur...@bitdance.com added the comment: This is a long-standing design choice in the email package. If you want to advocate for changing it, please join the email-sig mailing list (see mail.python.org). We are in the process of developing a new version, which will at least reject things like duplicate To headers. I'm closing this issue since as things stand this is not something that is likely to change. If you carry the day in a discussion on the email-sig, we can reopen the issue. In any case it is a feature request, not a bug. -- components: +Library (Lib) nosy: +r.david.murray status: open - closed type: behavior - feature request versions: +Python 3.3 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12111 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12112] The new packaging module should not use the locale encoding
New submission from STINNER Victor victor.stin...@haypocalc.com: The locale encoding is not portable, packaging should use UTF-8 instead. Attached patch is a draft to workaround LANG=C ./python -m test test_packaging failures. I'm not sure that my test in Metadata.write_file() of packaging.metadata is a good idea. Moreover, packaging should also maybe use the surrogateescape error handler, but I propose to decide that later and in another issue. We may use this error handler only to read files (like the Python makefile). For write a complete patch, *all* calls to open() in Lib/packaging/* (including Lib/packaging/tests/*) should be checked. distutils uses the locale encoding, which is UTF-8 on Mac OS X and most Linux setup (but never on Windows). But distutils cannot be fixed, whereas packaging is a new module and can be fixed. -- files: packaging_utf8.patch keywords: patch messages: 136280 nosy: haypo priority: normal severity: normal status: open title: The new packaging module should not use the locale encoding versions: Python 3.3 Added file: http://bugs.python.org/file22028/packaging_utf8.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12112 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12112] The new packaging module should not use the locale encoding
Changes by STINNER Victor victor.stin...@haypocalc.com: -- components: +Library (Lib) nosy: +eric.araujo, tarek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12112 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12112] The new packaging module should not use the locale encoding
STINNER Victor victor.stin...@haypocalc.com added the comment: See also issue #9561 (distutils: set encoding to utf-8 for input and output files) and #6011 (python doesn't build if prefix contains non-ascii characters). If you are curious, read also #8611 (Python3 doesn't support locale different than utf8 and an non-ASCII path (POSIX)) and #9425 (Rewrite import machinery to work with unicode paths). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12112 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12112] The new packaging module should not use the locale encoding
Tarek Ziadé ziade.ta...@gmail.com added the comment: Looks good, please commit this -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12112 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12113] test_packaging fails when run twice
New submission from Antoine Pitrou pit...@free.fr: $ ./python -m test -Fv -unetwork test_packaging [...] == ERROR: test_download (packaging.tests.test_pypi_dist.TestDistInfo) -- Traceback (most recent call last): File /home/antoine/cpython/default/Lib/packaging/tests/pypi_server.py, line 68, in wrapped func(server=server, *args, **kwargs) File /home/antoine/cpython/default/Lib/packaging/tests/test_pypi_dist.py, line 132, in test_download dist.download(self.mkdtemp()) File /home/antoine/cpython/default/Lib/packaging/pypi/dist.py, line 306, in download self._check_md5(filename) File /home/antoine/cpython/default/Lib/packaging/pypi/dist.py, line 339, in _check_md5 % (hashval.hexdigest(), expected_hashval)) packaging.pypi.errors.HashDoesNotMatch: got 8ae6ff9df26ff04232189724d520a17c instead of fe18804c5b722ff024cabdf514924fc4 == FAIL: test_hooks_get_run (packaging.tests.test_dist.DistributionTestCase) -- Traceback (most recent call last): File /home/antoine/cpython/default/Lib/packaging/tests/test_dist.py, line 234, in test_hooks_get_run 'post-test_dist']) AssertionError: Lists differ: ['finalize', 'pre-test_dist', ... != ['finalize', 'pre-test_dist', ... First list contains 4 additional elements. First extra element 4: finalize + ['finalize', 'pre-test_dist', 'run', 'post-test_dist'] - ['finalize', - 'pre-test_dist', - 'run', - 'post-test_dist', - 'finalize', - 'pre-test_dist', - 'run', - 'post-test_dist'] -- -- assignee: tarek components: Distutils2, Tests messages: 136283 nosy: alexis, eric.araujo, pitrou, tarek priority: high severity: normal stage: needs patch status: open title: test_packaging fails when run twice type: behavior versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12113 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12109] test_packaging monkeypatches httplib
Antoine Pitrou pit...@free.fr added the comment: Looks good here, thank you :) -- resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12109 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12113] test_packaging fails when run twice
Roundup Robot devnull@devnull added the comment: New changeset 3e0500881003 by Tarek Ziade in branch 'default': Issue #12113: make sure generated module is not reused on a second run http://hg.python.org/cpython/rev/3e0500881003 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12113 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12114] packaging.util._find_exe_version(): potential deadlock
New submission from STINNER Victor victor.stin...@haypocalc.com: The following code does deadlock if the subprocess writes a lot of output to stdout and stderr: pipe = Popen(cmd, shell=True, stdout=PIPE, stderr=PIPE) try: stdout, stderr = pipe.stdout.read(), pipe.stderr.read() finally: pipe.stdout.close() pipe.stderr.close() Popen.communicate() should be used to avoid the issue. Attached patch uses the .communicate() method and the context manager syntax (with). -- components: Library (Lib) files: packaging_popen_communicate.patch keywords: patch messages: 136286 nosy: haypo, tarek priority: normal severity: normal status: open title: packaging.util._find_exe_version(): potential deadlock versions: Python 3.3 Added file: http://bugs.python.org/file22029/packaging_popen_communicate.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12114 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12115] some tests need to be skipped on threadless systems
New submission from Tarek Ziadé ziade.ta...@gmail.com: == ERROR: packaging.tests.test_command_upload_docs (unittest.loader.ModuleImportFailure) -- Traceback (most recent call last): File /home/buildbot/buildarea/3.x.krah-fedora/build/Lib/unittest/case.py, line 407, in _executeTestPart function() File /home/buildbot/buildarea/3.x.krah-fedora/build/Lib/unittest/loader.py, line 32, in testFailure raise exception ImportError: Failed to import test module: packaging.tests.test_command_upload_docs Traceback (most recent call last): File /home/buildbot/buildarea/3.x.krah-fedora/build/Lib/unittest/loader.py, line 257, in _find_tests module = self._get_module_from_name(name) File /home/buildbot/buildarea/3.x.krah-fedora/build/Lib/unittest/loader.py, line 235, in _get_module_from_name __import__(name) File /home/buildbot/buildarea/3.x.krah-fedora/build/Lib/packaging/tests/test_command_upload_docs.py, line 14, in module from packaging.tests.pypi_server import PyPIServerTestCase File /home/buildbot/buildarea/3.x.krah-fedora/build/Lib/packaging/tests/pypi_server.py, line 36, in module import threading File /home/buildbot/buildarea/3.x.krah-fedora/build/Lib/importlib/_bootstrap.py, line 437, in load_module return self._load_module(fullname) File /home/buildbot/buildarea/3.x.krah-fedora/build/Lib/importlib/_bootstrap.py, line 141, in decorated return fxn(self, module, *args, **kwargs) File /home/buildbot/buildarea/3.x.krah-fedora/build/Lib/importlib/_bootstrap.py, line 342, in _load_module exec(code_object, module.__dict__) File /home/buildbot/buildarea/3.x.krah-fedora/build/Lib/threading.py, line 4, in module import _thread ImportError: No module named '_thread' -- assignee: tarek components: Library (Lib) messages: 136287 nosy: alexis, tarek priority: normal severity: normal status: open title: some tests need to be skipped on threadless systems versions: Python 3.3, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12115 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12113] test_packaging fails when run twice
Antoine Pitrou pit...@free.fr added the comment: Still fails on test_download. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12113 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10419] distutils command build_scripts fails with UnicodeDecodeError
Roundup Robot devnull@devnull added the comment: New changeset cc5cfeaa4a8d by Victor Stinner in branch 'default': Issue #10419, issue #6011: port 6ad356525381 fix from distutils to packaging http://hg.python.org/cpython/rev/cc5cfeaa4a8d -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10419 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12112] The new packaging module should not use the locale encoding
STINNER Victor victor.stin...@haypocalc.com added the comment: I ported a distutils fix (for non-ASCII path) into packaging: New changeset cc5cfeaa4a8d by Victor Stinner in branch 'default': Issue #10419, issue #6011: port 6ad356525381 fix from distutils to packaging http://hg.python.org/cpython/rev/cc5cfeaa4a8d -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12112 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12112] The new packaging module should not use the locale encoding
Antoine Pitrou pit...@free.fr added the comment: +if not isinstance(fileobject, StringIO): +encoding = codecs.lookup(fileobject.encoding).name IMO you should try to get the encoding attribute and silence the AttributeError instead. (also, I'm not even sure why you're adding this check) -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12112 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12113] test_packaging fails when run twice
Antoine Pitrou pit...@free.fr added the comment: Also, the hash value is different every time: packaging.pypi.errors.HashDoesNotMatch: got 043840092b5baf155fc94a77319c5f44 instead of fe18804c5b722ff024cabdf514924fc4 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12113 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12115] some tests need to be skipped on threadless systems
Roundup Robot devnull@devnull added the comment: New changeset 2c01bda139a7 by Tarek Ziade in branch 'default': Issue #12115: skipping all tests that need threading under a threadless environment http://hg.python.org/cpython/rev/2c01bda139a7 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12115 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12113] test_packaging fails when run twice
Tarek Ziadé ziade.ta...@gmail.com added the comment: my commit fixed only the first issue. the second one is a separate issue that needs more investigation -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12113 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12077] Harmonizing descriptor protocol documentation
Jay Parlar par...@gmail.com added the comment: Another problem is that the examples and text in the section Functions and Methods is no longer correct in 3.x. Namely the the references to unbound methods, and the example showing an unbound method being returned when accessing a method of a class. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12077 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12112] The new packaging module should not use the locale encoding
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12112 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12113] test_packaging fails when run twice
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12113 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12114] packaging.util._find_exe_version(): potential deadlock
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12114 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12116] io.Buffer*.seek() doesn't seek if seeking leaves us inside the current buffer
New submission from STINNER Victor victor.stin...@haypocalc.com:
Example:
with open(setup.py, rb) as f:
# read smaller than the file size to fill the readahead buffer
f.read(1)
# seek doesn't seek
f.seek(0)
print(f pos=, f.tell())
print(f.raw pos=, f.raw.tell())
Output:
f pos= 0
f.raw pos= 4096
I expect f.raw.tell() to be 0.
Extract of Modules/_io/buffered.c:
if (whence != 2 self-readable) {
Py_off_t current, avail;
/* Check if seeking leaves us inside the current buffer,
so as to return quickly if possible. Also, we needn't take the
lock in this fast path.
Don't know how to do that when whence == 2, though. */
/* NOTE: RAW_TELL() can release the GIL but the object is in a stable
state at this point. */
current = RAW_TELL(self);
avail = READAHEAD(self);
printf(current=% PY_PRIdOFF , avail=% PY_PRIdOFF \n, current,
avail);
if (avail 0) {
Py_off_t offset;
if (whence == 0)
offset = target - (current - RAW_OFFSET(self));
else
offset = target;
printf(offset=% PY_PRIdOFF \n, offset);
if (offset = -self-pos offset = avail) {
printf(NO SEEK!\n);
self-pos += offset;
return PyLong_FromOff_t(current - avail + offset);
}
}
}
I found this weird behaviour when trying to understand why:
with open(setup.py, 'rb') as f:
encoding, lines = tokenize.detect_encoding(f.readline)
with open(setup.py, 'r', encoding=encoding) as f:
imp.load_module(setup, f, setup.py, (.py, r, imp.PY_SOURCE))
is different than:
with tokenize.open(setup.py) as f:
imp.load_module(setup, f, setup.py, (.py, r, imp.PY_SOURCE))
imp.load_module() clones the file using something like fd = os.dup(f.fileno());
clone = os.fdopen(fd, r).
For tokenizer.open(), a workaround is to replace:
buffer.seek(0)
by
buffer.seek(0); buffer.raw.seek(0)
--
components: IO
messages: 136296
nosy: haypo, pitrou
priority: normal
severity: normal
status: open
title: io.Buffer*.seek() doesn't seek if seeking leaves us inside the current
buffer
versions: Python 3.3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12116
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
