pathlib 0.4
Hello,
I'm releasing pathlib 0.4 for Python 3.2 and later.
* Download: http://pypi.python.org/pypi/pathlib/
(you can probably use pip install pathlib or easy_install pathlib
instead)
* Documentation: http://readthedocs.org/docs/pathlib/en/latest/
* Issue tracker, repository: https://bitbucket.org/pitrou/pathlib/
What is pathlib
---
pathlib offers a set of classes to handle filesystem paths. It offers the
following advantages over using string objects:
* No more cumbersome use of os and os.path functions. Everything can be
done easily through operators, attribute accesses, and method calls.
* Embodies the semantics of different path types. For example, comparing
Windows paths ignores casing.
* Well-defined semantics, eliminating any warts or ambiguities (forward vs.
backward slashes, etc.).
Examples
Importing the module classes::
from pathlib import *
Listing Python source files in a directory::
p = Path('.')
[x for x in p if x.ext == '.py']
[PosixPath('test_pathlib.py'), PosixPath('setup.py'),
PosixPath('pathlib.py')]
Listing subdirectories::
[x for x in p if x.is_dir()]
[PosixPath('.hg'), PosixPath('docs'), PosixPath('dist'),
PosixPath('__pycache__'), PosixPath('build')]
Navigating inside a directory tree::
p = Path('/etc')
q = p['init.d/reboot']
q
PosixPath('/etc/init.d/reboot')
q.resolve()
PosixPath('/etc/rc.d/init.d/halt')
Querying path properties::
q.exists()
True
q.is_dir()
False
q.st_mode
33261
Opening a file::
with q.open() as f: f.readline()
...
'#!/bin/bash\n'
Regards
Antoine Pitrou.
--
http://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations/
[ANN] Summer School Advanced Scientific Programming in Python in Kiel, Germany
Advanced Scientific Programming in Python = a Summer School by the G-Node and the Institute of Experimental and Applied Physics, Christian-Albrechts-Universität zu Kiel Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists actually use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques, incorporating theoretical lectures and practical exercises tailored to the needs of a programming scientist. New skills will be tested in a real programming project: we will team up to develop an entertaining scientific computer game. We use the Python programming language for the entire course. Python works as a simple programming language for beginners, but more importantly, it also works great in scientific simulations and data analysis. We show how clean language design, ease of extensibility, and the great wealth of open source libraries for scientific computing and data visualization are driving Python to become a standard tool for the programming scientist. This school is targeted at Master or PhD students and Post-docs from all areas of science. Competence in Python or in another language such as Java, C/C++, MATLAB, or Mathematica is absolutely required. Basic knowledge of Python is assumed. Participants without any prior experience with Python should work through the proposed introductory materials before the course. Date and Location = September 2—7, 2012. Kiel, Germany. Preliminary Program === Day 0 (Sun Sept 2) — Best Programming Practices - Best Practices, Development Methodologies and the Zen of Python - Version control with git - Object-oriented programming design patterns Day 1 (Mon Sept 3) — Software Carpentry - Test-driven development, unit testing quality assurance - Debugging, profiling and benchmarking techniques - Best practices in data visualization - Programming in teams Day 2 (Tue Sept 4) — Scientific Tools for Python - Advanced NumPy - The Quest for Speed (intro): Interfacing to C with Cython - Advanced Python I: idioms, useful built-in data structures, generators Day 3 (Wed Sept 5) — The Quest for Speed - Writing parallel applications in Python - Programming project Day 4 (Thu Sept 6) — Efficient Memory Management - When parallelization does not help: the starving CPUs problem - Advanced Python II: decorators and context managers - Programming project Day 5 (Fri Sept 7) — Practical Software Development - Programming project - The Pelita Tournament Every evening we will have the tutors' consultation hour: Tutors will answer your questions and give suggestions for your own projects. Applications You can apply on-line at http://python.g-node.org Applications must be submitted before 23:59 UTC, May 1, 2012. Notifications of acceptance will be sent by June 1, 2012. No fee is charged but participants should take care of travel, living, and accommodation expenses. Candidates will be selected on the basis of their profile. Places are limited: acceptance rate last time was around 20%. Prerequisites: You are supposed to know the basics of Python to participate in the lectures. You are encouraged to go through the introductory material available on the website. Faculty === - Francesc Alted, Continuum Analytics Inc., USA - Pietro Berkes, Enthought Inc., UK - Valentin Haenel, Blue Brain Project, École Polytechnique Fédérale de Lausanne, Switzerland - Zbigniew Jędrzejewski-Szmek, Faculty of Physics, University of Warsaw, Poland - Eilif Muller, Blue Brain Project, École Polytechnique Fédérale de Lausanne, Switzerland - Emanuele Olivetti, NeuroInformatics Laboratory, Fondazione Bruno Kessler and University of Trento, Italy - Rike-Benjamin Schuppner, Technologit GbR, Germany - Bartosz Teleńczuk, Unité de Neurosciences Information et Complexité, Centre National de la Recherche Scientifique, France - Stéfan van der Walt, Helen Wills Neuroscience Institute, University of California Berkeley, USA - Bastian Venthur, Berlin Institute of Technology and Bernstein Focus Neurotechnology, Germany - Niko Wilbert, TNG Technology Consulting GmbH, Germany - Tiziano Zito, Institute for Theoretical Biology, Humboldt-Universität zu Berlin, Germany Organized by Christian T. Steigies and Christian Drews of the Institute of Experimental and Applied Physics, Christian-Albrechts-Universität zu Kiel , and by Zbigniew Jędrzejewski-Szmek and Tiziano Zito for the German Neuroinformatics Node of the INCF. Website: http://python.g-node.org Contact: python-i...@g-node.org -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software
Re: Disable use of pyc file with no matching py file
Terry Reedy wrote: On 1/30/2012 4:30 PM, Roy Smith wrote: Every so often (typically when refactoring), I'll remove a .py file and forget to remove the corresponding .pyc file. If I then import the module, python finds the orphaned .pyc and happily imports it. Usually leading to confusing and hard to debug failures. Is there some way to globally tell python, Never import a .pyc unless the corresponding .py file exits? Upgrade to 3.2. No. JM -- http://mail.python.org/mailman/listinfo/python-list
Re: Disable use of pyc file with no matching py file
On 01/30/2012 09:30 PM, Roy Smith wrote: Every so often (typically when refactoring), I'll remove a .py file and forget to remove the corresponding .pyc file. If I then import the module, python finds the orphaned .pyc and happily imports it. Usually leading to confusing and hard to debug failures. Is there some way to globally tell python, Never import a .pyc unless the corresponding .py file exits? Perhaps some environment variable I could set in my .login file? If you want to stay python only I have this function: def clean_orphaned_pycs(directory, simulate=False): Remove all .pyc files without a correspondent module and return the list of the removed files. If simulate=True don't remove anything but still return the list of pycs exists, join = path.exists, path.join rm = (lambda _: None) if simulate else remove removed = [] for root, dirs, files in walk(directory): for f in files: if f.endswith(.pyc): pyc = join(root, f) if not exists(pyc[:-1]): logger.debug(DELETING orphaned file %s % pyc) removed.append(pyc) rm(pyc) # ignore certain sub-dirs for i in reversed(range(len(dirs))): d = dirs[i] if d == .svn or d.endswith(.egg-info): dirs.pop(i) # dirs.remove(d) return removed -- http://mail.python.org/mailman/listinfo/python-list
Debugging / profiling subscripts
If for example I have a Python script which for some reasons needs to run another Python script as a subprocess, is there a way to debug / profile it? If I add an import pdb; pdb.set_trace() In my specific case I am running a program that, under certain conditions, has to restart from scratch, doing something like cmd = [python] + sys.argv Popen(cmd, shell=True) but the subprocess becomes than a black box.. -- http://mail.python.org/mailman/listinfo/python-list
Re: Disable use of pyc file with no matching py file
Jean-Michel Pichavant jeanmic...@sequans.com writes: Terry Reedy wrote: On 1/30/2012 4:30 PM, Roy Smith wrote: Is there some way to globally tell python, Never import a .pyc unless the corresponding .py file exits? Upgrade to 3.2. No. Terry's response was an answer to “Is there some way to [do what Roy Smith asked how to do]?”. Are you saying that upgrading to Python 3.2 doesn't work for that? -- \“Faith is belief without reason. I think as soon as you turn | `\ your back on reason, you turn your back on what makes us | _o__) human.” —Iain M. Banks, 2010-10-07 | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Disable use of pyc file with no matching py file
On Tue, 31 Jan 2012 11:26:10 +0100, Jean-Michel Pichavant wrote: Terry Reedy wrote: On 1/30/2012 4:30 PM, Roy Smith wrote: Every so often (typically when refactoring), I'll remove a .py file and forget to remove the corresponding .pyc file. If I then import the module, python finds the orphaned .pyc and happily imports it. Usually leading to confusing and hard to debug failures. Is there some way to globally tell python, Never import a .pyc unless the corresponding .py file exits? Upgrade to 3.2. No. Is that intended as No, I won't upgrade or No, Python 3.2 doesn't do the job? If the first one, then that's your decision, of course, but there really is no other straight-forward way to have Python ignore left-over .pyc files. Like it or not, there is no way to disable the use of .pyc files in older versions of Python. (I suppose you could write a patch for Python, and use your custom version, but that's it.) The ability to use .pyc files without providing source code is considered by some people a feature, not a bug, and in fact even in Python 3.2 the same applies. The difference in 3.2 is that you can't *accidentally* use old left-over .pyc files, you have to move and rename them before they can be used as sourceless modules. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
surprising result all (generator) (bug??)
I was just bitten by this unexpected behavior: In [24]: all ([i 0 for i in xrange (10)]) Out[24]: False In [25]: all (i 0 for i in xrange (10)) Out[25]: True -- http://mail.python.org/mailman/listinfo/python-list
TestFixtures 2.3.4 Released!
Hi All, I'm very happy to announce the first TestFixtures release built after a continuous integration testing process. The main benefit of this is that TestFixtures now works with Python 2.5 and Python 2.7. I thought they did before, but there were some annoying niggles. The CI stuff can all be seen here: http://jenkins.simplistix.co.uk/ ...with the specific build pipeline for TestFixtures here: http://jenkins.simplistix.co.uk/view/testfixtures/ The package is on PyPI and a full list of all the links to docs, issue trackers and the like can be found here: http://www.simplistix.co.uk/software/python/testfixtures Any questions, please do ask on the Testing in Python list or on the Simplistix open source mailing list... cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: surprising result all (generator) (bug??)
On Jan 31, 6:40 am, Neal Becker ndbeck...@gmail.com wrote: I was just bitten by this unexpected behavior: In [24]: all ([i 0 for i in xrange (10)]) Out[24]: False In [25]: all (i 0 for i in xrange (10)) Out[25]: True What does: import numpy all is numpy.all give you? -- Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: surprising result all (generator) (bug??)
Mark Dickinson wrote: On Jan 31, 6:40 am, Neal Becker ndbeck...@gmail.com wrote: I was just bitten by this unexpected behavior: In [24]: all ([i 0 for i in xrange (10)]) Out[24]: False In [25]: all (i 0 for i in xrange (10)) Out[25]: True What does: import numpy all is numpy.all give you? -- Mark In [31]: all is numpy.all Out[31]: True Excellent detective work, Mark! But it still is unexpected, at least to me. -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
Charles Yeomans wrote: To catch more than one exception type in an except block, one writes except (A, B, C) as e: I'm wondering why it was decided to match tuples, but not lists: except [A, B, C] as e: The latter makes more sense semantically to me -- catch all exception types in a list as opposed to catch this single thing composed of three exception types. On reflection, it seems to hint at a style that Python extensions were made in. (IIRC) the first operand in an `except` statement was originally just an arbitrary marker to identify the exception. Unique string values were customary, although the Python library defined things with standard exception names. Using a string means that general exceptions weren't to be collected in general sequences; `except Serious Error` was never meant to catch `raise r`. If only tuples were used for collections, it would create havoc for fewer of any weirdos who had used strange markers of their own devising. It looks like tuples were chosen as the most lightweight, or maybe least intrusive, sequence type to require to denote a collection of exceptions. You see a similar decision, with the opposite emphasis, with the string modulo operator. The second operand is supposed to be a tuple, but if the template string needs only one value, then the rules are relaxed and any single non-tuple value is used as-is. Mel. -- http://mail.python.org/mailman/listinfo/python-list
python zipfile v. native unzip
Does Python 2.7's zipfile module use its own algorithm or does it leverage the zip/unzip libraries that exist on the host? I ask because my host's native unzip program cannot handle files that, when unzipped, are larger than 2GB. Will using Python 2.7 get around this limitation? -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Jan 30, 2012, at 7:00 PM, Steven D'Aprano wrote: On Mon, 30 Jan 2012 12:41:00 -0500, Charles Yeomans wrote: To catch more than one exception type in an except block, one writes except (A, B, C) as e: I'm wondering why it was decided to match tuples, but not lists: except [A, B, C] as e: Simplicity. If you also allow lists, then why not allow arbitrary sequences? What about iterators, do you allow them? That could be awkward, because iterators can only be run through once. Dictionaries are also iterable, so once you allow arbitrary iterables, you get dicts. The whole thing becomes a mess. Better to keep it simple and only allow a single canonical collection type, and in Python, that type is tuple, not list. Tuples are that canonical collection type because they have a number of desirable properties: - Tuples are small and memory efficient, using the smallest amount of memory needed to hold their items. Lists typically carry a block of spare memory, to make insertions fast. - Consequently the Python virtual machine can create them rapidly and efficiently. - Tuples are immutable, so you don't have to worry about passing one to a function and having the function modify it behind your back. - Tuples are ordered, for the times where that matters. - Since the typical use-case is to iterate over the items in fixed order, there's no need to pay the extra expense for a dict or set. - Tuples are simple to write: in general you only need commas between items. Sometimes, to avoid ambiguity or change the precedence of calculation, you also need round brackets (parentheses for Americans). Except clauses are one of those times. - Frozensets and sets are ruled out for historical reasons: they didn't exist until Python 2.3. Besides, which would you rather write? (abc, def) frozenset([abc, def]) - Sets and lists are ruled out because they are mutable, both require much more memory, and sets have a heavier computational burden. The latter makes more sense semantically to me -- catch all exception types in a list as opposed to catch this single thing composed of three exception types. Then you are labouring under a misunderstanding. You're not catching a tuple, because tuples are never thrown. You're catching any of the exceptions that are contained in that tuple. Both lists and tuples *are* single things in themselves. Both lists and tuples are containers: A list is a single thing that contains other things. A tuple is a single thing that contains other things. I don't think of a tuple as a container, and I don't think it a misunderstanding on my part to think this. But I am aware that it is common to use tuples as immutable lists. I don't see that performance was really a consideration, given that one can use any expression in an except statement -- except IOError if today == 'Monday' else OSError as e or L = [] try: #code except tuple(L) as e: pass except Exception, e: L.append(e.__class__) In any case, though I appreciate your attempt at a post hoc justification, I was hoping for a positive explanation. Charles Yeomans -- http://mail.python.org/mailman/listinfo/python-list
Re: surprising result all (generator) (bug??)
On 01/31/12 06:40, Neal Becker wrote: I was just bitten by this unexpected behavior: In [24]: all ([i 0 for i in xrange (10)]) Out[24]: False In [25]: all (i 0 for i in xrange (10)) Out[25]: True You sure you transcribed that correctly? Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48) [GCC 4.4.5] on linux2 Type help, copyright, credits or license for more information. all([i0 for i in xrange(10)]) False all(iter([i0 for i in xrange(10)])) False all(i0 for i in xrange(10)) False So unless it's something in a newer version of Python, I suspect your examples aren't what you typed. -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Disable use of pyc file with no matching py file
Steven D'Aprano wrote: On Tue, 31 Jan 2012 11:26:10 +0100, Jean-Michel Pichavant wrote: Terry Reedy wrote: On 1/30/2012 4:30 PM, Roy Smith wrote: Every so often (typically when refactoring), I'll remove a .py file and forget to remove the corresponding .pyc file. If I then import the module, python finds the orphaned .pyc and happily imports it. Usually leading to confusing and hard to debug failures. Is there some way to globally tell python, Never import a .pyc unless the corresponding .py file exits? Upgrade to 3.2. No. Is that intended as No, I won't upgrade or No, Python 3.2 doesn't do the job? To answer Ben's mail as well, the No would be more of the don't do it. My answer was as argued as Terry's one anyway (it was quite intended). Steven, you often use analogies/similarities, here's one: A: My wheel is flat B: Buy a new car Buying a new car would solve A's problem : yes Should A buy a new car : probably no Python 3.2 is fine, but someone could run into several issues while migrating. This is quite a pretty huge decision to make (I dedicate this sentence to Rick, I hope he's trolling fine). JM -- http://mail.python.org/mailman/listinfo/python-list
Re: surprising result all (generator) (bug??)
On Jan 31, 7:24 am, Neal Becker ndbeck...@gmail.com wrote: In [31]: all is numpy.all Out[31]: True Excellent detective work, Mark! But it still is unexpected, at least to me. Agreed that it's a bit surprising. It's a consequence of NumPy's general mechanisms for converting arbitrary inputs to arrays: from numpy import asarray asarray([i 0 for i in range(10)]) array([False, True, True, True, True, True, True, True, True, True], dtype=bool) asarray(i 0 for i in range(10)) array(generator object genexpr at 0x4688aa8, dtype=object) So in the second case you get a 0-dimensional array of type 'object', whose only element is that generator object; not surprisingly, the generator object is considered true. As to why it's this way: best to ask on the NumPy mailing lists. It's probably something that's quite difficult to change without breaking lots of code, though. -- Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Tue, 31 Jan 2012 08:57:31 -0500, Charles Yeomans wrote: I don't think of a tuple as a container, and I don't think it a misunderstanding on my part to think this. Well, it is a misunderstanding, because tuples ARE containers. You might as well say I don't think of boxes as containers. What exactly are they if not containers? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
importing module versus using function?
Hi, I am confused on this quite bad!! If I have this typed in interactive python: import numpy def dummy(): y=numpy.arange(1,2,0.1) return y and then s = dummy() s array[1. , 1.1, 1.2] it works. But if I have a module called example.py, whose code is def dummy(): y=numpy.arange(1,2,0.1) return y and now if I do the following: import numpy from example import * s=dummy() The above sequence does not work. It gives an error saying global variable numpy not defined. I understand that when I import a module it gets imported after getting compiled and therefore, the module should have a statement import numpy at its start. But what if I just want to use the function 'dummy'? I tried the following: from example import dummy and then try to use the function, it still gives me 'global name numpy not defined'! Why is that happening? Any pointers please. Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: importing module versus using function?
On 1/31/12 3:08 PM, gujax wrote: Hi, I am confused on this quite bad!! If I have this typed in interactive python: import numpy def dummy(): y=numpy.arange(1,2,0.1) return y and then s = dummy() s array[1. , 1.1, 1.2] it works. But if I have a module called example.py, whose code is def dummy(): y=numpy.arange(1,2,0.1) return y and now if I do the following: import numpy from example import * s=dummy() The above sequence does not work. It gives an error saying global variable numpy not defined. I understand that when I import a module it gets imported after getting compiled and therefore, the module should have a statement import numpy at its start. But what if I just want to use the function 'dummy'? You need to import numpy in dummy.py. When a function needs to look up a name, it goes to the module's namespace in which the function is defined, not one of the many namespaces where the function is called from. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
configobj
I have a couple of questions about configobj, which I'm happily trying to use for this project. The first question is, how do I make a very long list of things? Suppose I have declared in a spec file. val = string_list now I would like to do val = one, two, three but that's not allowed, and also val = one, \ two, \ three doesn't work, is there a way to avoid to write everything on one line? The second question is, how do I avoid declaring twice the default value? For example supposing I have this spec: skip_pesky_pyc_paths = string_list I was giving for granted that (pseudocode ahead) conf = ConfigObj(spec=myspec) conf['skip_pesky_pyc_paths'] == [] but it's not the case, if it's not declared in the conf file it just doesn't find the key? Is there a magic option to make it create the key when they are not declared from the spec? Thanks, Andrea PS. and why by the way ConfigObj is not in the standard lib, instead of ConfigParser, which looked less complete and more cumbersome (to me at least) -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Jan 31, 2012, at 9:51 AM, Steven D'Aprano wrote: On Tue, 31 Jan 2012 08:57:31 -0500, Charles Yeomans wrote: I don't think of a tuple as a container, and I don't think it a misunderstanding on my part to think this. Well, it is a misunderstanding, because tuples ARE containers. You might as well say I don't think of boxes as containers. What exactly are they if not containers? Tuple is a heterogenous datatype that allows one to define objects ad hoc. That is to say, a tuple represents a single thing distinct from its components. For example, suppose you need to represent a location in text by line number and offset within a line. A tuple object makes it easy to do so without writing a class having no methods other than a constructor. Here, the components, a line number and an offset, define a new object distinct from the pieces. One can certainly view a tuple as a list, just as one can view a string as a list of characters, and sometimes that's useful; the Python dictum there should only be one way to do it doesn't imply that there is only one way to think of it. Nor am I the only person who sees such a distinction between tuple and list. Charles Yeomans -- http://mail.python.org/mailman/listinfo/python-list
Re: IDLE not setting current directory in its path
Thanks Terry, I see that the issue has been closed by you. However, I do not know how to run the patch on my Windows. Do I reinstall IDLE? Please suggest. I am using Python2.7 gujax On Jan 30, 10:55 pm, Terry Reedy tjre...@udel.edu wrote: On 1/30/2012 3:15 PM, Terry Reedy wrote: This issue is under consideration at http://bugs.python.org/issue13506 It should be fixed before the next Python releases. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Tue, Jan 31, 2012 at 11:23 AM, Charles Yeomans char...@declaresub.com wrote: On Jan 31, 2012, at 9:51 AM, Steven D'Aprano wrote: On Tue, 31 Jan 2012 08:57:31 -0500, Charles Yeomans wrote: I don't think of a tuple as a container, and I don't think it a misunderstanding on my part to think this. Well, it is a misunderstanding, because tuples ARE containers. You might as well say I don't think of boxes as containers. What exactly are they if not containers? Tuple is a heterogenous datatype that allows one to define objects ad hoc. That is to say, a tuple represents a single thing distinct from its components. For example, suppose you need to represent a location in text by line number and offset within a line. A tuple object makes it easy to do so without writing a class having no methods other than a constructor. Here, the components, a line number and an offset, define a new object distinct from the pieces. One can certainly view a tuple as a list, just as one can view a string as a list of characters, and sometimes that's useful; the Python dictum there should only be one way to do it doesn't imply that there is only one way to think of it. Nor am I the only person who sees such a distinction between tuple and list. Perhaps it'd be useful to look at how the Python language reference defines containers? Quote: Some objects contain references to other objects; these are called containers. Examples of containers are tuples, lists and dictionaries. The references are part of a container’s value. In most cases, when we talk about the value of a container, we imply the values, not the identities of the contained objects; however, when we talk about the mutability of a container, only the identities of the immediately contained objects are implied. So, if an immutable container (like a tuple) contains a reference to a mutable object, its value changes if that mutable object is changed. End quote. (Offtopic: How do I do an external block quote appropriately in an email?) Tuples are most certainly containers, precisely _because_ they're an ad-hoc way to define objects, where the only purpose of the object is to contain the values inside the tuple. But these are just words and it's beside the point. We should talk of things as if the word container didn't matter, because I don't think that's what you meant, but neither do I want to put words in your mouth. My interpretation is that you see tuples as an object where you can get meaningfully things by field, rather than just grab arbitrarily from a bag of things. This isn't the only way they are used, see the except statement (hee) and their use as keys in dictionaries. But it is true that their immutable length makes them very well suited to the task of representing product types. -- Devin -- http://mail.python.org/mailman/listinfo/python-list
Re: Killing threads, and os.system()
Le 31/01/2012 17:04, Dennis Lee Bieber a écrit : Of course, if that thread is stuck waiting for a call to os.system() to complete, then it can not do anything... os.system() is a rather limited, restrictive, call -- best used for quick one-of operations. If running Python 2.6+, I'd recommend converting from os.system() to subprocess.Popen(). .Popen() objects now have .terminate() and .kill() methods. Ok, I'll try that Popen. Indeed I think that my threads are stuck waiting os.system to complete. Thanks Laurent -- http://mail.python.org/mailman/listinfo/python-list
Re: surprising result all (generator) (bug??)
On 31/01/2012 12:40, Neal Becker wrote: I was just bitten by this unexpected behavior: In [24]: all ([i 0 for i in xrange (10)]) Out[24]: False In [25]: all (i 0 for i in xrange (10)) Out[25]: True Which version of Python? -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Jan 31, 2012, at 11:38 AM, Devin Jeanpierre wrote: On Tue, Jan 31, 2012 at 11:23 AM, Charles Yeomans char...@declaresub.com wrote: On Jan 31, 2012, at 9:51 AM, Steven D'Aprano wrote: On Tue, 31 Jan 2012 08:57:31 -0500, Charles Yeomans wrote: I don't think of a tuple as a container, and I don't think it a misunderstanding on my part to think this. Well, it is a misunderstanding, because tuples ARE containers. You might as well say I don't think of boxes as containers. What exactly are they if not containers? Tuple is a heterogenous datatype that allows one to define objects ad hoc. That is to say, a tuple represents a single thing distinct from its components. For example, suppose you need to represent a location in text by line number and offset within a line. A tuple object makes it easy to do so without writing a class having no methods other than a constructor. Here, the components, a line number and an offset, define a new object distinct from the pieces. One can certainly view a tuple as a list, just as one can view a string as a list of characters, and sometimes that's useful; the Python dictum there should only be one way to do it doesn't imply that there is only one way to think of it. Nor am I the only person who sees such a distinction between tuple and list. Perhaps it'd be useful to look at how the Python language reference defines containers? Quote: Some objects contain references to other objects; these are called containers. Examples of containers are tuples, lists and dictionaries. The references are part of a container’s value. In most cases, when we talk about the value of a container, we imply the values, not the identities of the contained objects; however, when we talk about the mutability of a container, only the identities of the immediately contained objects are implied. So, if an immutable container (like a tuple) contains a reference to a mutable object, its value changes if that mutable object is changed. End quote. (Offtopic: How do I do an external block quote appropriately in an email?) Tuples are most certainly containers, precisely _because_ they're an ad-hoc way to define objects, where the only purpose of the object is to contain the values inside the tuple. But these are just words and it's beside the point. We should talk of things as if the word container didn't matter, because I don't think that's what you meant, but neither do I want to put words in your mouth. My interpretation is that you see tuples as an object where you can get meaningfully things by field, rather than just grab arbitrarily from a bag of things. This isn't the only way they are used, see the except statement (hee) and their use as keys in dictionaries. But it is true that their immutable length makes them very well suited to the task of representing product types. I had read that bit of documentation, and don't entirely agree with it. Certainly many objects contain references to other objects, but are not considered containers. And I claim that tuples are most certainly not containers, when they are used to define an object as a collection of other objects, like the text-location example I offered earlier. On the other hand, it is certainly true that tuples quack like containers, as do strings. Charles Yeomans -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
Charles Yeomans wrote:
On Jan 31, 2012, at 9:51 AM, Steven D'Aprano wrote:
On Tue, 31 Jan 2012 08:57:31 -0500, Charles Yeomans wrote:
I don't think of a tuple as a container, and I don't think it a
misunderstanding on my part to think this.
Well, it is a misunderstanding, because tuples ARE containers. You
might as well say I don't think of boxes as containers. What
exactly are they if not containers?
Tuple is a heterogenous datatype that allows one to define objects
ad hoc.
And any object can be seen as a container for its component pieces --
some are just more general than others.
Compare:
location = (13, 4, 9)# line, word, char
time = (10, 15, 41) # hour, minute, second
result = ('this', 'that', 'huh') # result a, result b, result c
with:
record1 = Record('Ethan', 41, Male)
record2 = Record('Charles', 37, Male)
record3 = Record('Steven', 43, Male)
record4 = Record('Jennifer', 39, Female)
In this example, Records have a set layout and so it is more common to
think of a Record as a thing; location, time, and result, however, are
all random tuples created on the fly with no absolute restrictions on
what goes in which position.
lists, dicts, sets, and tuples are general purpose containers; strs (and
most user defined classes) are special purpose containers.
That is to say, a tuple represents a single thing distinct from its
components.
You could say that about a list as well. Doesn't change the fact that a
list is a container.
One can certainly view a tuple as a list, just as one can view a string
as a list of characters, and sometimes that's useful; the Python dictum
there should only be one way to do it doesn't imply that there is only
one way to think of it.
The 'dictum' is there should only be one *obvious* way to do it
(emphasis added).
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Jan 31, 2012, at 8:24 AM, Mel Wilson wrote: Charles Yeomans wrote: To catch more than one exception type in an except block, one writes except (A, B, C) as e: I'm wondering why it was decided to match tuples, but not lists: except [A, B, C] as e: The latter makes more sense semantically to me -- catch all exception types in a list as opposed to catch this single thing composed of three exception types. On reflection, it seems to hint at a style that Python extensions were made in. (IIRC) the first operand in an `except` statement was originally just an arbitrary marker to identify the exception. Unique string values were customary, although the Python library defined things with standard exception names. Using a string means that general exceptions weren't to be collected in general sequences; `except Serious Error` was never meant to catch `raise r`. If only tuples were used for collections, it would create havoc for fewer of any weirdos who had used strange markers of their own devising. It looks like tuples were chosen as the most lightweight, or maybe least intrusive, sequence type to require to denote a collection of exceptions. You see a similar decision, with the opposite emphasis, with the string modulo operator. The second operand is supposed to be a tuple, but if the template string needs only one value, then the rules are relaxed and any single non-tuple value is used as-is. Compatilbility; that makes sense. I came to python well after strings were used for exceptions. Thanks. Charles Yeomans -- http://mail.python.org/mailman/listinfo/python-list
Re: speaking at PyCon
On 29/01/2012 03:32, Eric Snow wrote: This is my first year speaking at PyCon, so I solicited speaking/preparation advice from a bunch of folks, particularly focusing on the PyCon speaking experience. I've compiled the results and put them online: http://ref.rtfd.org/speakers This is still rough, and feedback is welcome, as is more advice. :) For anyone speaking at the conference (or generally), I hope this will be helpful. Thanks! -eric Good general presentation tips, I have another suggestion: If you bring your own laptop, make sure to practice connecting it to the projector and have a special presentation account (which you also used for your practice and nothing else). -- mph -- http://mail.python.org/mailman/listinfo/python-list
Re: Disable use of pyc file with no matching py file
On Jan 30, 3:43 pm, Terry Reedy tjre...@udel.edu wrote: On 1/30/2012 4:30 PM, Roy Smith wrote: Every so often (typically when refactoring), I'll remove a .py file and forget to remove the corresponding .pyc file. If I then import the module, python finds the orphaned .pyc and happily imports it. Usually leading to confusing and hard to debug failures. Is there some way to globally tell python, Never import a .pyc unless the corresponding .py file exits? Upgrade to 3.2. -- Terry Jan Reedy Terry, I've noticed that the tutorial (section 6.1.3) hasn't been updated for PEP 3147; there's no way of telling that this is the behavior from reading the tutorial. The development doc for 3.3 hasn't been updated either. -- http://mail.python.org/mailman/listinfo/python-list
RE: contextlib.contextmanager and try/finally
Like Neil mentioned, a contextmanager generator is wrapped with an __exit__ method that is guaranteed to be called and that explicitly resumes or closes the generator. So as long as your contextmanager generator is properly written (i.e. it yields exactly once), the finally block will execute in a timely fashion. Is that true even in the face of something like sys.exit()? What happens if 1) sys.exit is called while in the same thread 2) sys.exit is called from another thread but while this thread is in context manager? Ramit Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology 712 Main Street | Houston, TX 77002 work phone: 713 - 216 - 5423 -- This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. -- http://mail.python.org/mailman/listinfo/python-list
RE: contextlib.contextmanager and try/finally
Prasad, Ramit wrote: Like Neil mentioned, a contextmanager generator is wrapped with an __exit__ method that is guaranteed to be called and that explicitly resumes or closes the generator. So as long as your contextmanager generator is properly written (i.e. it yields exactly once), the finally block will execute in a timely fashion. Is that true even in the face of something like sys.exit()? What happens if 1) sys.exit is called while in the same thread 2) sys.exit is called from another thread but while this thread is in context manager? sys.exit() uses the normal exception mechanism to terminate a program: import sys try: ... sys.exit() ... except SystemExit: ... print I'd rather not ... I'd rather not It won't derail a context manager: from contextlib import contextmanager @contextmanager ... def f(): ... try: ... yield ... finally: print bye ... import sys with f(): ... sys.exit() ... bye $ -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
Charles Yeomans char...@declaresub.com wrote: To catch more than one exception type in an except block, one writes except (A, B, C) as e: I'm wondering why it was decided to match tuples, but not lists: except [A, B, C] as e: The latter makes more sense semantically to me -- catch all exception types in a list as opposed to catch this single thing composed of three exception types. It may not be the only reason but the code would have to be slower and much more complex to handle lists. If you wanted you can write: except ((A,), ((B,), C)) as e: or other such complicated expression with nested tuples. If lists were allowed in a similarly nested structure there would be a danger that you could pass in a recursive list structure so the code would have to detect and avoid infinite loops. exceptions = [A, B, C] exceptions[1:1] = exceptions, ... except exceptions as e: # argh! Abitrarily nested tuples of exceptions cannot contain loops so the code simply needs to walk through the tuples until it finds a match. -- Duncan Booth http://kupuguy.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: contextlib.contextmanager and try/finally
On Tue, Jan 31, 2012 at 2:07 PM, Prasad, Ramit ramit.pra...@jpmorgan.com wrote: Like Neil mentioned, a contextmanager generator is wrapped with an __exit__ method that is guaranteed to be called and that explicitly resumes or closes the generator. So as long as your contextmanager generator is properly written (i.e. it yields exactly once), the finally block will execute in a timely fashion. Is that true even in the face of something like sys.exit()? Yes. What happens if 1) sys.exit is called while in the same thread Why don't you try it and find out? To answer the question, though, sys.exit() raises a SystemExit exception, which propagates out of the with block and calls the __exit__ method, which then throws the exception to the generator, which executes its finally clause and exits. The __exit__ method returns false, so the SystemExit exception continues to propagate, and if it is not caught, then the process exits. 2) sys.exit is called from another thread but while this thread is in context manager? Then the other thread raises SystemExit, and the current thread is unaffected. sys.exit only affects the thread it is called in. You can certainly come up with scenarios in which the finally clause does not execute, e.g. killing the interpreter with kill -9 or yanking out the power cord. Within the confines of the Python interpreter, though, it is guaranteed that the finally block will execute. -- http://mail.python.org/mailman/listinfo/python-list
Re: IDLE not setting current directory in its path
On 1/31/2012 11:27 AM, gujax wrote: Thanks Terry, I see that the issue has been closed by you. However, I do not know how to run the patch on my Windows. Do I reinstall IDLE? Please suggest. I am using Python2.7 Choices: 1. Wait for the next 2.7 release, which should be within a month. Easiest ;-). Will get you all other patches too. 2. Edit the files by hand, deleting lines marked - in the patch and adding lines marked +. Or change lines so they match the + lines. Harder, but you get the one change now instead of later. 3. Download and install TortoiseHG, turn your python installation (or just the idlelib directory) into a repository, apply the patch (or an edited version thereof), and perhaps delete the repository stuff. I would only do this if you want to learn to use (Tortoise)HG anyway, perhaps so you can apply other patches too, without waiting. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Disable use of pyc file with no matching py file
On 1/31/2012 3:20 PM, John Roth wrote: On Jan 30, 3:43 pm, Terry Reedytjre...@udel.edu wrote: On 1/30/2012 4:30 PM, Roy Smith wrote: Every so often (typically when refactoring), I'll remove a .py file and forget to remove the corresponding .pyc file. If I then import the module, python finds the orphaned .pyc and happily imports it. Usually leading to confusing and hard to debug failures. Is there some way to globally tell python, Never import a .pyc unless the corresponding .py file exits? Upgrade to 3.2. I tested before writing this. The caveat is that x.pyc in the same directly as x.py will not be ignored (for back compatibility). However, this only happens intentionally as .pyc files are put in __pycache__/ with name x.version.pyc, where version is 'cpython-32' or something similar for another version or implementation. I've noticed that the tutorial (section 6.1.3) hasn't been updated for PEP 3147; there's no way of telling that this is the behavior from reading the tutorial. The development doc for 3.3 hasn't been updated either. You are right. An oversight. Thanks for noticing. http://bugs.python.org/issue13915 Suggested rewrites are welcome. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Disable use of pyc file with no matching py file
On 1/31/2012 9:19 AM, Jean-Michel Pichavant wrote: A: My wheel is flat B: Buy a new car A better analogy would be Q. How do I make my old model car do something (it cannot do)? A. Get the free new model that has that feature added. Of course, there is a cost to giving up the old and familiar and learning and adjusting to the new, even when it is available gratis. A husband wearing an old sweater after his wife gives him a new one, and even retrieving it from the trash when she tosses it out, is a classic (and true) cartoon joke. But I am sure that 95% of readers here will be using 3.x withing 10 years. The only question for them is When?. This not-well-known new feature is one straw that some will put on the 'sooner' side of the balance. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On 1/31/2012 8:57 AM, Charles Yeomans wrote: In any case, though I appreciate your attempt at a post hoc justification, I was hoping for a positive explanation. I think the best you are going to get is that Python somewhat consistently*, for both practical and historical reasons#, uses tuples when the syntax allows an object or collection of objects. * except, isinstance, isubclass, ''%x, perhaps other places. In the last case, that creates a problem when one wants to interpolate a tuple as an object rather than having it viewed as a container of several objects to be interpolated. That was on # Python once treated tuples as different from lists in ways that is not true now. (Read the 1.5 docs if really interested.) -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
the script can not go through all the list, anyone could help, would be highly appreciated, thanks a lot !
the script can not go through all the list, anyone could help, would be highly appreciated, thanks a lot ! output: initial 7 lines: [9379, 'G', '0.1830'] [9378, 'G', '0.1752'] [9377, 'G', '0.1929'] [9375, 'G', '0.1950'] [9370, 'G', '0.1872'] [937, 'G', '0.1931'] [93, 'G', '0.1974'] processing: code_K = 93 rate_K = -1 len_K = 1 len_G = 7 code_K = 93 rate_K = 0.1830 code_G = 9379 rate_G = 0.1830 code_K = 93 rate_K = 0.1929 code_G = 9377 rate_G = 0.1929 code_K = 93 rate_K = 0.1929 code_G = 9370 rate_G = 0.1872 code_K = 93 rate_K = 0.1974 code_G = 93 rate_G = 0.1974 final 3 lines: [9378, 'G', '0.1752'] [9375, 'G', '0.1950'] [937, 'G', '0.1931'] code: .. # read data from .csv print initial, len(list_G), lines: for row_G in list_G: print row_G print processing: for row_K in list_K: code_K = str(row_K[0]) rate_K = -1 len_K = len(list_K) len_G = len(list_G) print code_K =, code_K, rate_K =, rate_K, len_K =, len_K, len_G =, len_G for row_G in list_G: code_G = str(row_G[0]) rate_G = str(row_G[2]) if re.match(code_K, code_G): if rate_K rate_G: rate_K = rate_G print code_K =, code_K, rate_K =, rate_K, code_G =, code_G, rate_G =, rate_G list_G.remove(row_G) print final, len(list_G), lines: for row_G in list_G: print row_G-- http://mail.python.org/mailman/listinfo/python-list
Re: configobj
On 1/31/2012 11:06 AM, Andrea Crotti wrote: I have a couple of questions about configobj, which I'm happily trying to use for this project. When asking about 3rd party modules, please include a url, so we can be sure of what you mean and even take a look. Is www.voidspace.org.uk/python/configobj.html what you mean? PS. and why by the way ConfigObj is not in the standard lib, instead of ConfigParser, Perhaps history. Informed suggestions for change are welcome. which looked less complete and more cumbersome (to me at least) Does ConfigParser have the same problems you found with ConfigObj. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Jan 31, 2012, at 7:12 PM, Terry Reedy wrote: On 1/31/2012 8:57 AM, Charles Yeomans wrote: In any case, though I appreciate your attempt at a post hoc justification, I was hoping for a positive explanation. I think the best you are going to get is that Python somewhat consistently*, for both practical and historical reasons#, uses tuples when the syntax allows an object or collection of objects. * except, isinstance, isubclass, ''%x, perhaps other places. In the last case, that creates a problem when one wants to interpolate a tuple as an object rather than having it viewed as a container of several objects to be interpolated. That was on # Python once treated tuples as different from lists in ways that is not true now. (Read the 1.5 docs if really interested.) I'll do that. Thanks. Charles Yeomans -- http://mail.python.org/mailman/listinfo/python-list
How can I verify if the content of a variable is a list or a string?
Hi, I'm writing a function which receive a list which elements are strings or new lists (sublists) containing strings. How can I verify if sone element of the list (which is contained in a variable) is a list or a string? I found the method isinstance(object,class) but I don't know which class should I use for. Thank you, regards Prof. Dr. Andrés Soto DES DACI UNACAR-- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Wed, Feb 1, 2012 at 9:03 AM, Duncan Booth duncan.booth@invalid.invalid wrote: Abitrarily nested tuples of exceptions cannot contain loops so the code simply needs to walk through the tuples until it finds a match. Is this absolutely guaranteed? The C API for CPython provides: (Py2) http://docs.python.org/c-api/tuple.html#PyTuple_SetItem (Py3) http://docs.python.org/dev/c-api/tuple.html#PyTuple_SetItem which doesn't have massive warnings on it saying USE THIS ONLY TO INITIALIZE A TUPLE (compare, for instance, _PyTuple_Resize which does carry a similar warning). Is the assumption then that we're all adults, and that mutating a tuple is like passing a null pointer to an API function (aka loaded gun in proximity to foot)? ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I verify if the content of a variable is a list or a string?
On 1/31/2012 7:44 PM, Andres Soto wrote: Hi, I'm writing a function which receive a list which elements are strings or new lists (sublists) containing strings. How can I verify if sone element of the list (which is contained in a variable) is a list or a string? I found the method isinstance(object,class) but I don't know which class should I use for. For 3.x, 'list' or 'str' (where 'str' means unicode). For 2.x, I believe 'basestring' includes unicode strings as well as byte strings. isinstance([], list) True isinstance([], str) False -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I verify if the content of a variable is a list or a string?
On Wed, Feb 1, 2012 at 12:44 AM, Andres Soto soto_and...@yahoo.com wrote: Hi, I'm writing a function which receive a list which elements are strings or new lists (sublists) containing strings. How can I verify if sone element of the list (which is contained in a variable) is a list or a string? I found the method isinstance(object,class) but I don't know which class should I use for. Thank you, regards Prof. Dr. Andrés Soto DES DACI UNACAR list and str my_list = [1, 2, 3] isinstance(my_list, list) True my_string = foobar isinstance(my_string, str) True -- http://mail.python.org/mailman/listinfo/python-list
'class' named tuple
I have the following program. I am trying to have index the attributes of an object using __getitem__. Reading them this way works great, but assigning them a value doesn't Is there a way to do such a thing? (Almost like a named tuple, but with custom methods) class LIter(object): def __init__(self,parent=None): super(LIter, self).__init__() self.toto = 3 self.tata = 'terto' def __getitem__(self,index): if index == 0 : return self.toto if index == 1 : return self.tata # [other methods] if __name__ == __main__: i = LIter() print i[0] print i[1] i.toto = 2 i.tata = 'bye' print i[0] print i[1] i[0] = -1 i[1] = 'salut' print i[0] print i[1] $ python iter.py 3 terto 2 bye Traceback (most recent call last): File iter.py, line 25, in module i[0] = -1 TypeError: 'LIter' object does not support item assignment -- http://mail.python.org/mailman/listinfo/python-list
Re: the script can not go through all the list, anyone could help, would be highly appreciated, thanks a lot !
On 01/02/2012 00:00, Para wrote: the script can not go through all the list, anyone could help, would be highly appreciated, thanks a lot ! output: initial 7 lines: [9379, 'G', '0.1830'] [9378, 'G', '0.1752'] [9377, 'G', '0.1929'] [9375, 'G', '0.1950'] [9370, 'G', '0.1872'] [937, 'G', '0.1931'] [93, 'G', '0.1974'] processing: code_K = 93 rate_K = -1 len_K = 1 len_G = 7 code_K = 93 rate_K = 0.1830 code_G = 9379 rate_G = 0.1830 code_K = 93 rate_K = 0.1929 code_G = 9377 rate_G = 0.1929 code_K = 93 rate_K = 0.1929 code_G = 9370 rate_G = 0.1872 code_K = 93 rate_K = 0.1974 code_G = 93 rate_G = 0.1974 final 3 lines: [9378, 'G', '0.1752'] [9375, 'G', '0.1950'] [937, 'G', '0.1931'] code: .. # read data from .csv print initial, len(list_G), lines: for row_G in list_G: print row_G print processing: for row_K in list_K: code_K = str(row_K[0]) rate_K = -1 len_K = len(list_K) len_G = len(list_G) print code_K =, code_K, rate_K =, rate_K, len_K =, len_K, len_G =, len_G for row_G in list_G: code_G = str(row_G[0]) rate_G = str(row_G[2]) if re.match(code_K, code_G): if rate_K rate_G: rate_K = rate_G print code_K =, code_K, rate_K =, rate_K, code_G =, code_G, rate_G =, rate_G list_G.remove(row_G) print final, len(list_G), lines: for row_G in list_G: print row_G rate_K is initially a number (rate_K = -1), but rate_G is a string. If you want to compare numeric values, make sure that they are both numbers. (In Python 3 it will complain if you say x y when one of them is a string and the other is a number; in Python 2 it will return a consistent but arbitrary result.) The re.match(code_K, code_G) checks whether code_G start with code_K. It is better to use code_G.startswith(code_K) instead. Also, you have: for row_G in list_G: ... list_G.remove(row_G) Don't add or remove items in a list over which you are iterating. -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Tue, Jan 31, 2012 at 5:53 PM, Chris Angelico ros...@gmail.com wrote: On Wed, Feb 1, 2012 at 9:03 AM, Duncan Booth duncan.booth@invalid.invalid wrote: Abitrarily nested tuples of exceptions cannot contain loops so the code simply needs to walk through the tuples until it finds a match. Is this absolutely guaranteed? The C API for CPython provides: (Py2) http://docs.python.org/c-api/tuple.html#PyTuple_SetItem (Py3) http://docs.python.org/dev/c-api/tuple.html#PyTuple_SetItem which doesn't have massive warnings on it saying USE THIS ONLY TO INITIALIZE A TUPLE (compare, for instance, _PyTuple_Resize which does carry a similar warning). Is the assumption then that we're all adults, and that mutating a tuple is like passing a null pointer to an API function (aka loaded gun in proximity to foot)? I don't know why the docs are written the way that they are, but if you check the code, you can see that PyTuple_SetItem will raise a SystemError if the reference count is anything other than 1. So I think that it is only meant to be used with similar caution and restraint. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Tue, Jan 31, 2012 at 6:09 PM, Ian Kelly ian.g.ke...@gmail.com wrote: On Tue, Jan 31, 2012 at 5:53 PM, Chris Angelico ros...@gmail.com wrote: On Wed, Feb 1, 2012 at 9:03 AM, Duncan Booth duncan.booth@invalid.invalid wrote: Abitrarily nested tuples of exceptions cannot contain loops so the code simply needs to walk through the tuples until it finds a match. Is this absolutely guaranteed? The C API for CPython provides: (Py2) http://docs.python.org/c-api/tuple.html#PyTuple_SetItem (Py3) http://docs.python.org/dev/c-api/tuple.html#PyTuple_SetItem which doesn't have massive warnings on it saying USE THIS ONLY TO INITIALIZE A TUPLE (compare, for instance, _PyTuple_Resize which does carry a similar warning). Is the assumption then that we're all adults, and that mutating a tuple is like passing a null pointer to an API function (aka loaded gun in proximity to foot)? I don't know why the docs are written the way that they are, but if you check the code, you can see that PyTuple_SetItem will raise a SystemError if the reference count is anything other than 1. So I think that it is only meant to be used with similar caution and restraint. Incidentally, I *think* that any correctly written C code attempting to nest a tuple inside itself would make the reference count of the tuple be at least 2 at the time of the call, and so it would fail. -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I verify if the content of a variable is a list or a string?
okok, my mistake is that I was using string in place of str. Thank you!! regards Prof. Dr. Andrés Soto DES DACI UNACAR From: Noah Hall enali...@gmail.com To: Andres Soto soto_and...@yahoo.com Cc: python-list@python.org python-list@python.org Sent: Tuesday, January 31, 2012 6:58 PM Subject: Re: How can I verify if the content of a variable is a list or a string? On Wed, Feb 1, 2012 at 12:44 AM, Andres Soto soto_and...@yahoo.com wrote: Hi, I'm writing a function which receive a list which elements are strings or new lists (sublists) containing strings. How can I verify if sone element of the list (which is contained in a variable) is a list or a string? I found the method isinstance(object,class) but I don't know which class should I use for. Thank you, regards Prof. Dr. Andrés Soto DES DACI UNACAR list and str my_list = [1, 2, 3] isinstance(my_list, list) True my_string = foobar isinstance(my_string, str) True -- http://mail.python.org/mailman/listinfo/python-list
Re: 'class' named tuple
On Tue, Jan 31, 2012 at 5:54 PM, Emmanuel Mayssat emays...@gmail.com wrote: I have the following program. I am trying to have index the attributes of an object using __getitem__. Reading them this way works great, but assigning them a value doesn't Is there a way to do such a thing? For assignment, use __setitem__. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: contextlib.contextmanager and try/finally
On 1 February 2012 09:02, Peter Otten __pete...@web.de wrote: Prasad, Ramit wrote: Is that true even in the face of something like sys.exit()? What happens if 1) sys.exit is called while in the same thread 2) sys.exit is called from another thread but while this thread is in context manager? sys.exit() uses the normal exception mechanism to terminate a program: import sys try: ... sys.exit() ... except SystemExit: ... print I'd rather not ... I'd rather not It won't derail a context manager: from contextlib import contextmanager @contextmanager ... def f(): ... try: ... yield ... finally: print bye ... import sys with f(): ... sys.exit() ... bye $ Note OTOH that os._exit() just terminates the process with no cleanup (this may be what you were thinking of): from contextlib import contextmanager @contextmanager ... def f(): ... try: ... yield ... finally: print bye ... import os with f(): ... os._exit(1) ... $ Tim Delaney -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterate from 2nd element of a huge list
On 01Feb2012 01:39, Paulo da Silva p_s_d_a_s_i_l_...@netcabo.pt wrote: | What is the best way to iterate thru a huge list having the 1st element | a different process? I.e.: | | process1(mylist[0]) | for el in mylist[1:]: | process2(el) | | This way mylist is almost duplicated, isn't it? Yep. What about (untested): process1(mylist[0]) for i in xrange(1,len(mylist)): process2(mylist[i]) Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ The truth is, I will not give myself the trouble to write sense long, for I would as soon please fools as wise men; because fools are more numerous, and every prudent man will go with the majority.- Hugh Henry Brackenridge -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterate from 2nd element of a huge list
On 1 February 2012 12:39, Paulo da Silva p_s_d_a_s_i_l_...@netcabo.ptwrote: Hi! What is the best way to iterate thru a huge list having the 1st element a different process? I.e.: process1(mylist[0]) for el in mylist[1:]: process2(el) This way mylist is almost duplicated, isn't it? If you are sure that mylist contains at least one element: mylist = [1, 2, 3] i = iter(mylist) print next(i) 1 for el in i: ... print el ... 2 3 Note: for older pythons, you may need i.next() instead of next(i). If mylist may be empty, you will need some error handling. Tim Delaney -- http://mail.python.org/mailman/listinfo/python-list
Re: contextlib.contextmanager and try/finally
On Tue, 31 Jan 2012 15:09:34 -0700, Ian Kelly wrote: On Tue, Jan 31, 2012 at 2:07 PM, Prasad, Ramit ramit.pra...@jpmorgan.com wrote: Like Neil mentioned, a contextmanager generator is wrapped with an __exit__ method that is guaranteed to be called and that explicitly resumes or closes the generator. So as long as your contextmanager generator is properly written (i.e. it yields exactly once), the finally block will execute in a timely fashion. Is that true even in the face of something like sys.exit()? Yes. [...] You can certainly come up with scenarios in which the finally clause does not execute, e.g. killing the interpreter with kill -9 or yanking out the power cord. Within the confines of the Python interpreter, though, it is guaranteed that the finally block will execute. Almost, but not quite. The finally block is not guaranteed to execute if the try block never exits -- infinite loops are still infinite loops. Also, unlike sys.exit, os._exit doesn't work through the exception mechanism, can't be caught, and simply exits immediately. import os try: ... os._exit(1) ... finally: ... print exiting ... steve@runes:~$ -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterate from 2nd element of a huge list
On 01/02/12 01:39, Paulo da Silva wrote: Hi! What is the best way to iterate thru a huge list having the 1st element a different process? I.e.: process1(mylist[0]) for el in mylist[1:]: process2(el) This way mylist is almost duplicated, isn't it? Thanks. Maybe (untested), it = iter(mylist) process1(it.next()) for el in it: process2(el) Duncan -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
On Wed, Feb 1, 2012 at 12:12 PM, Ian Kelly ian.g.ke...@gmail.com wrote: Incidentally, I *think* that any correctly written C code attempting to nest a tuple inside itself would make the reference count of the tuple be at least 2 at the time of the call, and so it would fail. Good, nice that that's certain :) Might be worth moving the [b]ecause tuples are supposed to be immutable warning up to the top of the page then, since the bulk of it applies to all those functions and not just resize. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterate from 2nd element of a huge list
Em 01-02-2012 01:39, Paulo da Silva escreveu: Hi! What is the best way to iterate thru a huge list having the 1st element a different process? I.e.: process1(mylist[0]) for el in mylist[1:]: process2(el) This way mylist is almost duplicated, isn't it? Thanks. I think iter is nice for what I need. Thank you very much to all who responded. -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterate from 2nd element of a huge list
Em 01-02-2012 03:16, Paulo da Silva escreveu: Em 01-02-2012 01:39, Paulo da Silva escreveu: Hi! What is the best way to iterate thru a huge list having the 1st element a different process? I.e.: process1(mylist[0]) for el in mylist[1:]: process2(el) This way mylist is almost duplicated, isn't it? Thanks. I think iter is nice for what I need. Thank you very much to all who responded. BTW, iter seems faster than iterating thru mylist[1:]! -- http://mail.python.org/mailman/listinfo/python-list
Re: Disable use of pyc file with no matching py file
In article mailman.5264.1328054130.27778.python-l...@python.org, Terry Reedy tjre...@udel.edu wrote: But I am sure that 95% of readers here will be using 3.x withing 10 years. The only question for them is When?. This not-well-known new feature is one straw that some will put on the 'sooner' side of the balance. We would love to move to 3.x, for the better unicode support, if nothing else. What's keeping us from doing so is the host of third-party modules and tools we depend on that don't yet support 3.x. Off the top of my head (it's possible some of these already have support): django (I understand it's been done, if not yet officially supported) pymongo mongoengine tornado pybeanstalk dateutils requests lxml blinker gunicorn I'm sure I've missed a few It's getting to the point where any open-source python project needs to either jump on the 3.x bandwagon or get consigned to obscurity. I'm guessing that's going to start happening in a big way in 2012. -- http://mail.python.org/mailman/listinfo/python-list
ANN: PySAL 1.3
On behalf of the PySAL development team, I'm happy to announce the official release of PySAL 1.3. PySAL is a library of tools for spatial data analysis and geocomputation written in Python. PySAL 1.3, the fourth official release of PySAL, includes a number of new features and enhancements: - The spatial regression module (spreg) has added: - Two Stage Least Squares - Spatial Two Stage Least Squares - GM Error (KP 98-99) - GM Error Homoskedasticity (Drukker et. al, 2010) - GM Error Heteroskedasticity (Arraiz et. al, 2010) - Spatial HAC variance-covariance estimation - Anselin-Kelejian test for residual spatial autocorrelation of residuals from IV regression - New utility functions and other helper classes - A new contrib module to support user contributed modules. The first contrib modules are: - Weights Viewer – A Graphical tool for examining spatial weights - World To View Transform – A class for modeling viewing windows, used by Weights Viewer - Shapely Extension – Exposes shapely methods as standalone functions - Shared Perimeter Weights – calculate shared perimeters weights along with many bug fixes and smaller enhancements. PySAL modules - - pysal.core — Core Data Structures and IO - pysal.cg — Computational Geometry - pysal.esda — Exploratory Spatial Data Analysis - pysal.inequality — Spatial Inequality Analysis - pysal.spatial_dynamics — Spatial Dynamics - pysal.spreg - Regression and Diagnostics - pysal.region — Spatially Constrained Clustering - pysal.weights — Spatial Weights - pysal.FileIO — PySAL FileIO: Module for reading and writing various file types in a Pythonic way - pysal.contrib — Contributed Modules Downloads -- Binary installers and source distributions are available for download at http://code.google.com/p/pysal/downloads/list Documentation - The documentation site is here http://pysal.org/1.3/contents.html Web sites - PySAL's home is here http://pysal.org/ The developer's site is here http://code.google.com/p/pysal/ Mailing Lists - Please see the developer's list here http://groups.google.com/group/pysal-dev Help for users is here http://groups.google.com/group/openspace-list Bug reports --- To search for or report bugs, please see http://code.google.com/p/pysal/issues/list License information --- See the file LICENSE.txt for information on the history of this software, terms conditions for usage, and a DISCLAIMER OF ALL WARRANTIES. Many thanks to all who contributed! Serge, on behalf of the PySAL development team. -- Sergio (Serge) Rey Professor, School of Geographical Sciences and Urban Planning Arizona State University http://geoplan.asu.edu/rey Editor, International Regional Science Review http://irx.sagepub.com -- http://mail.python.org/mailman/listinfo/python-list
Re: except clause syntax question
Chris Angelico wrote: On Wed, Feb 1, 2012 at 9:03 AM, Duncan Booth duncan.booth@invalid.invalid wrote: Abitrarily nested tuples of exceptions cannot contain loops so the code simply needs to walk through the tuples until it finds a match. Is this absolutely guaranteed? The C API for CPython provides: (Py2) http://docs.python.org/c-api/tuple.html#PyTuple_SetItem (Py3) http://docs.python.org/dev/c-api/tuple.html#PyTuple_SetItem which doesn't have massive warnings on it saying USE THIS ONLY TO INITIALIZE A TUPLE (compare, for instance, _PyTuple_Resize which does carry a similar warning). Is the assumption then that we're all adults, and that mutating a tuple is like passing a null pointer to an API function (aka loaded gun in proximity to foot)? Unfortunately, I can't remember the details now, but I once set out to create a recursive tuple by using the C API, and it turned out then that the C API went to some lengths to prevent anyone being able to do that. I did finally do it in some peculiar way, but it wasn't simple. The c.l.python archives might still have the post where I described it. Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterate from 2nd element of a huge list
On 01Feb2012 03:34, Paulo da Silva p_s_d_a_s_i_l_...@netcabo.pt wrote: | Em 01-02-2012 03:16, Paulo da Silva escreveu: | I think iter is nice for what I need. | Thank you very much to all who responded. | | BTW, iter seems faster than iterating thru mylist[1:]! I would hope the difference can be attributed to the cost of copying mylist[1:]. Do your timings suggest this? (Remembering also that for most benchmarking you need to run things many times unless the effect is quite large). Cheers, -- Cameron Simpson c...@zip.com.au DoD#743 http://www.cskk.ezoshosting.com/cs/ Any company large enough to have a research lab is large enough not to listen to it. - Alan Kay -- http://mail.python.org/mailman/listinfo/python-list
xhtml encoding question
I have to follow a specification for producing xhtml files.
The original files are in cp1252 encoding and I must reencode them to utf-8.
Also, I have to replace certain characters with html entities.
I think I've got this right, but I'd like to hear if there's something
I'm doing that is dangerous or wrong.
Please see the appended code, and thanks for any comments or suggestions.
I have two functions, translate (replaces high characters with entities)
and reencode (um, reencodes):
-
import codecs, StringIO
from lxml import etree
high_chars = {
0x2014:'mdash;', # 'EM DASH',
0x2013:'ndash;', # 'EN DASH',
0x0160:'Scaron;',# 'LATIN CAPITAL LETTER S WITH CARON',
0x201d:'rdquo;', # 'RIGHT DOUBLE QUOTATION MARK',
0x201c:'ldquo;', # 'LEFT DOUBLE QUOTATION MARK',
0x2019:rsquo;, # 'RIGHT SINGLE QUOTATION MARK',
0x2018:lsquo;, # 'LEFT SINGLE QUOTATION MARK',
0x2122:'trade;', # 'TRADE MARK SIGN',
0x00A9:'copy;', # 'COPYRIGHT SYMBOL',
}
def translate(string):
s = ''
for c in string:
if ord(c) in high_chars:
c = high_chars.get(ord(c))
s += c
return s
def reencode(filename, in_encoding='cp1252',out_encoding='utf-8'):
with codecs.open(filename,encoding=in_encoding) as f:
s = f.read()
sio = StringIO.StringIO(translate(s))
parser = etree.HTMLParser(encoding=in_encoding)
tree = etree.parse(sio, parser)
result = etree.tostring(tree.getroot(), method='html',
pretty_print=True,
encoding=out_encoding)
with open(filename,'wb') as f:
f.write(result)
if __name__ == '__main__':
fname = 'mytest.htm'
reencode(fname)
--
http://mail.python.org/mailman/listinfo/python-list
Installing pypi package twice
My system's default python is 2.6.5. I have separately installed 3.2.2 at /opt/python. I downloaded python-daemon-1.5.5 and installed with: $ tar xzf python-daemon-1.5.5 $ cd python-daemon-1.5.5 $ python setup.py build $ sudo python setup.py install How would I also install this package for 3.2.2? (I am assuming that python-daemon-1.5.5 is either version3-compatible or I can make it so). -- http://mail.python.org/mailman/listinfo/python-list
Installing pypi package twice
My system's default python is 2.6.5. I have also installed python3.2 at /opt/python. I installed a pypi package for 2.6.5 with: $ tar xzf package.tar.gz $ cd package $ python setup.py build $ sudo python setup.py install How can I also install this same package for 3.2? (I am assuming this package works with 3.2 or that I can make it work.) -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Descriptor as Instance Attribute
Thanks Ian for the explanation. Please see my comments below: The behavior is by design. First, keeping object behavior in the class definition simplifies the implementation and also makes instance checks more meaningful. To borrow your Register example, if the M descriptor is defined by some instances rather than by the class, then knowing that the object reg is an instance of Register does not tell me anything about whether reg.M is a valid attribute or an error. As a result, I'll need to guard virtually every access of reg.M with a try-except construct just in case reg is the wrong kind of register. I don't quite understand the above explanation. Sorry I'm not very familiar with the low level details, but from a user's point of view, if I defined reg.M, then it should be a valid access later on somehow. :-) Second, the separation of class from instance also helps you keep object behavior separate from object data. Consider the following class: class ObjectHolder(object): def __init__(self, obj): self.obj = obj Don't worry about what this class might be useful for. Just know that it's meant to hold and provide unrestricted access to arbitrary Python objects: holder = ObjectHolder(42) print(holder.obj) 42 holder.obj = range(5) print(holder.obj) [0, 1, 2, 3, 4] Since the class is meant to hold arbitrary objects, it's even valid that somebody might want to store a descriptor object there: holder.obj = property(lambda x: x.foo) print(holder.obj) property object at 0x02415AE0 Now suppose that Python invoked the descriptor protocol for descriptors stored in instance attributes: holder = ObjectHolder(None) holder.obj = property(lambda x: x.foo) print(holder.obj) Traceback (most recent call last): File stdin, line 1, in module AttributeError: 'ObjectHolder' object has no attribute 'foo' In this case, the ObjectHolder would fail to simply hold the property object as data. The mere act of assigning the property object, a descriptor, to an instance attribute would *change the behavior* of the ObjectHolder. Instead of treating holder.obj as a simple data attribute, it would start invoking the descriptor protocol on accesses to holder.obj and ultimately redirect them to the non-existent and meaningless holder.foo attribute, which is certainly not what the author of the class intended. OK I see some fundamental problems here now. And I think that's actually one of the limitations of descriptor: A descriptor only works when it is defined as class attribute and accessed from the instance. It really can be much more powerful if there can be a general way to define an attribute on either a class or an instance, but the access to it (either directly from class or from its instance) actually calls a function. It will make some kind of abstraction much more clean and simple in concept, like my example above, I have one class called register, and all of its instance represent different registers with different field, and assignment to its field automatically checks for validations, and read automatically fetches the value from the hardware. For the above reasons, I would probably implement your Register class as a set of related class sharing a common metaclass. The solution you came up with is probably fine to solve your specific problem, though. this like I said before is not conceptually simple enough, and it can confuses end user if they're not python expert. For years I loved python is because I can always figure out a best way to abstract a problem, and make end-user interface as simple as possible, I never failed before with python, but this time it seems python indeed have limitations here, or does there exist a better solution? To make you understand the problem I'm facing, I'd like to elaborate a bit more here. Registers in SoC peripherals have different field, and each field have different number of bits, different access capabilities (read only, write only, read write, ...), but all registers share some common attribute, like they're all 32 bits long. Also some common operations is shared, like distribute a value to each bit field, meaning that set the value of a register as a whole will automatically update each field. The definition of each register is in an XML file with all attribute for each field. And the preferred way to generate an representation of a register is to instantiate the Register class with its definition read from the xml file. This is the natural design, all register representation is an instance of Register class. So now the problem is, how can I have such a simple class with all its instance have different fields, which can be written and read directly (like reg.M = '101', or x = reg.M) with automated validation check and value fetch? Define a separate class for each register doesn't sounds feasible because there's hundreds of registers. Using metaclass to generate a class for each register also doesn't
Re: Iterate from 2nd element of a huge list
On 1 February 2012 03:16, Paulo da Silva p_s_d_a_s_i_l_...@netcabo.pt wrote: Em 01-02-2012 01:39, Paulo da Silva escreveu: Hi! What is the best way to iterate thru a huge list having the 1st element a different process? I.e.: process1(mylist[0]) for el in mylist[1:]: process2(el) This way mylist is almost duplicated, isn't it? Thanks. I think iter is nice for what I need. Thank you very much to all who responded. Nobody mentioned itertools.islice, which can be handy, especially if you weren't interested in the first element of the list: from itertools import islice: for el in islice(mylist, 1): process2(el) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: 'class' named tuple
On 1 February 2012 00:54, Emmanuel Mayssat emays...@gmail.com wrote: I have the following program. I am trying to have index the attributes of an object using __getitem__. Reading them this way works great, but assigning them a value doesn't Is there a way to do such a thing? (Almost like a named tuple, but with custom methods) class LIter(object): def __init__(self,parent=None): super(LIter, self).__init__() self.toto = 3 self.tata = 'terto' Add _attrs = 'toto', 'tata' def __getitem__(self, index): return getattr(self, _attrs[index]) def __setitem__(self, index, value) setattr(self, _attrs[index], value) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
[issue13506] IDLE sys.path does not contain Current Working Directory
Terry J. Reedy tjre...@udel.edu added the comment: I noticed that the script in ScriptBinding.py already dereferences interp and calls 3 interp functions directly, so I decided that calling a 4th on interp was fine. That lets restart_shell be specialized to the one purpose of being bound to Restart Shell on the menu. I added a docstring to that effect so no will will try to generalize it again to use when running a module. -- resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13506 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13903] New shared-keys dictionary implementation
Mark Shannon m...@hotpy.org added the comment: Raymond Hettinger wrote: Raymond Hettinger raymond.hettin...@gmail.com added the comment: Changing dictionaries is a big deal. You're changing many pieces at once (not a good idea) including changing tunable parameters that are well-studied (I spent a month testing whether 5/8 was better idea that 2/3 for resizing or when the ideal small dict size was 4, 8, or 16). You're changing the meaning of the fields in dictobject.h which will likely break any code that relied on those. The ideas may be good ones but they warrant a good deal of thought. Dicts weren't just slapped together -- the current code is the product to two decades of tweaking by engineers who devoted significant time to the task. It would be easy to unknowingly undo some of their work. OK. I'll write a PEP. By the way, I'm trying not changing the tunable parameters for the unshared-keys case, just the shared-keys case. Of course, they do interact with each other. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13903 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13912] ImportError using __import__ and relative level 1
Nick Coghlan ncogh...@gmail.com added the comment: It sounds like you may want runpy.run_module [1], rather than using imports at all. If you know how many levels up you want to go, it isn't hard to do your own munging of __name__ to create absolute module references to pass to runpy. The signature of __import__ is known to be unintuitive to the point of being insane - it's really designed for the convenience of the compiler and the interpreter, not for direct use by humans. [1] http://docs.python.org/release/2.6.7/library/runpy#runpy.run_module -- nosy: +ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13912 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13882] Add format argument for time.time(), time.clock(), ... to get a timestamp as a Decimal object
STINNER Victor victor.stin...@haypocalc.com added the comment: Patch version 6: - timestamp format is now a type instead of a string, e.g. time.time(int) - add int and datetime.timedelta formats, remove timespec format - complete the documentation - fix integer overflows, convert correctly time_t to PyLong There are now 5 timestamp formats: - int - float - decimal.Decimal - datetime.datetime - datetime.timedelta I consider the patch as ready to be commited, or at least ready for a review ;-) There is no more FIXME or known limitation. Well, now the most important part is to decide the API and the list of timestamp formats. The patch should be tested on Linux, FreeBSD and Windows, 32 and 64 bits to check assertions on type sizes: assert(sizeof(clock_t) = sizeof(size_t)); assert(sizeof(LONGLONG) = sizeof(size_t)); assert(sizeof(time_t) = sizeof(PY_LONG_LONG)); -- Added file: http://bugs.python.org/file24378/time_decimal-6.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13882 ___diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -808,13 +808,16 @@ as internal buffering of data. Availability: Unix. -.. function:: fstat(fd) +.. function:: fstat(fd, timestamp=None) Return status for file descriptor *fd*, like :func:`~os.stat`. Availability: Unix, Windows. -.. function:: fstatat(dirfd, path, flags=0) + .. versionchanged:: 3.3 + Added the *timestamp* argument. + +.. function:: fstatat(dirfd, path, flags=0, timestamp=float) Like :func:`stat` but if *path* is relative, it is taken as relative to *dirfd*. *flags* is optional and may be 0 or :data:`AT_SYMLINK_NOFOLLOW`. @@ -1696,7 +1699,7 @@ Files and Directories .. versionadded:: 3.3 -.. function:: lstat(path) +.. function:: lstat(path, timestamp=None) Perform the equivalent of an :c:func:`lstat` system call on the given path. Similar to :func:`~os.stat`, but does not follow symbolic links. On @@ -1706,6 +1709,9 @@ Files and Directories .. versionchanged:: 3.2 Added support for Windows 6.0 (Vista) symbolic links. + .. versionchanged:: 3.3 + The *timestamp* argument was added. + .. function:: lutimes(path[, times]) @@ -1955,7 +1961,7 @@ Files and Directories .. versionadded:: 3.3 -.. function:: stat(path) +.. function:: stat(path, timestamp=None) Perform the equivalent of a :c:func:`stat` system call on the given path. (This function follows symlinks; to stat a symlink use :func:`lstat`.) @@ -1975,6 +1981,11 @@ Files and Directories * :attr:`st_ctime` - platform dependent; time of most recent metadata change on Unix, or the time of creation on Windows) + :attr:`st_atime`, :attr:`st_mtime` and :attr:`st_ctime` values type is + :class:`float` by default, or :class:`int` if :func:`os.stat_float_times` is + ``False``. Set the optional *timestamp* argument to get another + :ref:`timestamp format timestamp-formats`. + On some Unix systems (such as Linux), the following attributes may also be available: @@ -2030,6 +2041,9 @@ Files and Directories Availability: Unix, Windows. + .. versionchanged:: 3.3 + Added the *timestamp* argument. + .. function:: stat_float_times([newvalue]) @@ -2055,6 +2069,9 @@ Files and Directories are processed, this application should turn the feature off until the library has been corrected. + .. deprecated:: 3.3 + Use *timestamp* argument of stat functions instead. + .. function:: statvfs(path) diff --git a/Doc/library/time.rst b/Doc/library/time.rst --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -95,6 +95,16 @@ An explanation of some terminology and c | local time | | | +-+-+-+ +.. _timestamp-formats: + +* Python supports the following timestamp formats: + + * :class:`int` + * :class:`float` + * :class:`datetime.datetime` + * :class:`datetime.timedelta` + * :class:`decimal.Decimal` + The module defines the following functions and data items: @@ -118,7 +128,7 @@ The module defines the following functio Unlike the C function of the same name, there is no trailing newline. -.. function:: clock() +.. function:: clock(format=float) .. index:: single: CPU time @@ -135,16 +145,27 @@ The module defines the following functio :c:func:`QueryPerformanceCounter`. The resolution is typically better than one microsecond. + Return the time as a floating point number by default, set the optional + *format* argument to get another :ref:`timestamp format timestamp-formats`. -.. function:: clock_getres(clk_id) + .. versionchanged:: 3.3 + Added the *format* argument. + + +.. function:: clock_getres(clk_id,
[issue13734] Add a generic directory walker method to avoid symlink attacks
Nick Coghlan ncogh...@gmail.com added the comment: Hmm, given the various *at() APIs that sort alphabetically next to their path based counterparts, perhaps we can make the naming consistency change go the other way? (i.e. listdirfd() and walkfd()). Even if POSIX puts the fd at the front, do we really have to follow them in a silly naming scheme? And as per the python-dev discussion, +1 for being consistent with os.walk() when it comes to symlinks to directories. Aside from the naming question, is there anything else holding this up? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13734 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13734] Add a generic directory walker method to avoid symlink attacks
Nick Coghlan ncogh...@gmail.com added the comment: Taking a closer look at the current naming scheme in the os module, fdopen() appears to be the only current function that uses the 'fd' prefix. All the other operations that accept a file descriptor just use 'f' as the prefix (fchmod, fchown, faccess, etc). Given that, flistdir() and fwalk() seem like the most consistent choices of name for APIs that aren't directly matching an underlying POSIX function name. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13734 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13734] Add a generic directory walker method to avoid symlink attacks
Antoine Pitrou pit...@free.fr added the comment: There's something I don't understand in the patch: why does _are_same_file examine st_mode? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13734 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13229] Improve tools for iterating over filesystem directories
Nick Coghlan ncogh...@gmail.com added the comment: With the release of 0.3, I'm pretty happy with the WalkDir design (previous versions coerced the output values to ordinary 3-tuples, now it will pass along whatever the underlying iterable produces with changing the type at all). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13229 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13903] New shared-keys dictionary implementation
Antoine Pitrou pit...@free.fr added the comment: As Victor I think the tunables could be changed separately, unless they truely get in the way of the shared-keys optimization. By the way, there will need a protection for the case where instances are used as bags of key/value pairs (dict-alikes with attribute access). Perhaps bound the size of the keys array to 100 entries and then switch into unshared mode. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13903 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13229] Improve tools for iterating over filesystem directories
Antoine Pitrou pit...@free.fr added the comment: (speaking of which, I'd just mention I've published pathlib as a standalone package: http://pypi.python.org/pypi/pathlib/ ) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13229 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13868] Add hyphen doc fix
Boštjan Mejak bostjan.me...@gmail.com added the comment: What's the purpose of this tracker if the author of an issue report has to beg for something to be fixed, despite the fact that (s)he even made a patch ? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13868 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10580] Minor grammar change in Python’s MSI installer
Boštjan Mejak bostjan.me...@gmail.com added the comment: Rather apply the patch, not change the title. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10580 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13899] re pattern r[\A] should work like A but matches nothing. Ditto B and Z.
Ezio Melotti ezio.melo...@gmail.com added the comment: The rule 1 makes sense, but it's not entirely obvious (people might consider bBaAzZ special too). The normal Python rules for backslash escapes but revert to the C behaviour of stripping the \ from unrecognised escapes is not obvious either, and from r'[\A]' people might expect: 1) same as \A, (beginning of the string); 2) a letter 'A'; 3) a '\' or a letter 'A' (especially if they write it as '[\\A]'); This is why I suggested to raise an error (and refuse the temptation to guess), but on the other hand, if you consider 'A' a normal letter like 'C', having an error for \A would be incoherent. It would have been better if \C raised an error too (I don't see why that would appear in a regex, since re.escape doesn't escape C and the user has no reason to add the \), but now it's too late for that. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13899 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13734] Add a generic directory walker method to avoid symlink attacks
Charles-François Natali neolo...@free.fr added the comment: Given that, flistdir() and fwalk() seem like the most consistent choices of name for APIs that aren't directly matching an underlying POSIX function name. Well, that seems OK for me. I guess the only reason fdlistdir() is named that way is because of fdopendir(3). I can make the change for fwalk(), and since 3.3 hasn't been released yet, I guess we can rename fdlistdir() too. There's something I don't understand in the patch: why does _are_same_file examine st_mode? It doesn't have to, that's actually useless. The only thing that bothers me is that it needs O(height of directory tree), so with really deep directory trees, we could run out of FDs. Not sure that could be a problem in practice, but that's something to keep in mind. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13734 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13734] Add a generic directory walker method to avoid symlink attacks
Antoine Pitrou pit...@free.fr added the comment: One other thing: is it deliberate to silence errors in the following snippet? +try: +names = fdlistdir(topfd) +except error as err: +if onerror is not None: +onerror(err) +return -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13734 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13903] New shared-keys dictionary implementation
Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13903 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13734] Add a generic directory walker method to avoid symlink attacks
Charles-François Natali neolo...@free.fr added the comment: It's to mimic os.walk()'s behavior: http://hg.python.org/cpython/file/bf31815548c9/Lib/os.py#l268 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13734 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13609] Add os.get_terminal_size() function
anatoly techtonik techto...@gmail.com added the comment: Terminal stuff is irrelevant to `shutil`, which is the module for 'High-level file operations' and deserve a separate module named 'console'. Why? Because terminal size is only relevant when you want to page output. The next step would be to detect if terminal supports color to see if ANSI sequences will be collapsed. Then you'll want to get user input without pressing 'enter' key. It is a whole new module. -- nosy: +techtonik ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13609 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13609] Add os.get_terminal_size() function
Antoine Pitrou pit...@free.fr added the comment: except Exception clauses in the tests are too broad. Otherwise, looks fine. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13609 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13609] Add os.get_terminal_size() function
STINNER Victor victor.stin...@haypocalc.com added the comment: termsize.diff.7: - shutil.get_terminal_size(): I would prefer columns and lines variable names instead of ccc and rrr - Right now the values are not cached, but this might change. why would it be cached? this sentence can just be removed - I would prefer os.get_terminal_size() instead of os.query_terminal_size(), I didn't know other function using the verb query in Python - To keep os simple, os.query_terminal_size() can return a simple, whereas shutil.get_terminal_size() returns a namedtuple - test_os.py: use @unittest.skipUnless() on TermsizeTests to check if os.query_terminal_size() is available - test.py, test_does_not_crash() catchs OSError, you may only ignore some error codes, not any. For example, skip the test if stdout is not a tty +try: +size = os.query_terminal_size(sys.__stdout__.fileno()) +except (NameError, OSError): +size = os.terminal_size(fallback) AttributeError should be catched here, not NameError. Or better, check if os has a query_terminal_size() function. +.. function:: get_terminal_size(fallback=(columns, lines)) Hum, you may document the default value: (80, 24). shutil.get_terminal_size() doesn't allow to choose the file descriptor. Would it be useful to allow to get the size of sys.stderr or another TTY? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13609 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13912] ImportError using __import__ and relative level 1
Brett Cannon br...@python.org added the comment: On Mon, Jan 30, 2012 at 18:33, Eric Snow rep...@bugs.python.org wrote: Eric Snow ericsnowcurren...@gmail.com added the comment: Jason: just a warning. importlib_backport is a relatively naive tool for generating the backport from the py3k source. It's also relatively fragile and at this point probably doesn't work with the default branch. I think Jason was thinking of http://pypi.python.org/pypi/importlib/ which is my personal backport of importlib from Python 2.7 all the way back to Python 2.3. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13912 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13496] bisect module: Overflow at index computation
Jesús Cea Avión j...@jcea.es added the comment: I think the patch is *NOT* correct. Does it work with http://bugs.python.org/msg148567, in a 32 bits python? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13496 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1625] bz2.BZ2File doesn't support multiple streams
Jesús Cea Avión j...@jcea.es added the comment: Éric, bz2 module in Python is documented as able to manage bz2 files. But that is not true, since it is unable to manage popular bz2 files. That looks like a bug to me. In fact, you can create those files from python, files that python can not unpack later. Moreover, Python 2.7 is going to live for a long time. If the refusal of backporting this to 3.2 and 2.7 is firm, I would beg to document this limitation in the 2.7/3.2 docs. It is serious enough. Please, reconsider backporting this to 2.7, at least. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1625 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13609] Add os.get_terminal_size() function
Zbyszek Szmek zbys...@in.waw.pl added the comment: Well, right now it's just one function. Functionality which you propose could of course be useful, but let's leave if for another day, another issue. See also http://bugs.python.org/issue13609#msg149627 and #444582 and #12442 -- shutil is growing in different directions, and terminal size can be one of them. Antoine Pitrou added the comment: except Exception clauses in the tests are too broad. Fixed. Otherwise, looks fine. STINNER Victor added the comment: - shutil.get_terminal_size(): I would prefer columns and lines variable names instead of ccc and rrr Changed (this was a leftover from the version when there was no named tuple). - I would prefer os.get_terminal_size() instead of os.query_terminal_size(), I didn't know other function using the verb query in Python Changed. I used os.g_t_s and shutil.g_t_s in docs so it should be clear which is which. - To keep os simple, os.query_terminal_size() can return a simple, whereas shutil.get_terminal_size() returns a namedtuple We would have two very similar functions returning something different. Also, the amount of code saved will be negligible, because the named tuple is mostly docs. I prefer to keep it in os to keep both functions similar. - test_os.py: use @unittest.skipUnless() on TermsizeTests to check if os.query_terminal_size() is available - To keep os simple, os.query_terminal_size() can return a simple, whereas shutil.get_terminal_size() returns a namedtuple AttributeError should be catched here, not NameError. Or better, check if os has a query_terminal_size() function. Fixed. I changed the tests a bit, e.g. to take into account that stty queries stdin, not stdout. Hum, you may document the default value: (80, 24). Done. shutil.get_terminal_size() is tied to COLUMNS and ROWS and thus makes most sense for stdout. But I guess that an optional parameter could be added: shutil.get_terminal_size(fd=None, fallback=(80, 24)) where fd could be either an integer, or an object with .fileno(). I don't know. - Right now the values are not cached, but this might change. why would it be cached? this sentence can just be removed Done. In theory it could be cached with watching for SIGWINCH, but I'll just remove the comment for now. Thanks for the review and comments! Patch version nine attached: termsize.diff.8 -- Added file: http://bugs.python.org/file24379/termsize.diff.8 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13609 ___diff --git a/Doc/library/os.rst b/Doc/library/os.rst --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1408,6 +1408,44 @@ .. versionadded:: 3.3 +.. _terminal-size: + +Querying the size of the output terminal + + +.. versionadded:: 3.3 + +.. function:: get_terminal_size(fd=STDOUT_FILENO) + + Return the size of the terminal window as ``(columns, lines)``, + tuple of type :class:`terminal_size`. + + The optional argument ``fd`` (default ``STDOUT_FILENO``, or standard + output) specifies which file descriptor should be queried. + + If the file descriptor is not connected to a terminal, an :exc:`OSError` + is thrown. + + This function is only present if an implementation is available for + this system (currently POSIX and Windows). + + :func:`shutil.get_terminal_size` is the high-level function which + should normally be used, ``os.get_terminal_size`` is the low-level + implementation. + +.. class:: terminal_size(tuple) + + A tuple of ``(columns, lines)`` for holding terminal window size. + + .. attribute:: columns + + Width of the terminal window in characters. + + .. attribute:: lines + + Height of the terminal window in characters. + + .. _os-file-dir: Files and Directories diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -450,3 +450,33 @@ -rw-r--r-- tarek/staff 37192 2010-02-06 18:23:10 ./known_hosts +Querying the size of the output terminal + + +.. versionadded:: 3.3 + +.. function:: get_terminal_size(fallback=(columns, lines)) + + Get the size of the terminal window. + + For each of the two dimensions, the environment variable, ``COLUMNS`` + and ``LINES`` respectively, is checked. If the variable is defined and + the value is a positive integer, it is used. + + When ``COLUMNS`` or ``LINES`` is not defined, which is the common case, + the terminal connected to :data:`sys.__stdout__` is queried + by invoking :func:`os.get_terminal_size`. + + If the terminal size cannot be successfully queried, either because + the system doesn't support querying, or because we are not + connected to a terminal, the value given in ``fallback`` parameter + is used. ``fallback`` defaults to ``(80, 24)``
[issue13897] Move fields relevant to coroutine/generators out of frame into generator/threadstate
Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13897 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13899] re pattern r[\A] should work like A but matches nothing. Ditto B and Z.
Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13899 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7856] cannot decode from or encode to big5 \xf9\xd8
Changes by Kang-Hao (Kenny) Lu kennyl...@csail.mit.edu: -- nosy: +kennyluck ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7856 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13609] Add os.get_terminal_size() function
anatoly techtonik techto...@gmail.com added the comment: What happens with COLUMNS env variables if terminal is resized after the Python script is executed. Will get_terminal_size() return new value? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13609 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13609] Add os.get_terminal_size() function
Zbyszek Szmek zbys...@in.waw.pl added the comment: the Python script is executed. Will get_terminal_size() return new value? Please see previous discussion and the docs (and the SUSv2 specs). The env. var. is for overriding. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13609 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13609] Add os.get_terminal_size() function
anatoly techtonik techto...@gmail.com added the comment: On Tue, Jan 31, 2012 at 7:48 PM, Zbyszek Szmek rep...@bugs.python.orgwrote: Zbyszek Szmek zbys...@in.waw.pl added the comment: the Python script is executed. Will get_terminal_size() return new value? Please see previous discussion and the docs (and the SUSv2 specs). The env. var. is for overriding. Could you just answer the question or provide a relevant link (unfortunately I don't have time to read SUSv2 specs)? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13609 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
