Announcing www.python-excel.org
Hi All, Google unfortunately has a knack of presenting prospective Python users who need to work with Excel files with information that is now really rather out of date. To try and help with this, I've setup a small website at: http://www.python-excel.org ...to try and list the latest recommended ways of working with Excel files in Python. If you work with excel files in Python, then please take a look and let me know what you think. cheers, Chris PS: If anyone reading this has a python-related blog, it might help Google if you could post a short entry about this new site. -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
Tutorial on working with Excel files in Python (without COM and cross platform!) at EuroPython 2009
Hi All, Too many people in the Python community *still* think the only way to work with Excel files in Python is using COM on Windows. To try and correct this, I'm giving a tutorial at this year's EuroPython conference in Birmingham, UK on Monday, 29th June that will cover working with Excel files in Python using the pure-python libraries xlrd, xlwt and xlutils. I'll be looking to cover: - Reading Excel Files Including extracting all the data types, formatting and working with large files. - Writing Excel Files Including formatting, many of the useful frilly extras and writing large excel files. - Modifying and Filtering Excel Files A run through of taking existing Excel files and modifying them in various ways. - Workshop for your problems I'm hoping anyone who attends will get a lot out of this! If you're planning on attending and have a particular problem you'd like to work on in this part of the tutorial, please drop me an email and I'll try and make sure I come prepared! All you need for the tutorial is a working knowledge of Excel and Python, with a laptop as an added benefit, and to be at EuroPython this year: http://www.europython.eu/ I look forward to seeing you all there! Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
xlutils 1.3.2 released!
Hi All, I'm pleased to announce a new release of xlutils. This is a small collection of utilities that make use of both xlrd and xlwt to process Microsoft Excel files. The list of utilities included in this release are: xlutils.copy Tools for copying xlrd.Book objects to xlwt.Workbook objects. xlutils.display Utility functions for displaying information about xlrd-related objects in a user-friendly and safe fashion. xlutils.filter A mini framework for splitting and filtering Excel files into new Excel files. xlutils.margins Tools for finding how much of an Excel file contains useful data. xlutils.save Tools for serializing xlrd.Book objects back to Excel files. xlutils.styles Tools for working with formatting information expressed in styles. A full list of changes since the last release can be found here: http://www.simplistix.co.uk/software/python/xlutils/changes To find out more, please read here: http://www.simplistix.co.uk/software/python/xlutils In case you're not aware, xlrd and xlwt are two excellent pure-python libraries for reading and writing Excel files. They run on any platform and, likely, any implementation of Python without the need for horrific things like binding to Excel via COM and so needing a Windows machine. If you use any of xlrd, xlwt or xlutils, the following google group will be of use: http://groups.google.com.au/group/python-excel Hope some of this is of interest, I'd love to hear from anyone who ends up using it! cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
TestFixtures 1.6.0 released!
Hi All, I'm pleased to announce the first advertised release of TestFixtures. This is a collection of helpers and mock objects that are useful when writing unit tests or doc tests. The modules currently included are: *Comparison* This class lets you instantiate placeholders that can be used to compared expected results with actual results where objects in the actual results do not support useful comparison. The comparision can be based just on the type of the object or on a partial set of the object's attributes, both of which are particularly handy when comparing sequences returned from tested code. *compare* A replacement for assertEquals and the failUnless(x() is True) pattern. Gives more useful differences when the arguments aren't the same, particularly for sequences and long strings. *diff* This function will compare two strings and give a unified diff of their comparison. Handy as a third parameter to unittest.TestCase.assertEquals. *generator* This function will return a generator that yields the arguments it was called with when the generator is iterated over. *LogCapture* This helper allows you to capture log messages for specified loggers in doctests. *log_capture* This decorator allows you to capture log messages for specified loggers for the duration of unittest methods. *replace* This decorator enables you to replace objects such as classes and functions for the duration of a unittest method. The replacements are removed regardless of what happens during the test. *Replacer* This helper enables you to replace objects such as classes and functions from within doctests and then restore the originals once testing is completed. *should_raise* This is a better version of assertRaises that lets you check the exception raised is not only of the correct type but also has the correct parameters. *TempDirectory* This helper will create a temporary directory for you using mkdtemp and provides a handy class method for cleaning all of these up. *tempdir* This decorator will create a temporary directory for the duration of the unit test and clear it up no matter the outcome of the test. *test_date* This is a handy class factory that returns datetime.date replacements that have a today method that gives repeatable, specifiable, testable dates. *test_datetime* This is a handy class factory that returns datetime.datetime replacements that have a now method that gives repeatable, specifiable, testable datetimes. *test_time* This is a handy replacement for time.time that gives repeatable, specifiable, testable times. *wrap* This is a generic decorator for wrapping method and function calls with a try-finally and having code executed before the try and as part of the finally. To find out more, please read here: http://pypi.python.org/pypi/testfixtures cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
PSF Community Awards go to Stephan Deibel Sean Reifschneider
[Also posted to the PSF blog: http://pyfound.blogspot.com/2009/06/second-quarter-community-service-awards.html] The Foundation tries to recognize those whose assistance has been significant in its growth and development as well as its day-to-day operations. This quarter's Community Service Award winners are two particularly noteworthy examples. Stephan Deibel: Stephan was last year's outgoing chairman after four years in harness. This year Stephan has stepped down as a director, after helping to ensure that the Foundation's bylaws were reorganized. Stephan developed pythonology.com to promote Python, and his work as founder of Wingware (wingware.com) and a developer of the Wing IDE has also had a significant impact. Sean Reifschneider Sean has master-minded the PyCon networking every time it's worked, and without the support of this always helpful and reliably competent tummy.com director our conferences simply would not have been the same. Our thanks and congratulations go to both these recipients. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
Python Essential Reference, 4th Edition
* * Python Essential Reference, 4th Edition * * by David Beazley I'm pleased to announce the release of the Python Essential Reference, 4th edition, soon to be appearing at a bookstore near you. More than a year in development, this edition covers Python 2.6, Python 3, and a wide variety of new library modules. Notable new features include - Greatly expanded coverage of advanced features such as generators, coroutines, context managers, decorators, metaclasses, and more. - Detailed coverage of concurrent programming including threads, the multiprocessing library, and coroutines. - A new chapter on testing, debugging, profiling, and performance optimization. - Completely revised examples and code samples appearing throughout the text. - New coverage of the database API, WSGI, ctypes, and other essential library modules. - Detailed information on Python 3 including compatibility notes, migration strategies, notable new features, and use of the 2to3 tool. If you liked previous editions of the Essential Reference, I think you will be pleased--this may be the best edition yet. Cheers, Dave -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
MailingLogger 3.3.0 Released!
I'm pleased to announce a new release of Mailinglogger.
Mailinglogger provides two handlers for the standard python
logging framework that enable log entries to be emailed either as the
entries are logged or as a summary at the end of the running process.
The handlers have the following features:
- customisable and dynamic subject lines for emails sent
- emails sent with an X-Mailer header for easy filtering
- flood protection to ensure the number of emails sent is not excessive
- support for SMTP servers that require authentication
- fully documented and tested
In addition, extra support is provided for configuring the handlers when
using ZConfig, Zope 2 or Zope 3.
The latest releases of ZConfig, in particular, provide a great way to
configure the python logging framework without having to resort to the
appalling .ini-based configuration stuff:
from ZConfig import configureLoggers
configureLoggers('''
... logger
...level INFO
...logfile
... PATH STDOUT
... format %(levelname)s %(name)s %(message)s
.../logfile
... /logger
... ''')
This release of MailingLogger adds the ability to use %(levelname)s in
subject-line formatting for SummarisingLoggers to include the highest
level logged.
For more information, please see:
http://www.simplistix.co.uk/software/python/mailinglogger
or
http://pypi.python.org/pypi/mailinglogger
cheers,
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations.html
ErrorHandler 1.0.0 Released!
I'm pleased to finally get around to announcing the release of ErrorHandler.
This is a handler for the python standard logging framework that can
be used to tell whether messages have been logged at or above a
certain level.
This can be useful when wanting to ensure that no errors have been
logged before committing data back to a database.
Here's an example:
First, you set up the error handler:
from logging import getLogger
from errorhandler import ErrorHandler
logger = getLogger()
e = ErrorHandler()
The handler started off being un-fired:
e.fired
False
Then you do whatever else you need to do, which may involve logging:
logger.info('some information')
e.fired
False
However, if any logging occurs at an error level or above:
logger.error('an error')
Then the error handler becomes fired:
e.fired
True
You use this as a condition to only do certain actions when no errors
have been logged:
if e.fired:
... print Not updating files as errors have occurred
Not updating files as errors have occurred
If your code does work in batches, you may wish to reset the error
handler at the start of each batch:
e.reset()
e.fired
False
For more information, please see:
http://www.simplistix.co.uk/software/python/errorhandler
or
http://pypi.python.org/pypi/errorhandler
cheers,
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations.html
Re: Is this pylint error message valid or silly?
Matthew Wilson wrote: Here's the code that I'm feeding to pylint: $ cat f.py from datetime import datetime def f(c=today): pylint infers that you intend users to pass a string. Human would guess the same at this point. if c == today: c = datetime.today() Now I guess that you actually intend c to be passed as a datetime object. You only used the string as a type annotation, not as a real default value. Something like 'record_date = None' is better. return c.date() and here you ask for the input's date, which strings do not have. And here's what pylint says: $ pylint -e f.py No config file found, using default configuration * Module f E: 10:f: Instance of 'str' has no 'date' member (but some types could not be inferred) Is this a valid error message? Is the code above bad? If so, what is the right way? I changed from using a string as the default to None, and then pylint didn't mind: $ cat f.py from datetime import datetime def f(c=None): if c is None: c = datetime.today() return c.date() $ pylint -e f.py No config file found, using default configuration I don't see any difference between using a string vs None. Both are immutable. I find the string much more informative, since I can write out what I want. Looking for comments. Matt -- http://mail.python.org/mailman/listinfo/python-list
Integer Division
a = 1 b = 25 a / b 0 float(a) / b 0.040001 from __future__ import division a = 1 b = 25 a / b 0.040001 In what simple way can I get just 0.04 ? -- Anjanesh Lekshmnarayanan -- http://mail.python.org/mailman/listinfo/python-list
Re: Integer Division
On Fri, Jun 19, 2009 at 12:22 AM, Anjanesh
Lekshminarayananm...@anjanesh.net wrote:
a = 1
b = 25
a / b
0
float(a) / b
0.040001
from __future__ import division
a = 1
b = 25
a / b
0.040001
In what simple way can I get just 0.04 ?
Note that what you are shown is the repr() of the float rather than
the str() of the float. The repr() value is what the number truly is
in binary, but str() applies more sensible rounding.
Example (remember that print() does an implicit str()):
from __future__ import division
print(1/25)
0.04
repr(1/25)
0.040001
Alternatively, you can use the decimal arithmetic library:
from decimal import Decimal
Decimal(1)/Decimal(25)
Decimal('0.04')
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newbie queue question
I've done some further testing on the subject: I also added some calculations in the main loop to see what effect they would have on speed. Of course, I also added the same calculations to the single threaded functions. They were simple summary functions, like average, sum, etc. Almost no interaction with the buffers was added, just retrieval of a single field's value. Single threaded, the calculations added another 4.3 seconds to the processing time (~18%) MultiThreaded, they added 1.8 seconds. CPU usage remained below 100% of one core at all times. Made me check the process affinity. I know the main thread uses way less CPU than DBF reading thread (4 secs vs 22 secs). So I think adding these calculations should have but a minimal impact on threaded execution time. Instead, the execution time increases!!! I'm beginning to think that Python's memory management / functions introduce quite a significant overhead for threading. I think I'll just write this program in one of the compilers today to verify just how stupid I've become. -- http://mail.python.org/mailman/listinfo/python-list
Re: walking a directory with very many files
On Fri, 19 Jun 2009 17:53:40 +1200 Lawrence D'Oliveiro l...@geek-central.gen.new_zealand wrote: In message 20090618081423.2e035...@coercion, Mike Kazantsev wrote: On Thu, 18 Jun 2009 10:33:49 +1200 Lawrence D'Oliveiro l...@geek-central.gen.new_zealand wrote: In message 20090617214535.10866...@coercion, Mike Kazantsev wrote: On Wed, 17 Jun 2009 23:04:37 +1200 Lawrence D'Oliveiro l...@geek-central.gen.new_zealand wrote: In message 20090617142431.2b25f...@malediction, Mike Kazantsev wrote: On Wed, 17 Jun 2009 17:53:33 +1200 Lawrence D'Oliveiro l...@geek-central.gen.new_zealand wrote: Why not use hex representation of md5/sha1-hashed id as a path, arranging them like /path/f/9/e/95ea4926a4 ? That way, you won't have to deal with many-files-in-path problem ... Why is that a problem? So you can os.listdir them? Why should you have a problem os.listdir'ing lots of files? I shouldn't, and I don't ;) Then why did you suggest that there was a problem being able to os.listdir them? I didn't, OP did ... Then why did you reply to my question Why is that a problem? with So that you can os.listdir them?, if you didn't think there was a problem (see above)? Why do you think that if I didn't suggest there is a problem, I think there is no problem? I do think there might be such a problem and even I may have to face it someday. So, out of sheer curiosity how more rediculous this topic can be I'll try to rephrase and extend what I wrote in the first place: Why would you want to listdir them? I can imagine at least one simple scenario: you had some nasty crash and you want to check that every file has corresponding, valid db record. What's the problem with listdir if there's 10^x of them? Well, imagine that db record also holds file modification time (say, the files are some kind of cache), so not only you need to compare listdir results with db, but also do os.stat on every file and some filesystems will do it very slowly with so many of them in one place. Now, I think I made this point in the first answer, no? Of course you can make it more rediculous by your I-can-talk-away-any-problem-I-can't-see-or-solve approach by asking why would you want to use such filesystems?, why do you have to use FreeBSD?, why do you have to work for such employer?, why do you have to eat? etc, but you know, sometimes it's easier and better for the project/work just to solve it, than talk everyone else away from it just because you don't like otherwise acceptable solution. -- Mike Kazantsev // fraggod.net signature.asc Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
MailingLogger 3.3.0 Released!
I'm pleased to announce a new release of Mailinglogger.
Mailinglogger provides two handlers for the standard python
logging framework that enable log entries to be emailed either as the
entries are logged or as a summary at the end of the running process.
The handlers have the following features:
- customisable and dynamic subject lines for emails sent
- emails sent with an X-Mailer header for easy filtering
- flood protection to ensure the number of emails sent is not excessive
- support for SMTP servers that require authentication
- fully documented and tested
In addition, extra support is provided for configuring the handlers when
using ZConfig, Zope 2 or Zope 3.
The latest releases of ZConfig, in particular, provide a great way to
configure the python logging framework without having to resort to the
appalling .ini-based configuration stuff:
from ZConfig import configureLoggers
configureLoggers('''
... logger
...level INFO
...logfile
... PATH STDOUT
... format %(levelname)s %(name)s %(message)s
.../logfile
... /logger
... ''')
This release of MailingLogger adds the ability to use %(levelname)s in
subject-line formatting for SummarisingLoggers to include the highest
level logged.
For more information, please see:
http://www.simplistix.co.uk/software/python/mailinglogger
or
http://pypi.python.org/pypi/mailinglogger
cheers,
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
Re: Integer Division
On Jun 19, 8:22 am, Anjanesh Lekshminarayanan m...@anjanesh.net wrote: a = 1 b = 25 a / b 0 float(a) / b 0.040001 Python typically stores floats in binary, not decimal. The value 0.04 isn't exactly representable in binary, so the division float(1)/25 can't produce 0.04: what you get instead is a very good approximation to 0.04. That's why you're seeing what you're seeing above. Note that 0.0400...001 is *also* just an approximation to the actual value stored, but it's a better approximation than 0.04. The exact value stored is (probably, assuming your machine is typical): 0.04832667268468867405317723751068115234375 or, if you prefer, it's: 5764607523034235/144115188075855872 In what simple way can I get just 0.04 ? The answer to that depends on what you want and why you want it. Do you want a string or a number? And if you want a number, does it have to be *exactly* 0.04, or can your application cope with a tiny error? If you're trying to get a string and just want things to look nice, use str() instead of repr(), or display the value using print. If you want more control over the number of digits displayed, use Python's string formatting: e.g., in Python 2.6: format(0.04, '.3f') # format with 3 places after the point '0.040' See the following for more information on format: http://docs.python.org/library/string.html#formatstrings If you're after the number and you *really* need to be able to manipulate the *exact* value 0.04 in Python (e.g., because you're doing financial work), you're probably better off using the Decimal module: http://docs.python.org/library/decimal.html See also: http://docs.python.org/tutorial/floatingpoint.html Mark -- http://mail.python.org/mailman/listinfo/python-list
Slow wxPyhon on Vista?
Hello there, this might be my first post here and it is slightly off topic since it is not about something I am developing. At work use a nifty little program called Task Coach. It helps me keep track of what I spend time on. Here is my problem. When I use it on a Vista box its user interface is very, very slow when it comes to some but not all operations. On any other Windows version it reacts instantaneously. I have not tried Task Coach on Linux or Solaris (yet). Is this how wxPython or Python works on Vista in general or is this the result of some local oddity on my employer's Vista configuration? TIA, /Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie queue question
Digging further, I found this: http://www.oreillynet.com/onlamp/blog/2005/10/does_python_have_a_concurrency.html Looking up on this info, I found this: http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock If this is correct, no amount of threading would ever help in Python since only one core / CPU could *by design* ever be utilized. Except for the code that accesses *no* functions / memory at all. This does seem to be a bit harsh though. I'm now writing a simple test program to verify this. Multiple data- independed threads just so I can see if more than one core can at all be utilized. :( -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie queue question
On 2009-06-19, =?windows-1252?Q?Jure_Erzno=9Enik?= jure.erznoz...@gmail.com wrote: If this is correct, no amount of threading would ever help in Python since only one core / CPU could *by design* ever be utilized. Except for the code that accesses *no* functions / memory at all. Don't multithread...multiprocess. By running multiple python instances, the operating system handles the processor scheduling for each so that you can use all available CPUs/cores. It also tends to make debugging easier. It does create more overhead -- significantly more on some OSs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Help: Group based synchronize decorator
Vishal Shetye vishal_she...@persistent.co.in (VS) wrote: VS I want to synchronize calls using rw locks per 'group' and my implementation is similar to VS http://code.activestate.com/recipes/465057/ VS except that I have my own Lock implementation. VS All my synchronized functions take 'whatGroup' as param. My lock considers 'group' while deciding on granting locks through acquire. VS What I could come up with is: VS - decorator knows(assumes) first param to decorated functions is always 'whatGroup' VS - decorator passes this 'whatGroup' argument to my lock which is used in acquire logic. VS Is it ok to make such assumptions in decorator? As long as you make sure that all decorated functions indeed adhere to that assumption there is nothing wrong with it. -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Once again, comparison wxpython with PyQt
Hans Müller wrote: Thanks for all your informative replies. If I understand you right, for a commercial, closed source program I only need a commercial PyQt license for ~ 500€ ? Why not ask the guys at riverbankcomputing? http://www.riverbankcomputing.co.uk/commercial/pyqt This page http://www.riverbankcomputing.co.uk/software/pyqt/license Says The commercial version of PyQt can be used with both the commercial and LGPL versions of Qt. As far as I know I also need a Qt Licenses which is ~3500€ per OS. Not true, now Qt is licensed under the LGPL. You have to abide by the LGPL, however, which means that the user has to be able to relink a modified form of the library, Qt, with your application should they wish. You should check whether the LGPL is appropriate for the way you want to ship your program. Jeremy -- Jeremy Sanders http://www.jeremysanders.net/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Tutorial on working with Excel files in Python (without COM and cross platform!) at EuroPython 2009
John Machin sjmac...@lexicon.net (JM) wrote: JM [1] No kidding: http://www.google.com/patents/about?id=QMwnEBAJ Apart from these patents probably being silly, why don't they just write the code in Python? :=) -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
ErrorHandler 1.0.0 Released!
I'm pleased to finally get around to announcing the release of ErrorHandler.
This is a handler for the python standard logging framework that can
be used to tell whether messages have been logged at or above a
certain level.
This can be useful when wanting to ensure that no errors have been
logged before committing data back to a database.
Here's an example:
First, you set up the error handler:
from logging import getLogger
from errorhandler import ErrorHandler
logger = getLogger()
e = ErrorHandler()
The handler started off being un-fired:
e.fired
False
Then you do whatever else you need to do, which may involve logging:
logger.info('some information')
e.fired
False
However, if any logging occurs at an error level or above:
logger.error('an error')
Then the error handler becomes fired:
e.fired
True
You use this as a condition to only do certain actions when no errors
have been logged:
if e.fired:
... print Not updating files as errors have occurred
Not updating files as errors have occurred
If your code does work in batches, you may wish to reset the error
handler at the start of each batch:
e.reset()
e.fired
False
For more information, please see:
http://www.simplistix.co.uk/software/python/errorhandler
or
http://pypi.python.org/pypi/errorhandler
cheers,
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
Excel Formulae in Python ;-)
Piet van Oostrum wrote: John Machin sjmac...@lexicon.net (JM) wrote: JM [1] No kidding: http://www.google.com/patents/about?id=QMwnEBAJ Apart from these patents probably being silly, why don't they just write the code in Python? :=) Would be cool, but there are things like Resolver One (http://www.resolversystems.com/product/) and DiscoveryScript (http://www.xefion.com/en/discoveryscript.html) ...that help. cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: CAD file format specifications?
On Thu, 18 Jun 2009 13:39:28 +0200, Anthra Norell wrote: I had a look at Blender. It looks impressive too. It might be an alternative to Sketch Up. I'll worry about that later. My immediate need is a file conversion utility. A cursory inspection of Blender's menu tabs and the various help options didn't turn up a file-conversion utility. So, my question is: How do I convert a bunch of three-dimensional coordinates defining lines into a file format Sketch Up can read (skp, dwg, dxf, 3ds, ddf or dem)? Frederic If you look in the File/Import menu in Blender, you will see all the file types which it can load. The importing is done with python scripts, so if you are going to write a converter you might just as well write an import script for Blender. The raw import script shows how simple it can be. -- http://mail.python.org/mailman/listinfo/python-list
Re: A question on scope...
MRAB a écrit : Wells Oliver wrote: NB : answering the OP (original post didn't show up on c.l.py ???) In writing out python classes, it seems the 'self' is optional, You mean, inside a method ? meaning that inside a class method, warning topic=semantic issue In Python, a class method is a method that operates on the class object itself instead of operating on an instance of the class. /warning self.foo = bar has the same effect as foo = bar. Is this right? It's obviously false, cf MRAB's answer. -- http://mail.python.org/mailman/listinfo/python-list
Status of Python threading support (GIL removal)?
See here for introduction: http://groups.google.si/group/comp.lang.python/browse_thread/thread/370f8a1747f0fb91 Digging through my problem, I discovered Python isn't exactly thread safe and to solve the issue, there's this Global Interpreter Lock (GIL) in place. Effectively, this causes the interpreter to utilize one core when threading is not used and .95 of a core when threading is utilized. Is there any work in progess on core Python modules that will permanently resolve this issue? Is there any other way to work around the issue aside from forking new processes or using something else? -- http://mail.python.org/mailman/listinfo/python-list
Re: sqlite3, OperationalError: no such column, shouldn't that ne a ProgrammingError?
Gabriel Rossetti wrote: Hello everyone, I get an OperationalError with sqlite3 if I put the wrong column name, but shouldn't that be a ProgrammingError instead? I read PEP 249 and it says : OperationalError Exception raised for errors that are related to the database's operation and not necessarily under the control of the programmer, e.g. an unexpected disconnect occurs, the data source name is not found, a transaction could not be processed, a memory allocation error occurred during processing, etc. It must be a subclass of DatabaseError. ProgrammingError Exception raised for programming errors, e.g. table not found or already exists, syntax error in the SQL statement, wrong number of parameters specified, etc. It must be a subclass of DatabaseError. and to me it sounds more like a programming error than an operational error. I agree. But I can't help you there. See _pysqlite_seterror() at http://oss.itsystementwicklung.de/trac/pysqlite/browser/src/util.c What I do is map SQLite error codes to DB-API exceptions. And SQLite reports your error as generic SQLITE_ERROR, which is mapped to OperationalError. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: preferring [] or () in list of error codes?
In article pan.2009.06.09.03.18...@remove.this.cybersource.com.au,
But practicality beats purity -- there are many scenarios where we make
compromises in our meaning in order to get correct, efficient code. E.g.
we use floats, despite them being a poor substitute for the abstract Real
numbers we mean.
In addition, using a tuple or a list in this context:
if e.message.code in (25401,25402,25408):
is so idiomatic, that using a set in it's place would be distracting.
Rather that efficiently communicating the programmer's intention, it
would raise in my mind the question that's strange, why are they using a
set there instead of a tuple?.
As a newby I'm really expecting a set here. The only reason my mind
goes in the direction of 3 items is that it makes no sense in
combination with ``in''. That makes this idiom one that should be killed.
point1 = (0,1,0)
point2 = (1,0,0)
point3 = (0,0,1)
for i in (point1,point2, point3):
???
I don't think so.
At least I would do
for i in [point1,point2,point3]:
statements
But I greatly prefer a set
for i in {point1,point2,point3}:
statements
Because a set is unorderded, this would convey to the
the compiler that it may evaluate the three statements concurrently.
For a list I expect the guarantee that the statements are
evaluated in order.
For a tuple I don't know what to expect. That alone is sufficient
reason not to use it here.
[Yes I know { } doesn't denote a set. I tried it. I don't
know how to denote a set ... ]
--
Steven
Groetjes Albert.
--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
alb...@spearc.xs4all.nl =n http://home.hccnet.nl/a.w.m.van.der.horst
--
http://mail.python.org/mailman/listinfo/python-list
Re: Allocating memory to pass back via ctypes callback function
I think I found the answer to my own question. Can anyone spot any issues with the following solution? The application I'm writing will be hitting these callbacks pretty heavily so I'm nervous about mucking up the memory management and creating one of those bugs that passes undetected through testing but nails you in production. def my_callback(p_cstring): answer = 'foobar' address = VENDOR_malloc(len(answer)+1) cstring = c_char_p.from_address( address ) cstring.value = answer p_cstring.contents = cstring return -- http://mail.python.org/mailman/listinfo/python-list
Re: Integer Division
Anjanesh Lekshminarayanan m...@anjanesh.net (AL) escribió: a = 1 b = 25 a / b AL 0 float(a) / b AL 0.040001 from __future__ import division a = 1 b = 25 a / b AL 0.040001 AL In what simple way can I get just 0.04 ? In addition to the answers others have given: 0.04 0.040001 IIRC, Python 3.1 will give 0.04. But this doesn't mean the answer will be mathematically equal to 0.04, just that it tries to make it less painful. -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Excel Formulae in Python ;-)
Chris Withers ch...@simplistix.co.uk (CW) wrote: CW Piet van Oostrum wrote: John Machin sjmac...@lexicon.net (JM) wrote: JM [1] No kidding: http://www.google.com/patents/about?id=QMwnEBAJ Apart from these patents probably being silly, why don't they just write the code in Python? :=) CW Would be cool, but there are things like CW Resolver One (http://www.resolversystems.com/product/) CW and CW DiscoveryScript (http://www.xefion.com/en/discoveryscript.html) CW ...that help. Actually, I meant writing the patent application in Python :=) instead of this pseudo-exact legalese with goto instructions for the control flow. -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie queue question
Jure Erznožnik jure.erznoz...@gmail.com (JE) wrote: JE Digging further, I found this: JE http://www.oreillynet.com/onlamp/blog/2005/10/does_python_have_a_concurrency.html JE Looking up on this info, I found this: JE http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock JE If this is correct, no amount of threading would ever help in Python JE since only one core / CPU could *by design* ever be utilized. Except JE for the code that accesses *no* functions / memory at all. It is not the design of the Python language, but of the *CPython* implementation. And yes, it will not benefit from more than one core. You should watch/read this: http://blip.tv/file/2232410 http://www.dabeaz.com/python/GIL.pdf -- Piet van Oostrum p...@cs.uu.nl URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: p...@vanoostrum.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Integer Division
Anjanesh Lekshminarayanan wrote: a = 1 b = 25 a / b 0 float(a) / b 0.040001 from __future__ import division a = 1 b = 25 a / b 0.040001 In what simple way can I get just 0.04 ? Your subject line says Integer Division but you're dealing with floats. IEEE standard float cannot exactly represent 0.04 In fact, no binary floating point value of finite precision can get that exact value. So if you really want 0.04, you need to use the decimal package. It's the only way you'll be sure of exactly representing any decimal fraction you need. Let's call val = float(1)/25If you're using future division (or Python 3.x), 1/25 will be exactly the same as val. In either case, the division cannot come out exactly, since there is no such exact float. On the other hand, some *string* representation of val may not come out exact. Technically, to represent val in decimal digits may take up to 59 or so characters. But the two standard conversions from float to string ( float.__str__ and float.__val__ ) do a little rounding themselves. So you can try them, and see that one of them works. (print calls the __str__ method, while the interpreter uses __repr__) The problem with this is that what works for one such val might fail for the next. And these __str__ and __val__ have been implemented multiple times, so testing on one Python version will not assure consistency on the next. The issue is that you're counting on two approximations canceling each other out. That'll work in some circumstances, but knowing what they are isn't reasonable. So what do people do? Simplest is to use a decimal package. I tried back in the early 80's to convince the IEEE committee to include decimal float in the standard, but was rejected. I had implemented (microcoded) arithmetic and transcendentals on a machine which offered only decimal floating point; no binary floating point. Next choice? Stick to integers, or implied fractions. For example, if dealing with dollars and cents, always calculate and store the value in pennies, and only insert the decimal when displaying. Finally, it's not hard (if you know something about the numbers) to get a pretty display version for yourself, rounding it as you're converting to string. Note that this affects other aspects of your program, such as comparisons between two floats. If you sum up 100 copies of .01, will you get exactly 1.0 ? I don't know, but in many cases, clearly not. This is not a new problem. I read about it in McCracken's Fortran book in 1967, where he said to use something on the order of: if abs(a-b) .01 ... when comparing reals. -- http://mail.python.org/mailman/listinfo/python-list
Re: preferring [] or () in list of error codes?
Albert van der Horst alb...@spenarnc.xs4all.nl writes:
But I greatly prefer a set
for i in {point1,point2,point3}:
statements
Agreed, for the reasons you cite. I think this idiom can be expected to
become more common and hopefully displace using a tuple literal or list
literal, as the set literal syntax becomes more reliably available on
arbitrary installed Python versions.
[Yes I know { } doesn't denote a set. I tried it. I don't know how to
denote a set ... ]
Try it in Python 3 and be prepared to be pleased
URL:http://docs.python.org/3.0/whatsnew/3.0.html#new-syntax.
--
\ “Too many Indians spoil the golden egg.” —Sir Joh |
`\ Bjelke-Petersen |
_o__) |
Ben Finney
--
http://mail.python.org/mailman/listinfo/python-list
Re: generator expression works in shell, NameError in script
Emile van Sebille a écrit : On 6/17/2009 3:54 PM ssc said... Wow! Didn't expect that kind of instant support. Thank you very much, I'll give both zip and enumerate a try. The code I've shown is actually copied pretty straight from a Django form class, but I didn't want to mention that as not to dilute the conversation. Don't think it matters, anyway. This is the relevant excerpt: from django.forms import Form class SignupForm(Form): titles = ['Dr', 'Miss', 'Mr', 'Mrs', 'Ms',] title_choices = [(0, '')] + list((titles.index(t)+1, t) for t in titles) Now that I look at it again, it seems odd to me to not have the code e.g. in __init__(...), but just 'class-global'. And as class is an executable statement, I imagine titles exists in a namespace that hasn't yet been completely defined. Still, that does not seem a reason for titles not to be not defined, as I do define it just in the line above. Well, again, titles will exists in the SignupForm namespace once it exists, which it doesn't yet when you get to title_choices and want to access it. The namespace itself is as defined at any point of the class statement's body as the local namespace of a function at any given point of the function's body. Else decorators (or their alternate pre '@' syntactic-sugar version) wouldn't work. FWIW, the following code works JustFine(tm): br...@bruno:~$ python Python 2.5.2 (r252:60911, Oct 5 2008, 19:24:49) [GCC 4.3.2] on linux2 class Foo(object): ... bar = ['a', 'b', 'c'] ... baaz = list(enumerate(bar)) as well as this one: class Foo(object): ... bar = ['a', 'b', 'c'] ... baaz = [(bar.index(t)+1, t) for t in bar] and this one: class Foo(object): ... bar = ['a', 'b', 'c'] ... baaz = list((b, b) for b in bar) but it indeed looks like using bar.index *in a generator expression* fails (at least in 2.5.2) : class Foo(object): ... bar = ['a', 'b', 'c'] ... baaz = list((bar.index(b), b) for b in bar) ... Traceback (most recent call last): File stdin, line 1, in module File stdin, line 3, in Foo File stdin, line 3, in genexpr NameError: global name 'bar' is not defined Looks like a bug to me :-/ -- http://mail.python.org/mailman/listinfo/python-list
Decorator question (how to test if decorated function is in a class)
Hi all,
I have been trying out to wrap my mind around the advantages of
decorators and thought I found a use in one of my experiments. (see code
after my sig).
Although it works, I think it should be able to do it better.
My particular problem is that I want to remove an argument (say always
the first one) when the decorated function is called.
However if the function is defined in a class the first argument is self
and thus the second argument should be removed.
Since I like to have a more general DRY approach I want to test if the
decorated function is defined in a class or not thus knowing that the
first argument is self and must be preserved.
I tried different approaches but all didn't work and looked very
unpythonic (looking at the attributes, descriptors namespaces, id() of
the function, the first argument and the decorator itself).
So is there a way to find out if the first argument is 'self' without
making a false positive if the first argument is a class instance passed
to a function?
--
MPH
http://blog.dcuktec.com
'If consumed, best digested with added seasoning to own preference.'
code:
class Decorator(object):
When used as Decorator(argument) returns a decorator
def __init__(self, key):
self.key = key
def __call__(self, function):
Return a decorator
def decorator(*arg_tuple, **arg_dict):
The decorator test if the arguments contain a valid key
if arg_tuple[0] == self.key:
# This means it is a simple function, so check and strip it
arg_list = arg_tuple[1::]
return(function(*arg_list, **arg_dict))
elif arg_tuple[1] == self.key:
# This means it is a class method, check it,
# strip (but retain self)
arg_list = arg_tuple
arg_list = arg_list[0:1] + arg_list[2::]
return(function(*arg_list, **arg_dict))
else:
# The key was invalid
return('invalid')
return(decorator)
class Test(object):
@Decorator('key')
def test(self, data):
return(data)
@Decorator('key')
def function_2(self, data):
return(data)
@Decorator('key')
def function_3(self, data):
return(data)
@Decorator('key')
def test_f(data):
return(data)
# Test starts here
test_c = Test()
print(test_c.test('key', 'data'))
print(test_f('key', 'data'))
print(test_c.test('invalid', 'data'))
print(test_f('invalid', 'data'))
--
http://mail.python.org/mailman/listinfo/python-list
Re: generator expression works in shell, NameError in script
On Jun 19, 1:45 pm, Bruno Desthuilliers bruno. 42.desthuilli...@websiteburo.invalid wrote: [...] but it indeed looks like using bar.index *in a generator expression* fails (at least in 2.5.2) : class Foo(object): ... bar = ['a', 'b', 'c'] ... baaz = list((bar.index(b), b) for b in bar) ... Traceback (most recent call last): File stdin, line 1, in module File stdin, line 3, in Foo File stdin, line 3, in genexpr NameError: global name 'bar' is not defined Looks like a bug to me :-/ I think it's working as intended: name resolution for nested scopes skips intermediate class scopes. The 'Discussion' section of PEP 227 (Statically Nested Scopes) is illuminating; here's a snippet: Names in class scope are not accessible. Names are resolved in the innermost enclosing function scope. If a class definition occurs in a chain of nested scopes, the resolution process skips class definitions. This rule prevents odd interactions between class attributes and local variable access. Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: persistent composites
On Jun 17, 3:53 pm, Rhodri James rho...@wildebst.demon.co.uk wrote: On Wed, 17 Jun 2009 16:06:22 +0100, Aaron Brady castiro...@gmail.com wrote: On Jun 16, 10:09 am, Mike Kazantsev mk.frag...@gmail.com wrote: On Tue, 16 Jun 2009 06:57:13 -0700 (PDT) Aaron Brady castiro...@gmail.com wrote: Making the charitable interpretation that this was the extent of c-l- py's support and enthusiasm for my idea, I will now go into mourning. Death occurred at oh-eight-hundred. Rest in peace, support enthusiasm. I've read this thread from the beginning, being tempted to insert remarks about shelve module or ORMs like SQLAlchemy, but that'd be meaningless without the problem description, which I haven't seen anywhere. Is it some trick idea like let's walk on our heads? More like bronze them, or hang them on a tackboard. You haven't convinced me that it's not a problem, or that it's an easy one. Unfortunately it's up to you to demonstrate that it is a problem, whichever of the many possible 'it's you're talking about. So far, the question Why would I want to use this? What's the use case? has gone unanswered, and I'm sure I'm not the only baffled by it. I can demonstrate it's a problem; many things are. The subject line says it all. SQL will persist (and share for IPC, after a fashion) sets of statically typed tuples. But we have need for sets of dynamically typed tuples, in volatile storage at the very least, or no one would like Python. Whether we have need for them in persistent storage remains to be seen. POSH Python Object SHaring has been at least one student's graduate thesis. It couldn't hurt to pursue it if one has time, as with many things. It's pretty likely my estimates of end user usefulness of 'KeepDB', say, derive sturdily and steadily from our estimated popularity of Python. So much for my motives, though not for my opportunities. My implementation has followed two separate paths: one using SQL for a back-end host; the other using hand-coded bytes on disk, complete with large file support using views, and on-disk (or in-buffer) memory management, which incidentally I spoke up about about a year ago. It was the topic of an honors lecture in the Data Structures course in my undergrad. For the record, I still can't beat a log-N balanced tree for finding free nodes, but I am operating on the assumption that the smallest sufficient region is the best to return from 'alloc'. You are not being any help, Rhodri, in your question. Its chief virtue is fidelity of programming, that is, what you write is most true to what you mean. If you have an object on disk, and want the third element of its 'coords' list attribute, it's a waste to do anything other than find its disk address, then the disk address of its 'coords' attribute, then the disk address of the third element of that, such as say, loading or worse, processing, the entire structure; even if, and in light of parallel considerations about economic market transparency and efficiency, that is to say, economy, maybe / especially/ if, you are having a lowly machine make do. You don't want to open every box in your basement to find your tennis racket; you want to open one box, find the racket in the index, then open all and only the boxes that 'nestedly' contain the racket. (Yes, that's / all/ and /only/, or 'if and only if'-- not to be confused with 'one and only'.) It's more economic. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this pylint error message valid or silly?
On Fri 19 Jun 2009 02:55:52 AM EDT, Terry Reedy wrote: if c == today: c = datetime.today() Now I guess that you actually intend c to be passed as a datetime object. You only used the string as a type annotation, not as a real default value. Something like 'record_date = None' is better. Thanks for the feedback. I think I should have used a more obvious string in my original example and a more descriptive parameter name. So, pretend that instead of c=today I wrote record_date=defaults to today's date. I know my way is unorthodox, but I think it is a little bit more obvious to the reader than record_date=None The None is a signal to use a default value, but that is only apparent after reading the code. Thanks again for the comments. Matt -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
This is a very long-running issue, that has been discussed many times. Here are the two sides to keeping the gil or removing it: Remove the GIL: - True multi-threaded programming - Scalable performance across a multi-core machine - Unfortunately, this causes a slow-down in single core/thread performance - Most of the slow down comes from Python's Reference counting, as every function is supposed to increase and decrease the references... which leads to a lot of synchronization and checking of locks. - It makes the core interpreter much more complex Keeping the GIL: - Processes should be used instead of threads. Look at Erlang, THE model of concurrency. It uses a process model for concurrency and message passing, rather than threads. This means you avoid deadlocks, livelocks, race conditions(not entirely, but they are reduced) - C extensions can still gain really impressive multi-threaded performance. My C extension, a wrapper around ffmpeg, releases the GIL for 90% or so of its processing time, so all other threads are still extremely responsive. - It allows python to stay simple, understandable, and offer good performance, if you know how to use it properly. Basically the two sides are that you it ends up slowing things down, it makes the interpreter more complex vs you should use processes instead, and be smart about how you use the GIL. -Tyler On Fri, Jun 19, 2009 at 2:52 AM, Jure Erznožnik jure.erznoz...@gmail.comwrote: See here for introduction: http://groups.google.si/group/comp.lang.python/browse_thread/thread/370f8a1747f0fb91 Digging through my problem, I discovered Python isn't exactly thread safe and to solve the issue, there's this Global Interpreter Lock (GIL) in place. Effectively, this causes the interpreter to utilize one core when threading is not used and .95 of a core when threading is utilized. Is there any work in progess on core Python modules that will permanently resolve this issue? Is there any other way to work around the issue aside from forking new processes or using something else? -- http://mail.python.org/mailman/listinfo/python-list -- Visit my blog at http://oddco.ca/zeroth/zblog -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
Jure Erznožnik wrote: See here for introduction: http://groups.google.si/group/comp.lang.python/browse_thread/thread/370f8a1747f0fb91 Digging through my problem, I discovered Python isn't exactly thread safe and to solve the issue, there's this Global Interpreter Lock (GIL) in place. Effectively, this causes the interpreter to utilize one core when threading is not used and .95 of a core when threading is utilized. Is there any work in progess on core Python modules that will permanently resolve this issue? Is there any other way to work around the issue aside from forking new processes or using something else? There is a group of people working on an alternative implementation to Python that, among other things, will not have a GIL: http://code.google.com/p/unladen-swallow/ There was even a successful attempt to remove the GIL from CPython, but it caused single threaded python code to be much slower. See more here: http://www.python.org/doc/faq/library/#can-t-we-get-rid-of-the-global-interpreter-lock Cheers, Ben -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this pylint error message valid or silly?
On Fri, 19 Jun 2009 14:36:01 +0100, Matthew Wilson m...@tplus1.com wrote: On Fri 19 Jun 2009 02:55:52 AM EDT, Terry Reedy wrote: if c == today: c = datetime.today() Now I guess that you actually intend c to be passed as a datetime object. You only used the string as a type annotation, not as a real default value. Something like 'record_date = None' is better. Thanks for the feedback. I think I should have used a more obvious string in my original example and a more descriptive parameter name. So, pretend that instead of c=today I wrote record_date=defaults to today's date. I know my way is unorthodox, but I think it is a little bit more obvious to the reader than record_date=None The None is a signal to use a default value, but that is only apparent after reading the code. It's a very common idiom, however, and in addition record_data is None is a cheap test, while record_data == defaults to today's date is an expensive test as well as easy to mistype. -- Rhodri James *-* Wildebeest Herder to the Masses -- http://mail.python.org/mailman/listinfo/python-list
Re: persistent composites
On Fri, 19 Jun 2009 14:24:34 +0100, Aaron Brady castiro...@gmail.com wrote: You are not being any help, Rhodri, in your question. To you, perhaps not. To me, it has at least had the effect of making what you're trying to do (write a pythonic object database) clearer. -- Rhodri James *-* Wildebeest Herder to the Masses -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
Digging through my problem, I discovered Python isn't exactly thread safe and to solve the issue, there's this Global Interpreter Lock (GIL) in place. It's the opposite: Python is exactly thread safe precisely because it has the GIL in place. Is there any other way to work around the issue aside from forking new processes or using something else? If you know that your (C) code is thread safe on its own, you can release the GIL around long-running algorithms, thus using as many CPUs as you have available, in a single process. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
Digging through my problem, I discovered Python isn't exactly thread safe and to solve the issue, there's this Global Interpreter Lock (GIL) in place. It's the opposite: Python is exactly thread safe precisely because it has the GIL in place. Is there any other way to work around the issue aside from forking new processes or using something else? If you know that your (C) code is thread safe on its own, you can release the GIL around long-running algorithms, thus using as many CPUs as you have available, in a single process. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this pylint error message valid or silly?
Am Fri, 19 Jun 2009 00:56:18 + schrieb Matthew Wilson: Here's the code that I'm feeding to pylint: $ cat f.py from datetime import datetime def f(c=today): if c == today: c = datetime.today() return c.date() And here's what pylint says: $ pylint -e f.py No config file found, using default configuration * Module f E: 10:f: Instance of 'str' has no 'date' member (but some types could not be inferred) Is this a valid error message? Is the code above bad? If so, what is the right way? I changed from using a string as the default to None, and then pylint didn't mind: $ cat f.py from datetime import datetime def f(c=None): if c is None: c = datetime.today() return c.date() $ pylint -e f.py No config file found, using default configuration I don't see any difference between using a string vs None. Both are immutable. I find the string much more informative, since I can write out what I want. Looking for comments. Matt Think of what happens if somebody calls some_result = f(yesterday) HTH Martin -- http://mail.python.org/mailman/listinfo/python-list
multiprocessing and process run time
Hi, I'm making use of the multiprocessing module, and I was wondering if there is an easy way to find out how long a given process has been running for. For example, if I do import multiprocessing as mp import time def time_waster(): time.sleep(1000) p = mp.Process(target=time_waster) p.start() Is there a way that I can then find how long p has been running for? I figured I can use p.pid to get the PID of the process, but I'm not sure where to go from there. Is there an easy way to do this? Thanks, Thomas -- View this message in context: http://www.nabble.com/multiprocessing-and-process-run-time-tp24112854p24112854.html Sent from the Python - python-list mailing list archive at Nabble.com. -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
On 19 juin, 11:52, Jure Erznožnik jure.erznoz...@gmail.com wrote: See here for introduction:http://groups.google.si/group/comp.lang.python/browse_thread/thread/3... Digging through my problem, I discovered Python isn't exactly thread safe and to solve the issue, there's this Global Interpreter Lock (GIL) in place. Effectively, this causes the interpreter to utilize one core when threading is not used and .95 of a core when threading is utilized. Is there any work in progess on core Python modules that will permanently resolve this issue? Is there any other way to work around the issue aside from forking new processes or using something else? hi, please read this carefully, http://www.ibm.com/developerworks/aix/library/au-multiprocessing/ index.html?ca=dgr-lnxw07Python- MultiS_TACT=105AGX59S_CMP=grsitelnxw07 there is a solution for Python on multi-core : multiprocessing api. Really nice. http://docs.python.org/library/functions.html Keep real threads for common tasks like network stuff for example. I recently complained too :-) http://groups.google.com/group/comp.lang.python/browse_frm/thread/ dbe0836d9602f322# Olivier -- http://mail.python.org/mailman/listinfo/python-list
Re: Integer Division
On 2009-06-19, Anjanesh Lekshminarayanan m...@anjanesh.net wrote: a = 1 b = 25 a / b 0 float(a) / b 0.040001 from __future__ import division a = 1 b = 25 a / b 0.040001 In what simple way can I get just 0.04 ? You can't. There _is_no_ 0.04 when using binary IEEE floating point (which is what's used for floats on any computer you're likely to run across). You could use some other data type (rationals, BCD floats, etc.). -- Grant Edwards grante Yow! Hello. I know at the divorce rate among visi.comunmarried Catholic Alaskan females!! -- http://mail.python.org/mailman/listinfo/python-list
Retrieving column values by column name with MySQLdb
Hi,
Is there a way of retrieving the value of columns in the rows returned by
fetchall, by column name instead of index on the row?
Code Snippet:
query=select * from employees
db=MySQLdb.connect(host=host,user=user,passwd=passwd,db=database)
cursor = db.cursor ()
cursor.execute (query)
rows = cursor.fetchall ()
for row in rows:
print row[0]
Instead of specifying the index of the row to retrieve the first column
(row[0]), I'd like to retrieve the value of the first column by column name.
Something like row.get('employee_id')
Is something of the sort possible with Mysqdb?
Thanks very much.
--
http://mail.python.org/mailman/listinfo/python-list
Re: persistent composites
In article be292347-1011-4bb6-b8e9-a5d738827...@u10g2000vbd.googlegroups.com, Aaron Brady castiro...@gmail.com wrote: You are not being any help, Rhodri, in your question. Maybe not, but honestly, you're getting pretty close to going back in my killfile. Although you're no longer trolling this group, I find your writing difficult to read at best; answering questions from people like Rhodri is about your only chance to reach people like me. Even without the killfile, I'm skipping about 60-80% of your posts. It's a free Usenet, of course, so you're under no obligation to pay any attention to me, but I think you ought to consider the merits of changing your opinion of Rhodri's questions. Side note: the rhetorical trick of inserting a person's name in a statement is often intended and received as a mildly patronizing insult. It's not at all clear to me whether that was your intent; if not, you might want to change your writing style a bit. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ as long as we like the same operating system, things are cool. --piranha -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling subprocess with arguments
So no one has an answer for why passing flags and the values the flags need
through subprocess does not work? I would like an answer. I've examined all
the examples I could find online, which were all toy examples, and not
helpful to my problem.
On Thu, Jun 18, 2009 at 7:40 PM, Tyler Laing trinio...@gmail.com wrote:
I've been trying any variation I can think of to do this properly, but
here's my problem:
I want to execute this command string: vlc -I rc
This allows vlc to be controlled via a remote interface instead of the
normal gui interface.
Now, say, I try this from subprocess:
p=subprocess.Popen('vlc -I rc test.avi'.split(' '), shell=False,
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
But I don't get the remote interface. I get the normal gui interface. So
how do I do it? I've tried passing ['vlc', '-I', 'rc'], I've tried ['-I',
'rc'] with executable set to 'vlc'. I've had shell=True, I've had
shell=False. I've tried all these combinations.
What am I doing wrong?
--
Visit my blog at http://oddco.ca/zeroth/zblog
--
Visit my blog at http://oddco.ca/zeroth/zblog
--
http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing and process run time
On Fri, 19 Jun 2009 07:40:11 -0700 (PDT) Thomas Robitaille thomas.robitai...@gmail.com wrote: I'm making use of the multiprocessing module, and I was wondering if there is an easy way to find out how long a given process has been running for. For example, if I do import multiprocessing as mp import time def time_waster(): time.sleep(1000) p = mp.Process(target=time_waster) p.start() Is there a way that I can then find how long p has been running for? I figured I can use p.pid to get the PID of the process, but I'm not sure where to go from there. Is there an easy way to do this? If you use unix-like platform (e.g. linux) you can just use 'ps -e -o pid,start_time | grep pid'. I'd used procpy to do ps-like stuff in python since it's a wrapper around the same procps library, prehaps it can get the same results as well, w/o having to invoke shell commands: http://code.google.com/p/procpy/ -- Mike Kazantsev // fraggod.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling subprocess with arguments
Hello,
The problem might be that, aside from creating the Popen object, to
get the command run you need to call 'communicate' (other options, not
used with the Popen object directly, are 'call' or 'waitpid' as
explained in the documentation). Did you do that?
Best regards,
Javier
2009/6/19 Tyler Laing trinio...@gmail.com:
So no one has an answer for why passing flags and the values the flags need
through subprocess does not work? I would like an answer. I've examined all
the examples I could find online, which were all toy examples, and not
helpful to my problem.
On Thu, Jun 18, 2009 at 7:40 PM, Tyler Laing trinio...@gmail.com wrote:
I've been trying any variation I can think of to do this properly, but
here's my problem:
I want to execute this command string: vlc -I rc
This allows vlc to be controlled via a remote interface instead of the
normal gui interface.
Now, say, I try this from subprocess:
p=subprocess.Popen('vlc -I rc test.avi'.split(' '), shell=False,
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
But I don't get the remote interface. I get the normal gui interface. So
how do I do it? I've tried passing ['vlc', '-I', 'rc'], I've tried ['-I',
'rc'] with executable set to 'vlc'. I've had shell=True, I've had
shell=False. I've tried all these combinations.
What am I doing wrong?
--
Visit my blog at http://oddco.ca/zeroth/zblog
--
Visit my blog at http://oddco.ca/zeroth/zblog
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: GUI(eclipse+pydev/SPE) freeze when doing python auto-completion under Linux
On Fri, Jun 19, 2009 at 12:22 AM, Wei, Xiaohaiwist...@gmail.com wrote: Thanks for your reply. where is the error log? I can not find it at /var/log Take a look at http://pydev.sourceforge.net/faq.html#how_do_i_report_a_bug (it gives the details on how to find the needed info). I have a virtual network card to bridge kvm virtual machine. will it cause problem? Not sure... What I know is that pydev creates a shell and connects to it through 127.0.0.1, so, if that fails, that could be a problem. as you said configuration of interpretor, how should I configure the interpretor? Please follow the getting started tutorial for that: http://fabioz.com/pydev/manual_101_root.html Cheers, Fabio Thanks James On Fri, Jun 19, 2009 at 9:32 AM, Fabio Zadrozny fabi...@gmail.com wrote: yes, the same problem even on an empty program. every file has the same problem. for example, if I new a file and input the following: import os os. after I input '.', it will pop up the window, and i can select the function of os module or continue input. but after that, no action can be taken for this file unless I switch to other files and then switch back. If it's in pydev, there's probably some problem in your interpreter configuration (when you do os. it'll spawn a shell to gather the completions -- but that should be pretty quick unless you have some firewall that's preventing the communication or the spawn didn't go well -- check your error log to see if you have any errors... In linux I've seen some problems when connecting to 127.0.0.1 while having ipv6 enabled or in a network card misconfiguration that could cause problems too). Cheers, Fabio On Thu, Jun 18, 2009 at 10:57 AM, Tyler Laing trinio...@gmail.com wrote: Do you experience the same problem even on an empty program file or is it limited to just one file? -Tyler On Wed, Jun 17, 2009 at 7:47 PM, Wei, James wist...@gmail.com wrote: On Jun 18, 10:45 am, Wei, James wist...@gmail.com wrote: When I am editing python program with SPE, I found that SPE will freeze when it is doing auto-completion. The behavior is very strange that I can not edit the file again. If I switch to another file and then switch back, I can edit it again. So I switch to eclipse+pydev, but I found the same thing happen. So I think it is not the problem of SPE or eclipse or pydev. If I disable auto-completion function in SPE or Eclipse+PyDev, it will not freeze any longer. Anybody can give me some hints? I am using Ubuntu 8.10 and updated to latest. All packages is installed through package manager. the only thing I googled related with it is http://www.nabble.com/-pydev---Users--jython-2.5b1-Froze-eclipse-on-autocomplete-td21394274.html but I think they are different. -- http://mail.python.org/mailman/listinfo/python-list -- Visit my blog at http://oddco.ca/zeroth/zblog -- Best wishes to you. Yours sincerely Xiaohai Wei wist...@ustc.edu -- http://mail.python.org/mailman/listinfo/python-list -- Best wishes to you. Yours sincerely Xiaohai Wei wist...@ustc.edu -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling subprocess with arguments
I can't use communicate, as it waits for the child process to terminate.
Basically it blocks. I'm trying to have dynamic communication between the
python program, and vlc.
On Fri, Jun 19, 2009 at 8:05 AM, Charles Yeomans char...@declaresub.comwrote:
On Jun 19, 2009, at 10:55 AM, Tyler Laing wrote:
So no one has an answer for why passing flags and the values the flags need
through subprocess does not work? I would like an answer. I've examined all
the examples I could find online, which were all toy examples, and not
helpful to my problem.
On Thu, Jun 18, 2009 at 7:40 PM, Tyler Laing trinio...@gmail.com wrote:
I've been trying any variation I can think of to do this properly, but
here's my problem:
I want to execute this command string: vlc -I rc
This allows vlc to be controlled via a remote interface instead of the
normal gui interface.
Now, say, I try this from subprocess:
p=subprocess.Popen('vlc -I rc test.avi'.split(' '), shell=False,
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
But I don't get the remote interface. I get the normal gui interface. So
how do I do it? I've tried passing ['vlc', '-I', 'rc'], I've tried ['-I',
'rc'] with executable set to 'vlc'. I've had shell=True, I've had
shell=False. I've tried all these combinations.
What am I doing wrong?
You might try
p = subprocess.Popen('vlc -I rc test.avi', ...
Charles Yeomans
--
Visit my blog at http://oddco.ca/zeroth/zblog
--
http://mail.python.org/mailman/listinfo/python-list
Re: Calling subprocess with arguments
On Fri, 19 Jun 2009 07:55:19 -0700
Tyler Laing trinio...@gmail.com wrote:
I want to execute this command string: vlc -I rc
This allows vlc to be controlled via a remote interface instead of the
normal gui interface.
Now, say, I try this from subprocess:
p=subprocess.Popen('vlc -I rc test.avi'.split(' '), shell=False,
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
But I don't get the remote interface. I get the normal gui interface. So
how do I do it? I've tried passing ['vlc', '-I', 'rc'], I've tried ['-I',
'rc'] with executable set to 'vlc'. I've had shell=True, I've had
shell=False. I've tried all these combinations.
What am I doing wrong?
Write a simple script:
#!/usr/bin/env python
import sys
open('/tmp/argv', 'w').write(repr(sys.argv))
And replace 'vlc' with a path to this script, then invoke it from a
shell, compare the results.
If it gets the right stuff, try the same with os.environ (prehaps vlc
keeps socket location there, just like ssh/gpg-agents?).
--
Mike Kazantsev // fraggod.net
signature.asc
Description: PGP signature
--
http://mail.python.org/mailman/listinfo/python-list
Re: Calling subprocess with arguments
On Fri, 19 Jun 2009 08:07:29 -0700 Tyler Laing trinio...@gmail.com wrote: I can't use communicate, as it waits for the child process to terminate. Basically it blocks. I'm trying to have dynamic communication between the python program, and vlc. Unfortunately, subprocess module doesn't allow it out-of-the-box, but you can use fnctl calls to perform non-blocking reads/writes on it's pipes, like this: flags = fcntl.fcntl(fd, fcntl.F_GETFL) fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NONBLOCK) After that, you can grab all the available data from the pipe at any given time w/o blocking. Try this recipe: http://code.activestate.com/recipes/576759/ -- Mike Kazantsev // fraggod.net signature.asc Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing and process run time
On 19 juin, 16:40, Thomas Robitaille thomas.robitai...@gmail.com
wrote:
Hi,
I'm making use of the multiprocessing module, and I was wondering if there
is an easy way to find out how long a given process has been running for.
For example, if I do
import multiprocessing as mp
import time
def time_waster():
time.sleep(1000)
p = mp.Process(target=time_waster)
p.start()
Is there a way that I can then find how long p has been running for? I
figured I can use p.pid to get the PID of the process, but I'm not sure
where to go from there. Is there an easy way to do this?
Something like :
import multiprocessing as mp
import time
q_out = mp.Queue()
def time_waster(qo):
time.sleep(5)
qo.put('stopped at ' + time.ctime(time.time()))
def main():
p = mp.Process(target=time_waster, args=(q_out,))
print start at: + time.ctime(time.time())
p.start()
p.join()
print q_out.get()
if __name__ == __main__:
main()
The Queue (the one from multiprocessing ! don't mix with the Queue
module) is used as a safe exchange object between the parent and all
the child.
Olivier
--
http://mail.python.org/mailman/listinfo/python-list
Re: Retrieving column values by column name with MySQLdb
On Fri, Jun 19, 2009 at 8:16 PM, jorma kalajjk...@gmail.com wrote:
Hi,
Is there a way of retrieving the value of columns in the rows returned by
fetchall, by column name instead of index on the row?
Code Snippet:
query=select * from employees
db=MySQLdb.connect(host=host,user=user,passwd=passwd,db=database)
cursor = db.cursor ()
cursor.execute (query)
rows = cursor.fetchall ()
for row in rows:
print row[0]
Instead of specifying the index of the row to retrieve the first column
(row[0]), I'd like to retrieve the value of the first column by column name.
Something like row.get('employee_id')
Is something of the sort possible with Mysqdb?
Thanks very much.
Use a DictCursor:
import MySQLdb.cursors
.
.
.
cursor = db.cursor (MySQLdb.cursors.DictCursor)
cursor.execute (query)
rows = cursor.fetchall ()
for row in rows:
print row['employee_id']
--
kushal
--
http://mail.python.org/mailman/listinfo/python-list
Re: fastest native python database?
Ethan Furman wrote:
This body part will be downloaded on demand.
Not sure what happened there... here's the text...
Howdy, Pierre!
I have also written a pure Python implementation of a database, one that
uses dBase III or VFP 6 .dbf files. Any chance you could throw it into
the mix to see how quickly (or slowly!) it runs?
The code to run the same steps are (after an import dbf):
#insert test
table = dbf.Table('/tmp/tmptable', 'a N(6.0), b N(6.0), c C(100)')
# if recs is list of tuples
for rec in recs:
table.append(rec)
# elif recs is list of lists
#for a, b, c in recs:
# current = table.append()
# current.a = a
# current.b = b
# current.c = c
#select1 test
for i in range(100):
nb = len(table)
if nb:
avg = sum([record.b for record in table])/nb
#select2 test
for num_string in num_strings:
records = table.find({'c':'%s'%num_string}, contained=True)
nb = len(records)
if nb:
avg = sum([record.b for record in records])/nb
#delete1 test
for record in table:
if 'fifty' in record.c:
record.delete_record()
# to purge the records would then require a table.pack()
#delete2 test
for rec in table:
if 10 rec.a 2:
rec.delete_record()
# again, permanent deletion requires a table.pack()
#update1 test
table.order('a')
for i in range(100): # update description says 1000, update code is 100
records = table.query(python='10*%d = a 10*%d' %(10*i,10*(i+1)))
for rec in records:
rec.b *= 2
#update2 test
records = table.query(python=0 = a 1000)
for rec in records:
rec.c = new_c[rec.a]
Thanks, I hope! :)
~Ethan~
http://groups.google.com/group/python-dbase
--
http://mail.python.org/mailman/listinfo/python-list
Re: Calling subprocess with arguments
Thanks mike, the idea that maybe some of the info isn't being passed is
certainly interesting.
Here's the output of os.environ and sys.argv:
ty...@surak:~$ cat environ
{'XAUTHORITY': '/home/tyler/.Xauthority', 'GNOME_DESKTOP_SESSION_ID':
'this-is-deprecated', 'ORBIT_SOCKETDIR': '/tmp/orbit-tyler', 'LESSOPEN': '|
/usr/bin/lesspipe %s', 'LOGNAME': 'tyler', 'USER': 'tyler', 'PATH':
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games',
'HOME': '/home/tyler', 'WINDOWPATH': '7', 'SSH_AGENT_PID': '3235', 'LANG':
'en_CA.UTF-8', 'TERM': 'xterm', 'SHELL': '/bin/bash', 'XDG_SESSION_COOKIE':
'c40b96e13a53e01daf91b3c24a37cc3f-1245418411.623618-1455422513',
'SESSION_MANAGER': 'local/Surak:/tmp/.ICE-unix/3078', 'SHLVL': '1',
'DISPLAY': ':0.0', 'WINDOWID': '62914615', 'COLUMNS': '80',
'GPG_AGENT_INFO': '/tmp/seahorse-WkObD8/S.gpg-agent:3260:1', 'USERNAME':
'tyler', 'GDM_XSERVER_LOCATION': 'local', 'GTK_RC_FILES':
'/etc/gtk/gtkrc:/home/tyler/.gtkrc-1.2-gnome2', 'LINES': '24',
'SSH_AUTH_SOCK': '/tmp/keyring-glzV2L/socket.ssh', 'DESKTOP_SESSION':
'default', 'GDMSESSION': 'default', 'DBUS_SESSION_BUS_ADDRESS':
'unix:abstract=/tmp/dbus-HXzG5Pt75p,guid=9fc00f8a914d6f800340ca634a3b93ac',
'_': '/usr/bin/python', 'GTK_MODULES': 'canberra-gtk-module',
'GNOME_KEYRING_SOCKET': '/tmp/keyring-glzV2L/socket', 'LESSCLOSE':
'/usr/bin/lesspipe %s %s', 'GNOME_KEYRING_PID': '3065', '__GL_YIELD':
'NOTHING', 'GDM_LANG': 'en_CA.UTF-8', 'HISTCONTROL': 'ignoreboth',
'XDG_DATA_DIRS': '/usr/local/share/:/usr/share/:/usr/share/gdm/', 'PWD':
'/home/tyler', 'COLORTERM': 'gnome-terminal', 'FROM_WRAPPER': 'yes',
'LS_COLORS':
'no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.svgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:'}
ty...@surak:~$ cat argv
['./testsub.py', '-I', 'rc']
On Fri, Jun 19, 2009 at 8:21 AM, Mike Kazantsev mk.frag...@gmail.comwrote:
On Fri, 19 Jun 2009 08:07:29 -0700
Tyler Laing trinio...@gmail.com wrote:
I can't use communicate, as it waits for the child process to terminate.
Basically it blocks. I'm trying to have dynamic communication between the
python program, and vlc.
Unfortunately, subprocess module doesn't allow it out-of-the-box, but
you can use fnctl calls to perform non-blocking reads/writes on it's
pipes, like this:
flags = fcntl.fcntl(fd, fcntl.F_GETFL)
fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)
After that, you can grab all the available data from the pipe at any
given time w/o blocking.
Try this recipe:
http://code.activestate.com/recipes/576759/
--
Mike Kazantsev // fraggod.net
--
http://mail.python.org/mailman/listinfo/python-list
--
Visit my blog at http://oddco.ca/zeroth/zblog
--
http://mail.python.org/mailman/listinfo/python-list
Re: Retrieving column values by column name with MySQLdb
jorma kala wrote:
Hi,
Is there a way of retrieving the value of columns in the rows returned by
fetchall, by column name instead of index on the row?
Code Snippet:
query=select * from employees
db=MySQLdb.connect(host=host,user=user,passwd=passwd,db=database)
cursor = db.cursor ()
cursor.execute (query)
rows = cursor.fetchall ()
for row in rows:
print row[0]
Instead of specifying the index of the row to retrieve the first column
(row[0]), I'd like to retrieve the value of the first column by column name.
Something like row.get('employee_id')
Is something of the sort possible with Mysqdb?
Mike gave you a good answer, though I think it's MySQL specific.
For a more generic solution:
cursor.execute(query)
name_to_index = dict(
(d[0], i)
for i, d
in enumerate(cursor.description)
)
rows = cursor.fetchall()
for row in rows:
print row[name_to_index['employee_id']]
Or in case you have lots of column-names, a simple lambda can
ease the typing required:
for row in rows:
item = lambda col_name: row[name_to_index[col_name]]
print item('employee_id')
The built-in sqlite3 module also has a way to tell results to
come back as a dict[1]
Note in each case the column-name indexing is case-sensitive.
Hope this helps,
-tim
[1]
http://docs.python.org/library/sqlite3.html#sqlite3.Connection.row_factory
--
http://mail.python.org/mailman/listinfo/python-list
Re: sqlite3, OperationalError: no such column, shouldn't that ne a ProgrammingError?
On Jun 19, 8:20 pm, Gerhard Häring g...@ghaering.de wrote: Gabriel Rossetti wrote: Hello everyone, I get an OperationalError with sqlite3 if I put the wrong column name, but shouldn't that be a ProgrammingError instead? I read PEP 249 and it says : OperationalError Exception raised for errors that are related to the database's operation and not necessarily under the control of the programmer, e.g. an unexpected disconnect occurs, the data source name is not found, a transaction could not be processed, a memory allocation error occurred during processing, etc. It must be a subclass of DatabaseError. ProgrammingError Exception raised for programming errors, e.g. table not found or already exists, syntax error in the SQL statement, wrong number of parameters specified, etc. It must be a subclass of DatabaseError. and to me it sounds more like a programming error than an operational error. I agree. But I can't help you there. See _pysqlite_seterror() athttp://oss.itsystementwicklung.de/trac/pysqlite/browser/src/util.c What I do is map SQLite error codes to DB-API exceptions. And SQLite reports your error as generic SQLITE_ERROR, which is mapped to OperationalError. Hi Gerhard, In http://www.sqlite.org/c3ref/prepare.html it says When an error occurs, sqlite3_step() will return one of the detailed error codes or extended error codes. The legacy behavior was that sqlite3_step() would only return a generic SQLITE_ERROR result code and you would have to make a second call to sqlite3_reset() in order to find the underlying cause of the problem. With the v2 prepare interfaces, the underlying reason for the error is returned immediately. Are you using sqlite3_prepare() or sqlite3_prepare_v2()? sqlite3_errcode() or sqlite3_extended_errcode()? Cheers, John -- http://mail.python.org/mailman/listinfo/python-list
Re: Decorator question (how to test if decorated function is in a class)
Martin P. Hellwig a écrit : Hi all, I have been trying out to wrap my mind around the advantages of decorators and thought I found a use in one of my experiments. (see code after my sig). Although it works, I think it should be able to do it better. My particular problem is that I want to remove an argument (say always the first one) when the decorated function is called. However if the function is defined in a class the first argument is self and thus the second argument should be removed. You'll have the same problem with a function defined outside a class then attached to the class, ie: def func(...): pass class Foo(object): pass Foo.meth = func Since I like to have a more general DRY approach I want to test if the decorated function is defined in a class or not thus knowing that the first argument is self and must be preserved. Short answer: this makes no sense. I tried different approaches but all didn't work and looked very unpythonic (looking at the attributes, descriptors namespaces, id() of the function, the first argument and the decorator itself). So is there a way to find out if the first argument is 'self' without making a false positive if the first argument is a class instance passed to a function? The solution to your problem is elsewhere... Read the doc about the descriptor protocol, and how it's implemented by the function type to turn functions into methods when they are looked up as attribute of a class... -- http://mail.python.org/mailman/listinfo/python-list
Re: Retrieving column values by column name with MySQLdb
On Fri, 19 Jun 2009 15:46:46 +0100 jorma kala jjk...@gmail.com wrote: Is there a way of retrieving the value of columns in the rows returned by fetchall, by column name instead of index on the row? Try this: db = MySQLdb.Connection(host=host,user=user,passwd=passwd,db=database) db.query(query) result = db.store_result() data = result.fetch_row(maxrows=0, how=1) -- Mike Kazantsev // fraggod.net signature.asc Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: Retrieving column values by column name with MySQLdb
On Fri, 19 Jun 2009 10:32:32 -0500 Tim Chase python.l...@tim.thechases.com wrote: Mike gave you a good answer, though I think it's MySQL specific. I don't have to deal with MySQL frequently but I've remembered that I used got the fields out somehow, and now, looking at the code, I wonder myself, why how is 1 and wtf is this how, anyway!? ;) I can't seem to find any mention of such methods in documentation and even python source, guess they are implemented directly in underlying C lib. Hope I learned to abstract from such syntax since then, I sure do... -- Mike Kazantsev // fraggod.net signature.asc Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this pylint error message valid or silly?
Matthew Wilson wrote: Thanks for the feedback. I think I should have used a more obvious string in my original example and a more descriptive parameter name. So, pretend that instead of c=today I wrote record_date=defaults to today's date. I know my way is unorthodox, Which is fine. Just do not by surprised when a fake default fakes out pylint, which by its nature, must embody the orthodoxy of its author ;-) To win general acceptance, that in turn must match the orthodoxy of the general audience. but I think it is a little bit more obvious to the reader than record_date=None The None is a signal to use a default value, but that is only apparent after reading the code. Thanks again for the comments. Matt -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
Jure Erznožnik wrote: See here for introduction: http://groups.google.si/group/comp.lang.python/browse_thread/thread/370f8a1747f0fb91 Digging through my problem, I discovered Python isn't exactly thread safe and to solve the issue, there's this Global Interpreter Lock (GIL) in place. Effectively, this causes the interpreter to utilize one core when threading is not used and .95 of a core when threading is utilized. You might want to read about The Problem with Threads: http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf and then decide to switch to an appropriate concurrency model for your use case. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling subprocess with arguments
On Fri, 19 Jun 2009 08:28:17 -0700 Tyler Laing trinio...@gmail.com wrote: Thanks mike, the idea that maybe some of the info isn't being passed is certainly interesting. Here's the output of os.environ and sys.argv: ... I'm afraid these doesn't make much sense without the output from the second results, from py itself. My suggestion was just to compare them - pop the py shell, eval the outputs into two sets, do the diff and you'll see it at once. If there's an empty set then I guess it's pretty safe to assume that python creates subprocess in the same way the shell does. -- Mike Kazantsev // fraggod.net signature.asc Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: Integer Division
Anjanesh Lekshminarayanan wrote: a = 1 b = 25 a / b 0 float(a) / b 0.040001 from __future__ import division a = 1 b = 25 a / b 0.040001 In what simple way can I get just 0.04 ? Short answer: use 3.1: 1//25 0 1/25 0.04 ;-) But you should really try to understand the answers others gave. tjr -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling subprocess with arguments
On Fri, 19 Jun 2009 22:00:28 +0600 Mike Kazantsev mk.frag...@gmail.com wrote: On Fri, 19 Jun 2009 08:28:17 -0700 Tyler Laing trinio...@gmail.com wrote: Thanks mike, the idea that maybe some of the info isn't being passed is certainly interesting. Here's the output of os.environ and sys.argv: ... I'm afraid these doesn't make much sense without the output from the second results, from py itself. My suggestion was just to compare them - pop the py shell, eval the outputs into two sets, do the diff and you'll see it at once. If there's an empty set then I guess it's pretty safe to assume that python creates subprocess in the same way the shell does. Just thought of one more really simple thing I've missed: vlc might expect it's remote to work with tty, so when py shoves it a pipe instead, it automatically switches to non-interactive mode. You can remedy that a bit by superclassing subprocess.Popen, replacing pipes with pty, but they are quite hard to work with, prehaps pexpect module would be of some use there: http://pypi.python.org/pypi/pexpect/ -- Mike Kazantsev // fraggod.net signature.asc Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
Jure Erznožnik wrote: See here for introduction: http://groups.google.si/group/comp.lang.python/browse_thread/thread/370f8a1747f0fb91 Digging through my problem, I discovered Python isn't exactly thread safe and to solve the issue, there's this Global Interpreter Lock (GIL) in place. Effectively, this causes the interpreter to utilize one core when threading is not used and .95 of a core when threading is utilized. Python does not have (or not have) GIL. It is an implementation issue. CPython uses it, to good effect. Is there any work in progess on core Python modules that will permanently resolve this issue? Is there any other way to work around the issue aside from forking new processes or using something else? Use one of the other implementations. Jython, IronPython, Pypy, ??? -- http://mail.python.org/mailman/listinfo/python-list
n00b confusion re: local variable referenced before assignment error
Writing a class which essentially spiders a site and saves the files locally. On a URLError exception, it sleeps for a second and tries again (on 404 it just moves on). The relevant bit of code, including the offending method: class Handler(threading.Thread): def __init__(self, url): threading.Thread.__init__(self) self.url = url def save(self, uri, location): try: handler = urllib2.urlopen(uri) except urllib2.HTTPError, e: if e.code == 404: return else: print retrying %s (HTTPError) % uri time.sleep(1) self.save(uri, location) except urllib2.URLError, e: print retrying %s % uri time.sleep(1) self.save(uri, location) if not os.path.exists(os.path.dirname(location)): os.makedirs(os.path.dirname(location)) file = open(location, w) file.write(handler.read()) file.close() ... But what I am seeing is that after a retry (on catching a URLError exception), I see bunches of UnboundLocalError: local variable 'handler' referenced before assignment errors on line 38, which is the file.write(handler.read()) line.. What gives? -- Wells Oliver we...@submute.net -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling subprocess with arguments
Sorry, XD. I'll ask the VLC people if they happen to know why VLC won't open up the remote interface. -Tyler On Fri, Jun 19, 2009 at 9:09 AM, Mike Kazantsev mk.frag...@gmail.comwrote: On Fri, 19 Jun 2009 22:00:28 +0600 Mike Kazantsev mk.frag...@gmail.com wrote: On Fri, 19 Jun 2009 08:28:17 -0700 Tyler Laing trinio...@gmail.com wrote: Thanks mike, the idea that maybe some of the info isn't being passed is certainly interesting. Here's the output of os.environ and sys.argv: ... I'm afraid these doesn't make much sense without the output from the second results, from py itself. My suggestion was just to compare them - pop the py shell, eval the outputs into two sets, do the diff and you'll see it at once. If there's an empty set then I guess it's pretty safe to assume that python creates subprocess in the same way the shell does. Just thought of one more really simple thing I've missed: vlc might expect it's remote to work with tty, so when py shoves it a pipe instead, it automatically switches to non-interactive mode. You can remedy that a bit by superclassing subprocess.Popen, replacing pipes with pty, but they are quite hard to work with, prehaps pexpect module would be of some use there: http://pypi.python.org/pypi/pexpect/ -- Mike Kazantsev // fraggod.net -- http://mail.python.org/mailman/listinfo/python-list -- Visit my blog at http://oddco.ca/zeroth/zblog -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b confusion re: local variable referenced before assignment error
On Fri, 19 Jun 2009 11:16:38 -0500 Wells Oliver we...@submute.net wrote: def save(self, uri, location): try: handler = urllib2.urlopen(uri) except urllib2.HTTPError, e: if e.code == 404: return else: print retrying %s (HTTPError) % uri time.sleep(1) self.save(uri, location) except urllib2.URLError, e: print retrying %s % uri time.sleep(1) self.save(uri, location) if not os.path.exists(os.path.dirname(location)): os.makedirs(os.path.dirname(location)) file = open(location, w) file.write(handler.read()) file.close() But what I am seeing is that after a retry (on catching a URLError exception), I see bunches of UnboundLocalError: local variable 'handler' referenced before assignment errors on line 38, which is the file.write(handler.read()) line.. What gives? Why not? Try fails, except calls retry and after the retry code execution continues to the undefined handler, since the try has failed here. You might want to insert return or avoid (possibly endless) recursion altogether - just wrap it into while loop with some counter (aka max_tries). Also, you can get rid of code duplication by catching some basic urllib2 exception, then checking if it's urllib2.HTTPError and it's code is 404, retrying (continue for the loop case) otherwise. -- Mike Kazantsev // fraggod.net signature.asc Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b confusion re: local variable referenced before assignment error
On Jun 19, 10:16 am, Wells Oliver we...@submute.net wrote: Writing a class which essentially spiders a site and saves the files locally. On a URLError exception, it sleeps for a second and tries again (on 404 it just moves on). The relevant bit of code, including the offending method: [snip] But what I am seeing is that after a retry (on catching a URLError exception), I see bunches of UnboundLocalError: local variable 'handler' referenced before assignment errors on line 38, which is the file.write(handler.read()) line.. What gives? 'Handler' is only assigned in the try statement, so if you error into your exception clause, nothing will have been bound to the name 'handler', causing the exception you're seeing. -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b confusion re: local variable referenced before assignment error
Wells Oliver schrieb: Writing a class which essentially spiders a site and saves the files locally. On a URLError exception, it sleeps for a second and tries again (on 404 it just moves on). The relevant bit of code, including the offending method: class Handler(threading.Thread): def __init__(self, url): threading.Thread.__init__(self) self.url = url def save(self, uri, location): try: handler = urllib2.urlopen(uri) except urllib2.HTTPError, e: if e.code == 404: return else: print retrying %s (HTTPError) % uri time.sleep(1) self.save(uri, location) except urllib2.URLError, e: print retrying %s % uri time.sleep(1) self.save(uri, location) if not os.path.exists(os.path.dirname(location)): os.makedirs(os.path.dirname(location)) file = open(location, w) file.write(handler.read()) file.close() ... But what I am seeing is that after a retry (on catching a URLError exception), I see bunches of UnboundLocalError: local variable 'handler' referenced before assignment errors on line 38, which is the file.write(handler.read()) line.. Your code defines the name handler only if the urllib2.urlopen is successful. But you try later to access it uncoditionally, and of course that fails. You need to put the file-stuff after the urlopen, inside the try-except. Also note that python has no tail-recursion-optimization, so your method will recurse and at some point exhaust the stack if there are many errors. You should consider writing it rather as while-loop, with breaking out of it when the page could be fetched. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling subprocess with arguments
It appears to be an issue specifically with VLC, not subprocess. Thank you guys. The remote interface works through sockets, which is perfectly fine... if I create a local socket, I can have it connect to the socket with command line arguments. On Fri, Jun 19, 2009 at 9:30 AM, Tyler Laing trinio...@gmail.com wrote: Sorry, XD. I'll ask the VLC people if they happen to know why VLC won't open up the remote interface. -Tyler On Fri, Jun 19, 2009 at 9:09 AM, Mike Kazantsev mk.frag...@gmail.comwrote: On Fri, 19 Jun 2009 22:00:28 +0600 Mike Kazantsev mk.frag...@gmail.com wrote: On Fri, 19 Jun 2009 08:28:17 -0700 Tyler Laing trinio...@gmail.com wrote: Thanks mike, the idea that maybe some of the info isn't being passed is certainly interesting. Here's the output of os.environ and sys.argv: ... I'm afraid these doesn't make much sense without the output from the second results, from py itself. My suggestion was just to compare them - pop the py shell, eval the outputs into two sets, do the diff and you'll see it at once. If there's an empty set then I guess it's pretty safe to assume that python creates subprocess in the same way the shell does. Just thought of one more really simple thing I've missed: vlc might expect it's remote to work with tty, so when py shoves it a pipe instead, it automatically switches to non-interactive mode. You can remedy that a bit by superclassing subprocess.Popen, replacing pipes with pty, but they are quite hard to work with, prehaps pexpect module would be of some use there: http://pypi.python.org/pypi/pexpect/ -- Mike Kazantsev // fraggod.net -- http://mail.python.org/mailman/listinfo/python-list -- Visit my blog at http://oddco.ca/zeroth/zblog -- Visit my blog at http://oddco.ca/zeroth/zblog -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
On 19 juin, 16:16, Martin von Loewis martin.vonloe...@hpi.uni-: If you know that your (C) code is thread safe on its own, you can release the GIL around long-running algorithms, thus using as many CPUs as you have available, in a single process. what do you mean ? Cpython can't benefit from multi-core without multiple processes. Olivier -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
CPython itself can't... but the c extension can. Mine did. On Fri, Jun 19, 2009 at 9:50 AM, OdarR olivier.da...@gmail.com wrote: On 19 juin, 16:16, Martin von Loewis martin.vonloe...@hpi.uni-: If you know that your (C) code is thread safe on its own, you can release the GIL around long-running algorithms, thus using as many CPUs as you have available, in a single process. what do you mean ? Cpython can't benefit from multi-core without multiple processes. Olivier -- http://mail.python.org/mailman/listinfo/python-list -- Visit my blog at http://oddco.ca/zeroth/zblog -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
If you know that your (C) code is thread safe on its own, you can release the GIL around long-running algorithms, thus using as many CPUs as you have available, in a single process. Olivier what do you mean ? Olivier Cpython can't benefit from multi-core without multiple Olivier processes. It can, precisely as Martin indicated. Only one thread at a time can hold the GIL. That doesn't mean that multiple threads can't execute. Suppose you have two threads, one of which winds up executing some bit of C code which doesn't mess with the Python run-time at all (say, a matrix multiply). Before launching into the matrix multiply, the extension module releases the GIL then performs the multiply. With the GIL released another thread can acquire it. Once the multiply finishes the first thread needs to reacquire the GIL before executing any calls into the Python runtime or returning. -- Skip Montanaro - s...@pobox.com - http://www.smontanaro.net/ when i wake up with a heart rate below 40, i head right for the espresso machine. -- chaos @ forums.usms.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
On 19 juin, 19:13, s...@pobox.com wrote: Olivier what do you mean ? Olivier Cpython can't benefit from multi-core without multiple Olivier processes. It can, precisely as Martin indicated. Only one thread at a time can hold the GIL. That doesn't mean that multiple threads can't execute. Suppose I don't say multiple threads can't execute(?). I say that with the Python library, I don't see (yet) benefit with multiple threads *on* multiple CPU/core. Ever seen this recent video/presentation ? : http://blip.tv/file/2232410 http://www.dabeaz.com/python/GIL.pdf you have two threads, one of which winds up executing some bit of C code which doesn't mess with the Python run-time at all (say, a matrix multiply). I don't know how to do that with common Python operations... Only one thread will be really running at a time in memory (meanwhile other thread are waiting). Are you refering to a specialized code ? Before launching into the matrix multiply, the extension module releases the GIL then performs the multiply. With the GIL released another thread can acquire it. Once the multiply finishes the first thread needs to reacquire the GIL before executing any calls into the Python runtime or returning. I don't see such improvement in the Python library, or maybe you can indicate us some meaningfull example...? I currently only use CPython, with PIL, Reportlab...etc. I don't see improvement on a Core2duo CPU and Python. How to proceed (following what you wrote) ? A contrario, I saw *real* improvement on parallel computing with the Py 2.6 multiprocessing module. Olivier -- http://mail.python.org/mailman/listinfo/python-list
Convert hash to struct
Hello, I am retrieving values from a database in the form of a dictionary so I can access the values as d['column'] and I was wondering if there is a way to convert the hash to a struct like format so i can just say d.column. Makes it easier to read and understand. Thanks Amita -- Amita Ekbote -- http://mail.python.org/mailman/listinfo/python-list
Re: Measuring Fractal Dimension ?
Evidently my posts are appearing, since I see replies. I guess the question of why I don't see the posts themselves \is ot here... On Thu, 18 Jun 2009 17:01:12 -0700 (PDT), Mark Dickinson dicki...@gmail.com wrote: On Jun 18, 7:26 pm, David C. Ullrich ullr...@math.okstate.edu wrote: On Wed, 17 Jun 2009 08:18:52 -0700 (PDT), Mark Dickinson Right. Or rather, you treat it as the image of such a function, if you're being careful to distinguish the curve (a subset of R^2) from its parametrization (a continuous function R - R**2). It's the parametrization that's uniformly continuous, not the curve, Again, it doesn't really matter, but since you use the phrase if you're being careful: In fact what you say is exactly backwards - if you're being careful that subset of the plane is _not_ a curve (it's sometimes called the trace of the curve. Darn. So I've been getting it wrong all this time. Oh well, at least I'm not alone: De?nition 1. A simple closed curve J, also called a Jordan curve, is the image of a continuous one-to-one function from R/Z to R2. [...] - Tom Hales, in 'Jordan's Proof of the Jordan Curve Theorem'. We say that Gamma is a curve if it is the image in the plane or in space of an interval [a, b] of real numbers of a continuous function gamma. - Claude Tricot, 'Curves and Fractal Dimension' (Springer, 1995). Perhaps your definition of curve isn't as universal or 'official' as you seem to think it is? Perhaps not. I'm very surprised to see those definitions; I've been a mathematician for 25 years and I've never seen a curve defined a subset of the plane. Hmm. You left out a bit in the first definition you cite: A simple closed curve J, also called a Jordan curve, is the image of a continuous one-to-one function from R/Z to R2. We assume that each curve comes with a fixed parametrization phi_J : R/Z -¨ J. We call t in R/Z the time parameter. By abuse of notation, we write J(t) in R2 instead of phi_j (t), using the same notation for the function phi_J and its image J. Close to sounding like he can't decide whether J is a set or a function... Then later in the same paper Definition 2. A polygon is a Jordan curve that is a subset of a finite union of lines. A polygonal path is a continuous function P : [0, 1] -¨ R2 that is a subset of a finite union of lines. It is a polygonal arc, if it is 1 . 1. By that definition a polygonal path is not a curve. Worse: A polygonal path is a function from [0,1] to R^2 _that is a subset of a finite union of lines_. There's no such thing - the _image_ of such a function can be a subset of a finite union of lines. Not that it matters, but his defintion of polygonal path is, _if_ we're being very careful, self-contradictory. So I don't think we can count that paper as a suitable reference for what the _standard_ definitions are; the standard definitions are not self-contradictory this way. Then the second definition you cite: Amazon says the prerequisites are two years of calculus. The stanard meaning of log is log base e, even though it means log base 10 in calculus. Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: Convert hash to struct
On Fri, 19 Jun 2009 13:17:24 -0500
Amita Ekbote amita.ekb...@gmail.com wrote:
I am retrieving values from a database in the form of a dictionary so
I can access the values as d['column'] and I was wondering if there is
a way to convert the hash to a struct like format so i can just say
d.column. Makes it easier to read and understand.
Are there enough clues here?
class MyDict(dict):
def __getattribute__(self, name):
return dict.__getattribute__(self, name)
def __getattr__(self, name):
return self.get(name, 42)
x = MyDict({'a': 1, 'b': 2, 'values': 3})
print x.a
print x.z
print x.values
Big question - what should the last line display? If you expect 3
and not built-in method values of MyDict object at 0xbb82838c then
you need to reconsider the above implementation. Thinking about the
question may change your opinion about this being a good idea after all.
--
D'Arcy J.M. Cain da...@druid.net | Democracy is three wolves
http://www.druid.net/darcy/| and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner.
--
http://mail.python.org/mailman/listinfo/python-list
Re: walking a directory with very many files
Lawrence D'Oliveiro wrote: In message %zv_l.19493$y61.5...@news-server.bigpond.net.au, Lie Ryan wrote: Yeah, it might be possible to just mv the file from outside, but not being able to enter a directory just because you've got too many files in it is kind of silly. Sounds like a problem with your file/directory-manipulation tools. try an `ls` on a folder with 1+ files. See how long is needed to print all the files. Ok, now pipe ls to less, take three days to browse through all the filenames to locate the file you want to see. The file manipulation tool may not have problems with it; it's the user that would have a hard time sorting through the huge amount of files. Even with glob and grep, some types of queries are just too difficult or is plain silly to write a full fledged one-time use program just to locate a few files. -- http://mail.python.org/mailman/listinfo/python-list
Re: Convert hash to struct
Amita Ekbote wrote:
Hello,
I am retrieving values from a database in the form of a dictionary so
I can access the values as d['column'] and I was wondering if there is
a way to convert the hash to a struct like format so i can just say
d.column. Makes it easier to read and understand.
Thanks
Amita
You may be able to update the class' dict:
class MyDB(object):
... def __init__(self, di):
... self.__dict__.update(di)
...
di = {'a':10, 'b': 20}
d = MyDB(di)
d
__main__.MyDB object at 0x7f756b0d0b90
d.a
10
d.b
20
but this might be a security risk, if you cannot trust the database or
its content.
It is much preferrable to use something like:
class MyDB(object):
... def __init__(self, di):
... self.a = di['a']
... self.b = di['b']
since you now have full control of what collumns can go in and whatnot.
--
http://mail.python.org/mailman/listinfo/python-list
Rich comparison methods don't work in sets?
Hello, everyone. I've noticed that if I have a class with so-called rich comparison methods (__eq__, __ne__, etc.), when its instances are included in a set, set.__contains__/__eq__ won't call the .__eq__ method of the elements and thus the code below: obj1 = RichComparisonClass() obj2 = RichComparisonClass() set1 = set([obj1]) set2 = set([obj2]) if obj1 == obj2: print Objects 1 and 2 are equivalent else: print Objects 1 and 2 aren't equivalent if set1 == set2: print Sets 1 and 2 are equivalent else: print Sets 1 and 2 aren't equivalent Would output: Objects 1 and 2 are equivalent Sets 1 and 2 aren't equivalent instead of: Objects 1 and 2 are equivalent Sets 1 and 2 are equivalent How can I work around this? The only solution that comes to my mind is to write a function like this: def same_sets(set1, set2): if len(set1) != len(set2): return False for element1 in set1: found = False for element2 in set2: if element1 == element2: found = True break; if not found: return False return True But I see it pretty ugly; also I'd also have to roll out my own contains (e.g., `element in set`) function. I expect and hope there's a pythonic way to do this. Does it exist? Thanks in advance! -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
OdarR wrote: I don't see such improvement in the Python library, or maybe you can indicate us some meaningfull example...? I currently only use CPython, with PIL, Reportlab...etc. I don't see improvement on a Core2duo CPU and Python. How to proceed (following what you wrote) ? I've seen a single Python process using the full capacity of up to 8 CPUs. The application is making heavy use of lxml for large XSL transformations, a database adapter and my own image processing library based upon FreeImage. Of course both lxml and my library are written with the GIL in mind. They release the GIL around every call to C libraries that don't touch Python objects. PIL releases the lock around ops as well (although it took me a while to figure it out because PIL uses its own API instead of the standard macros). reportlab has some optional C libraries that increase the speed, too. Are you using them? By the way threads are evil (http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf) and not *the* answer to concurrency. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Status of Python threading support (GIL removal)?
On Jun 19, 6:53 am, Ben Charrow bchar...@csail.mit.edu wrote: Jure Erznožnik wrote: See here for introduction: http://groups.google.si/group/comp.lang.python/browse_thread/thread/3... Digging through my problem, I discovered Python isn't exactly thread safe and to solve the issue, there's this Global Interpreter Lock (GIL) in place. Effectively, this causes the interpreter to utilize one core when threading is not used and .95 of a core when threading is utilized. Is there any work in progess on core Python modules that will permanently resolve this issue? Is there any other way to work around the issue aside from forking new processes or using something else? There is a group of people working on an alternative implementation to Python that, among other things, will not have a GIL:http://code.google.com/p/unladen-swallow/ That's not a foregone conclusion. Well it's not a foregone conclusion that unladen-swallow will succeed at all, but even if it does they only say they intend to remove the GIL, not that they necessarily will. The GIL actually solves two problems: the overhead of synchronizing reference counts, and the difficulty of writing threaded extensions. The unladen-swallow team only address the first problem in their plans. So, even if they do remove the GIL, I doubt GvR will allow it to be merged back into CPython unless makes extensions are just as easy to write. That is something I have serious doubts they can pull off. Which means a GIL-less unladen-swallow is likely to end being another fork like IronPython and Jython. Those projects already have no GIL. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Measuring Fractal Dimension ?
On Jun 19, 2009, at 2:43 PM, David C. Ullrich wrote: Evidently my posts are appearing, since I see replies. I guess the question of why I don't see the posts themselves \is ot here... On Thu, 18 Jun 2009 17:01:12 -0700 (PDT), Mark Dickinson dicki...@gmail.com wrote: On Jun 18, 7:26 pm, David C. Ullrich ullr...@math.okstate.edu wrote: On Wed, 17 Jun 2009 08:18:52 -0700 (PDT), Mark Dickinson Right. Or rather, you treat it as the image of such a function, if you're being careful to distinguish the curve (a subset of R^2) from its parametrization (a continuous function R - R**2). It's the parametrization that's uniformly continuous, not the curve, Again, it doesn't really matter, but since you use the phrase if you're being careful: In fact what you say is exactly backwards - if you're being careful that subset of the plane is _not_ a curve (it's sometimes called the trace of the curve. Darn. So I've been getting it wrong all this time. Oh well, at least I'm not alone: De?nition 1. A simple closed curve J, also called a Jordan curve, is the image of a continuous one-to-one function from R/Z to R2. [...] - Tom Hales, in 'Jordan's Proof of the Jordan Curve Theorem'. We say that Gamma is a curve if it is the image in the plane or in space of an interval [a, b] of real numbers of a continuous function gamma. - Claude Tricot, 'Curves and Fractal Dimension' (Springer, 1995). Perhaps your definition of curve isn't as universal or 'official' as you seem to think it is? Perhaps not. I'm very surprised to see those definitions; I've been a mathematician for 25 years and I've never seen a curve defined a subset of the plane. I have. Hmm. You left out a bit in the first definition you cite: A simple closed curve J, also called a Jordan curve, is the image of a continuous one-to-one function from R/Z to R2. We assume that each curve comes with a fixed parametrization phi_J : R/Z -¨ J. We call t in R/Z the time parameter. By abuse of notation, we write J(t) in R2 instead of phi_j (t), using the same notation for the function phi_J and its image J. Close to sounding like he can't decide whether J is a set or a function... On the contrary, I find this definition to be written with some care. Then later in the same paper Definition 2. A polygon is a Jordan curve that is a subset of a finite union of lines. A polygonal path is a continuous function P : [0, 1] -¨ R2 that is a subset of a finite union of lines. It is a polygonal arc, if it is 1 . 1. These are a bit too casual for me as well... By that definition a polygonal path is not a curve. Worse: A polygonal path is a function from [0,1] to R^2 _that is a subset of a finite union of lines_. There's no such thing - the _image_ of such a function can be a subset of a finite union of lines. Not that it matters, but his defintion of polygonal path is, _if_ we're being very careful, self-contradictory. So I don't think we can count that paper as a suitable reference for what the _standard_ definitions are; the standard definitions are not self-contradictory this way. Charles Yeomans -- http://mail.python.org/mailman/listinfo/python-list
timeit and __future__
I wanted to time something that uses with_statement, in python2.5. Importing __future__ in the statement or the setup doesn't work since it's not the beginning of the code being compiled. Other than using a separate module, I could only come up with this: timeit.template = 'from __future__ import with_statement\n' + timeit.template timeit should directly support importing __future__ stuff... -- http://mail.python.org/mailman/listinfo/python-list
dynamically associate radio buttons with droplists
Hi Folks, Not being very familiar with python, nor with cgi/http, I intend to have 3 of buttons in a webpage, each of them is associate with a file (so I have 3 files, too) What I would like to have is, when users choose a button, the droplist update automatically to load the contents of the associated file. Can someone educate me the approach of this? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Rich comparison methods don't work in sets?
On Fri 19 Jun 2009 03:02:44 PM EDT, Gustavo Narea wrote: Hello, everyone. I've noticed that if I have a class with so-called rich comparison methods (__eq__, __ne__, etc.), when its instances are included in a set, set.__contains__/__eq__ won't call the .__eq__ method of the elements and thus the code below: obj1 = RichComparisonClass() obj2 = RichComparisonClass() What does obj1 is obj2 return? I don't know anything about set internals. Matt -- http://mail.python.org/mailman/listinfo/python-list
Re: n00b confusion re: local variable referenced before assignment error
Wells Oliver wrote: Writing a class which essentially spiders a site and saves the files locally. On a URLError exception, it sleeps for a second and tries again (on 404 it just moves on). The relevant bit of code, including the offending method: class Handler(threading.Thread): def __init__(self, url): threading.Thread.__init__(self) self.url = url def save(self, uri, location): try: handler = urllib2.urlopen(uri) except urllib2.HTTPError, e: if e.code == 404: return else: print retrying %s (HTTPError) % uri time.sleep(1) self.save(uri, location) except urllib2.URLError, e: print retrying %s % uri time.sleep(1) self.save(uri, location) if not os.path.exists(os.path.dirname(location)): os.makedirs(os.path.dirname(location)) file = open(location, w) file.write(handler.read()) file.close() ... But what I am seeing is that after a retry (on catching a URLError exception), I see bunches of UnboundLocalError: local variable 'handler' referenced before assignment errors on line 38, which is the file.write(handler.read()) line.. What gives? Your save() method is recursive in the case of that error, which is a poor excuse for what should have been a loop. You should have some retry depth check, just in case you get hundreds of such errors on the same site. But ignoring that issue, the specific symptom is caused when returning from the recursive call. You still fall through to the end of your outer method call, and try to write stuff from that handler (also wiping out the file location in the process). Without a handler, that causes an exception. So you should follow those calls to self.save() with return's. Much better would be to write a loop in the first place, and break out of the loop when you succeed. Then the flow is much easier to follow, and you can easily avoid the risk of looping forever, nor of running out of stack space. Something like (untested): def save(self, uri, location): for tries in xrange(10):#try up to 10 times try: handler = urllib2.urlopen(uri) except urllib2.HTTPError, e: if e.code == 404: break else: print retrying %s (HTTPError) % uri time.sleep(1) continue except urllib2.URLError, e: print retrying %s % uri time.sleep(1) continue if not os.path.exists(os.path.dirname(location)): os.makedirs(os.path.dirname(location)) file = open(location, w) file.write(handler.read()) file.close() break Other refinements are possible, of course. For example, if you have any cleanup code at the end you may need an additional flag variable to tell whether you've succeeded or not. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is this pylint error message valid or silly?
On Jun 18, 8:56 pm, Matthew Wilson m...@tplus1.com wrote: Here's the code that I'm feeding to pylint: $ cat f.py from datetime import datetime def f(c=today): if c == today: c = datetime.today() return c.date() And here's what pylint says: $ pylint -e f.py No config file found, using default configuration * Module f E: 10:f: Instance of 'str' has no 'date' member (but some types could not be inferred) Is this a valid error message? Is the code above bad? If so, what is the right way? I changed from using a string as the default to None, and then pylint didn't mind: $ cat f.py from datetime import datetime def f(c=None): if c is None: c = datetime.today() return c.date() $ pylint -e f.py No config file found, using default configuration I don't see any difference between using a string vs None. Both are immutable. I find the string much more informative, since I can write out what I want. Looking for comments. Matt def midnight_next_day(initial_time=None): if initial_time == use today's date: initial_time = datetime.now() return initial_time.date() + timedelta(days=1) midnight_next_day() Traceback (most recent call last): File pyshell#27, line 1, in module midnight_next_day() File pyshell#26, line 6, in midnight_next_day return initial_time.date() + timedelta(days=1) AttributeError: 'NoneType' object has no attribute 'date' -- http://mail.python.org/mailman/listinfo/python-list
Re: Rich comparison methods don't work in sets?
Gustavo Narea wrote: I've noticed that if I have a class with so-called rich comparison methods (__eq__, __ne__, etc.), when its instances are included in a set, set.__contains__/__eq__ won't call the .__eq__ method of the elements and thus the code below: obj1 = RichComparisonClass() How is RichComparisonClass implemented? Did you provide a __hash__() method such that obj1 == obj2 implies hash(obj1) == hash(obj2)? This is necessary for instances of the class to work properly with sets and dicts. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Measuring Fractal Dimension ?
On Jun 19, 7:43 pm, David C. Ullrich ullr...@math.okstate.edu wrote:
Evidently my posts are appearing, since I see replies.
I guess the question of why I don't see the posts themselves
\is ot here...
Judging by this thread, I'm not sure that much is off-topic
here. :-)
Perhaps not. I'm very surprised to see those definitions; I've
been a mathematician for 25 years and I've never seen a
curve defined a subset of the plane.
That in turn surprises me. I've taught multivariable
calculus courses from at least three different texts
in three different US universities, and as far as I
recall a 'curve' was always thought of as a subset of
R^2 or R^3 in those courses (though not always with
explicit definitions, since that would be too much
to hope for with that sort of text). Here's Stewart's
'Calculus', p658:
Examples 2 and 3 show that different sets of parametric
equations can represent the same curve. Thus we
distinguish between a *curve*, which is a set of points,
and a *parametric curve*, in which the points are
traced in a particular way.
Again as far as I remember, the rest of the language
in those courses (e.g., 'level curve', 'curve as the
intersection of two surfaces') involves thinking
of curves as subsets of R^2 or R^3. Certainly
I'll agree that it's then necessary to parameterize
the curve before being able to do anything useful
with it.
[Standard question when teaching multivariable
calculus: Okay, so we've got a curve. What's
the first thing we do with it? Answer, shouted
out from all the (awake) students: PARAMETERIZE IT!
(And then calculate its length/integrate the
given vector field along it/etc.)
Those were the days...]
A Google Books search even turned up some complex
analysis texts where the word 'curve' is used to
mean a subset of the plane; check out
the book by Ian Stewart and David Orme Tall,
'Complex Analysis: a Hitchhiker's Guide to the
Plane': they distinguish 'curves' (subset of the
complex plane) from 'paths' (functions from a
closed bounded interval to the complex plane).
Definition 2. A polygon is a Jordan curve that is a subset of a
finite union of
lines. A polygonal path is a continuous function P : [0, 1] -¨ R2
that is a subset of
a finite union of lines. It is a polygonal arc, if it is 1 . 1.
By that definition a polygonal path is not a curve.
Right. I'm much more willing to accept 'path' as standard
terminology for a function [a, b] - (insert_favourite_space_here).
Not that it matters, but his defintion of polygonal path
is, _if_ we're being very careful, self-contradictory.
Agreed. Surprising, coming from Hales; he must surely rank
amongst the more careful mathematicians out there.
So I don't think we can count that paper as a suitable
reference for what the _standard_ definitions are;
the standard definitions are not self-contradictory this way.
I just don't believe there's any such thing as
'the standard definition' of a curve. I'm happy
to believe that in complex analysis and differential
geometry it's common to define a curve to be a
function. But in general I'd suggest that it's one
of those terms that largely depends on context, and
should be defined clearly when it's not totally
obvious from the context which definition is
intended. For example, for me, more often than not,
a curve is a 1-dimensional scheme over a field k
(usually *not* algebraically closed), that's at
least some of {geometrically reduced, geometrically
irreducible, proper, smooth}. That's a far cry either
from a subset of an affine space or from a
parametrization by an interval.
Then the second definition you cite: Amazon says the
prerequisites are two years of calculus. The stanard
meaning of log is log base e, even though means
log base 10 in calculus.
Sorry, I've lost context for this comment. Why
are logs relevant? (I'm very well aware of the
debates over the meaning of log, having frequently
had to help students 'unlearn' their log=log10
mindset when starting a first post-calculus course).
Mark
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is this pylint error message valid or silly?
On Jun 18, 8:56 pm, Matthew Wilson m...@tplus1.com wrote: Here's the code that I'm feeding to pylint: $ cat f.py from datetime import datetime def f(c=today): if c == today: c = datetime.today() return c.date() And here's what pylint says: $ pylint -e f.py No config file found, using default configuration * Module f E: 10:f: Instance of 'str' has no 'date' member (but some types could not be inferred) Is this a valid error message? Is the code above bad? If so, what is the right way? I changed from using a string as the default to None, and then pylint didn't mind: $ cat f.py from datetime import datetime def f(c=None): if c is None: c = datetime.today() return c.date() $ pylint -e f.py No config file found, using default configuration I don't see any difference between using a string vs None. Both are immutable. I find the string much more informative, since I can write out what I want. Looking for comments. Matt def midnight_next_day(initial_time=use today's date): if initial_time == use today's date: initial_time = datetime.now() return initial_time.date() + timedelta(days=1) midnight_next_day() Traceback (most recent call last): File pyshell#24, line 1, in module midnight_next_day() File pyshell#23, line 6, in midnight_next_day return initial_time.date() + timedelta(days=1) AttributeError: 'str' object has no attribute 'date' -- http://mail.python.org/mailman/listinfo/python-list
